I read recently that Stephen King said (and I'm seriously paraphrasin') Don't write down idea's because IF they are good idea's you will remember them anyway-and if you don't remember them-they weren't good idea's.
And here's why.
YOU as the writer won't/don't/can't know what is going to resonate with the reader and move them.
Sure, sometimes you'll think, hey this will work or that will work--but other times a little aside that has little conscious meaning to you as the writer will strike a nerve and move someone-it will resonate and rock their perception and the readers perception of your story is the readers reality.
I know because I have done it. Small tweaks in stories have changed characters into some readers favorites-and that would not have happened if I wasn't taking notes and later perusing them and putting that stuff into the book. I call it Tetrising things in.
It's all well and good for King to throw out that particular advice but the reality is we all forget things and will continue to forget things and at King's age - How many things has he already forgotten? How many sparkling gems has he potentially trod over? I suggest it isn't even a potential-its absolutely already happened countless times.
So I advise, in contrast to a million dollar writer (because he is wrong), take notes because YOU will forget.
The Muse can be fickle, don't ignore her, don't think she'll call back later with the same message, and don't forget to map the journey she is asking you to travel on.