A lot can go on in a story. The action and dialogue often motor on at at least a decent clip. But paired with scenes are sequels, and paired with sound should be (from time to time) silence.
You can learn a lot from what you hear when you are silent. The wind. Running water. The buzzing in your ears. Every setting will have different sounds. Why not build your world by listening through the silence to what calls out?
I love the sound in stories, but I can appreciate silence too. Maybe there should be just a little more of it.