You're thinking that rules and theory are naturally restrictive in that regard, they aren't, ultimately what creates the best works when an author accepts these standards are manipulation of the meaning behind theory and rules. That's how you can create something deftly composed in a rigid manner beautiful in structure and flow of language, but that takes a certain amount of talent that few people possess. There's willingly submitting to these rules and expanding outward from them, and then there's accepting these rules but choosing to build upon them by stretching their application, that gives you Shakespeare or Herman Melville, the former gives you Stephen King or hell, Lovecraft even.
I think the best way to demonstrate my point is to look at the difference between King and Poe, Poe has an inordinate amount of subtext in his works, he doesn't simply play off of fears but generally introduces themes such as the 'soul' of family, guilt, and madness, (specifically in Fall of the House of Usher) but he introduces these themes precisely and in totality, there's very little that isn't connected and doesn't affect the story's expression of morality. It's expertly crafted, but it's also very complex and indepth. Poe is someone who didn't exactly amaze with intuitive narrative, but his writing was spectacular and he made sure to push at the boundaries of rules with every word, he didn't limit himself with them but eventually used them to his own benefit.
I don't feel that with King, even with his better books such as The Stand I see this very stoic narrative that doesn't always factor in every event and exchange of dialog as meaningful. There's the desire to build plot, to develop character, but ultimately the story itself and the characters almost feel separate.
It's a strange sensation in writing that I tend to call 'Unintended Isolation', essentially the characters become independent from the plot as it seems to be that there's a thread of fate, or in fact destiny, which cements the eventual outcome of the story in stone and the characters simply meander along in concurrence with their eventual fate and accomplish tasks and experience events that eventually lead to the ending, but the characters themselves seem encapsulated between the beginning and end. The middle is their freedom, but it's ultimately lacking as the threads don't tie to the main narrative. This turns into mechanical writing.
To be noted is the difference between a book that emphasizes fate and this phenomena, I definitely don't mean novels that carry fate as a theme and explicitly factor it in as a major plot-point, but rather books that seem to have the beginning and end stick out starkly in comparison to the rest of the narrative. The beginning seems to be merely an introduction, the end a conclusion. The ideal book causes these two to flow together, the middle is simply a joining of the two and the transition between events is seamless.
Stephen King in my experience is very much a beginning/end author. His endings typically suck, yes, but despite that I always feel that his middle segments exist merely to reach the ending, rather than to establish a regular flow of relevant events.
But I guess that's why you prefer his short stories, which are typically much better, but I still feel that disconnect.
It may be preference, but often I miss that totality of Poe, of Melville, of Joyce, everything is meaningful and nothing is superfluous, nothing is meant for the purpose of pace, or carrying the story effectively, it's only through the inherent nature of writing that establishing that totality delivers satisfying work.