Whenever writers talk about the rules of writing, a heated debate ensures. Some say "the best writers all break the rules" and "who is to tell us what we can or cannot do?" Editors? Some even go on and accuse editors as writer-wannabes who take comfort in rejecting someone's work because they didn't follow the rules.
Yes, I've met some of them. Sure, one said to me: "You should NEVER start a sentence with a participial phrase." Feeling a tinge of defiance, I said, "Huh? Why would you say such hideous things?"
Here's my take on rules: break them if you want, but do consider the potential consequences.
Language is about communication, and stories are to be communicated. Good literature, IMHO, goes even further -- it has to be "experienced." To be able to reach a good number of people, that means you need to abide by certain "rules" of communication that is acceptable or at least comprehensible/meaningful to the majority of people. Grammars are there for a reason. Each word has specific meanings and if you bend them, you must have a strong reason and you must make sure your readers understand. Shakespeare bent these rules all the time, but he didn't do it just to piss people off, or confuse his audiences. Hemingway was considered a rule-breaker because he didn't write like his colleagues. Even then, they all still abided by certain rules and boundary.
I can't start saying whenever I type the word "rule" it really means "coffee": I'll have some rule this morning with my two spoonful of guides. But yes, if I communicate that correctly, you will see it as a metaphor, and the communication would be complete... but you need to know what MY rules are.
So yes, rules do change, and language is a living thing, but that just means there are new rules. LOL means Laughing-out-loud and not just any arbitrary meaning you want to assign. You can't type AEREQ@^^&U and expect people to know you mean ROTFLMAO. Same with grammar. You can't start typing shit like "he no want said bring on it i laugh cryed she; very.!" and expect people to read and enjoy your story.
Other than basic rules such as word meanings and grammar, the rest is just guidelines and best practices. Who says you can't start a sentence with a conjunction? Who says you can't finish a sentence with a preposition? Who says if you use "there are" or "he was" you're a bad writer? The problem is that bad writers do overuse these devices, and people associate these things with bad writing. But it's not the complete truth either. These alone do NOT bad writing make -- it's the combination of everything, the entire picture, that makes writing bad or good. Even if someone writes with perfect grammar and uses all the right words and follows all the "rules," there's no guarantee that the writing is any good. The reverse is also true: just because someone uses some of these "bad writing techniques" doesn't mean the writing is bad. Hemingway used "there was/were" all the time and he won the Nobel.
Most readers don't even notice these writerly "flaws" when the story is told beautifully -- when they're experiencing the story instead of reading words, all that stuff is secondary. On the other hand, when a writer is so worried about breaking or following rules, his or her writing may become stiff and uninspired. Rigid rule-following makes for rigid writing. We often tell writers, "Do not write in complete, grammatically correct sentences all the time, especially in dialogue."
The irony is that itself is a guideline, not a rule.
New writers should learn these rules and guidelines because they have worked for generations of writers before them. It doesn't mean everyone should paint with the same strokes. It just means "learn your craft, then you can focus on the art."
It bugs me when people say, "I have a great story, and so what if I don't follow the rules? I don't even have to know them. I can just wing it because, gosh, I have such a great story and people will love it." They're missing the point -- running before they can walk.
Once you master the basics, it becomes art. And there are no rules in art. But perception. Your art fails when you fail to communicate.
And that to me (yes, I started a sentence/paragraph with a conjunction; so sue me) is the key: failure to communicate.