All of the facets of grammar (sentence structure, subject-verb agreement, punctuation, etc.) are essential for written forms of communication. Transcribing English words and ideas into a written form needs order, structure, and consistency. However, grammar is nonessential to actually speaking English. The written word takes the sounds of a language and assigns them arbitrarily to symbols known as letters. But it is not enough to simply learn which sound goes with which letter. Words must be organized into sentences. A sentence must contain a complete thought. Complete thoughts must be separated from one another to maintain clarity and order. Grammar and punctuation rules are put into place to mimic the way that we naturally speak, and the ways in which we organize our thoughts mentally and verbally. However, grammar and punctuation were created solely to serve the purposes of the written word. Understanding grammar was never needed to learn how to speak effectively, and grammar itself did not exist prior to the written word. The need for grammar didn't arise until after the written word had already been developed.
Grammar is as important in speech as it is in writing and reading. Thinking about grammar helps us to formulate our sentences in a way that can be clearly understood when we talk to one another. In fact, when grammar was first conceived of, it was defined as a language which helps a person talk about their language, as well as help them speak it.
[P1] Grammar is important for reading and writing, but not for speaking.
Rejecting the premises
[Rejecting P1] Grammar is as important in speaking as it is in reading and writing.