To argue that language has any effect on cognition is to erase the fact that there are universal principles that govern all languages. For example, children typically acquire their first language(s) in the same stages in the same order, so they learn simple single-syllable utterances before multi-syllable ones, and so on. There are so many language rules common to all--or at least the vast majority of--languages that many researchers support the idea of a universal grammar that all children are born with.
But if the language a child learns influences how they think and learn, one would expect to also see variations in how they acquire language based on the specific language(s) they are learning.
The other problem with the concept of linguistic relativity is that its proponents situate it as a rejection of Sapir and Whorf's linguistic determinism--because the reality is that these linguists actually rejected the view that language is the only factor that determines worldview. Both Sapir and Whorf actually argued for merely a connection between a person's language and thought processes.
Proponents of linguistic relativity have therefore misconstrued the beliefs of the two scientists for which the Sapir-Whorf hypothesis is named in order to advance their own views.