Why do we grant students the right to be protected from controversial ideas? Citizens are not afforded the same protections in other public spaces. Why do university campuses need to be kept as spaces without offence and controversy?
We don't remove offensive books from student libraries, or erect firewalls preventing offensive and inflammatory content reaching them online. Why should we stop controversial and inflammatory ideas reaching them through planned speaker events? 
There are other ways of preventing violence from happening that doesn’t stifle free speech. An increased police presence and harsher punishments for those starting violence are an alternative way to keep attendees physically safe.