Keeping brothels illegal reduces the number of sex workers abused and/or exploited by pimps
Sex work is one of the most exploitative professions. The legalisation of brothels would legitimize the pimps, who have the most to gain from passing off legal protections to the state rather than creating a healthy working environment (which decriminalisation does not guarantee).
Beyond the debated illegality of sex workers themselves, there is inevitable criminal activity in the prostitution network. Often, the perpetrators are the pimps who monetarily benefit from the exploitation of women's bodies without performing sex acts themselves. Typically, those pimps are the targets of raids and criminal cases against brothels and sex work organizations because the goal of criminalising sex work is to prevent those exploitative behaviors and the attitudes and inequalities they generate. There is evidence to suggest that decriminalising sex work would only end up benefitting pimps, not helping reduce the stigma against sex workers themselves. Profits made from decriminalisation laws end up helping to subsidize pimps and validate those who use brothels. Decriminalising prostitution frequently leads to increased human trafficking and sex tourism. Another aim of decriminalisation is fostering an environment where sex workers can go to legal authorities for protection in dangerous situations. However, there is consistent evidence that legal institutions such as the police have ties to brothels and turn a blind eye to their illegality. Decriminalisation may reduce stigma, but it doesn't help protect the workers involved.
The criminality of pimps and exploitative users should not reflect back on sex workers and prevent them from achieving proper protection and social security. The sex workers aren't benefitting fully from this exploitative system. With decriminalisation, they would at least have some additional protections.