“We have a president who’s a racist,” Brown said