If it goose-steps like a Nazi, wears jackboots like a Nazi, and salutes like a Nazi...
Then it still deserves the same civil liberties granted to anybody else. While I won't defend Nazi ideas, I will defend their right to make their ideas known.
When Richard Spencer got punched, I felt no sympathy for him. He's a racist prick, no matter how much he tries to couch his ideas in "nonviolence" (as if you can have the ethnic cleansing he advocates without violence). Still, that doesn't mean I approved of the punching. I believe that people should be able to express their ideas, no matter how repugnant they are. Unpopular speech has to be protected precisely
because it's unpopular. It wasn't that long ago that the very idea of LGBT rights was deeply unpopular in mainstream America.
Or let's look at it pragmatically. What, exactly, did punching Spencer accomplish? Aside from raising his profile?