Court deputies had to break up a physical fight between a Brevard County judge and a public defender after an argument during a hearing on Monday.