What did jem tell scout he was afraid of after the group of men showed up?

In Harper Lee's novel, To Kill a Mockingbird Jem tells Scout that he fears, their father, Atticus will get in trouble. Jem fears this because he thinks the group of men were apart of a mob that was upset about Atticus helping Tom Robinson.