Why did Atticus think Jim and scout disobeyed him during the house fire?

Atticus starts to get angry with the children after the fire because he thought that they had gone back inside the house after he said to stay in front of the Radley's house. But he did not realize that even Scout, the one with the blanket on her, didn't know she had a blanket wrapped around her until Atticus had pointed it out.