When can you take the male mouse out or away from the female mouse?

Typically, yes. The father has been known to eat the children. I don't think the mother will, but that's also a possibility. Just try to keep them away from the father when they're born, maybe wait until they open their eyes and get fur.