Why would a man restrict his mistress phone number?

A man might restrict his mistress's phone number to maintain privacy and secrecy in their relationship, especially if he is involved with someone else or wants to avoid complications. This action could also be a way to control the communication and limit contact to specific situations or times. Additionally, he may fear being discovered by his partner, leading him to take measures to conceal their interactions.