Why is it difficult for a project manager coming in part way into a project?

Because a manager who was present when the project was initiated & planned will have a better understanding of the project and everything related to it.

If a new manager comes part way, he has to depend on the documents created or the transition provided by the old manager for all details. There can never be a 100% coverage in terms of transition or knowledge and hence the new manager will have to work twice as hard as the guy who started the project.

Moreover, team members will take some time to warm-up to the new manager and it can be difficult for him to handle the project efficiently.