A larger surface area makes diffusion happen more quickly?

Yes, it does. The surface area being larger increases the diffusion because there are more protein pathways for molecules to be shipped in and out.

As a cell grows, however, the surface area is smaller and diffusion can not happen at a rate fast enough to support the cell. When a cell reaches a certain size, they divide to avoid this problem among others that come with an oversized cell.