How big does a compost pile need to be?

Generally you'd expect a compost heap to be about three feet across and two or three feet high. Much smaller than that and whatever you're composting will probably not heat up into "fast compost", but will simply decompose slowly, the way leaves decompose on a forest floor.

That being said, it is perfectly legitimate to create "sheet compost" over a garden area rather than building a specific compost heap. To do that, you layer organic mulch thickly on top of whatever area you want to benefit from the compost (obviously not on top of tiny seeds or seedlings, though) and simply wait a year or more for it to break down into compost where it lies.