Is it a must for you to lift heavy weight for you to have big body?

No. Some people, especially if they regularly do physical labor, are very strong without training in gyms lifting weights. A lot of your ability to move heavy objects is genetically determined.

Realize that there is a difference between being strong and being skilled at demonstrating strength. To demonstrate strength by, for example, doing a bench press or a clean-and-jerk, requires technique that must be learned and mastered in a gym. Therefore, some people who are naturally very strong may not be able to demonstrate their strength in standard ways.