Why generally the high voltage side of transformer is delta connected?

I don't know what kind of transformer you are talking about, but most utility distribution Transformers are star connected on the primary side.

It might be different in some other countries, but in the UK, three-phase distribution transformers are always connected in delta on their primary (high voltage) side, and in star (wye) on the secondary (low voltage) side. The primary line voltage is 11 kV, while the secondary line/phase voltages are 400 V and 230 V.

In the United States and Canada, the primary winding is also connected in delta -however, the secondary side is also connected in delta, with one phase centre- tapped to provide the split-phase 240/120 V arrangement.

The reason for using a delta connection is because three-phase high-voltage lines are supplied as three line conductors -there is no provision for a neutral- so there is no practical advantage in having a star-connected primary.