What is the formula used to calculate the area of an irregular shape?

There is no set formula.

You have to "split" the irregular shape into regular shapes, use whatever dimensions you have and fit them into the formulas of the known shapes.

Find the area of each split shape. Lastly, you add all the areas to give you the totl area of the irregular shape