Would it matter if you used vinegar oil instead of vegetable oil to make a homemade lava lamp?

no it does not matter because their both the same. It's oil.

Clarification:

If the oil has vinegar in it, then no, you can't use it. The vinegar does not mix with the oil, and will not give you the desired result.