Can you replace a 6V 2.5 Ah battery with 4 AA 1.5 V batteries?

If four AA cells are connected in series, they're equivalent to a 6-volt battery,

otherwise they're not. You'd have to know how they're used in the specific device

where you're considering substituting the 6V battery.