How much postage do you need to mail a 5 X 5 inch envelope in the US?

I mailed 5 x 5 inch Christmas cards . most were returned to me , rubberbanded together , saying insufficient postage. I went to my postal annex and they said because they were square they required an extra 15 cent stamp. i asked why only some were returned and they said sometimes the postmaster will let them thru or possibly charge the recipient. Considering the size of regular cards is much larger I feel this is not fair.