Is it illegal to throw away your spouse's mail?

Mail should always only be opened by the addressee. This is just plain common sense courtesy. However, things like known utility bills and other things that are jointly held can be opened by either spouse.