Is it illegal to attend random college classes?

Yes, it is generally illegal to attend college classes without being officially enrolled or without permission from the instructor or institution.