What gives us the right to put other people down?

Nobody has that right. But people often think they do, or act as if they think they do. Some may feel a bad day or intoxication gives them that right. They may believe what others do gives them that right. Or they were just never taught to respect others and to mind their own business.