What is the study of human behavior?

Psychology studies human behavior, particularly focusing on individual behavior. If you are referring to human behavior as it relates to societal behaviors, the answer would be sociology.
Psychology.

The New Oxford Dictionary defines this as: The scientific study of the human mind and its functions, especially those affecting behavior in a given context.