What word refers to the categorizing of fiction?

The Word you're looking for is "genre." Genre is used to categorize different types of fiction based on themes, writing style, and other characteristics. Some common genres include romance, mystery, Science Fiction, and fantasy.