What do you answer when someone says this you're probably not into my sought of thing?

You can respond by saying, "I appreciate your honesty! I'm open to trying new things and would love to hear more about what you're into." This shows your willingness to engage while also respecting their interests.