What part of the brain feels?

Depends on which feelings you're interested in.

The paliocortex (aka the thalamus or brain stem) runs the "deepest" emotions,

the olfactory lobe locks in the strongest conscious emotions,

but every specialized section of the brain controls the feelings that are associated with that particular function. i.e. the visual lobe triggers emotion based on things you are seeing, have seen, and imagine that you see.