Is this guy falling for me and avoids me because of his growing feelings or is it something else?

the story is long but here it is

I have been friends with this guy for about 3 years and was dating and living with my ex at the time. About 2 years ago, we started having sex once and awile when things got bad with my ex. During this period, this guy made comments about wanting to be with me and me telling him the same thing. We talked on the phone alot and met up once and a wile for friends with benefits type sex.

So about 4 months ago I finally left my ex. I am 24 and this guy is 34. He has been married once but has no kids and left because she was unfaithful. So these past 4 months is what is getting me so mixed up. I have had deep feelings for him for at least a year now. These past 4 months we have met up for sex and talk on the phone. I havent specifically asked him how he feels for me. But like the past 2 months the sex has changed. Our sex for the last 2 1/2 years was quick so not to get caught and just fun sex nothing specific. Lately though when we meet up, he acts different. When we meet for just other types of sexual fun, he is all of a sudden kissing my neck, getting closer like grabbing me and leaning real close on me to kiss my neck. During actual sex, it is also seems more intense like kissing my neck and body, doing whatever possible to get me to get off, if he is having a hard time getting an erection, i see in his eyes frustration. During sex, he also has lately been staring deep in my eyes.

Even the other day he pushed me up against my car touching me staring deep in my eyes. Another time we were in the woods and I got leafs in my hair and he took every one of them out of my hair for me. When i call, if i need anything, money to borrow-he lets me. If I need help with something, he will come to help.

The only problem I have is are these signs that he is getting a stronger emotional feelings for me? Because after every one of these meets we have, he then avoids calling me or taking my calls for a few days after. Then after those few days he will answer my calls, or call me back from a message, and we will talk.

My main question is is this man falling for me and avoids me after these emotional and physical encounters out of fear of those feelings he has and maybe is scared I dont feel like he does and backs down during those days.

ANY INSIGHT WOULD HELP SO MUCH BECAUSE I HAVE ALREADY FALLEN COMPLETELY FOR HIM AND NEED TO KNOW IF I HAVE TO LET MY FEELINGS GO BEFORE I GET ANY FARTHER IN