Do Eye rhyme refers to words that both sound alike and look alike?

No, eye rhyme refers to Words that appear to rhyme based on their spelling, but do not actually sound alike when pronounced.