Is it illegal if you use a part of a poem in a song?

If you have the permission of the poet for copyrighted material, then no. If the material is copyrighted and you have no permission to use it, you cannot legally use it.

If, however, the poem is public domain--the copyright limit has expired--then by all means, use it. Keep in mind that using someone else's Words effectively is not always the easiest thing to do. It can be like trying to make Mopar parts fit on a Chevy.