Is the book was returned to the library yesterday a active voice?

No, the sentence "The book was returned to the library yesterday" is in passive voice because the subject (the book) is receiving the action (being returned) rather than performing the action.