Why did Northerners oppose the annexation of Texas?

Whenever there was opposition from the North regarding the western territories, it generally was about the issue of slavery. Texas, a huge territory, was open to the concept of slavery. Therefore, the North saw this as an opportunity the South would use to push their slave agenda and gain additional clout in Congress due to the three-fifths rule of population count given for slaves that had given the South a lot of political power.