Which part of the fish is responsible for movement?

All parts work together. Fish couldn't swim without fins any more than they could swim without their gills or digestive system.