Which branch of government can refuse appointments to the US Supreme Court?

The Legislative Branch (Congress) has the power to impeach and convict a US Supreme Court justice if he or she commits wrongdoing, but no one has the power to "get rid" of a member of the Court simply because they don't like his (or her) decisions or ideology.

If Congress believes a judge or justice has done something to merit removal, the US House of Representatives files articles of impeachment (like a grand jury indictment), and the Senate conducts a trial to determine whether the justice is guilty. If convicted at trial, the justice will be removed from office.