Is it grammatically correct to start a sentence with but?

Yes. There is no Word that cannot begin an English sentence. But that does not mean that it is always a good way to begin one.