Is burning a marshmallow a chemical change?

It is easier in this case to say why it isn't a physical change.

As you cannot turn a roasted marshmallow back into an un-roasted one, A chemical reaction must have occurred.

Cooking is normally a chemical change as you cannot un-cook something.