How can you exit a steam game that's frozen?

That means the host either lost connection, or you did. You can try and wait and see if it comes back, or you can use CTRL + ALT + DELETE.