Which country is closest to the US but does not share a border?

Russia.

The Diomede islands in the Bering Strait are separated by 2.4 miles. East Diomede Island is part of the state of Alaska, and West Diomede is governed by the Far Eastern Federal District of Russia.