What was the name of the first website?

Depends on your definitions of "public" and "website," but the first one was probably the World Wide Web Consortium's World Wide Web Project's start page.

Here's a copy: http://www.w3.org/History/19921103-hypertext/hypertext/WWW/TheProject.html