What is the connection between stone masonry and freemasonry?

it is rumored that at first, stone masons had secret meetings and held secrets of the labor. Eventually, they started accepting other people to their meetings, from different work backgrounds ... Since it wasn't restricted to stone masons anymore, this was changed to become "free masonry"