Why could Charles not marry a commoner and William does and Diana paid the price for the British throne?

You might need to ask Queen Elizabeth, because she is mostly in charge of weddings and has the final say. But most likely, it's another example of how times have changed, as well as how attitudes have changed. In America, as recently as the 1980s, people still wondered if a president who had been divorced (Ronald Reagan) could be elected-- he could, and he was, but if this were the 1950s, it might have prevented his election. The idea of not marrying a "commoner" does not seem to bother most modern people the way it might have 100 years ago. In fact, it seems to be more a custom, rather than a rule-- there have been other royals who married commoners without any penalty. In the case of Charles, the opposition to Camilla getting a royal title was more in deference to how popular Princess Diana had been, rather than because she did not come from royal lineage.