If you allow someone with poor credit history to be an authorized user on your credit card will it lower your credit score due to their poor history?

I was a authorized user on one of my exwifes accounts and did not realize it. I noticed my credit score was going down steadily and was not sure why. Then it made a big plunge so I checked my credit report and fiqured out I was a authorized user on one of her accounts that she had stopped paying on.I called the company and at first they would not take my name off of the account saying it would have to be settled first. I argued with them how could I be responsible for something I don't even see a statement on. It took several calls but I finally reached someone who saw my side of the story. They had trouble deleting my name from the account and had to get a supervisor involded to get me removed. My credit score jumped by more than a 100 points the next month.

EDIT: This would have been a good response if the question was "Will my credit be affected if I am added as an authorized user". I think the person answering the original question got the authorized user and account holder in reverse.

The answer below is correctly answers the question without confusion.

Also...a couple of credit card issuers have policies that hold Authorized Users responsible for their actions and safeguard the Primary Account holders credit file.

Credit scores are not affected by the actions of an authorized user.

Neither can an AU be held legally responsible for the account regardless of what they may be told by the card issuer.

The premise being that an AU has no control over how the account is handled as they do not have access to payment of the account or other issues such as increasing the credit line.