Is it possible to assess the quality of software if the customer keeps changing what it is supposed to do?

I will answer this as the person responsible for testing.

The customer can do what they want. My job is to tell them what is possible. I may not be able to test/retest in the given time. I then need to inform the customer. The customer can still go ahead with releasing the software.

There is another very important part to this. When I am part of a test team, in these situations, I tell the testers that they can’t later say, ‘..I told you so..’. What this means is if they think there is a significant risk, they need to inform the customer. They need to know the risk. It is not good enough to say, ‘I don’t know…’

Here is an example:

On one of my projects, a manager said, ‘we need to release next week’. But we hadn’t started testing the 64-bit version. Manager said, ‘I don’t care’.

At that point, I said, ‘let me get back by the end of the day if we have any reservations.’ I then told him that we will make sure basic functionality works. If there are blocking issues, we will revisit. I also made a list of what we will test and what we won’t. We worked together to refine the list.

It’s very important that I will never blame the manager for his decisions. I was part of that decision. If I felt strongly about risk, I would speak up. If I didn’t speak up, it was my mistake, not his.

Just to repeat, you dont’ have an option to say, ‘I don’t have time’. What you need to say is, ‘In the given time, I can test X. This then leaves the risk that customers may use this feature and find problems.’