It’s been a couple of years since we introduced SQP25 – a powerful adaptation of the NZTA’s Attribute Weight Setting Tool. In that period, it’s been used by scores of tender evaluation teams to take the guesswork (and hopefully, the nasty surprises) out of setting attribute weights. Through that experience, we’ve refined our thinking and found better ways to approach the tool and explain how it works. To find out more, please get in touch – we’ll be happy to demonstrate it.
Here are the five most important things you need to know about this tool.
- It’s not new. That’s right, the SQP25 is simply a variation on the traditional NZTA Attribute Weight Setting tool that was developed more than a decade ago to help procurement staff see the effect of different weightings on the premiums they might end up paying.
- SQP25 addresses the risks of ‘scrunched up’ scoring, which have previously not been focused on. When tender evaluators use subjective scoring, there is sometimes a tendency to score within very tight bands. While sometimes this may be justified (when the differences in attribute quality between tenderers are very small), on other occasions, this results in an unintended consequence. In these cases, even if the price weight is relatively low, price will dominate the decision. This effectively defeats the purpose of choosing Price Quality evaluation – Lowest Price Conforming would be far more efficient and transparent.
Scrunched up scoring is the revenge of a disaffected LPC advocate who’s been forced to use PQM! The SQP25 effectively calibrates the scoring to maintain appropriate ‘stretch’ in the scoring so that the right premium is paid for extra quality.
- It’s founded on statistical analysis of tender evaluations of attribute scores.
Our research showed that in over 95% of tender evaluations, the range between the worst and the best scores in each attribute was less than 25. That’s right, even though tender evaluators typically have a scale from 0 to 100 to use, once they have eliminated unsuitable suppliers using some form of prequalification process, the scores are in a much narrower band. It’s hard to change human nature, so we used this fact to calibrate the attribute weightings and the scoring within any tender evaluation system that uses both price and quality attributes. Read on….
- SQP25 cleverly knits together the uncontrolled multiple variables in the tender evaluation process – i.e. the elasticity in the scoring system and the balance between price and non-price attribute weightings. By developing fact-based scenarios for the hypothetical worst and best scoring tenderers in each attribute, and pinning those extremes to a 25-point difference in attribute scores and the dollar value that would be appropriate to pay for that difference, the SQP tool effectively calibrates the three key elements of the Supplier Quality Premium – the Engineer’s estimate (value of the contract); the elasticity in the scoring, and the Price/ non-Price weights.
- SQP25 drives procurement professionals to follow sound processes in planning their procurements. Rather than guessing at weightings prematurely (before they have analysed the potential effects of those weightings), tender evaluators must first:
- consider the key risks and opportunities associated with each attribute
- clearly articulate what they will score highly
- generate fact-based scenarios that describe the best and the worst in each attribute
- decide what premium, relative to the contract price, would be appropriate and acceptable for these extremes in scoring.
This information is then used by the SQP25 tool to determine the price and non-price attribute weights.
The SQP25 tool is simple, user-friendly, intuitive and logical. It’s not rocket science, and it’s not a departure from the excellent processes that have been developed by the NZTA over the past few decades to underpin the Price Quality Method (which is, in the author’s humble opinion, by far the most robust, process-driven, fair and defensible evaluation mechanism that exists anywhere in international procurement today).
All the SQP25 tool does is reverse the process, so that it starts where it should – with a quantified, project-specific risk analysis.
Yes, if you are used to arbitrarily guessing at what the weights should be for tenders (or recycling them from the last RFT), this is going to restrict your powers. And if, by using those legacy methods, you’ve on occasion found yourself manipulating your scoring later so you get the answer that gut feel tells you should be right, you’ll be uncomfortable that the up-front analysis required to use SQP25 will give you less freedom to relitigate your scoring.
But if you’re dedicated to undertaking robust analysis in developing your procurement tools, and not retrospectively changing your scoring somehow when the answer isn’t what you expect or hope, then you’ll welcome the clarity and the logic within the SQP25 process.
We welcome comments and feedback on the SQP25 rationale, its tools and its process.
If you’d like to try using it, please contact us for a copy – we’re giving it away! And we’d love to show you how it works.
Alternatively, check out this link to find out more or download the SQP25 spreadsheet to try it out. Use the password CBSQP2518 to open the file once you have downloaded it.
