It is easy for procurement professionals to understand that the best indicator of how a contractor will do in the future is how they have performed in the past, but very few are working with a credible, objective and fact-based system for collecting and utilizing this information on a regular basis. And given the specific reference to ‘significant or persistent deficiencies in performance’ as grounds for excluding vendors under the trade agreements, the issue is taking on heightened importance.
This article explores how extensive consultation with industry coupled with internal re-education and internal change management processes have enabled the City of Ottawa to establish a defensible system for building a history of vendor performance over time, allowing future vendor selection decisions to include a historical performance perspective.
The Catalyst
In April 2011, Council directed staff to implement a procedure for reporting supplier performance to Supply Services. In response to this, as well as a recommendation from a 2012 Audit of Construction Supervision, the City of Ottawa has created a robust Vendor Performance Management (VPM) program. Since 2015 the City has been monitoring, evaluating and recording contractor and consultant performance on the delivery of construction related contracts for design, construction and contract administration services. They are now beginning to use vendor scores in bid evaluation and contact award decisions, to ensure the best value contractors are selected for future contracts.
So how did the City get to this point, and what have been some of the critical learnings along the way?
Program Objectives and Scope
Recognizing the importance of collaboration with vendors, key objectives for the VPM program include:
o Improve communication between staff that manage contracts and vendors
o Improve overall vendor (consultants and contractors) performance on City contracts, in a measurable way
o Provide feedback to vendors with the goal of performance excellence
o Build a history of vendor performance over time, driving improvements and allowing future vendor selection decisions to include historical performance perspective
o Support the contract administration process used to address non-performance
Development Challenges
Prior to embarking on this initiative, the City was in the position of many Canadian organizations – only paper based evaluations were being conducted, it was inconsistent, there were no formal rules around the process, the results were not being analyzed nor was there a centralized database for capturing the evaluation information. Engaging with multiple user groups across the City enabled the development of standard processes and templates, configuration of a centralized database and a ‘phase-in’ strategy to speed up the implementation.
Given the time and resources involved, the City elected to focus the first phase of the Program on construction related contracts, including engineering and design consultant contracts over $15,000, as well as general construction contracts over $100,000.
Consultation with Industry
Preliminary research on other organizations embarking on similar initiatives helped to shape the way forward for the City of Ottawa. Prior to implementation, the City reviewed other public sector vendor management systems and explored lessons learned by those organizations. They also consulted with numerous industry associations on the contractors’ side of the equation, including Ottawa and Canadian Construction Associations, General Contractors Association of Ottawa and the National Capital Heavy Construction Association. They also consulted with the Ontario Association of Architects and Ottawa Regional Association of Architects, the Consulting Engineers of Ontario and the Ontario Association of Landscape Architects. These key stakeholders provided detailed feedback, and identified five overarching requirements from their perspective:
1. Clearly defined scoring criteria
2. Multiple controls to ensure consistency and fairness
3. A chance to review and discuss the evaluation prior to it being entered into the system
4. The evaluation to reflect certain extra costs and delays outside their control
5. An appeal process.
The City adopted these suggestions, ensuring specific processes were in place to deal with each of these areas of concern.
The Scoring
In close consultation with industry, the City developed 8 Key Performance indicators (KPIs) for General Contractors, 6 for Consultants, and 8 for Contract Administration, each with associated point weightings. Individual vendor performance evaluation scores are tracked, and all low scores must have supporting comments and documentation. Each of the three areas (Contractor, Consultant, Contract Admin) has an associated VPM Expectations Document which clearly outlines the requirements, how they will be scored and describes the five performance levels ranging from ‘outstanding’ to ‘not satisfactory’. Those scoring below ‘satisfactory’ are monitored, and various remedial steps may be required depending on the severity of the performance issues encountered. It is the Overall Vendor Score (OVS), a weighted-average over the previous 3 years, that is used in future vendor selection decisions.
The ‘No Surprises’ Strategy
Embedded in the new VPM process is the underlying philosophy of ongoing communication and consultation with vendors as the project unfolds. In addition to sending and tracking emails and letters as soon as issues arise, formal performance discussions have been made mandatory as part of the program, and include the Project Kick-Off Meeting, Progress Meetings, Interim Evaluations, and the Project Close Out Meeting.
The Project Kick-Off Meeting in particular ensures sharing of the City’s performance expectations with the vendor by providing them with the VPM Expectations Overview document at the very beginning of the project. This provides the basis for a discussion with the vendor about how their performance will be evaluated and at what intervals. The vendor is required to sign off on the Expectations Document and the evaluation strategy. The approved Expectations Document, along with any significant documentation related to performance issues identified, are to be posted to the MERX system and visible to the vendor being evaluated. The final evaluation is saved in PDF format from MERX and stored in the vendor performance folder of the project file on the City’s records management system.
The Process
The normal process flow for project managers is depicted in the following chart.*
*from City of Ottawa – VPM Business Process Guide, Feb 2018
Where Are They Now?
At the end of Q1 2018 the City has completed 1116 final evaluations. Of these, only 4% have resulted in scores that fell below the ‘satisfactory level’. As of the end of Q1 2018 there have been only 12 appeals, and vendors had an average of 5.24 projects evaluated and a median of 2.
Overall, vendors are very satisfied with the objective and transparent VPM process, which has enabled the City to bring meaningful past performance data into the RFX evaluation process. In early 2018 the City began implementing this historical information as part of its procurement processes, enabling them to reward the best performing vendors and putting others on notice that improvements are required if they wish to remain competitive. The less visible beneficiaries of the VPM process are, of course, the citizens of Ottawa.
Ongoing Improvements
The City continues to consult with industry on an ongoing basis, to enhance the VPM program moving forward. In particular, they are working on improvements to the KPI wording for clarity, and maintaining the delicate balance between transparency and confidentiality when using the scores in contract award decisions. They have also prepared a ‘VPM Business Process Guide’ as a companion reference to the VPM program, which is available to City staff, vendors and other stakeholders.
The Procurement School extends many thanks to Will McDonald, Chief Procurement Officer, City of Ottawa for his help with this article.
It is easy for procurement professionals to understand that the best indicator of how a contractor will do in the future is how they have performed in the past, but very few are working with a credible, objective and fact-based system for collecting and utilizing this information on a regular basis. And given the specific […]
By The Procurement School On September 21, 2021, Justin Trudeau, Canada’s Liberal party leader, was re-elected with a minority Government. As Trudeau will be leading the country for the next 18 months, we thought we would look at what re-election means for public procurement. It seems the Liberals are investing heavily into keeping up with […]
As the effects of the global pandemic unfold, governments the world over are looking to procurement leaders to restructure supply chains that are agile and resilient as well as leverage technology to gain procurement efficiencies. This has led to increased positions and career opportunities for procurement professionals — thereby furthering the need for, and increasing […]
Integrated Project Delivery is a construction delivery methodology successfully in use for decades in the US and parts of Europe. It is currently gaining traction in Canada. what is integrated project delivery, or IPD? There are essentially four construction delivery method options a project team can choose from. These four include: design-build, design-bid-build, construction management […]
Author: Debby Shapero ProppWhether you are contracting to licence or acquire custom software, hardware, or related services, IT procurement creates unique challenges and is often associated with high costs, complexity, and a relatively high rate of failure. Improperly informed IT decisions can be costly. These challenges can be daunting even to the most seasoned practitioners. […]
When a contract is not performed as promised, the other party may have the right to claim damages for breach of contract. The amount of liquidated damages payable will depend on the terms of the contract and, if the contract is unclear, then on what can be established as ‘reasonable and provable losses’ flowing from […]
My husband, the cook in our household, often speaks of the holy trinity of French cuisine – the mirepoix – a base of sautéed onions, celery and carrots that form the foundation of many French recipes. In contracts, the trinity of risk allocation is the limitation of liability, indemnity and insurance clauses. Together, this trinity […]
As a company that provides education in public procurement, this is perhaps the most common question that arises. While there are many similarities, the nuances of public legislation and free trade requirements create a procurement environment that is more regulated, demands a greater level of transparency, and requires vigilance to stay on top of the constantly […]
In the last issue of The Legal Edge, we reported on the pivotal Supreme Court of Canada decision in Bhasin v. Hrynew, 2014 SCC 71, which expanded the organizing principle of the duty of good faith to include a “duty to act honestly” in general contractual dealings. While acting honestly and making sure that you […]
As procurement professionals know, contractual discretion is a double-edged sword: it leaves you in the driver’s seat, but almost certainly will lead to challenges when that discretion is exercised to the detriment of the other party to the contract. While the Supreme Court of Canada’s decision in Bhasin v Hrynew laid out some organizing principles […]
