Innovative Vendor Performance Management at the City of Ottawa

It is easy for procurement professionals to understand that the best indicator of how a contractor will do in the future is  how they have performed in the past, but very few are working with a credible, objective and fact-based system for  collecting and utilizing this information on a regular basis. And given the specific reference to ‘significant or persistent  deficiencies in performance’ as grounds for excluding vendors under the trade agreements, the issue is taking on  heightened importance. 

This article explores how extensive consultation with industry coupled with internal re-education and internal change  management processes have enabled the City of Ottawa to establish a defensible system for building a history of vendor  performance over time, allowing future vendor selection decisions to include a historical performance perspective. 

The Catalyst 

In April 2011, Council directed staff to implement a procedure for reporting supplier performance to Supply Services. In  response to this, as well as a recommendation from a 2012 Audit of Construction Supervision, the City of Ottawa has  created a robust Vendor Performance Management (VPM) program. Since 2015 the City has been monitoring,  evaluating and recording contractor and consultant performance on the delivery of construction related contracts for  design, construction and contract administration services. They are now beginning to use vendor scores in bid  evaluation and contact award decisions, to ensure the best value contractors are selected for future contracts. 

So how did the City get to this point, and what have been some of the critical learnings along the way?  

Program Objectives and Scope 

Recognizing the importance of collaboration with vendors, key objectives for the VPM program include: 

o Improve communication between staff that manage contracts and vendors  

o Improve overall vendor (consultants and contractors) performance on City contracts, in a measurable  way 

o Provide feedback to vendors with the goal of performance excellence 

o Build a history of vendor performance over time, driving improvements and allowing future vendor  selection decisions to include historical performance perspective 

o Support the contract administration process used to address non-performance 

Development Challenges 

Prior to embarking on this initiative, the City was in the position of many Canadian organizations – only paper based  evaluations were being conducted, it was inconsistent, there were no formal rules around the process, the results were  not being analyzed nor was there a centralized database for capturing the evaluation information. Engaging with  multiple user groups across the City enabled the development of standard processes and templates, configuration of a  centralized database and a ‘phase-in’ strategy to speed up the implementation.

Given the time and resources involved, the City elected to focus the first phase of the Program on construction related  contracts, including engineering and design consultant contracts over $15,000, as well as general construction contracts  over $100,000. 

Consultation with Industry 

Preliminary research on other organizations embarking on similar initiatives helped to shape the way forward for the  City of Ottawa. Prior to implementation, the City reviewed other public sector vendor management systems and  explored lessons learned by those organizations. They also consulted with numerous industry associations on the  contractors’ side of the equation, including Ottawa and Canadian Construction Associations, General Contractors  Association of Ottawa and the National Capital Heavy Construction Association. They also consulted with the Ontario  Association of Architects and Ottawa Regional Association of Architects, the Consulting Engineers of Ontario and the  Ontario Association of Landscape Architects. These key stakeholders provided detailed feedback, and identified five overarching requirements from their perspective: 

1. Clearly defined scoring criteria 

2. Multiple controls to ensure consistency and fairness 

3. A chance to review and discuss the evaluation prior to it being entered into the system 

4. The evaluation to reflect certain extra costs and delays outside their control 

5. An appeal process. 

The City adopted these suggestions, ensuring specific processes were in place to deal with each of these areas of  concern. 

The Scoring 

In close consultation with industry, the City developed 8 Key Performance indicators (KPIs) for General Contractors, 6 for  Consultants, and 8 for Contract Administration, each with associated point weightings. Individual vendor performance  evaluation scores are tracked, and all low scores must have supporting comments and documentation. Each of the three  areas (Contractor, Consultant, Contract Admin) has an associated VPM Expectations Document which clearly outlines  the requirements, how they will be scored and describes the five performance levels ranging from ‘outstanding’ to ‘not  satisfactory’. Those scoring below ‘satisfactory’ are monitored, and various remedial steps may be required depending  on the severity of the performance issues encountered. It is the Overall Vendor Score (OVS), a weighted-average over  the previous 3 years, that is used in future vendor selection decisions.  

The ‘No Surprises’ Strategy 

Embedded in the new VPM process is the underlying philosophy of ongoing communication and consultation with  vendors as the project unfolds. In addition to sending and tracking emails and letters as soon as issues arise, formal  performance discussions have been made mandatory as part of the program, and include the Project Kick-Off Meeting,  Progress Meetings, Interim Evaluations, and the Project Close Out Meeting. 

The Project Kick-Off Meeting in particular ensures sharing of the City’s performance expectations with the vendor by  providing them with the VPM Expectations Overview document at the very beginning of the project. This provides the  basis for a discussion with the vendor about how their performance will be evaluated and at what intervals. The vendor  is required to sign off on the Expectations Document and the evaluation strategy. The approved Expectations  Document, along with any significant documentation related to performance issues identified, are to be posted to the  MERX system and visible to the vendor being evaluated. The final evaluation is saved in PDF format from MERX and  stored in the vendor performance folder of the project file on the City’s records management system. 

The Process 

The normal process flow for project managers is depicted in the following chart.*

 

*from City of Ottawa – VPM Business Process Guide, Feb 2018

Where Are They Now?  

At the end of Q1 2018 the City has completed 1116 final evaluations. Of these, only 4% have resulted in scores that fell  below the ‘satisfactory level’. As of the end of Q1 2018 there have been only 12 appeals, and vendors had an average of  5.24 projects evaluated and a median of 2. 

Overall, vendors are very satisfied with the objective and transparent VPM process, which has enabled the City to bring  meaningful past performance data into the RFX evaluation process. In early 2018 the City began implementing this  historical information as part of its procurement processes, enabling them to reward the best performing vendors and putting others on notice that improvements are required if they wish to remain competitive. The less visible  beneficiaries of the VPM process are, of course, the citizens of Ottawa. 

Ongoing Improvements 

The City continues to consult with industry on an ongoing basis, to enhance the VPM program moving forward. In  particular, they are working on improvements to the KPI wording for clarity, and maintaining the delicate balance  between transparency and confidentiality when using the scores in contract award decisions. They have also prepared a  ‘VPM Business Process Guide’ as a companion reference to the VPM program, which is available to City staff, vendors  and other stakeholders. 

The Procurement School extends many thanks to Will McDonald, Chief Procurement Officer, City of Ottawa for his help with this article.

Disclaimer: The views and opinions expressed in this article are those of the Subject Matter Experts and do not necessarily reflect the official policy or position of The Procurement School.