Beyond Mission Impossible: How to Truly Evaluate a Consulting Firm

An impossibile mission

Prefer to listen? 🎧 Here's a brief audio overview of this article, generated with artificial intelligence:

One morning, on your desk, you find a pair of glasses. You put them on, and suddenly you hear a voice saying: "Your mission, should you choose to accept it, is to completely, rigorously, and impartially evaluate the consulting firm you have collaborated with. This message will self-destruct in 5, 4, 3, 2, 1...".

Okay, maybe that's a bit exaggerated. Perhaps this isn't a true "Mission Impossible," but in many ways, it's quite similar. Objectively evaluating the work of a consulting firm, with which you have collaborated or are currently collaborating, seems like a simple task, almost accessory at times.

But it's not. And if you want to do it well, and do good for your company and your project, you need to be able to transform what might be a personal and partial judgment into something impartial: in short, into a repeatable process.

Elements of a repeatable process can include (but are not an exhaustive list):

·       Standardization: The use of predefined metrics or Key Performance Indicators (KPIs).

·       Documentation: The importance of recording observations and data.

·       Frequency: How often should this evaluation be done?

·       Involvement: Who should be involved in the evaluation process? (Not just the person who chose the consultancy, but perhaps also the team that worked with them).

Why Evaluating Is Never Personal (But It Is Crucial)

You must remember that you chose this firm using the criteria we already discussed in a previous article. You didn't rely on chance; in fact, you followed a process even then. It's crucial to remember that there's nothing personal about what you're doing. Since you collaborate with other people, and since human as well as professional relationships are involved, it's clear that the task is anything but simple.

You must always keep in mind the importance that the work done together with this consulting firm represents:

·       for your project

·       for your company

·       for your work and for your colleagues

·       and also for the consulting firm itself, for those who work there and for their managers, because it is not a one-way but a two-way relationship.

Essentially, it's about being honest: honest with yourself, honest with the people you work with, honest with your company, and honest with the company you are collaborating with.

The Importance of an Evaluation Methodology

Always remember that no one ever said it would be an easy task. In fact, no one even told you it was a task you had to perform, but everyone, I believe, considers it something taken for granted. I also doubt that any of you have taken a corporate course to be prepared for this aspect of your job, to have objective evaluation criteria, to have a methodology that is not just based on your good sense, but that can also come from the experience of others, and, why not, from tested and retested methodologies in the field.

We will divide the evaluation into four macro-groups. Of course, this is not a limitation, but just an aid to schematize things.

The Four Evaluation Scenarios

The four groups are:

1. A situation where everything went wonderfully well, exactly as you expected.

2. A situation where there were ups and downs, but all in all, things went quite well.

3. A situation where, objectively, things went more badly than well, even if there was something good.

4. The last case, where frankly, nothing went as it should have.

Four different cases, for each of which I will provide my point of view and my experience gained over years in which I have encountered all four situations.

Case 1: Everything was fantastic!

This seems to be the simplest case: everyone is happy, the work was done correctly with the right level of quality. It seems there's nothing to do, an easy job. Yet, even in this situation, in my opinion, there's work to be done. Everything went perfectly, fantastic, but the question is:

·       Could you have done better?

·       Could you have chosen a firm that could have worked even better?

·       Could the firm that worked with you have done better?

To give a concrete example, imagine a case where the firm in question delivered a job on time, but this work cost them less time than budgeted. They could have delivered a week early, and in that week, you could have integrated their work and tested it more calmly, perhaps discovering more issues. Another case is if the company had already developed the same functionality in the past with an innovative result, but it was not proposed to you due to simple inertia. So, the fact that the consulting firm worked well paradoxically does not exempt you from the need to still judge what has been done. The old adage "there is nothing well done that cannot be done better" applies perfectly to this case.

Case 2: Something went well, something didn't.

In this case, it is still necessary to evaluate what went wrong.

·       Was it a problem of understanding?

·       Of inadequate tools?

·       Of project requirements changing along the way?

There are many reasons why there may have been hitches, and they need to be analyzed. I advise you to put the pros and cons in two columns, and for each, assign a kind of "weight" or importance. For example, if a consultant delivered 90% of the project but failed in communication with the team, causing internal delays, how do you balance these aspects? You can proceed by assigning a "weight" to delivery (i.e., 40%) and a weight to communication (i.e., 20%), and checking how the different items impact the overall total. In the end, it is necessary to sum up, draw conclusions, and see if the positive aspects are able to compensate for the negative ones. If not, you must also, in this case, re-evaluate the firm you selected and again go back to checking the criteria you used for selection. Always remember that when you make an evaluation, after the evaluation you must always go back to evaluating your criteria, because you were the one who chose that consulting firm. And if something went wrong, the solution is often found on your desk.

Case 3: Things did not work out as they should have.

Now we go into an even worse case, where the things that worked are objectively and decidedly few and unsatisfactory. Something worked, but it's not enough to save the situation. In this case, it is inevitable that you will have to go back to your criteria and reasonably modify them. It won't be easy to identify what to change and what to keep, but it's necessary to do so. This because imagine what would happen if you had to replicate the same poor result you had on another project, and another, and yet another. It's not about finding a culprit, but rather about finding the error.

Case 4: Totally failed collaboration.

The fourth and last case, which I hope never happens to you but unfortunately is possible, is one where absolutely nothing of the work done can be saved. It was a totally disastrous collaboration. In this case, it's no longer about re-evaluating your criteria individually (you could do that too, of course), but rather about starting from a blank slate, because if nothing worked, there's something systemic to correct, something deeper. And even if it seems time-consuming (because you already have a thousand commitments to follow for your daily work), in the end, I advise you to see it as an investment that always pays off in the medium to long term.

Some possible actions are:

·       A thorough retrospective analysis to identify the roots of the failure.

·       Reviewing the entire vendor selection process.

·       Considering a radical change in the approach to managing consultancies.

The Key to Success: Your Criteria

Always remember: your criteria are your best allies. Once they work, you can reuse them again and again and again, even when you have changed roles in the company, even when the company's structure changes. Your criteria will be there to support you, if you have worked correctly on them. In all four cases, as you could see, from the evaluation you return to the criteria, and then from the criteria you move on to a new evaluation, in a virtuous cycle of continuous improvement, a kind of ascending spiral of learning.

In summary: always evaluating the work of the consulting firm with which you have collaborated is a necessary activity. It must be carried out with the same attention, rigor, and discipline with which you approach the use of your selection criteria.

And even in this case, if you have any doubts about how to approach the evaluation, do not hesitate to contact me. My job is to help companies like yours in all phases of working with a consulting firm: from selection to evaluation.

Thank you and see you soon.