Metrics & Evaluation In Legal Procurement

This article was published by PASA – Procurement and Supply Australasia.

Lawcadia have produced a 5-week series on buying legal services in Australasia, written specifically for PASA. Each week, we are bringing you the next chapter. Here’s the fourth instalment…

The last three articles in this series introduced the topic of legal procurement, outlined the drivers behind its growing influence and assessed the ways in which organisations define the value that legal procurement can bring. This article will focus specifically on the metrics used in legal procurement because what is measured will guide the focus of the suppliers (law firms) and how the legal team evaluate the services provided.

Tracking, measurement, and control: essential elements in almost every category and business unit, and yet completely new territory for most legal departments. According to the Buying Legal Council in their 2016 Legal Procurement Survey “legal services used to be unpredictable and immeasurable. This is no longer the case”. ¹

In this digital era, data is becoming more sought after and available, and whilst some legal teams are starting to use data and metrics to help inform decision-making, the industry needs to be cautious.

“I find that the data available to make dependable or truly informed decisions is very weak in the legal market,” says Friedrich Blasé, Global Director of Pangea3, a Thomson Reuter legal outsourcing company based in New York.²

So, defining what to measure is one thing, and then ensuring rigour and validity over the data that can be made available is another key issue that legal procurement needs to address.

Legal procurement: what sophisticated companies measure

The Buying Legal Council, the international trade organisation for professionals tasked with sourcing legal services and managing legal services supplier relationships, conducted a research survey in 2016 that examined the purchasing behaviour of its members and other legal procurement professionals. [Nb. The abridged version of the research is immensely interesting and can be found here.] This research highlighted the most important aspects that legal procurement professionals are seeking to measure in the legal category.

The research identified that legal procurement specialists in large organisations, equivalent to Fortune 100 through to Fortune 1000, look to measure matters and law firms, as well as metrics that might indicate excessive billing, law firm staffing issues and law firm billing issues. ¹

We will now explore each of these in turn.

Matter level measurements

Examining the legal category at a top level is not sufficient, as the ‘devil is in the detail’. Therefore, it is important that legal procurement drill down to an individual matter level to measure:

1. Matter cost
2. Internal client satisfaction
3. Average blended rate (if fees are based on hourly rates)
4. Outcome
5. Matter duration
6. Phase length¹

As noted by the Buying Legal Council, we are still in the early days of legal procurement, and as more robust data becomes available on a matter-by-matter basis, more sophisticated analysis will emerge.

Law firm measurements

All legal procurement professionals surveyed by the Buying Legal Council measured their total spend per firm, but there are many more layers to examine, including:

1. Satisfaction with the law firm (other than outcome)
2. Availability of the law firm and ease of communication with the lawyers
3. The quality of the working relationship
4. Volume of active matters
5. Average blended rate
6. Matter cost vs benchmark/estimate
7. Matter duration vs benchmark/estimate
8. Timekeepers per matter (billable lawyers working on the matter)
9. Average years of experience per Partner¹

Metrics to measure billing issues

The Buying Legal Council were surprised to find that not many legal procurement professionals look to metrics to identify potential billing issues, however this is an area that can be valuable. Red flags include:

1. Individual lawyers billing significant amounts over a year
2. Lawyers billing in excess of 40 hours per week
3. Individual lawyers billing more than 8-12 hours per day ¹

Metrics around points one and two can identify that it would be of greater benefit to the organisation to recruit and expand their own team, or alternatively, to contract in an experienced in-house lawyer.

Point three is often overlooked, but it is important to understand – do you really want over-worked, exhausted lawyers providing business critical advice?

Metrics that in-house counsel sit up for

One observation that we will make is that many of the metrics highlighted above are essential for procurement professionals, however for seasoned in-house counsel we have also found that there are some specific metrics that they get very excited about.

The first of these is work-in-progress (WIP) estimates. Law firms will not provide WIP estimates on their own initiative, and there are few in-house counsel who demand them. If the in-house legal team received and compiled accurate WIP estimates on a monthly basis they could start to proactively manage and control their legal spend in a much more strategic and targeted way.

The second metric is around scope control – what is the current anticipated cost of out-of-scope work pertaining to a specific matter. Again, as above, this information starts to allow for tracking, measurement and control. Importantly, it puts the onus on law firms to be proactive about managing out-of-scope work when it arrives, not at the point of billing the client (which is what happens at the moment).

When working with our own corporate and government clients, we are very focused on providing in-house counsel with these important metrics.

Post-matter evaluation

Evaluating law firms on pricing accuracy and service delivery following matter completion is a key aspect of supplier relationship management, however it is very rarely done.

In the legal industry at present, there is very little formal evaluation of law firms post-matter completion. Law firms crave this type of information and they pay very good money to professional research consultants to find out what their clients think of them.

Interestingly, many lawyers argue quite assertively that it is impossible to assess law firms accurately – they are not a restaurant or hotel after all.

We disagree with this last statement for four reasons:

1. Law firms will be judged and evaluated informally whether they like it or not, that is human nature and it will happen.
2. Formal evaluations allow for law firms to enhance their knowledge and improve their performance and this can only be useful when it is done on a matter-by-matter basis across the board, or at the very least on a consistent, standard basis.
3. Evaluation is a key part of the legal procurement life-cycle so that legal teams can continuously learn and use this knowledge for future engagements.
4. Lawyers, like professional services and other service industries can, and should, be evaluated.
5. Knowing that they will be rated at the end of a matter incentivises the lead lawyer to provide great service and accurate, transparent pricing for their client. Used in this way, an evaluation tool, such as a scorecard or Lawcadia’s rating system, can be a mechanism for holding the law firm accountable for their pricing and service delivery.

According to 2015 research, legal heavy-weight Lexis Nexis found that eight-in-10 lawyers think they deliver above-average service and value but only 40% of clients say that’s what they get.³ This highlights a clear disparity between what lawyers think they are delivering versus the client’s experience. Law firms and their lawyers would greatly benefit from consistent feedback so that they understand and appreciate the real picture.

Parting words

As this is new, unchartered territory, measuring and effectively utilising data in the legal category is still a challenging area, but with the growth of technology and applications, we expect that the future will hold a much greater level of transparency and accountability.

 

Works Cited

¹ Buying Legal Council. (2016). 2016 Legal Procurement Survey.

² Melnitzer, J. (2 May, 2016). Using data to select external counsel. Retrieved 8 December 2016.

³ Lexis Nexis. (2015). The Bellwether Report 2015: the Age of the Client. Retrieved 14 December 2016.