In March, a standards review of the Crown Prosecution Service discovered that ‘errors of analysis and judgement’ had subsequently led to 7% of cases being wrongly prosecuted or withdrawn. The CPS used human reviewers, which included unit heads, who only spotted one in four cases with errors, increasing ‘anxiety, inconvenience and distress to victims and witnesses’ and leading to those who should not be charged facing court.
Like many organisations, the CPS has to manage mountains of information – much of which is increasingly unstructured in format. For example, a typical case could have anything in excess of 60,000 e-mails, IM transcriptions, social network transcriptions and additional evidence that all needs to be sorted, analysed and verified for use in court. The strength or weakness of that disclosed information is important as it could be the difference between a case going to court or not, and between justice being delivered for the victim and his or her family, or a defendant being wrongly prosecuted for a crime he didn’t commit. As public servants, prosecutors usually want to avoid bringing poor cases based on poor evidence to court and feel a duty to ensure that justice is served.
Of course analysis errors aren’t unique to the CPS: any business that relies on significant human review in its processes will suffer from similar issues. This indicates that the problems highlighted in the report are not exclusively the fault of the CPS. They are issues with being human.
The problem with people
The CPS’s reliance on people to sort and analyse information is the number one reason errors occur. Firstly it is not feasible for a team of humans to analyse every piece of data efficiently and effectively. Take the current News Corporation case for example in which News Corp provided over 300 million e-mails to the Metropolitan Police. We estimate that if every member of staff at the Met, from the receptionists to commissioners, spent 10 hours a day, every day, it would take the entire Met Police force approximately four years just to get through those e-mails. It is impractical and certainly not cost or time-effective.
Furthermore, the problem with using people is that people are simply not good at repetitive tasks, particularly ones that require concentration over long periods, such as the serial review of documents. When you consider that the average sustained attention span is only about 20 minutes, it is easy to see how critical information may be missed or passed over. Now the challenge is to see how such tasks can be done better.
Extracting the human machine to reduce errors
The CPS does not need to look far to find a solution. Fully removing the human element for evidence gathering is not feasible, nor is it necessary. However, minimising the amount of human input is. The CPS, like other areas of the legal sector, needs to shift its focus towards a computer-assisted review strategy where technology is central. Private practices are increasingly adopting technologies to help review and prioritise documents in order to extract the relevant from the irrelevant pieces. Unlike humans, technology has proven itself at running repetitive tasks. By de-prioritising the irrelevant materials, lawyers can be presented with a more manageable sample to review and analyse making the whole process much more efficient.
However, legacy knowledge management systems, founded on keyword searches that ‘flag’ documents based on relevance to a keyword or phrase, are also limited in their ability to find all pertinent information. Traditional keyword search technologies do not recognise a change in nuance, and can ‘miss’ important documents that do not contain the word searched for. A typical keyword based search review can ‘miss’ approximately 80% of relevant documents.
Context and meaning
The CPS cannot, therefore, rely on documents that have been found using keyword search alone and instead needs a solution that can understand the context and meaning of each individual document within a sample. Concept search is already emerging as the preferred solution for document analysis. Rather than simply identifying specific keywords and phrases, concept search technology ‘understands’ the meaning of documents and recognises patterns within them.
Concept search helps employees effectively access key information through predictive information management and is driven by machine learning. This would enable the CPS to search, categorise and analyse information provided over any means. Further, by viewing individual pieces of information in context, the ensuing judgements on that information can be more accurate.
Risk-free CPS
While the CPS and other legal practices continue to rely solely on human review, the risk of errors in case documents and evidence will remain high. Instead, legal practitioners need to adopt computer assisted-review technologies that don’t just analyse documents based on keywords, but can actually understand the meaning and context of individual documents in relation to any given case. Only then can the CPS mitigate any risk of case withdrawal or wrongful prosecution.
Howard Sklar is senior counsel at Recommind. He is an expert in eDiscovery and search solutions: www.recommind.com
