If the Civil Justice Council’s Advisory Group on online dispute resolution (ODR) recommends the adoption of ODR for civil disputes, it will represent an opportunity to introduce new combinations of justice and technology that go well beyond mere automation of the status quo.
With few exceptions, leveraging technology will still depend on various forms of human input and administration. Optimization will come when we combine the power of people and computers, rather than focusing too heavily on one or the other in isolation.
In this fourth, and final article, I consider one particularly aspirational shift relating to the use of artificial intelligence (AI) to add new capacity to overburdened justice systems. Rather than take a big step and make grand promises about the benefits of robotic lawyers and judges, my vision is based on relatively simple technology that is practical, and that will build a system that is achievable in the short term. In keeping with this aspirational but practically minded theme, I also consider the potential to make monitoring and measurement a part of an ODR system’s architecture. Lastly, I consider the need for designers to embrace a range of new skills and approaches that we would not traditionally associate with justice system design. This multidisciplinary approach will likely be critical to the successful design and implementation of a civil justice ODR system.
I. Justice System Artificial Intelligence
A successful civil justice ODR system will address the needs of many users if it is designed to be accessible, fast and oriented toward a customized user experience. Yet, in this era of justice system resource scarcity, it must also minimize impacts on human resources. Not only are human resources expensive, but they can also create bottlenecks that slow justice processes down.
These seemingly incongruent goals can be achieved through the use of scalable technologies to do the work. Appropriate use of AI is one way to use technology to provide services directly to users.
Reliance on AI will not eliminate the need for human resources or the delivery of human services to users, nor should it be. Humans on their own can do many things very well. Computers on their own can also do many things very well. Combining these respective strengths in an appropriate way will achieve much better results while helping to manage the outlay of resources.
Artificial Intelligence and Expert Systems
Artificial intelligence that mimics a human expert can take the form of an ‘expert system’. Expert systems can help support or guide human decision-making processes. Their recommendations or conclusions are based on information input by users.
Justice system users desperately need support in their justice interactions, but often cannot afford the costs of expert legal assistance. Moreover, court registry staff are often restricted from providing much guidance for fear of giving inappropriate advice or because they are too busy just trying to cope with perpetual system backlogs.
A web-based expert system would be available for use 24 hours a day, 7 days a week to anyone with a computer or mobile device. Its services could help users to resolve their disputes early in the process or to prepare for subsequent dispute resolution phases. Functions could include:
1. Problem diagnosis: Providing the structure to help the user self-identify the type of problem or dispute they are having. The level of diagnostic specificity is up to the system designers.
2. Information: Once the user’s problem is diagnosed, the system can deliver specific information to help the user better understand the relevant facts, laws and processes. It might help them to manage the dispute or to understand what a reasonable outcome might look like.
3. Self-help: Once the user’s problem is diagnosed and some information has been provided, the system can offer interactive tools or other forms of guidance to support active management or attempted resolution.
4. Triage and streaming: If the user is unable to achieve a self-directed resolution, the expert system can ‘stream’ the dispute into the appropriate next step within the broader ODR process. It can also provide some triage criteria and suggest expedited processes in the case of urgency or additional sources of support if the user’s needs appear to require it.
Knowledge Engineering
Expert systems do not actually ‘think’; they are programmed with specialized information to mimic human experts. This expert knowledge is derived from a process called knowledge engineering.
Knowledge engineers work with subject matter experts to acquire their knowledge and then structure it in a computer-readable format. That knowledge then forms the expert system’s knowledge base and makes the technology seem intelligent.
Simple, Practical AI
AI technologies may come in many forms and use a range of different reasoning types. However, a relatively basic, deductive and rule-based expert system could contribute significant value to an ODR system. It would also be much simpler than other forms of artificial intelligence to build and maintain.
In a simple expert system, an intelligent questionnaire could serve as the user interface. By answering the system’s questions, users would interact with the expert knowledge. This knowledge would be structured to perform the problem diagnosis, information, self-help, streaming and triage functions created through the knowledge engineering process.
We can leave open the possibility for highly advanced AI technologies in the future that may allow natural language input or use reasoning processes similar to the human brain in the delivery of justice services. In the meantime, basic AI can be integrated into ODR systems to provide improved services for users without dramatically increasing system resource pressures.
II. Monitoring and Measurement
If new technologies such as ODR are deployed within a justice system, what will be the indications of success? Moreover, how will we know to improve on its shortcomings or resolve failures? Let’s consider the potential to build monitoring and measurement mechanisms directly into ODR’s technological architecture.
Setting a Performance Baseline
An important step in the ODR system design process will be identification of key performance indicators.
KPIs will serve as the framework for measurement and evaluation and bring aspirational design goals or objectives into tangible form. The development of KPIs will also play a critical role in helping to identify the needs of the technological infrastructure required to collect real evidence, in the form of data, to measure and monitor them.
Evidence Based Monitoring
Because ODR relies heavily on technology-based channels, it presents excellent opportunities to collect business intelligence and evaluation data.
Business intelligence data may include basic information derived from web site analytics showing how many people are accessing the system, how many ‘complete’ specific processes and where they ‘drop off’.
Other data might indicate the number of cases in the system at any given time or over a specific period, the average time to resolution or the average monetary value of disputes. Disputes may also be broken down by type, helping system administrators to understand areas of highest demand and frequency.
Evaluation data can also be collected through technological means in an ODR system. This data is likely to be more qualitative in nature, and can be collected directly from users.
For instance, evaluative data might show whether users understand the system, whether they find it easy to use, whether it’s affordable and or whether the process seems fair. Evaluative questions might also be asked of users as they exit the ODR system to see whether they would use it again.
Designing a System to be Measured and Monitored
Implementation of ODR presents an excellent opportunity to set new standards for measurement and monitoring of justice processes. It is generally understood that justice is information-intensive (to a fault).
So much information is generated that it can be very difficult to manage for justice system administrators, particularly with respect to collection, collation and analysis. Consistency and data quality can also give way to human fallibilities.
One-off evaluation exercises can be helpful, but provide only a ‘snapshot’ in time, missing the opportunity to reveal trends and relative measures. The time and expense of evaluation may also limit their use in justice systems.
As a result of the difficulties associated with measurement and evaluation, it can be very difficult to collect and compile reliable evidence that shows, with any degree of certainty, how justice systems ae performing at a given time. Similarly, it may be nearly impossible to identify or monitor specific trends.
The lack of performance-related evidence can also inhibit new attempts at reform. Rather than analyze new proposals or initiatives against solid evidence, these discussions often consist of anecdotal evidence shared by a collection of well-meaning but slightly biased committee members’ recent trials or most memorable (or annoying) cases within the previous five years.
If system designers build business intelligence and evaluation data collection mechanisms into the ODR system’s technological architecture, information can be compiled automatically and displayed in real-time dashboards for system administrators. At any point in time, it should be possible to monitor the functioning and performance of the ODR system, based on evidence collected and displayed continually by the system. This data can also be accumulated into very detailed reports.
Continuous Improvement
Based on ongoing feedback collected through the system’s built-in evidence-based evaluation and monitoring mechanisms, ODR can be treated as a real-time research experiment. Measurement and performance management can become a larger part of justice system administration.
The ODR system’s human administrators can continuously update workflows or processes, and collect the evidence to validate or disprove their assumptions or hypotheses. These new practices focused on ongoing performance measurement and management can then support the broader goal of continuous improvement.
The aspirational goal of continuous improvement, in combination with ongoing monitoring and measurement, should create fertile ground for a civil justice ODR system that gets better the more it is used.
III. Asssembling New Teams to Create New Justice Processes
If a decision is taken to create an ODR system for civil dispute resolution, it will be important to consider whom to involve in the implementation effort. What new team members should participate in implementation and what role should current justice stakeholders take?
New Team Members and New Vision
The number of judges and lawyers calling for new justice system technologies may be growing, but they are still in the minority. Within this segmentation of justice stakeholders calling for change, fewer still will fully subscribe to the view that disruptive new technologies and equally disruptive new processes are needed to help address challenges to the current system.
Many senior judges and justice officials will naturally prefer incremental, experimental procedural change rather than technology-driven disruption. Such views among these actors are understandable: they play important roles as administrators, obsessed with keeping the current system operating and reacting to periodic crises. It will be rare to find an administrator who is also a radical ODR visionary or an expert in client-focused internet technologies.
Accordingly, it will be necessary to bring in new actors with new perspectives and new skills. The new agents of change best equipped to support the implementation of ODR will not fit the traditional persona of senior lawyer, academic or judge who sits on a blue-ribbon committee. They will have different skills, new perspectives and novel skills from a range of disciplines.
Multidisciplinary Collaborators
To implement ODR within a formal justice system, ODR experts will obviously be involved. Many of these people come from the world of e-commerce. Relative to the traditional justice stakeholders, these actors will hold distinctly different perspectives on issues like legal authority, transparency of processes, natural justice considerations and precedent. They may also have difficulties in understanding the nature and complexity of court file administration and case management.
Diverse views on dispute resolution between private e-commerce and the formal justice system should be compared and evaluated with one another. Each side may offer insights from their respective paradigms. Improvements can be shared in both directions.
User experience experts, interface designers and others with experience in user-centric design will be key to the development work. Workflow process modellers and information technology experts should also be on the core ODR implementation team. These non-traditional justice contributors will help to design a system that will work not only for the justice system itself, but also for its users.
Innovation-minded lawyers and judges could collaborate with ODR system designers to create the rules and processes that link the ODR technology to the civil justice system. Rule drafters should be reminded constantly that the rules will exist to complement the ODR system, and not to constrain its disruptive potential.
If the ODR system employs basic AI or any form of automated guidance or triage, knowledge engineers will be required to help to acquire and structure the expert knowledge through collaborations with subject-matter experts.
Senior lawyers, academics and judges who are innovation-minded and who understand the challenges facing justice and the potential of technology should certainly be involved even if they lack expertise in ODR. In fact, they should become the champions who will build support among their more conservative colleagues.
The Role of Key Justice System Administrators
Obviously, it is preferable to involve key justice system administrators in the implementation of an ODR system. The creation of silos between an ODR implementation project team and the people who manage the existing system will create an endless number of reasons for the initiative to fail.
Senior judges and justice officials can participate in project governance boards or advisory boards in a directive capacity or in a less formal consultative role. Judges and civil servants can also participate in workshops and specialized consultative working groups.
In addition to gaining a range of operational insights from these stakeholders, these efforts will also help ‘normalize’ new ideas and processes associated with ODR, foster greater knowledge of ODR and share the ‘ownership’ in the newly created system. Wider acceptance and ownership should help to support a more successful system shift into a new era of technological acceptance that will be necessary to make civil justice ODR a reality.
IV. Conclusion
If the Civil Justice Council’s Advisory Group recommends ODR, it could signal a commitment to embrace new forms of justice technologies and set a precedent for other courts and jurisdictions to follow. As noted throughout this series of articles, implementation will not be easy. It is much more complicated than merely setting out to ‘do ODR’. In spite of these challenges, the Advisory Group has every reason to succeed.
This series of articles was meant to examine a range of issues and challenges concerning the implementation of ODR in a public justice system. At the very least, it should help to normalize the subject matter and support the acceptance of such technologies by a wider range of justice stakeholders and decision makers.
I would like to congratulate the Civil Justice Council for creating the ODR Advisory Group to explore the costs and benefits of ODR in the public justice context. The Society for Computers and Law deserves my gratitude for hosting these articles and supporting a timely exploration of this subject.
Darin Thompson is a lawyer with the Ministry of Justice in British Columbia, Canada. He currently serves as the Acting Legal Officer for the BC Civil Resolution Tribunal, a new, fully online tribunal that will begin operations in 2015, handling small claims and condominium disputes. He has helped to initiate multiple projects using ODR and is a member of the Canadian delegation to the United Nations Working Group on ODR.
In 2014/15, Darin will serve as an adjunct professor, co-instructing new Legal Information Technology courses at the University of Victoria Faculty of Law and Osgoode Hall Law School.
