The AI Revolution
In the 1990s and 2000s, as everything from banking to buying books moved online, “bricks and mortar” businesses were suddenly required to have an online presence and to design new business processes to accommodate the new online world. Through a practice called business process engineering (BPE), banks had to design online services and appropriate authentication procedures, retailers designed online payment processes, and businesses in many sectors implemented processes for back office activities (such as invoicing and accounts payable) and customer facing activities (such as responding to queries and complaints). 1
The generation of native online businesses that flourished during this period enjoyed a natural advantage in this environment. Their business processes had been engineered from the start for the online world whereas the established players often ran their old pre-internet processes and the new online processes in parallel. Over time, through business process re-engineering (BPR), these parallel processes were redesigned so that the bricks and mortar businesses could compete in the new environment.
Today, the revolution in AI poses a similar challenge to that posed by the internet revolution. AI will be able to drive efficiencies through further labour arbitrage and by generating and allowing customers to react to richer insights into their business processes. A new wave of BPE will create processes centred around AI, and no doubt in time, old processes will need to be redesigned through BPR.
In almost every outsourcing that we have seen over the past two or three years, the transformation plan includes the deployment of automation and AI to drive efficiency. Rules-based automation has been around for some time (it is not the focus of this article), but the autonomous nature of AI and the specific requirements for training an AI solution present a new set of challenges for the outsourcing practitioner.
This article asks a simple question: is the traditional outsourcing agreement, “engineered” for the pre-AI world, suitable in a world in which AI displaces both people and rules-based methods of automation?
The answer to that apparently simple question is somewhat complex. This article addresses the question by: (1) looking at the ways in which outsourcing agreements may need to be re-engineered for the deployment of AI, whether one is looking to amend an existing outsourcing agreement through change control or contracting for an entirely new outsourcing; and (2) focusing on the impact of AI on a few key provisions in outsourcing agreements:
- IP and customer data;
- liability, relief and remediation;
- the exit schedule;
- maintenance arrangements; and
- the charging schedule.
AI will necessarily affect other provisions in the agreement (for example, the data protection provisions), with greater consequences in some sectors than in others (for example, regulated outsourcings in the financial services sector).2
This article will then consider the downstream impact of AI, within an outsourcing context of the customer’s other third party vendors, and will conclude with some observations on contracting for disruptive technologies (such as AI) more generally.
Ownership – IP and Customer Data
A common type of AI system involves a dynamic, learning piece of software that is trained on data.3 The more data that is available for training, the more the AI should improve, and therefore the more efficient the outsourced process should become.
Such potential outcomes raise a number of questions for the customer:
– will the customer provide some or all of the training data for the AI?
– does the customer have the necessary dataset?
– would the customer benefit from an AI that has been trained on third party data, or even the data of other customers of the supplier?
– is the customer willing to share any improvements to the AI that result from training on its data?; and
– would this be relinquishing a competitive advantage or even give competitors an insight into the customer’s business?
At one end of the spectrum, a customer might take the view that it wishes to totally segregate any AI deployed in its outsourcing. A separate, “plain vanilla” instance of the AI is accordingly deployed on segregated servers and then trained on customer data (perhaps in combination with data sets that are licensed in from third parties). All improvements to the AI are owned by the customer and cannot be redeployed elsewhere.
At the other end of the spectrum, a customer might agree to deploy AI as a service (AIaaS) and contribute customer data to the data lake on which it is trained. The supplier then owns the AI and any improvements to it, and can deploy it elsewhere. This means that any valuable insights from the customer data are (indirectly) available to potential competitors but the customer also benefits from improvements to the AI that come from training on other customers’ data.
Rather than opting for one end of the spectrum or the other, it is becoming more common to take a layered approach to this issue. The customer may be willing to contribute certain low-value data and relinquish ownership of general improvements to the AI, while segregating high-value data and retaining ownership of the consequent improvements.
The layered approach in practice
In certain circumstances it may be technically possible for the supplier to retain ownership of general improvements in natural language processing and use customer data for that purpose.
High-value risk modelling, behavioural or commercially sensitive data is kept segregated.
Improvements to the AI that come out of training on this high-value data are owned by the customer and cannot be deployed to other customers of the supplier.
In a traditional outsourcing contract normally each party licenses their background IP for the purposes of the agreement, and then any foreground IP created during the term is assigned to the customer. Customer data must be held securely (or perhaps not held by the supplier at all) and processed only for the purposes of the outsourcing. Such a traditional approach to IP and customer data does not cater for the complexity inherent in the layered approach described above. For this reason changes may be necessary.
Changes that may be required
When negotiating a change control note for an existing outsourcing agreement, or the IP and customer data provisions in a new agreement, a customer may need to address:
– who owns improvements in the AI;
– the purposes for which customer data is licensed; and
– the arrangements for provision or hosting of customer data and specific arrangements for its segregation.
Responsibility – Liability, Relief and Remediation
A great deal has already been written on the autonomous nature of AI and its implications for liability. This is a complex question and beyond the scope of this article.4
In the outsourcing context, the autonomous nature of AI raises a series of questions for the customer:
- to what extent should the supplier be liable for errors caused by AI?
- in what circumstances should the supplier be granted relief from liability?; and
- how, in practice, can errors or breaches caused by AI be remedied?
It is common for an outsourcing agreement to contain a “Relief Events” or an “Excusing Cause” mechanism, whereby the supplier is excused from liability where the customer has caused the supplier to be in breach of its obligations (for example, by the customer failing to satisfy certain dependencies). AI is only as good as the data on which it has been trained, so if the customer has provided the training data, should this be grounds for supplier relief?
Similarly, an outsourcing agreement often includes a remediation procedure in respect of breaches. Commonly the supplier is required to conduct a root cause analysis and then to prepare and implement a remediation plan.
How would this apply in the case of AI? The functioning of AI is often opaque.5 It may not be possible to identify a root cause and therefore remedy the issue. The supplier may be unwilling to share detailed information on the functioning of an AI solution because it views the solution itself as proprietary. Further, the remedy may in fact be to replace the AI with a more traditional process for a period of time; in which case, who bears this cost and who will provide the personnel to take over this activity?
These problems highlight the need to address such issues in the documentation.
Changes that may be required
When negotiating a change control note to an existing outsourcing agreement, or the liability, relief and remediation provisions in a new agreement, a customer may need to:
– discuss at the outset with the supplier the extent to which the supplier is guaranteeing outcomes from the use of AI and how liability for breaches caused by AI will be allocated;
– pay careful attention to the way in which the Relief Events or Excusing Causes are characterised;
– consider how in practice errors caused by AI will be remedied and who bears this cost; and
– be clear on what information the supplier is required to share with regard to the functioning of AI in the event of a breach.
Exit
Most outsourcing agreements contain detailed provisions for termination assistance and for conducting an orderly transition to a replacement supplier (or to the customer if the function is being brought back in-house) in accordance with an agreed exit plan.
In a traditional outsourcing, this will generally require that the supplier migrates the data to a replacement supplier and deletes any customer data in its possession following exit (and, in some cases, to transfer related licences or equipment). Suppliers will often be required to facilitate knowledge transfer and to maintain a service library or know-how repository for this purpose. In many jurisdictions, exit arrangements will also include procedures for managing transfer of employees under TUPE or its equivalent.
While deployment of AI may reduce the burden in terms of managing transfers of employees (because headcount may be reduced), it may make exit and continuity more complex in other ways. This is because:
- the supplier may be unwilling to provide information in relation to proprietary AI or license it to a replacement supplier (who is likely to be a competitor);
- it may not be possible to provide the training data for use by a replacement supplier because it belongs to the customer/supplier or multiple third parties; and
- it may not be possible to extract customer data that sits in a data lake.
Such problems:
- increase the risk of disruption on exit and also increase the risk of “lock-in” to a proprietary supplier solution;
- already exist in software procurement more generally – a fact that underscores the possibility that we may see some convergence between AI-enabled outsourcing agreements and software development and supply agreements (a theme discussed in more detail below); and
- can be somewhat mitigated through forward planning and transparency, in particular, by identifying potential “lock-ins” and possible replacement technologies and any gaps in the know-how that can be shared on exit.
Again, such problems highlight the need to address such issues in the documentation.
Changes that may be required
When negotiating a change control note to an existing outsourcing agreement or a new agreement, a customer may need to:
– ensure that the exit plan clearly identifies any software or know how that will not be provided to a replacement supplier (or ensure that the current draft exit plan is revised to address this issue);
– potentially, pre-agree the terms on which any proprietary solution will be licensed following termination;
– conduct due diligence to assess the risk of “lock-in” to a supplier and identify potential replacement solutions; and
– ensure that, in respect of any customer data that cannot be deleted, the supplier has a continuing obligation to maintain the confidentiality and security of such data, and ensure that such provisions clearly set out the purposes (if any) for which such data may be used.
Maintenance
The centrality of software to an AI-enabled outsourcing blurs the lines between an outsourcing and a software procurement. When some or all of a business process is being conducted by a piece of software, rather than by humans, the outsourcing agreement will need to cater for the maintenance of that software. This may require additional provisions more commonly seen in a software maintenance contract, such as:
- an incident management framework, with incident classifications and response and fix service levels;
- provisions for the deployment of upgrades and patches, potentially with commitments as to system downtime and maintenance windows;
- provisions for secure access to customer systems by supplier personnel performing maintenance (if the AI is deployed on customer systems); and
- software warranties.
While many IT outsourcings (ITOs) already contain these provisions, business process outsourcings (BPOs) often do not.
Changes that may be required
When negotiating a change control note to an existing outsourcing agreement or a new agreement, a customer may need to:
– consider who will maintain the AI software and how maintenance will be conducted;
– agree an incident management framework with the supplier; and
– add a maintenance schedule or update the existing schedule.
Charging – Commercial Models
BPOs are often charged on a per “FTE” (that is, full-time equivalent) basis with detailed arrangements for forecasting, transparency of costs and benchmarking. At its core, AI delivers efficiencies because people are displaced by software, allowing for alternative commercial models.
Charging for AI solutions in an outsourcing might be based on combinations of flat or stepped licence fees, volume (for example, a charge per process) or on another abstract measure of value (for example, charges based on savings achieved).
Flat fees may allow the customer to realise gains when operating at scale, volume-based fees will create a more predictable relationship between the scale of activities and the customer’s cost base, while mechanisms such as gain-sharing allow for sharing of risk. Careful attention will need to be paid when drafting the charging schedule to ensure that the unit of count and charging metrics by reference to which charges are calculated are clearly defined and that economies of scale and efficiency gains can be locked in. For example, on an outsourcing of payment processing, charges might be calculated by reference to the number of transactions processed. If the unit of count for calculation of charges is the “transaction”, the charging schedule would need to clearly define what constitutes a transaction and may need to address issues such as transactions that are reversed or never closed and linked transactions.
Traditionally, financial performance is monitored and addressed through transparent reporting and periodic benchmarking. Because AI has the potential to break the link between the supplier’s cost base and the charges under an outsourcing, careful consideration will need to be given to how financial performance is monitored and best value is achieved. In particular, the customer will need to:
- consider how costs are modelled in financial reports; and
- ensure that, in practice, the financial reporting accurately captures the agreed charging metrics in the agreement, especially if that reporting is automated and drives the supplier’s invoicing.
Benchmarking is usually carried out by reference to comparable transactions, but what should constitute a comparable transaction in the AI context? For example, would a traditional BPO constitute a comparable transaction? Would deployment of “off-the-shelf” automation solutions be included in the benchmarking?
Changes that may be required
When negotiating a change control note to an existing outsourcing agreement, or the charging, financial reporting and benchmarking schedules in a new agreement, the customer may need to:
– carefully define units of count and charging metrics;
– include clear transparency and financial reporting obligations; and
– agree a benchmarking process that is suitable to the solution that is being deployed and identifies the relevant comparators.
Downstream Effects
It is not just the supplier’s commercial model that will be affected by the deployment of AI. An outsourcing is often one component of the customer’s operations, which is supported by a number of third party software vendors.
Software is often licensed on a per-user model that is no longer appropriate in the AI context because a single AI solution can do the work of many human users. This may allow the customer to achieve savings elsewhere by scaling back its licensing of such software. However, careful due diligence must be carried out to validate this assumption. Issues that will need to be considered include the following:
- it is becoming increasingly common for software providers to explicitly license on the basis of human users, in which case additional licences for non-human users may be required (usually with a different commercial model). At worst, deployment of AI without checking on the licensing parameters may place the customer in breach of its agreement with the software vendor, perhaps giving rise to termination and under-licensing scenarios of the type explored by the High Court in the well-known case of SAP UK Ltd v Diageo Great Britain Ltd [2017] EWHC 189 (TCC); and
- because AI (and also rules-based automation solutions) is able to operate at a much higher intensity than a human, it may actually mean that the customer is using third party software beyond the specified volume tolerances. This can potentially place the customer outside of warranty with its software vendors and could potentially lead to system downtime, affecting the entire outsourced process.
Action required
- Before deploying AI in an outsourcing, the customer should conduct due diligence on related software licences to ensure that they are suitable for deployment alongside AI.
Some final observations
AI is but one of a number of highly disruptive technologies (such as the Internet of Things, Distributed Ledger Technologies, Smart Contracts and Quantum Computing) to have found focus among computer scientists, technology lawyers and a large number of businesses across a whole range of industry sectors.
In the headlong rush to embrace such technologies, it is important to bear in mind that such technologies are called disruptive technologies for a reason. They are highly disruptive to existing markets and business models. Businesses need to prepare with lawyers active in these markets such that the legal documentation facilitates implementation of these technologies, requiring a substantive reappraisal of traditional contractual risk allocation in many areas of outsourcing and technology contracting.
————————–
NOTES
[1] The author would like
to thank Professor Peter McBurney, Professor of Computer Science at Kings College London and Head of Technology Consulting, Norton Rose Fulbright LLP, and Dr Michael Sinclair, Consultant, Technology and Innovation, Norton Rose Fulbright LLP, for their comments on this article.
[2] More information on these topics can be found at Norton Rose Fulbright LLP’s website,
[3] This article refers only to such machine learning systems when it refers to AI.
[4] for a detailed discussion, see Norton Rose Fulbright LLP
[5] The opacity of AI decision-making also raises specific challenges in contexts where an individual is entitled to an explanation of an automated decision that affects them, for example under the GDPR
