The second SCL Irish Group event built on the first event – an introduction to AI – by exploring one of the known shortcomings of AI, bias. This was an exploration of whether artificial intelligence (AI) or machine learning (ML) could be developed in ways that could counteract much of the gender, demographic and racial biases that are present in traditional coded software technology and design. For example. giant smartphones that don’t fit in women’s hands, virtual assistant AI that fails to understand accents or women’s voices, technology services that fail to cater for people living with a disability and health care algorithms that put lives at risk because the data is derived from a non-diverse dataset.
In our capacity as lawyers, during this time of progress, it is incumbent on us to try to understand these risks. Outside identifying and managing legal risks, however, is there a place for lawyers to challenge product teams to ensure that such biases are not carried over into AI or ML software? Can we play our part so that the technology of the future can be more socially inclusive? Are we engaging with our clients at an early product design stage and asking those difficult questions? To borrow and tweak Emma Watson’s gender equality motivational phrase; given the lack of any meaningful EU or domestic regulation or oversight in this field at present: if not now, when? And if not us, then who?
We asked Dr. Suzanne Little of DCU School of Computing and Principal Investigator at the Insight Centre for Data Analytics and Co-Director of the SFI Centre for Research and Training in AI to tackle the subject of bias in AI with interactions from the floor. Here are some of the key take-aways:
1. Automation gets a bad rap, but…
AI has made huge strides in the fields of medical devices (makes them safer), data analytics (can handle greater volumes of data), security (can more easily detect cybersecurity threats). However, in the field of driverless cars, the high point of automation to date, the Uber car that fatally hit a woman in 2018 failed because “the car failed to identify her properly as a pedestrian”. She had been walking and wheeling her bike on the footpath. This scenario had not been considered during the design phase and cost a person their life.
2. AI or ML and the lure of Big Data
AI is any technique that allows computers to mimic human intelligence and ML is a subset of AI which generally uses statistical techniques to learn from data fed to it. AI is driven by data and we humans have generated a lot of it. It is commonly thought that if we have more data we will ‘know’ enough about it and create a perfect, unbiased AI system. As mere mortals we are guilty of thinking that data equals fact. Data is an aggregation of facts, statistics, things known or assumed as facts, collected information or even just a set of 1s and 0s. Many people think it is black and white, precise and 100% accurate. People are complicated but computers are binary. But what happens when we place our trust in the incorruptible-ness of data as opposed to the people that create it?
3. Consequences of AI Bias
The many real-life examples that have arisen over the last few years highlight the consequences of AI bias. They are wide ranging with the most serious resulting in the perpetration of social injustices. The below are some examples:
I. the Natural Language models that display gender bias in word association (e.g. the word doctor co-occurs more frequently with male pronouns than female ones) even in languages with no gender (Finnish); to
II. Tay (Microsoft Chatbot) that became extremely racist within hours because it was designed to machine learn from Twitter’s users but was infiltrated by 4Chan trolls; to
III. the COMPAS algorithm used by judges in the USA to determine the reoffending propensity of a criminal defendant which was found to be racially biased against black defendants who were incorrectly judged to be at a higher risk of reoffending than their white counterparts; to
IV. facial recognition technology that works best for white males.
4. How does AI become biased?
1. AI is built and taught by people;
2. People choose the data;
3. People create the data;
4. Data is an imperfect representation of reality
5.Users (people) think ‘data’ is trustworthy
6. Users (people) don’t understand machine learning confidence – or should we say over-confidence (!).
Therefore, if AI is driven by data and people create and choose that data and people are biased then it seems inevitable that the AI will be biased. A prime example of this bias arose a week before our event when news of Apple’s ‘sexist’ credit card broke. It appeared to favour men because it gave them significantly higher credit limits than their wives. Embarrassingly, even Steve Wozniak agreed saying “that he had also been given a much higher credit limit than his wife, even though the pair have no separate cards, accounts or assets.” The credit card issuer, Goldman Sachs, is now under investigation by the Department of Financial Services and has stated publicly that “Any algorithm that intentionally or not results in discriminatory treatment of women or any other protected class violates New York law.” Whilst this is to be lauded, it highlights the fact that the vast majority of these algorithms remain unaccountable to anyone but their proprietors and until there is more widespread regulation or indeed a movement within the industry itself to become more transparent and subject their coding practices to outside objective scrutiny we’re going to see or more insidiously not see future scenarios where bias has been at play in our technology.
5. Can we fix it?
Bias is, according to the Oxford English dictionary defined as “a strong feeling in favour of or against one group of people, or one side in an argument, often not based on fair judgement”. The underlined part of this sentence is what should concern us. Whether algorithmic bias or the ‘coded gaze’ a phrase coined by a Joy Buolamwini, a computer scientist and digital activist based at MIT, is intentional or not by the relevant coder is irrelevant. Increasingly, these algorithms are being deployed in many aspects of our lives, for example, in law enforcement, employee recruitment processes, eligibility for health care insurance, car or home insurance, or as in the previous paragraph, credit cards. The lack of transparency in the code is largely based on the fact that it is proprietary to the authors of that code.
So how do we balance the risks of AI bias against the reward incentives for intellectual property creators and maintain fairness, inclusiveness and transparency in this technology? Should we start with the premise that all AI is inherently untrustworthy and should not replace human decision makers? Or should be mandate transparency in AI systems such that all results should be explainable? Or should we treat AI like we do pharmaceuticals and have public and well-defined evaluation protocols?
6. Final thoughts
A lot of food for thought but it was agreed that the more people that are party to the conversation around software design and development the more inclusive that software is going to be. Developers need to think about the social impact of the technology they are creating and understand who will suffer the consequences. And, in this author’s opinion, so do their lawyers.
Kate McKenna is Senior IT/IP Counsel at The Stars Group
