AI and Ethics

Artificial intelligence (AI) applications have seen rapid adoption in both the private and public sectors, sparking debate on AI and ethics – specifically how to define clear ethical boundaries for the use of AI.

This is not the first time humanity has faced such a challenge. Indeed, just like nuclear fission and DNA manipulation, AI has the power and potential to both benefit or harm individuals and a large fraction of society. Therefore clear principles of ethics need to be agreed upon in order to avoid potential misuse and help us find a path through this uncharted territory.

Ethical concerns range from the fundamental human rights – which are seen as the accepted moral entitlements of every individual, including privacy, security and equal opportunities – to ethical principles such as justice and autonomy, as AI should not be used to further widen inequalities in our societies.

Key questions to consider:

• How do we ensure AI supports human values?
• Who is accountable for AI misuse or unforeseen consequences?
• How do we make sure transparency in AI development and deployment is guaranteed (i.e. making it accessible to understand how the decision making on an AI system is carried out)?
• What processes can be put in place to make AI safe, fair, non-discriminatory and accessible?

Experts across various disciplines are seeking to understand the answers to these questions. The growing sophistication of AI algorithms makes them computationally expensive to develop, train and deploy, meaning that only a few well-funded organizations – such as big tech companies and governments – can afford to use them.

In an era defined by the rise of authoritarian governments ¹, where computational power and the ability to harness Big Data is in the hands of the few,² there is a growing concern that this technology, if left without clear governance, can exaggerate power imbalances in our societies.

The development and adoption of digital technology is occurring at breakneck speed, with innovation moving faster than the ability of governing bodies to adequately adapt and respond. This leads some commentators to refer to this period as a technocracy³, since technology is forcing us to rethink our laws.

Here, AI poses an even more complicated problem than nuclear fission or DNA manipulation: who is “accountable” for artificial intelligence, for the decisions it can take and the outcomes it may cause? If these algorithms are so complex to understand and make decisions without consulting a human first, should they be accountable or should only humans be responsible for their use? Like medication, should each AI system come with its own “prescription” explaining how to use it, on what kind of data, under which circumstances, and what the potential outcomes and limitations are?

We can see how this debate is encompassing a whole range of topics – from social science to philosophy, law and governance – making it one of the big challenges of our time. Will data scientists, lawmakers and social scientists be able to find suitable solutions?

In the following sections, we will take a look at concrete examples of applications that may pose ethical concerns. We will progress to examine the European Union’s definition of ethics and the framework developed to address this complicated subject.

Examples of applications that pose ethical concerns

Predictive policing

Predictive policing being deployed to classify crimes using algorithms to analyze large criminal justice and historical crime data sets raises ethical concerns. Aiming to predict how likely certain individuals may be more prone to commit further crimes than others triggers debate. ⁴ This application poses ethical issues which are even more striking when you consider that these historical data sets have been proven to have many major flaws, including racial bias.⁵

A big concern with this type of application is that the models are often trained on far from ideal data sets, which may be biased with respect to race, sex and status. This makes the algorithms inherently racist or sexist and potentially favouring of the already privileged.

These types of data sets not only contain errors, but also mirror the human decisions made in the past up until today – therefore representing a skewed, often racist and otherwise discriminative society. An “ideal” data set would therefore be one which represents the reality we would hope to live in, with equal representation of everyone.

We must remember that AI models need to be trained to discover patterns on data similar to that which we intend to use the model on. The data on which the models are trained is called a training set. Part of this same data set is then used to validate that the model is accurate – and so if the data set is racially biased, then the most accurate algorithm you train is going to be racially biased as well.⁶

AI algorithms are meant to learn features and patterns in the data, before applying them to another set of data. Just like humans can learn how to multiply numbers using examples given in a school textbook before applying the same logic to answer exercise questions, AI systems follow the same procedure.

If we learn mathematical multiplication the wrong way, we will incorrectly apply it to all the questions in the exercise book. Similarly if an AI system finds that biased and discriminative patterns are most helpful to perform the task, it will look for them in any new data set, leading to biased results that reinforce the issue.

Job promotions

Predictive policing is complicated and far from our day-to-day lives, so let’s apply this same idea to a different problem: promotions within a company.

Imagine you are working for a bank in Switzerland which used to be a small family business before recent expansion. After moving its headquarters to Zurich city centre, they hire a diverse group of employees with more international backgrounds.

The head of HR aims to be as unbiased as possible when promoting staff and awarding bonuses to workers, so he decides to use an AI system that automatically selects who to promote. This sounds like a good idea, so the AI specialist starts informing the algorithm on the criteria to use, e.g. the number of projects that were successful per person. The algorithm is also trained on past data, trying to find patterns for identifying employees that have merited a promotion before.

The AI system the specialist is using is very “clever” and is left rather “unsupervised” in this task. The AI specialist gets very good results when testing the algorithm on past data, meaning it tallies with what the head of HR has been reasoning so far, and so it is deemed ready to be deployed company-wide.

The tested algorithm is then applied, but something in the results seems quite off: only men with Swiss German surnames are selected for promotions. This seems particularly odd given the diversity of the employees.

What happened? Shouldn’t an AI system be less biased than us humans?

Left unsupervised to learn patterns in the data set, the AI system discovered that in the past – back when the company was not yet situated in Zurich – the main characteristics of employees receiving promotions were men with Swiss German surnames. This resulted in the same reasoning being applied to the new data set.

Had the head of HR not immediately noticed the strange results, it would have determined promotions in an unfair way, leading to outrage within the company. Now just imagine if the company also attempted to remove the gender pay gap with a similar AI approach – it could definitely go in the opposite direction!

Going back to the predictive policing example, these types of applications have sparked a big debate around AI and ethics in recent years. By relying on a racially biased training set, improving the accuracy of the algorithm will increase the bias; the prior prejudices of decision makers will be taken over and historically disadvantaged and vulnerable groups will be further discriminated against.⁷

Taking all of this into account, businesses, governments, data scientists and other AI practitioners will need to ask themselves whether they want to use a data set that reflects the given reality, or if they would rather adapt it to one that reflects a desired reality.

Autonomous cars

Growing debate over the ethical concerns of AI also throws up another example from the world of autonomous cars.

The idea that we can relax, enjoy the ride and focus on other things while driving is a positive innovation from almost every point of view. It is also likely to make driving an overall much safer experience.

While we can train cars to behave correctly in clear and predictable scenarios, we might not be able to prevent every accident. And what if a car arrives at a point where the AI has to make an impossible ethical choice. How should the AI react? Could a human make a better choice?

For example, take a scenario whereby a parent and child are on the sidewalk and accidentally trip, causing both to unexpectedly fall into the road. What if there’s not enough distance for the autonomous car to brake safely and avoid a collision, with the only other option being to steer dangerously to the side and crash into a wall? How could the AI decide between hitting the wall with a possibly catastrophic result for the driver and potentially killing a parent and child?

This type of conflict is referred to as the “trolley problem”, after a series of ethical and psychological experiments carried out in the late 1970s.⁸ The main question posed is the following: should the AI (or oneself in the trolley problem) let a fatal scenario take its course or actively participate in it, changing the endangered person’s fate – and in doing so sending somebody else to their demise? Which part is more unethical: the passive acceptance or the active action?

The trolley problem became popular again in 2018 when researchers from MIT and other institutes gathered 40 million decisions on how to react to a specific scenario from millions of people in 233 countries and territories. The study showed how decisions are very much dependent on cultural and demographic variables, therefore suggesting that we do not all have the same concept of what is ethically right or wrong in such a situation.

Expanding on our example of the autonomous car, how does the vehicle decide what to do if there are two people falling into the street – but this time from both sides of the road – with the AI unable to avoid both of them? Should we program the car with clear instructions, a rule set to inform the AI system about the “right” decision in such situations? Ultimately, what is the right decision? Would it be right for everyone? Who is then accountable for the car’s accident? And going one step further, who is responsible for an accident between two autonomous cars?

Online advertising

One of the most well-known applications of AI that we have all experienced is targeted advertising on social media.

Based on what you have looked for on search engines and interacted with on social media pages, an AI system identifies your preferences and habits. Informed by data generated from like-minded people, it can then customize what content you see and which advertisements you will be exposed to. This maximizes your engagement with them and the profits of the companies paying to feature on a specific platform.

You will see advertisements for products you are most likely to buy, and the smarter the algorithm, the closer it gets to the holy grail of what you will spend money on. For instance, if you love playing football, follow stars from the game and make many interactions linked to the sport, you will be more likely to receive publicity on the latest TV subscription packed with live sports events. This is targeted advertising in a nutshell.

These AI systems have become so good that it sometimes feels like social media and search engines are reading our minds and listening to our conversations. This is because we do actually follow patterns when we interact with different platforms on the web, which make us quite predictable for a clever algorithm. Moreover, social media platforms do hold a lot of data about us – such as gender, age, personal traits and who our friends are – which can also help to better instruct the algorithms behind it, in turn making them smarter.

In recent years, targeted advertising has sparked global concern because of its application on influencing voters in election campaigns and on polarizing political views.⁹ The lack of transparency on who is behind these ads and how your personal data is being used to make you a target of certain political campaigns is a topical issue and causes a huge discrepancy between who is actually creating the data and who is profiting from it.

We are indeed giving away a lot of personal information, but we’re in the dark as to how it’s used to inform the platform and AI algorithm. Most of the time we’re not even aware of what actual information we are even giving up. This creates a growing lack of equality between the user (us) and the platform we are using.

Can we avoid unethical AI applications?

The above examples touch on some of the main topics of AI and ethics: bias, accountability (intended as responsibility) and transparency. However, AI ethics has many more dimensions, from safety, privacy, fairness, justice, equity, freedom and autonomy, to trust, sustainability, dignity, solidarity and data governance.¹⁰

However, what does data governance (i.e. the principles and rules governing data flows and data management within and across countries) have to do with AI? Isn’t that a separate issue?

In the predictive policing and employee promotion examples, we have seen how data is central to algorithm development, and that certain data sets may be used for reasons that are very different from what they were originally intended. We can see how data governance and AI governance are intrinsically connected and cannot be easily disentangled.

To this end, Timnit Gebru (a former researcher at Microsoft) and her colleagues proposed in their research paper that every data set should be well documented, using “data sheets for data sets” – a standardized way to document the motivation, composition and recommended use of each data set. This is intended to motivate the AI community towards more transparent AI systems creation, capable of increasing accountability.

In recent years, many public and private organizations have published guidelines on ethical AI use. While there seems to be an agreement on the main principles of AI and ethics globally, there remains a lack of consensus on their interpretation, how they should be guaranteed, and by who.¹⁰

For instance, the European Union published Ethics Guidelines for Trustworthy AI, which was put together by a group of experts and addressed the following points:

• Human agency and oversight
• Technical robustness and safety
• Privacy and data governance
• Transparency
• Diversity, non-discrimination and fairness
• Societal and environmental wellbeing
• Accountability

We’ve previously addressed the topics of transparency and accountability, so let’s now explore the other points in more detail.

Human agency and oversight of AI should ensure the support of human autonomous decision making. Indeed, AI should be helping us make decisions that foster fundamental rights, and not substituting humans in decision making. This will enable us to maintain an oversight of artificial intelligence – so rather than AI watching us, we can ensure it is used for good by different governance mechanisms.

Technical robustness and safety requires that algorithms should be developed with the idea of preventing, or at the very least minimizing, unintentional harm. Furthermore, AI systems should be robust enough to be safe from hacking and attacks to the model or data breaches.

Moreover, algorithms should be able to process the data in a way that should protect the privacy of the data. This should be guaranteed by clear data governance, such as the GDPR in the EU.¹¹

The avoidance of unfair bias is then accounted for by the principles of diversity, non-discrimination and fairness. Here we have seen how predictive policing examples may lead to biased algorithms and results.

Similar examples of racial bias can be found in facial recognition algorithms, which tend to work best on white males. A study on the accuracy of commercial facial recognition algorithms to detect and classify people’s faces was carried out by the MIT media lab. The Gender Shades study found that the investigated algorithms performed poorly on women with darker skin, highlighting the fact that AI should be trained and tested on racially diverse data sets to avoid bias.

Furthermore, we should keep an eye on environmental wellbeing when developing and deploying AI systems that rely heavily on data centre servers and their energy consumption, which is often linked to escalating carbon emissions.

The ecological impact of training an AI system of course depends heavily on the size and type of the training data, the learning approach and the AI models used. However, some investigations show that training a single AI model “in the cloud” can potentially emit as much carbon dioxide as five cars in their lifetimes¹² – this is immense! Trying to minimize AI’s impact on the planet’s climate and going for the “greener” option wherever possible should become a key consideration.

The concepts listed in the EU’s guidelines need to be taken into account for the safe and ethical use of AI. The authors also advocate for a continuous evaluation of AI during its whole life cycle as unattended consequences may be very hard to predict, especially when these algorithms become evermore advanced.

We have seen how AI and ethics is a very broad issue, encompassing many competing topics, making any proposed solution complex. We hope you have a better idea of what is meant by AI and ethics and how this is an issue that touches each and every one of us.