CDEI publishes review into bias in algorithmic decision-making

On 27 November 2020, the Centre for Data, Ethics and Innovation (CDEI) published its review into bias in algorithmic decision-making. While the report focusses on case studies in the financial services, local government, policing and recruitment sectors, it is of interest to the increasing number of organisations across a range of sectors who are using AI and similar solutions that are based on algorithms in their decision-making processes, as well as regulators seeking to govern such decision-making. We helped contribute to the review, in particular feeding into their analysis of the regulatory environment and data protection implications.

Key findings of the report include:

• organisations should be actively using data to identify and mitigate bias. Organisations, and in particular their leadership, should make sure that they understand the capabilities and limitations of algorithmic tools, and carefully consider the steps that are needed to ensure fair treatment of individuals;
• organisations are responsible for their decisions - whether they have been made by an algorithm or a team of humans - and the report contains guidance for organisational leaders and boards to enhance accountability; and
• there is a need for a broader ecosystem of industry standards and professional services to help organisations address algorithmic bias.

In the report, the CDEI also:

• calls for the government to place a mandatory transparency obligation on all public sector organisations using algorithms that have an impact on significant decisions affecting individuals; and
• recommends that the UK Government issues guidance that clarifies the application of the Equality Act to algorithmic decision-making. This should include guidance on the collection of data to measure bias, as well as the lawfulness of bias mitigation techniques.

The CDEI has already begun to conduct further work in this space, including initiating a programme of work on AI assurance and carrying out active public engagement to build understanding of the values that citizens want reflected in new models of data governance.

As AI continues to develop at pace, within a legal and regulatory landscape that was not designed with it in mind, guidance from bodies like the CDEI and the Information Commissioner’s Office (which has published guidance on the use of AI and on explaining the use of AI, see our blog posts here and here) is to be welcomed. Regulators, policymakers and government should, however, seek to further coordinate their efforts in this space, so that organisations have a clear and consistent body of guidance on how to operate current and new AI technology in a legally compliant way.