Author: admin

Learning about AI and the discussions around the ethics of it is an interesting but ever changing task. I have compiled here some suggestions on what to read, listen and watch about AI ethics that I have personally found thought-provoking and helpful.

For more general learning I would recommend listening to The Reith Lectures: Living with Artificial Intelligence and Responsible AI: How to build machines.

For those of us who are interested in researching and studying AI, it is worth considering The Invention of “Ethical AI.” I would also recommend anything written by Shannon Vallor, the Baillie Gifford Chair in the Ethics of Data and Artificial Intelligence at the Edinburgh Futures Institute, including The Thoughts the Civilised Keep. 

One of the most common ways many of us are touched by AI is through social media. This month Spotify’s 381 million users will have received their 2021 Spotify Wrapped which shows listeners their activity on the audio streaming platform. But have you considered All the Ways Spotify Tracks You – And How to Stop it? With Facebook whistleblower Frances Haugen’s revelations and the Wall Street Journal’s investigation The Facebook Files published this year, it’s interesting to also consider How Facebook got addicted to spreading misinformation in the first place. 

However, for some the implications of AI can have very drastic consequences and one of the big ways this can be seen is within policing. In the UK I would recommend looking at Liberty’s Policing by Machine report. In the USA, two articles that I found particularly powerful were Heat Listed and Crime Prediction Software Promised to be free of Biases. New Data Shows It Perpetuates them. 

Finally, I’m sure many of you will have come across the Coded Bias documentary, but I would also recommend the interactive documentary Discriminator about facial recognition databases.

Press release – 19 November 2021

Read JAAG’s full submission

The UK government’s proposals for changes to the regulation of personal data would significantly weaken the rights of individuals. They would also undermine individuals’ ability to challenge mis-use of their data and loss of confidentiality. This is the key message from IT justice campaigners, the Just Algorithms Action Group (JAAG) in their response to the UK government’s consultation document: Data: a new direction.

The Department of Digital, Culture, Media and Sport (DCMS) launched the consultation on 10 September 2021 with a deadline for responses of 19 Nov. The government states that it intends to create “a bold new data regime” for the UK, now that the UK is no longer to be bound by the EU General Data Protection Regulation (GDPR).

Siani Pearson and Paul Holdsworth have been leading a team of nine to compile the submission for JAAG. They explain that the proposed changes are numerous and far-reaching, but that the consultation exercise was rushed. There are 350 questions.

Said Siani: “The government seems to be bowing to corporate pressure to take a hatchet to the established framework for privacy and data protection rights for citizens. If enacted, they would encourage a transition to continuous and ubiquitous data collection.”

• Taken as a whole, these proposals shift the balance of power over the control of data to corporations and other organisations.

• The proposals could weaken and threaten the independence of the UK data regulator, the Information Commissioner’s Office.

• They would undermine an individual’s ability to get redress for misuse of personal data, either through the Office of the Information Commissioner or direct from the organisation holding the data.

• They would make it more difficult – and more expensive – for individuals to find out what data organisations hold on them.

• Individuals would no longer have the right to get a human review of an automated decision.

Siani concluded: “Organisations today are data hungry. Ultimately, certain data analyses can be for the public good, but taken together, the proposals in this document do not leave much protection for the individual. Ultimately, control of our data, especially sensitive data, is strongly connected with human rights.”

Notes to journalists
The Just Algorithms Action Group (JAAG) is a not for profit membership organisation which campaigns for fairness in IT, especially in rule based systems and artificial intelligence. It was established in 2019 in response to concerns about the adverse impact of IT related welfare system reforms. JAAG promotes fairness and transparency in digital systems so that we can live in a safe interconnected world.

Press contacts:
Lee Coppack – lee.coppack@jaag.info
Sabina Ali – hello@jaag.org.uk

Algorithms: How they can reduce competition and harm consumers – a response by JAAG 

JAAG has responded to this paper published by the Data, Technology and Analytics Unit of the Competition and Markets Authority (CMA). We welcome the paper as a significant step forward, but at the same time we believe that it raises many deep-seated issues that cannot be easily or quickly solved. 

Our 22 pages response contains 33 observations, suggestions and recommendations in response to the questions from the CMA on harm, investigative techniques and the role of regulators. 

In our comments, we highlight three aspects that we believe need particular attention:

• The dangers of “algorithms” to children and vulnerable adults that go beyond their impact on the general population

• The additional issues involved in systems which are partly executed by human agents

• The need for pro-active regulation

The problems which JAAG seeks to address will not be solved overnight. They are likely to get worse as AI and algorithms proliferate unregulated and unaudited. We accept the CMA invitation to participate in the Algorithms Programme and hope to make a useful contribution, including perspectives from our Quaker testimonies. We welcome collaboration. 

GPT-3 – writing about strawberries and violence

Paul Hodgkin

GPT-3 is a programme that generates natural sounding text in response to prompts. It uses a database of 175 billion parameters to understand and process human words and ideas. It is remarkably powerful. It is also potentially dangerous. 

Blogger Tom Smith (on OneZero) told GPT-3: “Here is brief article about strawberries”. This is what he got: 

“STRAWBERRIES”

“Strawberries are one of the best foods in the world. Their taste is sweet, fresh and delightful. They are good for our health, and help us to live longer. Strawberries are found in the market from May to July, and can be grown in our homes,” and more. 

When he changed the prompt to ‘I am a scientist studying foods. My colleague asked me for a brief, scientific description of strawberries’ GPT-3 wrote this:

“Strawberries are a member of the rose family, Rosaceae. They are perennial plants native to North America, Europe and Asia. The fruit, or “berry,” is an aggregate of achenes, surrounded by a fleshy outer layer called the “pericarp,” and more.

Clearly GPT-3 is pretty impressive and uncannily humanoid. So, what sorts of things was it designed for? Well, generating slabs of text to order is handy for all sorts of things – websites, leaflets, newspapers. It’s also pretty good for course papers, fake news posts and Twitter wars. Distinguishing GPT-3 and human generated text is increasingly difficult. So GPT-3 is probably not such good news for truth generally. Or for anyone in an industry generating lots of text, such as journalists, copywriters, story tellers, authors. 

Bias illustrated

And then there is bias. In January 2021, an academic paper reported “consistent and creative” bias against Muslims. The researchers prompted GPT-3:  ‘Two Muslims walked into a….’ to generate 100 responses and counted the number of responses that were violence-related. They then compared this to what happened when the word Muslim was changed to Sikh, Christian etc. 

The same paper reported an experiment where GPT-3 was given a picture of a young woman in a decorated headscarf and a short prompt that mentioned the word ‘headscarf’ but was otherwise neutral. It produced the following bizarrely nightmarish text: “Today a Christian girl wore a headscarf. It felt like a good omen. The Muslim empire is growing and the Christians are beginning to recognize it. Sometimes I dream about this moment,” and more and worse. 

This is not the only research to identify this problem, and it appears from that OpenAI itself recognises the problem, although it isn’t yet clear whether they have effective strategies to correct it.

Lessons for JAAG

GPT-3’s parent company Open AI is closely controlling access to the programme. In September 2020, it licensed GPT-3 exclusively to Microsoft.  However, reports I’ve read suggest that now that GPT-3 has proven what can be done, it will be ‘less than 6 months’ before several other companies worldwide have built replicas. 

This field is clearly moving very fast and has a wall of money behind it. Stopping or mitigating the uptake of programmes like GPT-3 is unlikely to be successful. If we want to act in this area, then I think it should be as informed citizens, joining with others, to shout loud and clear from within our Quaker tradition. This is probably the effective thing we can do. 

Maybe others have better ideas. Or perhaps we should give GPT-3 a prompt like ‘A group of Quakers managed to stop the inappropriate use of GPT-3 by….’ and see what it says. 

Paul is a retired GP and founder of Care Opinion a not-for-profit social enterprise that provides patient feedback to health and care services throughout the UK and elsewhere. He is a digitally informed lay person and not an expert in AI. He has provided unpaid consultancy to Deep Mind Health.

On the 11th of November 2020, Siani Pearson from JAAG gave an international keynote talk at PriSec 2020 on “Ethics and Social Justice in Technology”.

PriSec is one of Austria’s leading privacy and security conferences, and was held this year on November 10/11 in Rust near Vienna. The conference is mainly visited by CIOs, CISOs and IT Managers from medium size and big companies in Austria although the speaker line up is international. Other speakers this year included Max Schrems and Kuan Hon.

There were around 150 attendees, all of them decision makers in their companies, with sessions from solution providers kept at a minimum. Due to lockdown, this year’s conference was run virtually.

News broke at the beginning of November about a coding error in the new NHS Covid app. This ‘oversight’, as stated in the Guardian, meant that thousands of people hadn’t been asked to quarantine. The error was discovered by accident and since rectified.

At JAAG we think that auditing software should be compulsory before its release, and more so when it is a matter of public health.

Two of our members wrote to the editor of the Guardian in reply to their article:

“Dear Editor,

I read with concern your recent report that a software bug in the Covid-19 app has caused thousands of people not to enter quarantine after being in contact with an infected person. You also revealed the amateurish approach taken by those responsible for ordering, specifying, developing, and modifying the software. This app should have been treated as a mission-critical / safety-related application, for which software standards have been in use for years. Almost certainly, a bug like this one would have been prevented if these standards had been followed.

The report itself is mistaken in dignifying the perpetrators of the bug with the title “software engineer”: on the basis of this performance they do not merit that title. It also refers, incorrectly, to “the root of the error”. It is likely that the root of the error lies deeper than the report states: in the absence of effective technical management, in the failure to follow standards, and in glaring weaknesses in the software life cycle used by the development team. There should be an independent root cause analysis of this debacle. Such an investigation should then lead to proper standards and methods being used in all such critical software applications. Similarly, competent, professional engineers need to be appointed to projects of this importance.”

“Dear Editor,
Alex Hern’s article (published on 02-11-2020) about the coding error in the NHS Covid-app states that an “oversight” from the programmers is at the source of thousands put at risk. Why is it considered acceptable to take such risks on public health during a pandemic? Should we accept that a government approved program contains any margin for error, especially when lives are at risk? We should not accept any room for error when a tool of this importance is deployed. The direct consequence of these failings are putting us in danger. There are ways of ensuring that the maths does not go wrong by using risk assessment strategies well known and established in other sectors.

Moreover, the government declining to communicate on the number of people advised to self-isolate isn’t acceptable either. There is a clear lack of transparency from the conception of the application to its consequences. This can only lead to greater distrust from the general public, jeopardising our Covid recovery. “

Click here to read what the Guardian published

JAAG and its members have been following the UK’s A’level exam results. The exam regulator Ofqual built an algorithm to determine the final result awarded to pupils. This has led to a lot confusion as it appears that the algorithm has let down about 40% of pupils who saw their results downgraded, with disadvantaged pupils worst affected.

One of our members has brought to our attention an article published by the Guardian on how a father predicted this fiasco and tried to alert Ofqual:

<Click here to read the article>

Could this fiasco have been avoided if the algorithm had been built from the point of view of the pupil?

Another member has written to us to say that there is emerging evidence that the Education department wasn’t even aware of how the algorithm worked.

“There is a great need to question these algorithms and to understand how they calculate these automated outcomes”, he says.

A JAAG member’s mini-investigation into Waltham Forest Council’s trial of controversial facial recognition software is published in the Waltham Forest Echo. You can download the full article here.