Case study: dynamic noise identification

This case study is based on the second case from the Princeton Dialogues on AI and Ethics Case Studies. It was initially translated to Dutch and has now been translated back to English.

1. BACKGROUND INFORMATION

Dynamic Sound Recognition (DSR) is a technology can recognize complex sounds based on a short audio clip. At first, this technology was used to quickly recognize songs and then determine the title and artist. However, the technology has advanced and can now recognize much more than just music.

The company Epimetheus is developing a DSR app, AudioSho, that can recognize music, advertisements, nature sounds and human voices. On top of recognition, Epimetheus has also added an AI system to AudioSho, which attempts to provide more information about the sound found. For example, voices are enriched with information about the speaker and birdsong with the bird's Wikipedia page.

Epimetheus is becoming widely known and is beginning to add specific categories, for example voice actors, to AudioSho. Several companies are bidding on AudioSho and one winner emerges, Cronus. Before Cronus finalizes the sale, its lawyers want Epimetheus to prove that they have done everything possible to minimize the risk of potential harmful effects from AudioSho.

2. WHAT ROLE DOES AI PLAY IN THIS CASE STUDY? AND HOW IS THIS AI PUT TOGETHER?

The DSR software in AudioSho is the core of the case. For this software, an audio signal is converted into a fingerprint of the fragment. A fingerprint is created by analysing the fragment and seeing what frequencies are present where. The fingerprint is then looked up in Epimetheus' database to see what it most resembles. The AI system then looks at what comes out of the database and tries to find relevant information about the most likely match. Recognition of specific categories, such as recognizing voice actors, is also done by an AI system. 

3. IN WHAT WAYS WERE AGREEMENTS MADE BETWEEN THE RELEVANT PARTIES IN THIS CASE STUDY?

• Epimetheus has agreed with Cronus to examine its software to see if there are no harmful effects. As a result, Cronus will not face any surprises.
• The people who are being recognized and whose voices are being enriched have not given permission for this. That information is all gathered from the Internet, though, such as YouTube and Facebook.

4. WHAT ARE THE RISKS INVOLVED IN THIS CASE?

• Misidentification

While testing the software, Sybel, one of the testers, runs into a problem. Sybel is transgender and the software classifies her voice as male and also shows old videos from before her transformation. This is a clear violation of one's right to be forgotten and a problem Epimetheus had not yet considered.

• Cultural dependence

AudioSho's recognition groups were created in Western Europe by English-speaking developers, so the software works best for English speakers. This is a problem if the software is going to be used globally by Cronus.

• Categorization problems

The core of what AudioSho can do is reduce a fragment to a fingerprint, then categorize and enrich it. Problems arise when the categories AudioSho uses are too broad, or when people are miscategorized. The categories also may be too specific, so that people who have no public role are directly identified, or too broad so that the system is no longer useful. Also, small groups are sometimes not properly recognized.

• Unanticipated uses

Although the developers mean no harm, AudioSho can be used with malicious intent. One can be identified on the street based on one conversation snippet, or a transgender person can be recognized. Being recognized as transgender can be dangerous because transphobia is still unfortunately a serious issue.

5. HOW ARE THESE RISKS MITIGATED IN THIS CASE STUDY?

• Misidentification

Epimetheus tried to improve the classification of transgender people by collecting more data, but this did not work. After this, they decided to just turn off gender recognition.

• Cultural dependency

To deal with cultural issues, Epimetheus decided to turn off certain categorization at the local level. With this, they try to avoid "wrong" outcomes by making them unavailable.

• Categorization problems

Although AudioSho is very accurate, errors do occur. No matter how much data was added, the errors could not be eradicated. Epimetheus does not do much to address the problem of categories not quite fitting the groups being categorized.

• Unforeseen uses

Epimetheus does not have a direct solution to malicious usage, apart from disabling certain categories. There is no easy solution to this.

6. DOES PRIVACY PLAY A ROLE IN THIS CASE STUDY?

With the additional features Epimetheus has added to AudioSho, more and more information about people is being processed. For public personas such as actors, athletes and politicians, it is still acceptable that they can be quickly identified. However, for people who do not have a public persona, like Sybil, always being identified is not an advantage. It limits the anonymity that people had in public spaces, and by processing a person's characteristics without their explicit consent, there is almost certainly a violation of their privacy.