As artificial intelligence becomes a part of everyday life, developers need to make sure that the models it learns from provide an accurate reflection of the real world.
Like a young child, AI doesn’t know about the history or societal dynamics that have shaped the world to be the way it is. And just as children sometimes make strange or inappropriate remarks without knowing any better, AI learns patterns from the world naively without understanding the broader sociotechnical context that underlies the data it learns from.
Unlike children, however, AI is increasingly being asked to make decisions in high-stakes contexts, including finding criminal suspects, informing loan decisions and assisting medical diagnoses. For AI ‘parents’ who want to make sure their ‘children’ do not learn to reflect societal biases and act in a discriminatory manner, it is important to consider diversity throughout the development process.
Diversity impacts several stages of the AI development lifecycle. First off, diversity in training and evaluation data is key for many tasks where the goal is to make sure the AI performs well for people of different backgrounds. For example, the seminal ‘Gender Shades’ paper [PDF] highlighted how facial-processing technologies can have lower accuracy rates for black women compared with other groups. ‘Gender Shades’ and subsequent research have attributed such biases to lack of sufficient diversity and representation in the datasets used to develop these technologies. This type of bias can also occur in humans: studies have shown that people have a harder time recognising people of a different race. But psychological research has also shown that such biases are lower if individuals have more contact with diverse people when growing up.
It is not enough, however, to simply make dataset diversity a goal, and achieving it is non-trivial in practice. As I discuss in a forthcoming research paper, collecting sufficiently large, diverse datasets is very challenging, especially in sensitive contexts like human-centric computer vision where existing public datasets often suffer from diversity and privacy problems.
At Sony AI, we have several AI ethics initiatives focusing specifically on ethical data collection. Our goal is to develop best practices and techniques optimising for fairness, privacy, and transparency.
The level of diversity within the companies where AI is developed and deployed is important as well. Company culture plays a key role in fostering or hampering diverse AI teams. It is common for employers to attribute diversity issues to pipeline issues – that there are not enough women or under-represented minority students who study computer science or related fields in school. This emphasis on pipeline issues fails to account for the high rates of attrition of women and minorities from positions in the tech sector. A recent study examining the reasons why women and minoritised individuals leave AI teams found that attrition is primarily due to toxic work environments, experiences of prejudice and lack of growth opportunities.
Having diverse AI ‘parents’ is key, since spotting potential problems in AI development requires an understanding of how the technology might interact with society in harmful ways. For example, law enforcement’s use of AI is extremely controversial in the US because of the country’s history of biased policing practices. AI developers studying AI ethics in the US are well acquainted with the failure modes of trying to develop AI for US law enforcement, but every country has its own societal inequities that can be exacerbated by AI. Counteracting such harms requires greater awareness and understanding of the contexts where AI is deployed.
While this is not an easy challenge, it is one I and my colleagues at Sony are embracing. Through the years, Sony has consistently been recognised as one of the ‘World’s Most Ethical Companies’ for its long-time commitment to responsible business practices. Diversity is one of Sony’s core company values given the global and multicultural nature of the company and its businesses. Sony published its AI Ethics Guidelines [PDF] in 2018, and announced that it would screen all of its AI products for ethical risks. Ethics by design is key to our approach, which includes AI ethics assessments at every stage of the AI lifecycle, from ideation and design to development and deployment.
As AI becomes increasingly integrated into our everyday lives, technology companies and others developing AI must reflect on how they are ‘parenting’ it: what representations of the world it is learning from and whose values it reflects. To build a future of more just, equitable AI development, diversity must be at the heart of AI solutions.
Source: E&T
