Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

• Xenophobia is a key driver of discrimination and conflict
• Many ML fairness frameworks do not measure or mitigate xenophobic harms
• Aim to bridge the gap between AI and xenophobia
• Identify distinct types of xenophobic harms
• Review potential interplay between AI and xenophobia in various application domains
• Recommendations for inclusive, xenophilic design of future AI systems

Paper Content

Introduction

• AI is being used more and more in our daily lives
• There is a need to ensure that the risks and benefits of AI are distributed fairly
• AI systems can be biased against certain characteristics, such as race and gender
• AI systems need to take into account structural and historical power asymmetries
• This paper focuses on xenophobia, which is discrimination against the foreign
• Xenophobia is a growing problem, especially during the Covid-19 pandemic
• AI can be used to mitigate or amplify xenophobia
• Existing AI fairness strategies focus on legally protected groups
• AI systems need to make explicit normative choices when distinguishing between us and them
• AI can be used to detect hate speech and dangerous speech
• This paper reviews the impact of xenophobia in social media, healthcare, immigration, employment, and large pre-trained models
• The paper makes a moral argument for inclusive, xenophilic systems

On xenophobia

• Xenophobia is a form of hostility or prejudice directed towards foreigners, immigrants or those construed as “others”
• It can manifest as fear, dislike or hate towards people who are perceived to be different
• It has been associated with misassociations, stereotyping and cognitive bias
• It may be attitudinal prejudice or systematically biased institutional and structural processes
• Xenophobia is distinct from racism
• It is orientated around the notion of “civic ostracism”
• It penalises individuals on the basis of their foreignness
• It may result in discriminatory material disadvantages
• It may deny individuals proper ethical recognition
• It may restrict the effective exercise of individuals’ rights
• It may manifest differently than racism or sexism

Practical considerations

• Development of technological solutions can address discrimination against those perceived to be foreign
• Design of inclusive and xenophilic AI solutions can help address discrimination

Social media

• Social media can amplify xenophobia
• Low barriers to entry and difficulty in moderation can lead to fake news and hateful speech
• Social media can also provide a medium for positive and inclusive views
• Social media can shape culture, both positively and negatively
• There is a strong link between social media and hate crime
• Harms in the digital sphere can extend to the real world
• Social media can be used to shape public opinion and exclude minority views
• Deepfakes can amplify xenophobic narratives

Promise.

• Recent advances in understanding and mitigating social biases in AI models
• Development of benchmarks for identifying malignant stereotypes
• Need to transcend ethno-nationalistic exclusionary cultural narratives
• Historical AI approach centered around supervised learning
• Need to imbue AI systems with capacities for understanding and reasoning
• Multi-faceted and case-specific solutions needed
• Multi-lingual models, self-supervised learning, data curation, human feedback, etc.
• Holistic participatory approach needed to avoid technosolutionism

Immigration

• AI systems may exacerbate or mitigate xenophobic harms in the management of migration flows.
• Migration between culturally distinct countries and regions is often accompanied by a growing resentment of immigrants and refugees.
• States have already been quick to employ new digital and AI technologies to control migration.
• AI systems may create bias and harm by restricting rights of movement based on assumptions of risk or threat.
• Immigrants from marginalised communities face additional barriers when attempting to integrate into society.

Risks. numerous technological solutions incorporating ai

• AI systems have been proposed for border control and migrant assessment
• Data from refugees, asylum-seekers and migrants may be used for geopolitical, socio-economic, demographic and other goals
• AI systems have been proposed for automatic assessment of immigration forms and facial recognition technology
• Governments have seen migrants in terms of an algorithmic computation of ‘risks’

Healthcare

• Xenophobia has been identified as a determinant of health
• Policies that restrict health services to foreigners can create a hostile environment
• Repeated xenophobia in healthcare can erode trust and affect care-seeking behaviours
• Medical xenophobia can manifest via negative attitudes and practices by health workers
• Xenophobia and racism can impact health via early exposure to adverse childhood experiences
• WHO defines health as physical, mental and social well-being, not just absence of disease

Employment

• AI systems are increasingly used in decision-making around employment
• Xenophobia can manifest in this sector and impact AI-driven decision-support systems
• Unequal access to employment opportunities and unfair compensation can lead to social inequality
• Labour market attachment is important for individuals and groups, especially those that have been historically marginalised
• Immigrants can suffer xenophobic harms in the employment process, regardless of citizenship and immigration status
• Technology should complement and help coordinate initiatives to ensure fair employment outcomes

Risks.

• 6 categories of potential harms from large language models
• Intersection of material, representational and wider societal harms
• Potential for direct xenophobic discrimination, information hazards, misinformation, malicious uses, subtle reinforcement of stereotypes, and disparate socioeconomic and environmental impact
• Studies have identified concerning aptitude of systems towards accurately emulating extremist views and ethnic biases
• Need for more focused investments in data curation to address underlying risks

Promise

• Assistive technologies can improve candidate experience and help with diversity and inclusion.
• ML models can be used to audit and improve hiring practices.
• Explainability and fairness frameworks can be used to improve hiring processes.
• Simulations can be used to design policies and processes.
• Need for transparency and fairness for candidates of different backgrounds and immigration status.

Stereotypes in large pretrained models

• AI development involves systems that are general in scope
• LLMs can be fine-tuned and adapted to a range of applications
• LLMs are used as building blocks in complex AI systems
• Development of base models involves large-scale pre-training on large datasets
• Safety and ethics concerns due to difficulty of anticipating harmful biases and stereotypes
• LLMs may exhibit varying degrees of positive or negative sentiment
• Text-to-image models amplify demographic stereotypes
• Multimodal systems introduce another layer of complexity
• Important to evaluate models and training data prior to wider release

Towards xenophilic ai systems 4.1 a moral imperative

• Xenophobia has potential to negatively impact people’s lives across a range of digital services and contexts
• ML systems may be used or redesigned to address xenophobic bias in AI
• Building better systems to promote civic inclusion and deescalate othering dynamics is termed “xenophilic technology”
• Tackling xenophobia in AI requires commitment and technical innovation
• Problem of on-screen xenophobia was known but continued to “hide in plain sight”
• Xenophobic content encouraged offscreen discrimination, perpetuated symbolic violence, and contributed to an atmosphere of civic precarity
• Problem of xenophobic bias in AI services is likely widespread
• Scope and salience of resulting harm is not yet fully understood or appreciated
• Solution to the motivational question is to engage with people affected by xenophobia
• AI services have global reach and powerful shaping effect on social relationships
• Compound or structural phenomenon of people exposed to automated xenophobic bias across digital services
• Technologists should partner with domain experts to take the lead in this area and respect human rights

Measurement and mitigation

• Xenophobia in AI systems is not only a technical problem.
• There is a lack of mitigations in practice to address xenophobia in AI systems.
• There is a need to move away from hegemonic machine learning fairness approaches.
• Mitigating xenophobia may help to reassess how technical ML fairness is defined and operationalised.

Measuring xenophobia.

• Central to most machine learning fairness mitigation approaches is the ability to measure and quantify the degree to which a given system violates fairness constraints.
• A thorough assessment of such harms may require multiple scopes of analysis and a consideration of AI systems not only in isolation, but also within the pathways of their projected use, with humans in the loop.
• Most approaches to measuring fairness rely on an understanding of groups and identities across which performance should be measured, coupled with the availability of the corresponding information in the underlying data.
• There are deep issues with the overall quality of race and ethnicity data across observational databases.
• Fairness approaches rooted in comparing model performance and impact across groups need to be more cognizant of the normative nature of defining group identities.
• There is a pressing need for the development of frameworks and benchmarks for assessing xenophobic impact.
• Commitment to intersectional analysis may also act as a driver of technical innovation.
• Counterfactual and contrastive methodologies may prove useful for more concretely identifying the pathways of discrimination.
• Individual fairness may provide a potentially usefully lens to apply to processes in the interim.
• Investigations aimed at identifying xenophobic bias and (un)fairness need to not only improve and extend the current frameworks and categorizations, but also re-center the question of discrimination onto the perpetrators.
• Wider community participation is key when creating benchmarks, soliciting feedback, and empowering the affected communities to inform both policy as well as technical system development.

Mitigation strategies.

• Xenophobia in data used to train AI systems has adverse impacts on vulnerable and marginalized groups
• Current strategies to mitigate xenophobic harms are lacking
• Alternative approaches can provide partial safety and robustness guarantees
• Technical problem specifications rarely map onto socially acceptable mitigation outcomes
• Transparency and model audits are needed to complement fairness assessments
• AI explainability research can help identify model harms and vulnerable groups

Conclusions

• Xenophobic bias is yet to be formally recognised and assessed in AI system development
• Xenophobic attitudes may manifest in data and AI systems
• Potential xenophobic harms include direct material disadvantage, representational harms, and wider societal harms
• AI should promote civic inclusion and oppose “othering” of marginalised groups
• Interdisciplinary action, participatory frameworks, and technical innovation are needed to create xenophilic technology