Link to paper
The full paper is available here.
You can also find the paper on PapersWithCode here.
Abstract
- LLMs represent a major advance in AI research
- LLMs come with ethical and social challenges
- Auditing is a promising governance mechanism to ensure AI systems are ethical, legal, and technically robust
- Existing auditing procedures don’t address the governance challenges posed by LLMs
- Article proposes a three-layered approach to audit LLMs in feasible and effective ways
- Article discusses limitations of auditing LLMs
Paper Content
Introduction
- Auditing is a governance mechanism used to identify and mitigate risks associated with AI systems
- Procedural regularity and transparency contribute to good governance
- Proactivity in the design of AI systems helps identify risks and prevent harm
- Operational independence between the auditor and the auditee contributes to objectivity and professionalism of the evaluation
- Previous work on AI auditing has focused on ensuring specific applications meet predefined requirements
- Foundation models are effective across many different tasks and display emergent capabilities when scaled
- LLMs pose ethical and social challenges such as perpetuating harmful stereotypes, leaking personal data, spreading misinformation, plagiarism, and misuse of copyrighted material
- Auditing procedures should be designed to capture the risks posed by LLMs
- A three-layered approach combining governance, model, and application audits should be used to audit LLMs
- Outputs of the audits should ensure LLMs are designed and deployed in ethical, legal, and technically robust ways
The need to audit llms
- Previous research on LLMs and their ethical and social challenges
- Need for auditing procedures to capture risks LLMs pose
- Potential objections to approach addressed
The opportunities and risks of llms
- LLMs represent a major advance in AI research
- NLP researchers and practitioners have been developing software to analyse, manipulate, and generate natural language since the 1950s
- Deep learning, neural architectures, and computational power have revolutionised the field
- LLMs can approximate human performance on some benchmarks
- LLMs are highly adaptable to various downstream applications
- Scaling the model can result in emergent gains on a wide array of tasks
- LLMs are accessible via open-source libraries, democratising the gains from deep language modelling
- LLMs can introduce representational and allocational harms, compromise privacy, produce misleading information, be co-opted by users with bad intent, and incur high environmental costs
The governance gap
- LLMs pose methodological and normative challenges
- LLMs are developed and adopted in two stages
- It is difficult to assess LLMs independent of context
- Performance of LLMs can be unpredictable
- LLMs force AI labs and policymakers to face hard questions
- Audits help identify risks, inform design, and inform public discourse
Addressing initial objections
- Auditing procedures should be established at different stages of supply chains.
- Technology providers are responsible for taking precautions regarding foreseeable risks.
- Auditing procedures should be established at the application level and upstream.
- Designing LLM auditing procedures is difficult due to practical and conceptual components.
The merits and limits of existing ai auditing procedures
- Six claims about how AI auditing procedures should be designed
- Four LLM properties that undermine existing AI auditing procedures
Six claims about auditing of llms
- AI auditing tools and procedures have already been developed
- It is essential to distinguish between compliance audits and risk audits
- It is useful to distinguish between external and internal audits
- It is useful to distinguish between governance audits and technology audits
- It is useful to distinguish between functionality, model, and impact audits
- It is important to distinguish between ex-ante and ex-post audits
- AI auditing procedures focusing on compliance alone are unlikely to provide adequate assurance for LLMs
- External audits are required to ensure that LLMs are ethical, legal, and technically robust
- Auditing procedures must include elements of both governance and technology audits
- Model audits will play a key role in identifying and communicating LLMs’ limitations
- Auditing procedures must include elements of continuous ex-post auditing
Properties of llms that auditing procedures must account for
- LLMs enable a wide range of applications, making it difficult to audit.
- LLMs can learn after deployment, making ex-ante assessments ineffective.
- LLMs do not require a model of the real world, making it hard to audit without access to training data.
- LLMs are sometimes only accessible via an API, limiting third-party auditing.
Auditing llms: a three-layered approach
- Identify LLM-related risks
- Practically feasible to implement
- Justifiable cost-benefit ratio
A blueprint for llm auditing
- Governance audits assess organizational procedures, accountability structures, and quality management systems
- Model audits assess capabilities and limitations of LLMs before adaptation and deployment
- Application audits assess ethical alignment, legal compliance, and impact over time
- Governance, model, and application audits must be coordinated for effective assurance
Governance audits
- Technology providers should undergo governance audits to assess their procedures, incentive structures, and management systems
- Evidence shows that these features influence the design and deployment of technologies
- Risk-mitigation strategies work best when adopted transparently, consistently, and with executive-level support
- Technology providers are responsible for identifying and managing risks associated with their LLMs
- Governance audits should review the adequacy of organisational governance structures
- Audit trails should provide chronological documentary evidence of the development of an LLM’s capabilities
- Mapping roles and responsibilities within organisations that design LLMs facilitates the allocation of accountability for system failures
- Auditors require privileged access to facilities, documentation, and personnel
- Results of governance audits should be tailored to different audiences
Model audits
- Model audits should assess LLMs’ capabilities and limitations before deployment.
- Model audits focus on LLMs’ capabilities and characteristics, not organisational procedures.
- Model audits should inform continuous redesign and communicate capabilities and limitations to external stakeholders.
- Model audits should focus on socially and ethically relevant characteristics that are predictably transferable and meaningfully operationalisable.
- Examples of characteristics to focus on include performance, robustness, information security and truthfulness.
- Performance can be assessed using standardised benchmarks.
- Robustness can be assessed using evaluation toolkits, benchmark datasets and open-source platforms.
- Information security can be assessed by minimising memorisation of training data and testing for exposure.
- Truthfulness can be assessed using TruthfulQA.
- Model audits should also review training datasets for gaps and biases.
- Model audits require privileged access to LLMs and their training datasets.
Application audits
- Products and services built using LLMs should undergo application audits.
- Application audits have two components: functionality audits and impact audits.
- Functionality audits check if the application is legal and ethical and if it is aligned with the intended use of the LLM.
- Impact audits focus on how the application’s outputs impact different user groups and the environment.
- Pre-deployment assessments and post-deployment monitoring are necessary.
- Pre-deployment assessments can use empirical evidence or plausible scenarios.
- Post-deployment monitoring can be done periodically or automated.
- Application audits should include elements of continuous oversight.
- Quantitative and qualitative assessments can be used to assess potential harms.
- Application audits should be employed selectively.
- Results of application audits should be publicly available.
Connecting the dots
- Governance, model, and application audits must be connected into a structured process
- Model audits produce reports summarising LLMs’ properties and limitations
- Application audits produce output logs documenting the impact of different applications
- Governance audits check technology providers’ software development processes
- Different independent auditors can perform the three different types of audits
- Different institutional arrangements may be preferable in different jurisdictions or sectors
Limitations and avenues for further research
- Conceptual problems related to constructing validity of model audits
- Lack of institutional ecosystem to support independent third-party audits
- Not all LLM-related social and ethical risks can be practically addressed on the technology level
Lack of methods and metrics to operationalise normative concepts
- Difficulty of operationalizing normative concepts like robustness and truthfulness is a bottleneck to developing effective auditing procedures
- Lack of standardised evaluation metrics is a crucial challenge when implementing AI auditing procedures
- Construct validity problems arise when attempting to operationalize characteristics like performance, robustness, information security and truthfulness
- Pragmatist operationalisations of concepts like truthfulness and robustness promote fairness, accountability, and transparency
- Developing metrics to capture the essence of thick normative concepts is difficult and has many pitfalls
Lack of an institutional ecosystem
- Blueprint recommends three audits but does not identify who should conduct them
- Different institutional ecosystems are emerging in different jurisdictions and sectors
- Five institutional arrangements for structuring independent audits are relevant
- Private service providers, government agency, industry body, non-profit organisations, international organisation
- Institutional ecosystem needed to implement and enforce blueprint
Not all risks from llms can be addressed on the technology level
- Blueprint for auditing LLMs designed to contribute to good governance
- Most risks cannot be reduced to zero
- Risks from deliberate misuse create offensive-defensive asymmetry
- Not all risks associated with LLMs can be addressed on the technology level
- Social and political reform needed to complement technically oriented mechanisms
Conclusion
- LLMs have significant governance challenges
- Existing AI auditing procedures are not well-equipped to assess LLMs
- Three-layered approach to auditing LLMs: governance, model and application audits
- Governance audits evaluate technology providers’ accountability structures and quality management systems
- Model audits assess LLMs’ capabilities and limitations
- Application audits assess products and services built on top of LLMs
- Tools and methods used to conduct audits on each level
- Interlinking available tools and methods
- Feasibility and effectiveness of approach depend on two factors
- Limitations of approach
- Avenues for further research
- Technology providers should subject themselves to governance and model audits
- Establish and fund an independent industry body
- Lessons for auditing more capable and general future AI systems
- Blueprint not intended to replace existing governance mechanisms
- Adopt, adjust and expand to meet different stakeholders and contexts