Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

  • LLMs represent a major advance in AI research
  • LLMs come with ethical and social challenges
  • Auditing is a promising governance mechanism to ensure AI systems are ethical, legal, and technically robust
  • Existing auditing procedures don’t address the governance challenges posed by LLMs
  • Article proposes a three-layered approach to audit LLMs in feasible and effective ways
  • Article discusses limitations of auditing LLMs

Paper Content

Introduction

  • Auditing is a governance mechanism used to identify and mitigate risks associated with AI systems
  • Procedural regularity and transparency contribute to good governance
  • Proactivity in the design of AI systems helps identify risks and prevent harm
  • Operational independence between the auditor and the auditee contributes to objectivity and professionalism of the evaluation
  • Previous work on AI auditing has focused on ensuring specific applications meet predefined requirements
  • Foundation models are effective across many different tasks and display emergent capabilities when scaled
  • LLMs pose ethical and social challenges such as perpetuating harmful stereotypes, leaking personal data, spreading misinformation, plagiarism, and misuse of copyrighted material
  • Auditing procedures should be designed to capture the risks posed by LLMs
  • A three-layered approach combining governance, model, and application audits should be used to audit LLMs
  • Outputs of the audits should ensure LLMs are designed and deployed in ethical, legal, and technically robust ways

The need to audit llms

  • Previous research on LLMs and their ethical and social challenges
  • Need for auditing procedures to capture risks LLMs pose
  • Potential objections to approach addressed

The opportunities and risks of llms

  • LLMs represent a major advance in AI research
  • NLP researchers and practitioners have been developing software to analyse, manipulate, and generate natural language since the 1950s
  • Deep learning, neural architectures, and computational power have revolutionised the field
  • LLMs can approximate human performance on some benchmarks
  • LLMs are highly adaptable to various downstream applications
  • Scaling the model can result in emergent gains on a wide array of tasks
  • LLMs are accessible via open-source libraries, democratising the gains from deep language modelling
  • LLMs can introduce representational and allocational harms, compromise privacy, produce misleading information, be co-opted by users with bad intent, and incur high environmental costs

The governance gap

  • LLMs pose methodological and normative challenges
  • LLMs are developed and adopted in two stages
  • It is difficult to assess LLMs independent of context
  • Performance of LLMs can be unpredictable
  • LLMs force AI labs and policymakers to face hard questions
  • Audits help identify risks, inform design, and inform public discourse

Addressing initial objections

  • Auditing procedures should be established at different stages of supply chains.
  • Technology providers are responsible for taking precautions regarding foreseeable risks.
  • Auditing procedures should be established at the application level and upstream.
  • Designing LLM auditing procedures is difficult due to practical and conceptual components.

The merits and limits of existing ai auditing procedures

  • Six claims about how AI auditing procedures should be designed
  • Four LLM properties that undermine existing AI auditing procedures

Six claims about auditing of llms

  • AI auditing tools and procedures have already been developed
  • It is essential to distinguish between compliance audits and risk audits
  • It is useful to distinguish between external and internal audits
  • It is useful to distinguish between governance audits and technology audits
  • It is useful to distinguish between functionality, model, and impact audits
  • It is important to distinguish between ex-ante and ex-post audits
  • AI auditing procedures focusing on compliance alone are unlikely to provide adequate assurance for LLMs
  • External audits are required to ensure that LLMs are ethical, legal, and technically robust
  • Auditing procedures must include elements of both governance and technology audits
  • Model audits will play a key role in identifying and communicating LLMs’ limitations
  • Auditing procedures must include elements of continuous ex-post auditing

Properties of llms that auditing procedures must account for

  • LLMs enable a wide range of applications, making it difficult to audit.
  • LLMs can learn after deployment, making ex-ante assessments ineffective.
  • LLMs do not require a model of the real world, making it hard to audit without access to training data.
  • LLMs are sometimes only accessible via an API, limiting third-party auditing.

Auditing llms: a three-layered approach

  • Identify LLM-related risks
  • Practically feasible to implement
  • Justifiable cost-benefit ratio

A blueprint for llm auditing

  • Governance audits assess organizational procedures, accountability structures, and quality management systems
  • Model audits assess capabilities and limitations of LLMs before adaptation and deployment
  • Application audits assess ethical alignment, legal compliance, and impact over time
  • Governance, model, and application audits must be coordinated for effective assurance

Governance audits

  • Technology providers should undergo governance audits to assess their procedures, incentive structures, and management systems
  • Evidence shows that these features influence the design and deployment of technologies
  • Risk-mitigation strategies work best when adopted transparently, consistently, and with executive-level support
  • Technology providers are responsible for identifying and managing risks associated with their LLMs
  • Governance audits should review the adequacy of organisational governance structures
  • Audit trails should provide chronological documentary evidence of the development of an LLM’s capabilities
  • Mapping roles and responsibilities within organisations that design LLMs facilitates the allocation of accountability for system failures
  • Auditors require privileged access to facilities, documentation, and personnel
  • Results of governance audits should be tailored to different audiences

Model audits

  • Model audits should assess LLMs’ capabilities and limitations before deployment.
  • Model audits focus on LLMs’ capabilities and characteristics, not organisational procedures.
  • Model audits should inform continuous redesign and communicate capabilities and limitations to external stakeholders.
  • Model audits should focus on socially and ethically relevant characteristics that are predictably transferable and meaningfully operationalisable.
  • Examples of characteristics to focus on include performance, robustness, information security and truthfulness.
  • Performance can be assessed using standardised benchmarks.
  • Robustness can be assessed using evaluation toolkits, benchmark datasets and open-source platforms.
  • Information security can be assessed by minimising memorisation of training data and testing for exposure.
  • Truthfulness can be assessed using TruthfulQA.
  • Model audits should also review training datasets for gaps and biases.
  • Model audits require privileged access to LLMs and their training datasets.

Application audits

  • Products and services built using LLMs should undergo application audits.
  • Application audits have two components: functionality audits and impact audits.
  • Functionality audits check if the application is legal and ethical and if it is aligned with the intended use of the LLM.
  • Impact audits focus on how the application’s outputs impact different user groups and the environment.
  • Pre-deployment assessments and post-deployment monitoring are necessary.
  • Pre-deployment assessments can use empirical evidence or plausible scenarios.
  • Post-deployment monitoring can be done periodically or automated.
  • Application audits should include elements of continuous oversight.
  • Quantitative and qualitative assessments can be used to assess potential harms.
  • Application audits should be employed selectively.
  • Results of application audits should be publicly available.

Connecting the dots

  • Governance, model, and application audits must be connected into a structured process
  • Model audits produce reports summarising LLMs’ properties and limitations
  • Application audits produce output logs documenting the impact of different applications
  • Governance audits check technology providers’ software development processes
  • Different independent auditors can perform the three different types of audits
  • Different institutional arrangements may be preferable in different jurisdictions or sectors

Limitations and avenues for further research

  • Conceptual problems related to constructing validity of model audits
  • Lack of institutional ecosystem to support independent third-party audits
  • Not all LLM-related social and ethical risks can be practically addressed on the technology level

Lack of methods and metrics to operationalise normative concepts

  • Difficulty of operationalizing normative concepts like robustness and truthfulness is a bottleneck to developing effective auditing procedures
  • Lack of standardised evaluation metrics is a crucial challenge when implementing AI auditing procedures
  • Construct validity problems arise when attempting to operationalize characteristics like performance, robustness, information security and truthfulness
  • Pragmatist operationalisations of concepts like truthfulness and robustness promote fairness, accountability, and transparency
  • Developing metrics to capture the essence of thick normative concepts is difficult and has many pitfalls

Lack of an institutional ecosystem

  • Blueprint recommends three audits but does not identify who should conduct them
  • Different institutional ecosystems are emerging in different jurisdictions and sectors
  • Five institutional arrangements for structuring independent audits are relevant
  • Private service providers, government agency, industry body, non-profit organisations, international organisation
  • Institutional ecosystem needed to implement and enforce blueprint

Not all risks from llms can be addressed on the technology level

  • Blueprint for auditing LLMs designed to contribute to good governance
  • Most risks cannot be reduced to zero
  • Risks from deliberate misuse create offensive-defensive asymmetry
  • Not all risks associated with LLMs can be addressed on the technology level
  • Social and political reform needed to complement technically oriented mechanisms

Conclusion

  • LLMs have significant governance challenges
  • Existing AI auditing procedures are not well-equipped to assess LLMs
  • Three-layered approach to auditing LLMs: governance, model and application audits
  • Governance audits evaluate technology providers’ accountability structures and quality management systems
  • Model audits assess LLMs’ capabilities and limitations
  • Application audits assess products and services built on top of LLMs
  • Tools and methods used to conduct audits on each level
  • Interlinking available tools and methods
  • Feasibility and effectiveness of approach depend on two factors
  • Limitations of approach
  • Avenues for further research
  • Technology providers should subject themselves to governance and model audits
  • Establish and fund an independent industry body
  • Lessons for auditing more capable and general future AI systems
  • Blueprint not intended to replace existing governance mechanisms
  • Adopt, adjust and expand to meet different stakeholders and contexts