Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

  • Natural language processing (NLP) seeks to build systems that understand language.
  • Recent works have trained language models on narrative datasets to extract critical information.
  • This work investigates if these models are truly understanding the text or just learning a heuristic.
  • Results suggest that training can lead to deeper language understanding.
  • Findings have consequences for cognitive neuroscience and NLP.

Paper Content

Introduction

  • Language models trained on millions of documents have improved NLP benchmarks
  • Researchers have proposed training language models on narrative datasets to build systems with deeper language understanding
  • Prior work has used human brain recordings to interpret representations of pretrained language models
  • Researchers have shown that pretrained language models can predict large parts of the brain regions that underlie language comprehension
  • This work investigates if training language models for deeper language understanding improves alignment with human brain activity
  • Results show improved alignment with human brains’ deep understanding of characters, emotions and motions
  • Improved alignment is seen when providing longer input contexts (20 to 1000 words) to the language models
  • Research on disentangling contributions of different types of information to brain alignment
  • Toneva et al. (2020) show supra-word meaning is predictive of fMRI recordings
  • Caucheteux et al. (2021) and Reddy & Wehbe (2021) aim to disentangle alignment due to syntactic and semantic processing
  • Relationship between lower perplexity and higher brain alignment across NLP models (Schrimpf et al., 2021; Goldstein et al., 2022)
  • No simple relationship between language modeling loss and brain alignment (Pasquiou et al., 2022)
  • Merlin & Toneva (2022) introduced perturbations to disentangle contribution of next word prediction and semantic knowledge towards brain alignment

Methods

Brain representations

  • Brain dataset consists of fMRI recordings of 8 participants reading a chapter of Harry Potter
  • Chapter is divided into four runs with short breaks in between
  • Dataset chosen for its narrative world and large sample size
  • fMRI recordings sampled at 2 second intervals
  • Brain representation for each participant is a matrix with n TRs and v i voxels

Nlp representations

  • 8 models used
  • 4 “base models” trained with language modeling
  • 4 “booksum models” trained with BookSum dataset
  • Models designed to take advantage of long contexts
  • Base models have good performance on other long-context benchmarks
  • BookSum dataset for long-range summarization
  • NLP representations extracted for each word in Harry Potter fMRI dataset

Aligning brain and nlp representations.

  • General alignment approach used in several works
  • Function to predict fMRI recordings using NLP representations
  • Function parameterized as linear function, regularized using ridge penalty
  • Cross-validation used to train and test performance
  • Evaluation metrics: 20v20 classification accuracy and Pearson correlation

Training nlp models for deep narrative understanding

  • NLP models trained for deeper understanding of long narratives have improved brain alignment over those trained for language modeling
  • 4 booksum models and 4 base models were evaluated using 20v20 brain alignment scores
  • Booksum models are significantly better aligned to brain representations due to deeper understanding of text
  • Confounding variables were eliminated to ensure improved brain alignment was due to deeper language understanding
  • Brain alignment peaks around context length of 500 words
  • Brain alignment improves only for longer input contexts (20 to 1000 words)
  • Finetuning improves brain alignment for different layers across models
  • Improved brain alignment in booksum models is due to deeper understanding, not language modeling
  • 20v20 accuracy is significantly higher for booksum model than LM model across all brain language regions

Relationship between brain alignment, language modeling and deep understanding

  • Training language models on BookSum leads to improved brain alignment
  • Language modeling (LM) ability may contribute to brain-NLP alignment
  • Experiments conducted to investigate if improved brain alignment is due to deeper understanding or improved LM ability
  • Harry Potter text dataset used to measure both brain alignment and LM ability
  • Improved brain alignment not due to better LM ability for 3 of the 4 models
  • Brain alignment significantly greater for booksum models across all brain language ROIs
  • Brain alignment higher for Characters than other discourse features
  • Training NLP models for deeper understanding results in improved brain alignment for all discourse features

Conclusions and future work

  • Understanding of characters and other discourse features is a significant factor
  • Deeper understanding of a text contributes to brain-NLP alignment independently from language modeling
  • LM models achieve poorer representations for Characters and other discourse features, compared to deeper understanding models
  • Training methods for narrative understanding are a first step towards developing language models with deep language understanding
  • Investigate why brain alignment differs across discourse features
  • Explore the mechanisms behind deeper understanding of texts in NLP models
  • Reduce dimensionality of NLP representations
  • Construct TR-level NLP representations
  • Concatenate TR-level NLP representations
  • Learn a function to predict brain activity
  • Evaluate alignment between NLP predictions and true fMRI activity
  • Label words in the Harry Potter chapter
  • Map words to fMRI TRs
  • Sample 160 TRs for each discourse feature
  • Extract NLP encoding predictions and brain activity
  • Compute Pearson correlation
  • Booksum model has significantly greater brain alignment than the base model
  • Improved brain alignment is not simply due to domain similarity or summarization learning objective
  • Deeper understanding emerges when training models to summarize narrative texts
  • Booksum models have better brain alignment than LM-booksum counterparts