Link to paper
The full paper is available here.
You can also find the paper on PapersWithCode here.
Abstract
- Natural language processing (NLP) seeks to build systems that understand language.
- Recent works have trained language models on narrative datasets to extract critical information.
- This work investigates if these models are truly understanding the text or just learning a heuristic.
- Results suggest that training can lead to deeper language understanding.
- Findings have consequences for cognitive neuroscience and NLP.
Paper Content
Introduction
- Language models trained on millions of documents have improved NLP benchmarks
- Researchers have proposed training language models on narrative datasets to build systems with deeper language understanding
- Prior work has used human brain recordings to interpret representations of pretrained language models
- Researchers have shown that pretrained language models can predict large parts of the brain regions that underlie language comprehension
- This work investigates if training language models for deeper language understanding improves alignment with human brain activity
- Results show improved alignment with human brains’ deep understanding of characters, emotions and motions
- Improved alignment is seen when providing longer input contexts (20 to 1000 words) to the language models
Related work on brains and language
- Research on disentangling contributions of different types of information to brain alignment
- Toneva et al. (2020) show supra-word meaning is predictive of fMRI recordings
- Caucheteux et al. (2021) and Reddy & Wehbe (2021) aim to disentangle alignment due to syntactic and semantic processing
- Relationship between lower perplexity and higher brain alignment across NLP models (Schrimpf et al., 2021; Goldstein et al., 2022)
- No simple relationship between language modeling loss and brain alignment (Pasquiou et al., 2022)
- Merlin & Toneva (2022) introduced perturbations to disentangle contribution of next word prediction and semantic knowledge towards brain alignment
Methods
Brain representations
- Brain dataset consists of fMRI recordings of 8 participants reading a chapter of Harry Potter
- Chapter is divided into four runs with short breaks in between
- Dataset chosen for its narrative world and large sample size
- fMRI recordings sampled at 2 second intervals
- Brain representation for each participant is a matrix with n TRs and v i voxels
Nlp representations
- 8 models used
- 4 “base models” trained with language modeling
- 4 “booksum models” trained with BookSum dataset
- Models designed to take advantage of long contexts
- Base models have good performance on other long-context benchmarks
- BookSum dataset for long-range summarization
- NLP representations extracted for each word in Harry Potter fMRI dataset
Aligning brain and nlp representations.
- General alignment approach used in several works
- Function to predict fMRI recordings using NLP representations
- Function parameterized as linear function, regularized using ridge penalty
- Cross-validation used to train and test performance
- Evaluation metrics: 20v20 classification accuracy and Pearson correlation
Training nlp models for deep narrative understanding
- NLP models trained for deeper understanding of long narratives have improved brain alignment over those trained for language modeling
- 4 booksum models and 4 base models were evaluated using 20v20 brain alignment scores
- Booksum models are significantly better aligned to brain representations due to deeper understanding of text
- Confounding variables were eliminated to ensure improved brain alignment was due to deeper language understanding
- Brain alignment peaks around context length of 500 words
- Brain alignment improves only for longer input contexts (20 to 1000 words)
- Finetuning improves brain alignment for different layers across models
- Improved brain alignment in booksum models is due to deeper understanding, not language modeling
- 20v20 accuracy is significantly higher for booksum model than LM model across all brain language regions
Relationship between brain alignment, language modeling and deep understanding
- Training language models on BookSum leads to improved brain alignment
- Language modeling (LM) ability may contribute to brain-NLP alignment
- Experiments conducted to investigate if improved brain alignment is due to deeper understanding or improved LM ability
- Harry Potter text dataset used to measure both brain alignment and LM ability
- Improved brain alignment not due to better LM ability for 3 of the 4 models
- Brain alignment significantly greater for booksum models across all brain language ROIs
- Brain alignment higher for Characters than other discourse features
- Training NLP models for deeper understanding results in improved brain alignment for all discourse features
Conclusions and future work
- Understanding of characters and other discourse features is a significant factor
- Deeper understanding of a text contributes to brain-NLP alignment independently from language modeling
- LM models achieve poorer representations for Characters and other discourse features, compared to deeper understanding models
- Training methods for narrative understanding are a first step towards developing language models with deep language understanding
- Investigate why brain alignment differs across discourse features
- Explore the mechanisms behind deeper understanding of texts in NLP models
- Reduce dimensionality of NLP representations
- Construct TR-level NLP representations
- Concatenate TR-level NLP representations
- Learn a function to predict brain activity
- Evaluate alignment between NLP predictions and true fMRI activity
- Label words in the Harry Potter chapter
- Map words to fMRI TRs
- Sample 160 TRs for each discourse feature
- Extract NLP encoding predictions and brain activity
- Compute Pearson correlation
- Booksum model has significantly greater brain alignment than the base model
- Improved brain alignment is not simply due to domain similarity or summarization learning objective
- Deeper understanding emerges when training models to summarize narrative texts
- Booksum models have better brain alignment than LM-booksum counterparts