Publications

Quantifying elicitation of latent capabilities in language models

E Donoway, H Joren, A Somani, H Sleight, J Michael, M Deweese, ... Advances in Neural Information Processing Systems 38, 150412-150443, 2026

Best-of-n jailbreaking

J Hughes, S Price, A Lynch, R Schaeffer, F Barez, A Somani, S Koyejo, ... Advances in Neural Information Processing Systems 38, 73137-73221, 2026

The hot mess of AI: How does misalignment scale with model intelligence and task complexity?

A Hägele, AP Gema, H Sleight, E Perez, J Sohl-Dickstein arXiv preprint arXiv:2601.23045, 2026

Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks

H Cunningham, J Wei, Z Wang, A Persic, A Peng, J Abderrachid, ... arXiv preprint arXiv:2601.04603, 2026

Natural emergent misalignment from reward hacking in production rl

M MacDiarmid, B Wright, J Uesato, J Benton, J Kutasov, S Price, ... arXiv preprint arXiv:2511.18397, 2025

Agentic misalignment: How llms could be insider threats

A Lynch, B Wright, C Larson, SJ Ritchie, S Mindermann, E Hubinger, ... arXiv preprint arXiv:2510.05179, 2025

Towards safeguarding llm fine-tuning apis against cipher attacks

J Youstra, M Mahfoud, Y Yan, H Sleight, E Perez, M Sharma arXiv preprint arXiv:2508.17158, 2025

Inverse scaling in test-time compute

AP Gema, A Hägele, R Chen, A Arditi, J Goldman-Wetzler, ... arXiv preprint arXiv:2507.14417, 2025

Chain of thought monitorability: A new and fragile opportunity for ai safety

T Korbak, M Balesni, E Barnes, Y Bengio, J Benton, J Bloom, M Chen, ... arXiv preprint arXiv:2507.11473, 2025

Unsupervised elicitation of language models

J Wen, Z Ankner, A Somani, P Hase, S Marks, J Goldman-Wetzler, ... arXiv preprint arXiv:2506.10139, 2025

Reasoning models don't always say what they think

Y Chen, J Benton, A Radhakrishnan, J Uesato, C Denison, J Schulman, ... arXiv preprint arXiv:2505.05410, 2025

Failures to find transferable image jailbreaks between vision-language models

R Schaeffer, D Valentine, L Bailey, J Chua, C Eyzaguirre, Z Durante, ... International Conference on Learning Representations 2025, 44669-44704, 2025

Adaptive deployment of untrusted llms reduces distributed threats

J Wen, V Hebbar, C Larson, A Bhatt, A Radhakrishnan, M Sharma, ... International Conference on Learning Representations 2025, 42650-42693, 2025

Looking inward: Language models can learn about themselves by introspection

FJ Binder, J Chua, T Korbak, H Sleight, J Hughes, R Long, E Perez, ... International Conference on Learning Representations 2025, 3710-3756, 2025

Language models learn to mislead humans via rlhf

J Wen, R Zhong, A Khan, E Perez, J Steinhardt, M Huang, S Bowman, ... International Conference on Learning Representations 2025, 74670-74692, 2025

Forecasting rare language model behaviors

E Jones, M Tong, J Mu, M Mahfoud, J Leike, R Grosse, J Kaplan, ... arXiv preprint arXiv:2502.16797, 2025

Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming

M Sharma, M Tong, J Mu, J Wei, J Kruthoff, S Goodfriend, E Ong, A Peng, ... arXiv preprint arXiv:2501.18837, 2025

Attacking audio language models with best-of-n jailbreaking

J Hughes, S Price, A Lynch, R Schaeffer, F Barez, S Koyejo, H Sleight, ...

Alignment faking in large language models

R Greenblatt, C Denison, B Wright, F Roger, M MacDiarmid, S Marks, ... arXiv preprint arXiv:2412.14093, 2024

Many-shot jailbreaking

C Anil, E Durmus, N Panickssery, M Sharma, J Benton, S Kundu, J Batson, ... Advances in Neural Information Processing Systems 37, 129696-129742, 2024

Jailbreak defense in a narrow domain: Limitations of existing methods and a new transcript-classifier approach

TT Wang, J Hughes, H Sleight, R Schaeffer, R Agrawal, F Barez, ... arXiv preprint arXiv:2412.02159, 2024

A dataset of questions on decision-theoretic reasoning in Newcomb-like problems

C Oesterheld, E Cooper, M Kodama, LC Nguyen, E Perez arXiv preprint arXiv:2411.10588, 2024

Rapid response: Mitigating llm jailbreaks with a few examples

A Peng, J Michael, H Sleight, E Perez, M Sharma arXiv preprint arXiv:2411.07494, 2024

Sabotage evaluations for frontier models

J Benton, M Wagner, E Christiansen, C Anil, E Perez, J Srivastav, ... arXiv preprint arXiv:2410.21514, 2024

Latent adversarial training improves robustness to persistent harmful behaviors in llms

A Sheshadri, A Ewart, P Guo, A Lynch, C Wu, V Hebbar, H Sleight, ... arXiv preprint arXiv:2407.15549, 2024

When do universal image jailbreaks transfer between vision-language models?

R Schaeffer, D Valentine, L Bailey, J Chua, C Eyzaguirre, Z Durante, ... Workshop on Responsibly Building the Next Generation of Multimodal …, 2024

Sycophancy to subterfuge: Investigating reward-tampering in large language models

C Denison, M MacDiarmid, F Barez, D Duvenaud, S Kravec, S Marks, ... arXiv preprint arXiv:2406.10162, 2024

Bias-augmented consistency training reduces biased reasoning in chain-of-thought

J Chua, E Rees, H Batra, SR Bowman, J Michael, E Perez, M Turpin arXiv preprint arXiv:2403.05518, 2024

Debating with more persuasive llms leads to more truthful answers

A Khan, J Hughes, D Valentine, L Ruis, K Sachan, A Radhakrishnan, ... ICML, 2024

Learning from natural language feedback

A Chen, J Scheurer, JA Campos, T Korbak, JS Chan, SR Bowman, K Cho, ... Transactions on machine learning research, 2024

Sleeper agents: Training deceptive llms that persist through safety training

E Hubinger, C Denison, J Mu, M Lambert, M Tong, M MacDiarmid, ... arXiv preprint arXiv:2401.05566, 2024

Alignment faking in large language models. arXiv

R Greenblatt, C Denison, B Wright, F Roger, M MacDiarmid, S Marks, ... Preprint posted online December 18, 2024

Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs

A Ewart, A Sheshadri, PH Guo, A Lynch, C Wu, V Hebbar, H Sleight, ... Workshop on Socially Responsible Language Modelling Research, 2024

Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting

M Turpin, J Michael, E Perez, S Bowman Advances in Neural Information Processing Systems 36, 74952-74965, 2023

Towards evaluating ai systems for moral status using self-reports

E Perez, R Long arXiv preprint arXiv:2311.08576, 2023

Specific versus general principles for constitutional ai

S Kundu, Y Bai, S Kadavath, A Askell, A Callahan, A Chen, A Goldie, ... arXiv preprint arXiv:2310.13798, 2023

Towards understanding sycophancy in language models

M Sharma, M Tong, T Korbak, D Duvenaud, A Askell, SR Bowman, ... ICLR, 2023

Vision-language models are zero-shot reward models for reinforcement learning

J Rocamonde, V Montesinos, E Nava, E Perez, D Lindner ICLR, 2023

Studying large language model generalization with influence functions

R Grosse, J Bae, C Anil, N Elhage, A Tamkin, A Tajdini, B Steiner, D Li, ... arXiv preprint arXiv:2308.03296, 2023

Performance of a neural network using automatically uncovered failure cases

EJ Perez, SS Huang, NJ MCALEESE-PARK, G Irving US Patent App. 18/160,860, 2023

Measuring faithfulness in chain-of-thought reasoning

T Lanham, A Chen, A Radhakrishnan, B Steiner, C Denison, ... arXiv preprint arXiv:2307.13702, 2023

Question decomposition improves the faithfulness of model-generated reasoning

A Radhakrishnan, K Nguyen, A Chen, C Chen, C Denison, D Hernandez, ... arXiv preprint arXiv:2307.11768, 2023

Discovering language model behaviors with model-written evaluations

E Perez, S Ringer, K Lukosiute, K Nguyen, E Chen, S Heiner, C Pettit, ... Findings of the association for computational linguistics: ACL 2023, 13387-13434, 2023

Inverse scaling: When bigger isn't better

IR McKenzie, A Lyzhov, M Pieler, A Parrish, A Mueller, A Prabhu, ... TMLR, 2023

Training language models with language feedback at scale

J Scheurer, JA Campos, T Korbak, JS Chan, A Chen, K Cho, E Perez arXiv preprint arXiv:2303.16755, 2023

Improving code generation by training with natural language feedback

A Chen, J Scheurer, T Korbak, JA Campos, JS Chan, SR Bowman, K Cho, ... arXiv preprint arXiv:2303.16749, 2023

Pretraining language models with human preferences

T Korbak, K Shi, A Chen, R Bhalerao, CL Buckley, J Phang, SR Bowman, ... ICML 2023, 2023

The capacity for moral self-correction in large language models

D Ganguli, A Askell, N Schiefer, TI Liao, K Lukošiūtė, A Chen, A Goldie, ... arXiv preprint arXiv:2302.07459, 2023

Constitutional AI: Harmlessness from AI Feedback

Y Bai, S Kadavath, S Kundu, A Askell, J Kernion, A Jones, A Chen, ... arXiv preprint arXiv:2212.08073, 2022

Measuring progress on scalable oversight for large language models

SR Bowman, J Hyun, E Perez, E Chen, C Pettit, S Heiner, K Lukošiūtė, ... arXiv preprint arXiv:2211.03540, 2022

Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned

D Ganguli, L Lovitt, J Kernion, A Askell, Y Bai, S Kadavath, B Mann, ... arXiv preprint arXiv:2209.07858, 2022

Few-shot adaptation works with unpredictable data

JS Chan, M Pieler, J Jao, J Scheurer, E Perez ACL 2023, 2022

Language models (mostly) know what they know

S Kadavath, T Conerly, A Askell, T Henighan, D Drain, E Perez, ... arXiv preprint arXiv:2207.05221, 2022

RL with KL penalties is better viewed as Bayesian inference

T Korbak, E Perez, CL Buckley EMNLP 2022, 2022

Training Language Models with Language Feedback

J Scheurer, JA Campos, JS Chan, A Chen, K Cho, E Perez ACL 2022 Workshop on Learning with Natural Language Supervision, 2022

Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions

A Parrish*, H Trivedi*, E Perez*, A Chen, N Nangia, J Phang, SR Bowman ACL 2022 Workshop on Learning with Natural Language Supervision, 2022

Finding and Fixing Undesirable Behaviors in Pretrained Language Models

E Perez New York University, USA, 2022

Red teaming language models with language models

E Perez, S Huang, F Song, T Cai, R Ring, J Aslanides, A Glaese, ... EMNLP 2022, 2022

True Few-Shot Learning with Language Models

E Perez, D Kiela, K Cho NeurIPS 2021, 2021

Case-based reasoning for natural language queries over knowledge bases

R Das, M Zaheer, D Thai, A Godbole, E Perez, JY Lee, L Tan, ... EMNLP 2021, 2021

Rissanen data analysis: Examining dataset characteristics via description length

E Perez, D Kiela, K Cho ICML 2021, 2021

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

P Lewis, E Perez, A Piktus, F Petroni, V Karpukhin, N Goyal, H Küttler, ... NeurIPS 2020, 2020

Unsupervised Question Decomposition for Question Answering

E Perez, P Lewis, W Yih, K Cho, D Kiela EMNLP 2020, 2020

Finding Generalizable Evidence by Learning to Convince Q&A Models

E Perez, S Karamcheti, R Fergus, J Weston, D Kiela, K Cho EMNLP 2019, 2019

Supervised multimodal bitransformers for classifying images and text

D Kiela, S Bhooshan, H Firooz, E Perez, D Testuggine arXiv preprint arXiv:1909.02950, 2019

ELI5: Long Form Question Answering

A Fan, Y Jernite*, E Perez*, D Grangier, J Weston, M Auli Association for Computational Linguistics (ACL) 2019, 2019

Retrospective for: "FiLM: Visual Reasoning with a General Conditioning Layer"

E Perez NeurIPS ML Retrospectives Workshop, 2019

Feature-wise transformations

V Dumoulin, E Perez, N Schucher, F Strub, H Vries, A Courville, Y Bengio Distill 3 (7), e11, 2018

Visual Reasoning with Multi-hop Feature Modulation

F Strub, M Seurin, E Perez, H De Vries, J Mary, P Preux, A Courville, ... Proceedings of the European Conference on Computer Vision (ECCV), 784-800, 2018

FiLM: Visual Reasoning with a General Conditioning Layer

E Perez, F Strub, H De Vries, V Dumoulin, A Courville AAAI 2018, 2018

HoME: a Household Multimodal Environment

S Brodeur, E Perez*, A Anand*, F Golemo*, L Celotti, F Strub, J Rouat, ... ICLR 2018 Workshop, 2017

Learning Visual Reasoning Without Strong Priors

E Perez, H De Vries, F Strub, V Dumoulin, A Courville ICML 2017 Workshop, 2017

Semi-supervised learning with the deep rendering mixture model

T Nguyen, W Liu, E Perez, RG Baraniuk, AB Patel arXiv preprint arXiv:1612.01942, 2016

others. 2022. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned

D Ganguli, L Lovitt, AA Jackson Kernion, Y Bai, S Kadavath, B Mann, ... arXiv preprint arXiv:2209.07858, 1

Sycophancy to subterfuge: Investigating reward-tampering in large language models, 2024

C Denison, M MacDiarmid, F Barez, D Duvenaud, S Kravec, S Marks, ... URL https://arxiv. org/abs/2406.10162 2, 0

Simple probes can catch sleeper agents, 2024

M MacDiarmid, T Maxwell, N Schiefer, J Mu, J Kaplan, D Duvenaud, ... URL https://www. anthropic. com/news/probes-catch-sleeper-agents, 0

A dataset of rated conceptual arguments

E Cooper, C Oesterheld, LC Nguyen, A Kastner, E Perez

Generate-Feedback-Refine: How Much Does Model Quality in Each Role Matter?

X Pan, J Phang, G Davidson, E Perez ICLR 2025 Third Workshop on Deep Learning for Code, 0

Plan B: Training LLMs to fail less severely

J Stastny, N Warncke, D Xu, A Lynch, F Barez, H Sleight, E Perez