CS336 2026 · Lecture 12已发布

Evaluation

日期: Wed May 6
讲师: Percy
课程资料: 10 项
内容产物: 4 项

课程简介

本讲从困惑度、知识问答和偏好评测出发，逐步扩展到代码、终端、网络安全等 Agent 任务以及安全与专业领域评测。重点讨论指标饱和、评审模型偏差、数据污染、测试环境缺陷和排行榜过拟合，建立“先检查任务与样本，再解释分数”的评测习惯。

课程资料lecture_12.py打开链接↗论文Holistic Evaluation of Language ModelsHELM↗论文Chatbot Arena - An Open Platform for Evaluating LLMs by Human Preference偏好评测↗论文SWE-bench - Can Language Models Resolve Real-World GitHub Issues?Agent 评测↗论文Measuring Massive Multitask Language UnderstandingMMLU↗论文GPQA - A Graduate-Level Google-Proof Q&A Benchmark高难知识评测↗论文HellaSwag - Can a Machine Really Finish Your Sentence?常识推理↗论文HarmBench - A Standardized Evaluation Framework for Automated Red Teaming安全评测↗论文AlpacaEval 2.0自动偏好评测↗视频课程视频课程视频↗

Assignment 3: Scaling截止

Assignment 4: Data已发布

CS336 Lecture 12：Evaluation（评估）

学习目标

完成本讲后，你应该能够：

理解评估在语言模型开发中的核心地位，以及“抽象概念 → 具体指标”这一转化过程的核心挑战。
清晰区分困惑度（perplexity）与各类下游基准测试（exam、chat、agentic、reasoning、safety）的适用场景与局限性。
掌握各类评估方法的关键设计选择、优缺点与常见陷阱（如数据污染、LLM 评委偏差、基准饱和）。
理解“评估的是方法、模型还是智能体”这一元问题，并能够根据目标选择恰当的评估策略。

课程主线

本讲围绕一个核心问题展开：给定训练好的模型，它有多“好”？ 课程首先指出评估看似机械（定义 prompts → 获得回复 → 计算准确率），实则深刻——因为评估设定了北极星指标，潜移默化地塑造了 AI 的发展方向。随后，课程从最基础的概率视角出发，逐步介绍五类评估方法：

困惑度（Perplexity）：语言模型作为分布的最自然度量。
考试基准（Exam Benchmarks）：MMLU、MMLU-Pro、GPQA、Humanity's Last Exam。
聊天基准（Chat Benchmarks）：Chatbot Arena、AlpacaEval、WildBench。
智能体基准（Agentic Benchmarks）：SWE-bench、Terminal-Bench、CyBench、MLE-Bench。
纯推理与安全基准：ARC-AGI 系列与 HarmBench、AIR-Bench 等。

最后，课程上升到元层面，讨论评估的生态效度（ecological validity）、有效性（validity）以及“我们到底在评估什么”这一根本问题。

核心概念与关键原则

1. 困惑度（Perplexity）

定义：困惑度衡量语言模型 p 对数据集 D 赋予的概率质量。形式上，Perplexity = (1/p(D))^(1/|D|)，其中 |D| 是 token 总数。困惑度与对数损失（log loss）、似然度（likelihood）本质上是同一概念的不同表述。

历史脉络：

经典范式（2010s）：在 Penn Treebank、WikiText-103、One Billion Word Benchmark 等数据集上进行分布内评估（in-distribution evaluation）——在训练集划分上训练，在测试集划分上测试。2016 年纯 CNN+LSTM 在 One Billion Word Benchmark 上取得突破性进展，确立了神经网络在语言建模中的主导地位。
范式转变（2019，GPT-2）：在 WebText（从 Reddit 链接出去的 40GB 文本）上训练，但在标准的、其他研究者使用的数据集上进行零样本评估（out-of-distribution evaluation）。这开启了“在大规模数据上训练，在标准基准上评估”的新思考方式。

“困惑度即一切”的论点：

设真实分布为 t，模型为 p。可获得的最佳困惑度是真实分布的熵 H(t)，且仅当 p = t 时达到。
若 p = t，则模型可以对一切进行建模：以问题为条件生成答案，解决所有任务。
因此，持续压低困惑度将最终“达到 AGI”。

对困惑度的批判：

它可能比你需要的更多：困惑度对每一个 token 的预测都进行惩罚，但并非所有 token 同等重要。例如，“斯坦福大学创立于1885年”中，“1885”蕴含世界知识，但“创立”这类词的信息量较低。
解决思路：测量条件困惑度——以提示为条件，仅计算回复部分的困惑度，将注意力聚焦在相关 token 上。
有些基准是伪装的困惑度：
- LAMBADA：填空题（cloze task），本质是经过精心挑选位置的下一个 token 预测，要求长上下文才能解析。
- HellaSwag：多项选择句子补全，在某种程度上也是困惑度。

运行困惑度排行榜的信任问题：

如果提交者只返回测试数据的 logprob，无法验证这些概率是否有效（是否求和为1）。一个总是返回 logprob=0 的模型可以获得完美困惑度，但并非有效分布。
相比之下，下游任务的评估更直接：黑盒 LM，给提示得回复，直接计算准确率即可。

总结：困惑度在语言模型开发中仍被大量使用（尤其在缩放定律中，因为它随规模平滑变化），但需要额外的基准来捕捉现实世界情况。

2. 考试基准（Exam Benchmarks）

动机：考试是人类测试的一种方式，可以精确控制科目与难度，并设计有明确标准答案的题目，便于自动评分。

MMLU（Massive Multitask Language Understanding）

57 个学科（数学、美国历史、法律、道德等），多项选择。
由本科生和研究生从网上公开资料收集。
尽管名字含“语言理解”，实际主要测试知识与推理，而非纯粹语言理解。
评估方式：对 GPT-3 使用少样本提示（few-shot prompting），在上下文中给出若干“问题-答案”示例，然后让模型预测最后一个问题的答案。
结果：小模型勉强高于随机水平；GPT-3 远高于随机，鼓舞人心。如今该基准已基本饱和（90+ 分）。

MMLU-Pro

动机（2024）：MMLU 已取得很大进展，需要更难的任务。
改进：移除有噪声和琐碎的问题；选项从 4 个扩展到 10 个；评估时使用思维链（chain of thought）。
结果：模型准确率回落到 33%，但目前已回升至约 88-90%，接近饱和。

GPQA（Graduate-Level Google-Proof Q&A）

核心思想：如果一个问题能通过查 Google 解决，那对语言模型可能太容易（因为模型在互联网上训练过）。因此需要真正困难的问题。
构建过程：61 位 Upwork 博士承包商出题 → 专家验证并反馈 → 出题人修改 → 另一位专家再审。
“DIAMOND”子集：两位专家意见一致且一位非专家能答出，才被认为是好题。非专家被给予 Google 访问权限，30 分钟内仅达 34%（略高于随机）。
结果：人类博士专家 65%，GPT-4 当时 39%，目前已达 94%。

Humanity's Last Exam（HLE，2025）

动机：模型变得太强，需要“尽一切努力”创造真正难的考试。
规模与形式：2500 题，多模态，多学科，包含选择题与简答题。
构建过程：众包，提供 50 万美元奖金池与署名权奖励；由前沿模型进行多轮筛选。
防污染措施：保留未公开的私有测试集。
结果：发布时模型得分仅为个位数；截至本讲，顶尖模型（Mythos）也仅 64.7%。

关键观察：随着模型能力提升和现有基准饱和，考试难度持续攀升。选择题形式在难度上没有上限，但限制了可提出的问题类型。

主要问题：考试基准不能反映现实世界使用情况——少有人向语言模型提出考试式问题，大多数使用是开放式的，问题可能表述不规范且无唯一正确答案。

3. 聊天基准（Chat Benchmarks）

动机：多数人不会向 AI 助手提出选择题。开放式问题（如“甜菜山羊奶酪沙拉配什么香草合适？”）无标准答案，无法通过精确匹配评分。

Chatbot Arena（现名 Arena AI）

核心思想：使用人类偏好评估模型。

流程：

互联网随机用户输入提示，获得两个匿名模型（A 和 B）的回复。
用户评判 A 更好、B 更好、两者都好或两者都差。
收集大量成对比较数据，计算 ELO 排名。

ELO 评分模型：假设模型 A 击败模型 B 的概率是 1 / (1 + 10^((ELO_B - ELO_A)/400))。用成对比较数据拟合 ELO 参数，最大化数据似然。

优点：

真实世界提示：用户免费使用模型，有动机做有用的事。
不需要所有模型比较所有提示（类似国际象棋的稀疏对局），只要图连通即可推导排名。
动态性：可随时间加入新提示与新模型并更新排名。

缺点：

用户分布不确定（来 LMSYS 网站的是什么人群？），可能包含垃圾发送者、操纵者。
二元偏好混淆了风格与正确性——国际象棋 ELO 有效是因为“赢”是唯一标准，而“哪个更好”定义模糊。
人类评者的局限：提问者通常不知道答案，如何判断正确性？存在谄媚（sycophancy）偏差——讨好的答案可能优于正确的直言。

AlpacaEval（2023）

805 条指令，指标为相对于基线模型的胜率：模型回复 vs. GPT-4 预览版回复，由 GPT-4 预览版评判哪个更好。
这就是 LLM-as-a-judge 的实例，是当前非常流行的评估方法。
已知问题：LLM 评委偏爱较长回复，导致排行榜游戏（经微调模型仅因回复更长而获得高评分）。后续版本使用回归方法去偏。

评估指标的验证：AlpacaEval 与 Chatbot Arena 的相关性高达 0.98，这表明可以用低成本指标替代昂贵的人类评估。但相关性基于特定模型集合，对更强模型可能不成立。

WildBench

从 100 万条人机对话中采样 1024 条。
类似 AlpacaEval 使用 LLM 评委，但核心创新是使用生成的、针对特定提示的检查清单（checklist）——为“这个回复好不好”这一模糊任务提供明确评分标准。
与 Chatbot Arena 有良好相关性。

开放式回复评估的总结原则：

成对比较（pairwise comparisons）比绝对评分（7/10 vs 8/10）信号更强。
注意偏差：人类和 LLM 评委都有不同类型偏差。
使用多个评委：如果一个模型得到人类评委和所有 LLM 评委的一致好评，那它可能确实更好。
定义评分标准或检查清单：能极大提升评估的可靠性与明确性，无论对人类还是 LLM 评委都是如此。

4. 智能体基准（Agentic Benchmarks）

定义：评估 LLM 做了什么（而非说了什么）。智能体 = 语言模型 + 智能体脚手架（scaffold），脚手架是决定如何调用 LLM、可访问哪些工具的逻辑。

SWE-bench

任务：给定代码库和 GitHub issue 描述，提交一个拉取请求（PR）修复问题。
评估：通过单元测试（修复前不通过的测试需要通过，且不破坏其他测试）。
结果：SWE-bench Verified 从 2024 年约 16% 升至 93%。

Terminal-Bench

环境：计算机终端，通过在终端中输入命令完成任务。
数据：229 个任务由 93 人众包，其中 89 个构成 Terminal-Bench 2.0。
特点：环境简单通用；人类完成任务需 1 小时到一周以上。
重要观察：相同模型在不同脚手架下准确率不同——智能体评估评估的是模型与脚手架的组合。

CyBench（网络安全）

40 个夺旗（CTF）任务：智能体需查看源代码、访问网络服务器，最终侵入系统提取独特字符串（Flag）。
发布时最好模型约 10%，现已基本被完全解决。
简单脚手架：连续记忆缓冲区，将行动与环境反馈拼接后送入模型；但这会导致上下文快速增长，需要更复杂的上下文管理。

MLE-Bench

75 个 Kaggle 竞赛任务：智能体查看数据、阅读描述、编写代码、训练模型、提交结果并获评分。

现代智能体脚手架的四个关键组件：

显式规划（explicit planning）：维护待办事项清单并逐项勾选，而非仅靠意识流式思维链。
层级委托（hierarchical delegation）：主智能体以简洁上下文调用子智能体；子智能体完成工作后只返回结果，实现封装。
持久化记忆（memory）：显式读写文件，因为上下文窗口有限，无法容纳全部历史。
流程管理式的上下文工程：何时委派、何时尝试新策略、何时写入持久化记忆，通常针对特定模型进行优化。

5. 纯推理基准（Pure Reasoning）

动机：以上所有任务都需要语言与世界知识。能否将推理从知识中分离出来，测量更“纯粹”的智力？

ARC-AGI 系列

始于 2019 年，任务设计为人类 100% 可解（以某种合理时限为限）、但对 AI 具有挑战性。
每个任务都是“独一无二的雪花”，记忆或迁移难以直接应用。
视觉图形推理任务（如图片模式补全），不依赖语言。

发展轨迹：

ARC-AGI-1（2019）：预训练模型（GPT-3 时代）完全没有推动进展。
2024 年：OpenAI o1/o3 推理模型出现后，进展突然起飞，ARC-AGI-1 基本被解决。
ARC-AGI-2（2025）：更多多步推理，尚未完全解决但看起来在解决路上。
ARC-AGI-3（2026）：交互式环境。分数极低，仍有很大空间。

关键观察：

预训练的知识对解决 ARC-AGI 是否必要？没有预训练也没有推理模型的爆发，但预训练知识的作用在 ARC-AGI 大部分历史中不直接可见。
目标（分离推理与知识）极难实现，可能不存在“纯粹”推理——任务仍来自某种先验背景。
方法局限于人类推理（设计目标是人类可解），不延伸到超人推理（如拿更多金牌或解决开放数学问题）。

6. 安全基准（Safety）

类比：汽车安全性经过数十年才确定测试标准（撞击测试、安全气囊等）。AI 安全目前没有好答案。

HarmBench：用意在造成伤害的提示测试模型，期望模型拒绝（refuse）。
AIR-Bench：从欧盟、中国、美国公司政策中的监管框架出发，建立分类体系（314 个风险类别，5694 个提示）覆盖可能出错之事。
越狱（Jailbreaking）：语言模型通常被训练拒绝有害指令，但可通过巧妙构造提示绕过。GCG 是一种坐标梯度优化算法，可自动优化提示以绕过安全限制，且攻击可从开源模型迁移到闭源模型。

安全的复杂性：

高度依赖语境：涉及政治、法律、社会规范，在不同国家有所不同。
风险多样化：幻觉（医疗、法律、金融场景）、谄媚、教唆犯罪、不平等、丧失批判性思维——有些与能力正相关（更准确减少幻觉），有些与能力冲突。
双重用途：能力强的网络安全智能体既可被用于入侵系统，也可用于渗透测试提升安全性。

评估的元考量

生态效度（Ecological Validity）

评估在多大程度上捕捉现实世界使用？

考试基准（GPQA）远离现实使用。
Chatbot Arena 提示来自真实用户，但用户分布不受控。

尝试提高生态效度的基准：

GDPVal（OpenAI）：基于美国 GDP 前九大行业的 44 个职业，由约 14 年经验的专业人士创建任务（护士、房地产经纪人、影视剪辑师等）。
MedHELM：121 个临床任务来自 29 位临床医生，代表临床医生实际会问的问题（vs. 标准化考试）。
Clio（Anthropic）：使用 LLM 分析真实用户数据，分享人们使用 Claude 做哪类事情的总体模式。出于隐私考虑不能直接看数据，但可以让模型分析并总结一般模式。

难点：真实感与隐私往往冲突——理想上需要真实查询流样本，但这涉及隐私考量。

有效性（Validity）：我们如何知道评估有效？

训练集与测试集重叠（数据污染）

基础模型出现前：ImageNet、SQuAD 等有明确训练/测试划分，规则清晰。
今天：模型在互联网上训练，你不知道数据里有什么。

应对路径：

推断污染：利用基准中问题顺序应该随机的性质。如果模型偏好与基准一致的特殊顺序，可能是在该基准上训练的。
鼓励报告规范：模型提供者应报告训练集与测试集重叠情况（类似统计学中报告置信区间的实践）。
使用新鲜评估：LiveCodeBench、UncheatableEval 抓取模型截止日期后的新网页作为评估集。注意时间戳也不总是安全（内容可能被复制）。
私有评估集：公司内部代码库或你的个人未公开发布写作（如被拒稿的论文）。对困惑度评估尤其有利，因为只需要数据即可评估对数概率。

数据集质量

SWE-bench 原始版本中有些单测不严谨，修复后产生 SWE-bench Verified。
其他基准（GSM8K、MMLU）经仔细审核后也发现了问题，如“图中没有曲线，无法回答该问题”。
智能体基准更难评估：测试用例可能不完整，通过测试但解决方案仍不可用；有研究表明智能体仅输出空响应即可获得 38% 分数。
Docent：使用 LLM 检查智能体轨迹以检测问题——一种定性的补充方法。
通用建议：无论开发新基准还是运行已有基准，都应查看实际输出并审计，确认测量的是想要测量的东西。

构建评估策略的原则

没有唯一真正的评估——选择取决于你要回答什么问题：
- 用户/公司：为特定用例（如客服聊天机器人）做购买决策。
- 研究者：测量模型的原始能力（如“智能”）。
- 商业/政策：理解益处与危害。
- 开发者：获得改进模型的反馈。
明确游戏规则：我们在评估方法还是模型/系统？
- 基础模型前：评估方法（固定训练/测试划分，只有算法可变）。
- 今天：主要评估模型/系统（一切皆可变）。
- 例外：NanoGPT speedrun，固定数据和目标验证损失，仅评估训练速度。
权衡取舍：很难同时做到高难度、高真实感和高生态效度；也找不到完全避免污染的评估。根据目标选择愿意妥协的维度。

课程作业关联

本讲的内容直接关联到课程项目的评估环节。在训练完成语言模型后，你需要：

使用困惑度作为主要的内在评估指标（注意训练集/测试集的划分以及模型是否见过测试数据）。
在标准基准（如 MMLU）上进行零样本或少样本评估，以与其他模型对比。
注意数据污染的潜在风险——你的训练数据可能已经包含了这些基准的测试集。

常见误区

认为困惑度越低模型一定越好：困惑度是所有 token 的平均，可能包含大量不相关 token 的噪声。低困惑度不等于在特定任务上表现好。
轻信基准分数：基准可能存在数据污染（模型见过测试数据）或数据质量问题（测试用例不完整、问题本身有误）。应始终持保留态度并考虑审计。
混淆 LLM 评委偏长回复与质量高：LLM 评委常偏爱较长回复，导致排行榜游戏。应使用去偏后的指标或多个裁判交叉验证。
混淆风格与正确性：成对偏好比较无法区分模型是说得好听还是说得正确。人类评委也易受谄媚偏差影响。
忽视智能体脚手架的影响：在智能体基准中，相同模型在不同脚手架下表现差异巨大。评估的是“模型+脚手架”的联合系统。
认为 ARC-AGI 是纯推理：任务仍来自某种先验背景，不存在“纯粹”的推理；且其设计局限在人类推理范围内，不覆盖超人推理。
对安全基准的过度简化：安全高度依赖语境，单一数字无法全面代表安全性。不同国家、不同场景的安全定义不同。

复习清单

[ ] 能解释为什么说“困惑度既是一切，又不是一切”。
[ ] 能列举三个考试基准并说明它们的设计动机、评估方式及饱和状态。
[ ] 能说明 Chatbot Arena 的 ELO 排名工作机制及其主要局限。
[ ] 能解释 LLM-as-a-judge 的偏差问题，以及检查清单/评分标准如何帮助缓解。
[ ] 能定义“智能体脚手架”并列举其四个关键组件。
[ ] 能比较困惑度评估与下游任务评估在信任/验证上的本质差异。
[ ] 能说明数据污染的四种应对路径及其各自的局限性。
[ ] 能区分“评估方法”与“评估模型/系统”的目标差异，并知道各自的适用场景。
[ ] 能从生态效度、难度、有效性三个维度评估一个新的基准测试。

评估：从困惑度到智能体——如何度量语言模型的“好”

引言

当我们训练完一个语言模型，最自然的问题就是：它有多好？在CS336课程的学习路径中，我们已经走过了架构设计、优化器选择、训练循环实现、并行化加速、缩放定律推导和推理优化等完整环节。但正如课程中反复强调的，数据塑造模型行为——在代码上训练的模型擅长代码，在DNA序列上训练的模型无法说英语。而数据的选择依赖于我们对“理想行为”的定义，这正是评估（Evaluation）的用武之地。

评估看似是一个机械过程：定义提示词、发送给模型、获得回复、计算准确率。然而，评估实际上是一个深刻且影响深远的话题。它设定了北极星指标，模型开发者无论开源还是闭源，都将评估视为进步的衡量标准。通过精心设计评估方式，我们实际上在潜移默化地塑造模型能力的发展方向。评估的核心挑战在于：将抽象概念（如“擅长对话”“擅长推理”）转化为由具体提示词或环境支撑的具体指标。

本文将系统梳理课程中讨论的各类评估方法，从经典困惑度到考试基准、聊天基准、智能体基准、纯推理基准和安全基准，探讨评估的现实性、有效性以及最终目的。

困惑度：经典范式的信任与局限

语言模型本质上是对词元序列的分布建模。从这个角度看，最自然的评估方式就是困惑度（Perplexity）——测试模型对给定数据集赋予的概率质量。在训练时我们最小化训练集上的困惑度，因此评估时测量测试集上的困惑度似乎是顺理成成的。

在2010年代，这种做法是语言建模研究的标准范式。经典数据集包括Penn Treebank、WikiText-103和One Billion Word Benchmark。人们通过单纯的困惑度降低来衡量进展，例如2016年一篇著名论文展示了将CNN和LSTM应用于十亿词基准可带来巨大困惑度降低，这一决定性结果表明纯神经网络是正确方向。

2019年GPT-2的出现打破了这种分布内评估范式。GPT-2在一个名为WebText的数据集上训练（从Reddit链接出去的40GB网站内容），但评估却是在标准数据集上进行的零样本评估——这是分布外评估的先例。在PTB这样的小数据集上，GPT-2获得了35的困惑度，而当时最先进水平是46。这种范式转移开启了对语言模型泛化能力的关注。

那么困惑度是否足够？课程中提出了一个被称为“Perplexity is all you need”的信念（介于信仰与科学之间）：假设存在某个真实分布t，模型为p，最佳困惑度就是真实分布的熵，且仅当p=t时实现。如果你拥有了真实分布，就能解决所有任务——以问题为条件生成解答。这个信念驱动的方向是：通过不断降低困惑度，最终达到AGI。在GPT-3之前，人们并不清楚语言模型会产生如此巨大的影响，正是这种信念推动了缩放。

然而，困惑度可能比你需要的更多。例如句子“斯坦福大学创立于1885年”，困惑度会惩罚所有token的预测，包括“1885”（这个捕捉了世界知识的token）和“创立”（相对次要的token）。解决办法是测量条件困惑度，以提示为条件，只关注回复部分的困惑度。

值得注意的是，某些基准测试本质上就是披着外衣的困惑度。LAMBADA是填空题，目标是找出填补句子的词，尽管以准确率衡量，本质却是下一个token预测——只不过经过精心挑选，需要长上下文才能解析。HellaSwag是多项选择形式的句子补全，同样是困惑度的变体。

使用困惑度排行榜还有一个信任问题：提交的语言模型需要返回logprob，你必须相信这些概率有效且加和为1，否则模型可以总是返回logprob为0来获得完美分数。相比下游任务可以直接检查回复内容，困惑度涉及到分布的有效性检查，这更困难。

尽管如此，困惑度在语言模型开发中仍被大量使用，因为它随规模平滑变化，适合建立缩放定律。

考试基准：从MMLU到Humanity's Last Exam

考试是人类测试知识的方式，语言模型基准测试文化建立在这种理念之上。考试的好处在于可控制科目和难度，设计出有明确答案的题目，易于评分。

MMLU（Massive Multitask Language Understanding）是2020年的基准，涵盖57个学科，由学生从网上搜集来源构建。尽管名字含“语言理解”，它主要测试知识和推理。评估方式为GPT-3的少样本提示：构造包含问题-答案示例的上下文，最后让模型预测答案。当时小模型仅勉强高于随机水平，而GPT-3远高于随机，相当鼓舞人心。如今MMLU已基本饱和，分数来到90多分。

大约2024年，人们认识到这个基准太简单了，于是MMLU-Pro诞生：去除噪声琐碎问题，选项从4个扩大到10个，更适合使用思维链。模型准确率回落到33%，但如今又升至约88-90%。

GPQA（Google-Proof QA）尝试更严格：通过Google能解决的问题太容易。他们雇了61位来自Upwork的博士承包商，有人出题、专家验证、出题人修改、另一位专家重审。在非专家可以访问Google的情况下，博士专家只能拿到65%准确率，GPT-4当时仅39%。如今GPQA又到了94%。

Humanity's Last Exam是年度基准，目标是创造真正困难的多模态考试，涵盖众多学科。它众包题目，提供奖金与署名权激励，通过前沿模型多轮筛选，并保留未公开的私有测试集以防止数据污染。2026年的表现仍然是个位数，最高也只有64.7%。

考试基准的演进趋势清晰：随着模型能力提升和现有基准饱和，不断向更难题目推进。但考试与真实使用相距甚远——没有人会向语言模型提出HLE式的问题。大多数时候人们的提问是开放式的，不一定有唯一正确答案，甚至表述不太规范。

聊天基准：开放输出的评估困境

大多数人不会向AI助手提多项选择题，而是提出真实世界中的开放式问题。例如“我想做甜菜山羊奶酪沙拉，哪些香草合适，哪些不合适？”回复同样是开放式的，无法通过精确匹配检查，甚至没有标准答案。

Chatbot Arena（现名Arena AI）开创了一种方法：任何互联网用户可进来聊天，然后从两个匿名模型获得回复，用户评判哪个更好。通过大量成对比较数据，可以用ELO评分系统计算排名。ELO评分的想法是：特定模型击败另一个模型的概率是两者ELO分数差的函数。这种方法的优点包括获得真实世界提示（用户有动力使用有用的语言模型），且不需要让所有模型面对相同提示（ELO的精髓在于稀疏采样的对局仍可推导排名）。

但问题也存在：互联网随机用户的分布不可知；可能存在垃圾信息发送者和操纵结果者；二元偏好混淆了风格与正确性；提问者可能不知道答案却要判断哪个回复正确；存在谄媚问题——讨人喜欢的答案可能比正确但直言不讳的答案获得更高权重。

AlpacaEval采取不同路线：以基线模型（如GPT-4预览版）为参照，用LLM作为评委判断模型回复与基线哪个更好。这里立即产生偏差问题。最初AlpacaEval的一个已知问题是LLM评委偏爱较长回复，导致排行榜游戏——许多模型仅因回复更长而获得高分。后来论文用简单回归方法消除了偏差。

如何评估一个指标本身？常见健全性检查是查看它与其他指标的相关性。AlpacaEval与Chatbot Arena的相关系数为0.98，说明可以用AlpacaEval替代Arena评估，但相关性仅在特定模型集合内成立。

WildBench从人类与聊天机器人对话中收集大量样本，使用LLM作为裁判。主要创新是使用生成的、特定于提示的检查清单——让LLM在明确标准下评分，比模糊的“这个回复好不好”的定义更为明确。

开放式回复评估的总结：成对比较（尤其对相似回复）比绝对评分（7/10还是8/10）信号更强；始终注意人类和LLM裁判的偏差；通过多个裁判评估可增强信心；定义评分标准或检查清单对提高可靠性至关重要。

智能体基准：评估做了什么

智能体（Agent）是语言模型加上脚手架（Scaffold）——决定如何调用语言模型、可访问哪些工具的逻辑。我们此前评估的是模型“说了什么”（聊天），现在评估“做了什么”。

SWE-bench给出代码库和GitHub issue描述，需要提交拉取请求，评估方式是通过预设的单元测试。这些测试设计为修复前不通过，目标是通过它们且不破坏其他功能。评估非常直接。2024年准确率约16%，现已升至93%。

Terminal-Bench对环境做了泛化——基本就是一个计算机终端。任务由来自世界各地的93人众包，通常需要一个多礼拜时间。顶尖模型是前沿模型，且相同的模型配上不同脚手架会有不同准确率。

CyBench包含40个网络安全夺旗（Capture The Flag）任务，目标是侵入服务器提取独特字符串。这类竞赛原本由人类进行，基准发布时最佳模型约10%，如今已完全解决。

MLE工程基准包含Kaggle竞赛任务——处理数据、训练模型等。智能体可查看数据集、阅读描述、编写代码、训练模型、提交结果并获得评分。

关键洞见是：显式规划（维护待办事项清单）有助解决困难任务；层级委托（智能体以简洁上下文调用子智能体，只返回结果）实现封装；记忆（显式读写文件）尤其是上下文中无法容纳全部信息时至关重要；更极端的上下文工程涉及何时委派、何时尝试新策略、写什么入持久化记忆——这些通常针对特定语言模型进行优化。评估智能体时，实际上是在评估语言模型与脚手架两者。

纯推理基准：分离知识与推理

前述任务都需要语言和世界知识。问题是能否把推理（流体智力）从知识和事实记忆中分离出来？ARC-AGI项目始于2019年，目标正是如此：任务设计为人类100%可解决但对AI具有挑战性，每个任务都是独特的“雪花”，记忆或解决相似问题没有帮助。

ARC-AGI-1的示例显示了图形模式补全任务，人类看一眼就能解出。预训练语言模型（如GPT-3）无法推动进展，正是设计预期。2024年OpenAI发布o1和o3推理模型后，进展开始起飞，ARC-AGI-1基本被解决。ARC-AGI-2发布后也在被解决的路上，ARC-AGI-3则在2026年发布，现在是非常低的分数。

推理模型解锁了这类进展，但预训练的知识是否必要？没有预训练不会有推理模型的爆发。ARC-AGI方法局限于人类推理（明确目标是在合理时间内100%由人类可解），并不延伸到超人推理（如解决开放数学问题），但它清晰暴露了当前模型的短板。

安全基准：什么是安全的AI

车的安全评级有清晰定义（撞墙测试、安全气囊效果），AI的安全则没有标准答案。人们从不同角度探索。

HarmBench用意在造成伤害的内容提示模型，期望模型拒绝。AIR-Bench考察欧盟、中国和美国公司政策中的所有监管框架，建立涵盖所有可能出错之处的分类体系。

越狱是攻击安全限制的子问题：早期工作用GCG坐标优化算法自动优化提示绕过安全限制，生成近乎乱码的文本，加上逐步毁灭人类计划，OpenAI模型就会照做。这些攻击在开源模型上优化后可以迁移到其他模型。

安全问题的复杂性在于：许多方面高度依赖语境（政治、法律、社会规范因国家而异）；风险类型多样——幻觉（在医疗、法律或金融场景中高影响）、谄媚、教唆犯罪、不平等、相关性能力等。还有双重用途方面：Cybersecurity智能体可被用于入侵系统或渗透测试并让系统更安全，这使得讨论安全是一把双刃剑。

其实怎么知道评估是有效的？

关于评估有效性，我们面临数据污染的根本问题。基础模型之前，机器学习101告诉我们在训练集和测试集上训练测试，生活简单。现在模型在互联网上训练，且训练数据未公开，你不知道其中是否包含测试数据。

数据污染比“不小心在测试集上训练”微妙得多——可能不是直接包含测试集，而是题目来源于互联网资料，而资料被训练过。有几种应对路线：尝试推断模型是否见过测试数据（利用的问题顺序随机性——如果模型偏好与基准一致的特定顺序，很可能在基准上训练过）；报告训练集与测试集重叠情况（如同统计学中报告置信区间）；使用新鲜评估（如LiveCodeBench和UncheatableEval抓取截止日期后的新网页，但时间戳也不总安全——GitHub仓库可能包含源自其他仓库的内容）；保留私有评估集（公司使用内部代码库，个人用从未发表的写作）。

数据集质量也是重要问题。SWE-bench需要修复为SWE-bench Verified版本，因为原有任务和单元测试有缺陷。MMLU和GSM8K也经历了仔细审核发现问题（如“这段文本中没有给出曲线，你无法回答这个问题。宝宝穿袜子了吗？不，根本看不出来”）。智能体基准更难评估，因为不能只看问题和答案——测试用例可能不完整，通过所有测试用例但解决方案不可用的情形存在。TorchBench基准显示如果一个智能体只输出空响应就能拿到38%的分数，凸显了基准中琐碎问题需要避免。

评估的意义

没有一种评估能适用于所有情况。评估的目的大致可分为几类：用户或公司要在特定用例中选择模型；研究者要测量模型的原始能力；基于商业或政策原因理解效益与危害；模型开发者想获得反馈来改进模型。不同目标会引导选择不同基准或组合策略。

另外还要区分评估的是方法还是模型。基础模型时代之前，研究者评估方法——训练测试划分固定，只有算法可变。今天，我们主要评估模型和系统，一切皆可变。也有例外，如NanoGPT speedrun刻意评估“训练模型到特定验证损失所需的时间”，保持数据固定，评估的是方法中的算法速度。评估方法鼓励算法创新，评估模型/系统对下游用户更有用。

最终，关键是要明确定义规则：选择你想衡量的东西，并清晰宣布目标。不同基准在难度、真实性、有效性上各异，经常存在权衡——很难同时非常真实、非常困难且具有生态效度。根据目标，你将不得不选择最愿意妥协的因素。

结语

从困惑度到考试基准、聊天基准、智能体基准、纯推理基准和安全基准，评估语言模型的方法不断演进。课程给出的综合结论是：没有唯一真正的评估标准，选择应该基于你要衡量什么，并明确宣布游戏规则（评估方法是模型还是智能体），同时注意难度、真实性和有效性之间的权衡。

下周课程将转向训练数据——这个塑造模型行为的关键环节。在讨论数据选择前，理解评估定义了我们希望模型具备的行为，是必要的基础。正如课程所强调的，评估塑造AI发展，是通过认真思考评估方式，我们在引导模型能力前进的方向。

CS336 2026 Lecture 12：完整中文逐字稿

视频：官方视频
来源：英文人工字幕提取后翻译为中文；保留段落起始时间戳，只修正明显的断句、重复和语序。
说明：这是逐字稿，不是讲义摘要；相邻且语义连续的片段已合并为阅读段落；术语保留必要英文。

正文

[00:05.720] 今天我们要讨论评估。到目前为止，在这门课上，我们已经完成了训练语言模型所需的全部工作。我们定义了架构、优化器、主训练循环，展示了如何通过内核和并行化来加速训练。然后我们学习了缩放定律，也稍微了解了一些如何加速推理的方法。所以现在唯一缺失的部分就是讨论你用来训练的数据。我们将在下周花时间讨论这个。数据将塑造模型的行为。如果你在代码上训练，那么你的模型大概会擅长代码。如果你只训练DNA序列，它可能无法说英语。但在我们讨论数据之前，我们需要先讨论我们希望模型具备什么样的行为，而这正是评估的主题。评估问的是一个非常简单的问题——给定我们训练好的模型，它有多好？那么，什么是“好”？评估看起来可能是一个相当机械的过程。你定义一些提示词，将这些提示词发送给你的模型，得到一些回复，然后计算准确率。

[01:25.470] 那么为什么我们还要用一整节课来讨论评估呢？但实际上，评估是一个非常深奥的话题，而且也非常重要，因为评估塑造了AI的发展。评估设定了北极星指标，模型开发者——无论是开源的还是闭源的——每个人都把评估视为进步的衡量标准。因此，通过仔细思考你的评估方式，你实际上是在潜移默化地塑造你的模型能力的发展方向。评估的难点在于以下这一点。通常，你开始时会有一些抽象的概念：我希望我的模型做什么？也许我希望我的模型擅长对话，或者擅长推理。好的，这些是抽象概念。而评估实际上就是把这个抽象概念转化为一个具体指标的过程，这个指标大概由一些具体的提示词或环境来支撑。这就是核心挑战，我们将看到许多例子，展示这个从抽象概念到具体指标的过程

[02:36.190] 是如何体现的。那么让我们回到一个问题上：是什么让一个模型变得优秀。只是表面上看。也许一个模型如果在基准测试中表现好，它就是好的。有一个网站叫Artificial Analysis，它已经成了衡量所谓模型“智能”的一个标准参考。如果你访问Artificial Analysis AI，你可以看到从GPT-5.5往下排的各种模型排名——嗯，在这个排名后面还有很多模型。所以这是一个度量方式。它包含了一堆不同的数据集，其中一些我们稍后会讨论。或者也许一个模型如果既在基准测试中表现好，又运行便宜，那它就是好的，因为成本也应该重要。

[03:26.160] 这张图展示了之前的智能指数，同时横轴是根据推理成本计算的运行特定模型需要花费多少钱。你可以看到，正如你可能预料的那样，两者是相关的。你支付的运行费用越高，智能指数也越高，但两者并非完全对齐。但也许一个模型是好是坏，就看人们是否更喜欢它。稍后我们也会更深入探讨。这就是Arena AI（以前叫Chatbot Arena）所采用的观点。你可以基于人们是否喜欢这些模型来形成一个排名。或者也许一个模型是好是坏，取决于人们是否真的选择使用它，并因此为之付费。OpenRouter就是这样一种服务，它允许用户通过一个中心端点使用多种不同的模型。因此，他们收集了大量关于人们在使用哪些模型的统计数据。

[04:34.410] 他们发布了一些统计数据，这些数据看起来非常有意思，比如通过OpenRouter使用最多的几个模型。当然，这并不能代表所有模型的使用情况，但它能让你对OpenRouter上的人们在做什么有个概念。所以这更像是一种经济视角。我不知道什么是“好”，但如果人们在为它付费，那它一定不错。好的，这些答案都不一定是正确的。我也不知道是否有正确答案，但我只是想让大家思考一下，关于什么是“好”，可以有多种不同的思考方式。现在让我们开始看看评估语言模型的不同具体方法。最合适的起点莫过于回到语言模型的核心概念，即它是在词元序列上的一个分布P(x)。从这个角度看，有一种非常自然的评估分布的方法，那就是困惑度（perplexity），或者似然度（likelihood），或者对数损失（log loss）。

[05:46.540] 这些都是相互关联的概念。基本上就是说，我有一个测试数据集d，我要看看我的语言模型赋予这个d多少概率质量。然后我对其进行归一化和除法，使得数字更具可解释性。但基本上就是这么回事。我的概率语言模型赋予一个数据集多少概率质量。好的，这很自然。事实上，在训练时，我们最小化了训练集上的困惑度。所以显而易见的做法就是测量测试集上的困惑度，而这也是人们在整个2010年代的语言建模研究中多年来的传统做法。如果你拿起一篇语言建模论文，这就是他们所做的。那时候，标准数据集包括Penn Treebank、WikiText-103，还有一个从机器翻译来源获取的十亿词基准（One Billion Word Benchmark）。这些都处于经典的评估范式内，即分布内评估。

[06:58.750] 你在该数据集的某个训练集划分上训练，然后在该数据集的某个测试集划分上评估。人们确实取得了很大进展，而这些进展仅仅是通过困惑度的降低来衡量的。例如，有一篇2016年的著名论文，展示了将CNN和LSTM应用到十亿词基准上，可以获得巨大的困惑度降低。请记住，在那个时候，人们还在讨论n-gram模型，或许还有混合模型。而这是第一个决定性的结果，表明纯神经网络显然是正确的方向。好的，当时的生活非常简单。你在训练集划分上训练。在测试集划分上测试困惑度。然后到了2019年，OpenAI的GPT-2出现了，他们在一个名为WebText的数据集上训练，这是从Reddit链接出去的40GB文本网站内容。而他们的评估方式实际上有所不同。

[08:10.920] 他们在其他所有人都在评估的标准数据集上进行了零样本评估。所以这是分布外评估。这是他们论文中的一张表，展示了他们训练不同模型的情况。最大的模型有15亿参数。他们展示了相当多的改进，尤其是在小型数据集上。例如，PTB，这是一个很小的数据集。他们获得了35的困惑度，而当时的最先进水平是46。现在，如果你有足够的分布内数据，你可能比不上最先进水平。但尽管如此，这已经相当令人印象深刻了，因为这个模型根本没有在十亿词数据集上训练过。我不知道，也许有一些重叠，因为我不清楚他们是否做了仔细的训练测试去污染检查。这开启了一个新的思考范式，即语言模型不仅仅用于分布内场景，而是思考我们如何在一个大型数据集上训练

[09:31.480] 并在标准基准上进行评估，这在如今我认为已经是非常普遍的做法了。但在当时，这被认为是一种新鲜事物。我还会论证，既然我们在讨论困惑度，那么困惑度就是你们所需要的一切。我不希望大家把这个观点看得太严肃，但我认为这是一种驱动了大量语言模型研究的思维方式。基本的论证是这样的。假设存在某个真实分布——我们称之为t——而你在训练这个模型p。那么你能获得的最佳困惑度是多少呢？最佳困惑度就是那个真实分布的熵，而这只有在p等于t的时候才能实现。所以通过最小化困惑度，你本质上就是在不断压低困惑度。而唯一的最优解就是当你真正得到真实分布的时候。当你拥有了真实分布，你就大功告成了。你就能对一切进行建模。你可以解决所有问题，你可以做诸如以问题为条件、生成解答这类事情。

[10:52.000] 你可以以问题为条件，生成答案。所以这个论证就是，通过不断降低困惑度，我们最终将达到AGI。我的意思是，这听起来可能，我不知道，要么有点异想天开。但从某种意义上说，我认为这确实是许多人当时不断缩放、不断扩展语言模型的强大驱动力，而那个时代，收益可能还不那么明显。在GPT-3之前，我认为人们并不清楚语言模型会产生如此巨大的影响。真正驱动人们的正是这种信念：如果你把困惑度降下来，好事情就会发生。所以当然，困惑度也许不是一切。有一种表述方式是，困惑度可能比你需要的更多。举个例子，如果你有一个句子，比如“斯坦福大学创立于1885年”，困惑度会说，要对所有token的预测进行惩罚。具体来说，它会问，在给定其余部分的情况下，“1885”的概率是多少？而那个token可能相当有用。

[12:07.980] 这基本上就是一个问答示例，只不过是以句子补全的形式呈现的。所以它捕捉了关于世界的知识。但其他的token——比如一个句子的第一个词，甚至“创立”这个词——可能就没那么有意思了，我觉得。而困惑度并不在乎这些。它会为每一个偏离真实分布的比特向你收取比特费用。所以这里有一个解决办法，那就是你总是可以测量条件困惑度。你可以以某个提示为条件，然后测量剩余回复的困惑度。这样，总的来说，你就可以把注意力集中在那些你认为相关的token上，而较少关注那些多少有些附带性的token。好的。另外，有一点很重要需要记住，有些基准测试，即使它们并不是明确的困惑度度量，实际上也只是披着外衣的困惑度。比如说，想想那些封闭式任务。有一篇LAMBADA论文，其实是很久以前的，2016年的，基本上就是填空题。

[13:29.650] 你有一个上下文，你有一个句子。“你诚实地认为我会想要你有一个____。”目标是找出那个词。尽管这是以准确率来衡量的，但它本质上是一个下一个token预测问题。而且在这里，它不是所有词的困惑度，而是经过精心挑选的。如果你读了这篇论文，会发现那些词确实是精心挑选出来的，使得你确实需要大量的长上下文才能解析出来。所以这实际上是早期GPT论文真正抓住的东西，因为他们对长上下文建模感兴趣，认为那是解锁推理能力以及所有这些事物的关键。而且，通过聚焦于那些需要长距离依赖才能解析的位置，实际上是一种将困惑度聚焦到你关心的某种现象上的方法。这是另一个流行的经典数据集，你会在一些语言模型相关的读物中看到。

[14:46.890] HellaSwag。那么什么是HellaSwag？他们去了各种不同的数据集和来源，想出了——它是多项选择，但实际上是句子补全。在某种程度上，它就是困惑度。这里有一个句子：“一个女人带着一个桶和一条狗在外面。那只狗在到处跑。她____。”然后你要填入哪个句子能最好地补全它。所以它不完全是困惑度，它是多项选择。但在某些方面，它是困惑度。所以有一点要提醒大家。如果你真的被说服了，认为困惑度就是那个度量标准，而且你打算推出一个排行榜，让人们围绕困惑度进行竞争，这里只想提醒一句，那就是困惑度排行榜会是什么样子？人们会向你提交一个语言模型，你需要计算一个logprob。你把测试数据发给他们，他们给你返回一个logprob。那么，你看看这个结果，嗯，这可能会让人觉得有点可疑，因为你需要相信他们还回来的

[16:01.650] 概率是有效的，并且它们加起来等于1。否则，我可以随便实现一个LM，它总是返回1，或者说logprob为0。那我就能够得到非常好的困惑度，但显然，那不是有效的分布。所以在这样的约定下，要检查某人是否真的在内部拥有一个对数概率并不容易，所以这里面存在信任问题。也许你不得不查看代码之类的东西。对于下游任务，这就直接多了，因为你可以有一个黑盒LM。我给你一个提示，你给我一个回复，我就可以直接使用那个回复并计算准确率。所以困惑度是根本不同的，因为它涉及到分布，你必须小心一些。更不用说，自回归模型有很多问题。但如果你有VAE，或者其他一些你甚至无法计算分布的模型——你只能计算一个界——那么你真的还得相信那个数学推导，确信这确实是一个有效的界。

[17:11.060] 好的。所以关于困惑度的总结是，它在语言模型开发中实际上仍然被大量使用。它有一些很好的性质。我们看到它通常被用于缩放定律，因为它也随着规模平滑变化，这对于获得缩放定律是很重要的。但归根结底，特别是如果你不是信徒的话，那么你需要一些基准来真正捕捉现实世界的情况，从而说服人们你的语言模型是好的。对于信徒来说，如果我给你看一个低得惊人的困惑度，他们就会被说服。好的，关于困惑度有什么问题吗？好的。希望大家现在都成为信徒了。好的，那么我们来谈谈考试基准。考试，正如你们所有学生所知道的，是我们用来测试人类的一种方式。所以你可以采用同样的思路来测试语言模型。

[18:30.730] 考试的好处在于，你可以对科目、对难度进行非常仔细的控制。你可以设计出具有明确无误的标准答案的题目，这使得它们很容易评分。所以考试有很多好的性质。因此，很多语言模型的基准测试文化就是建立在这种考试理念之上的。显然，已经有很多基准了，但我只想重点介绍一些我认为对语言模型发展产生了影响的基准。这个就是MMLU，即Massive Multitask Language Understanding，来自Hendrycks等人，2020年。在那个时候，人们还不清楚语言模型是否真的是通用的任务解决器。那大概是GPT-3问世的时候。我想，很多人仍然认为语言模型只是一般的语言模型——你能生成流畅的英语。

[19:36.340] 但他们当时有这样一个想法：我们要走在潮流前面。我们要涵盖57个学科，让一组学生去网上搜寻各种来源，真正构建出一个非常全面的数据集。尽管MMLU这个名字里有“语言理解”这个词，但它实际上主要测试的是知识和推理能力，而不那么侧重于纯粹的语言理解。他们的评估方式是使用GPT-3，采用少样本提示。基本上就是说，我构造了这样一个提示。以下是关于高中数学的问题。还有一些上下文中的示例，问题、答案、问题、答案，然后是最后一个问题，模型需要预测答案。这看起来非常普通。但在当时，我认为，你要构造这样一个相当复杂的问题，并且配上上下文示例，然后期望语言模型能做出合理的回答，这是相当激进的。果然，一开始，模型的表现并不好。一些小模型也就勉强高于随机水平。而较大的模型——也就是GPT-3——实际上远高于随机水平，所以这还是挺鼓舞人心的。

[21:13.910] 然后随着时间推移，正如我们所知，这个基准基本上已经饱和了。这是一个网站，我不确定它是否包含所有数据，但大概从3.5 Turbo开始，这大概是在——我猜是2024年初——哦，实际上是2023年——到GPT-4。然后现在，已经到90多分了。这里还有一个链接，你之后可以点开看看，在那里你可以实际浏览这些模型的预测结果，看看模型到底在哪些地方出错、哪些地方答对，总是很有趣的。好的。大约在2024年，当MMLU看起来已经取得了很多进展的时候，人们就想——你后面还会看到这个主题——好吧，这个基准太简单了。我们试着让它更难一些。首先，他们确实注意到里面有一些有噪声的和琐碎的问题，所以把这些问题去掉。四个选项可能有点太容易了，所以扩大到10个。那时，用思维链来解决这些基准已经更流行了，而GPT-3出来的时候，这还不是个事儿。

[22:46.300] 而且很明显，其中一些问题如果能有一些推理步骤会更好。所以这个新基准更难了。现在模型的准确率又回落到33。但你可能从目前的进展可以看出，它起步很低，现在又回到了，我猜大概是88，接近90了，好的。所以这个基准可能也有点太容易了。好的，还有另一个基准叫做GPQA，全称是Google-Proof QA。这里的想法是，我们怎样才能得到一个真正好的基准？想法是这样的：如果你能通过查Google解决一个问题，Google拥有整个互联网，而语言模型是在互联网上训练的。所以如果你能直接解决它，那可能就太容易了。所以他们找了一些现在是博士水平的人。我觉得MMLU可以看作是——也许有些题目是本科水平，但也许也有研究生水平。

[24:00.050] 但现在，更明确地说，这些题目是刻意设计得很困难的。到了这个时候你会说，好吧，我怎么才能想出真正难的题目？你实际上需要投入相当多的人力才能得到真正难的题目。所以他们雇了61位来自Upwork的博士承包商。有人写一道题。然后经过专家验证，专家会给出反馈。然后出题人去修改题目。另一位专家再看一遍题目。然后题目就完成了。接着你把它发给一些被给予Google访问权限的非专家。然后你看看他们能不能解出这道题。在这些题目中有一个子集叫做DIAMOND集，你可能在别人的排行榜上看到过。如果两位专家意见一致，并且有一位非专家能够答出这道题，那么它就被认为是一个好的DIAMOND题。

[25:01.240] 所以这是一个相当严格的审阅流程。而博士专家也只能拿到65%。这我想大多是四选一的选择题。所以这高于随机水平。但如果你考试只拿65%，这可不是个好成绩。而且即使是专家，即使给他们30分钟并可以使用Google，他们也只比随机水平高一点，但高得不多。那时候GPT-4大概是39%，现在看看它表现如何。GPQA现在又到了94。所以这些基准的有效期我觉得并不是太长。我得说，有些基准即使在大模型上已经饱和了，它们对开发较小的模型或者拟合缩放定律等等仍然是有用的。然后，是的。你怎么知道这些测试在被用于训练时，你的模型实际上没有跟这些测试数据有接触？是的，这是一个非常重要的问题。你怎么知道这些问题不在训练集里？简短的回答是，我们不知道，因为我并不知道训练集里有什么。

[26:37.150] 我稍后会回到这个数据污染的问题，以及解决它的方法。但你总是应该——我很高兴你问了这个——你总是应该对这些数字持一点保留态度，因为这只能靠信任。也许再多说一点。我认为训练数据污染有点微妙。因为很可能不是直接把这些测试集拿来训练。但题目可能来源于其他资料。而那些资料可能被训练过。所以通常数据污染是一个比“不小心在测试集上训练”微妙得多的过程。还有别的问题吗？同样的问题。好的，没问题。保持怀疑总是好的。好的，我要讲的最后一个考试基准，名字有点吓人，叫“人类最后的考试”(Humanity's Last Exam)。这是去年的。他们说，好吧，这些模型变得太强了。

[27:39.500] 让我们尽一切努力创造一个真正难的东西。所以这是多模态的，涵盖很多学科。仍然是选择题，我猜还有一些简答题。这是众包的。他们提供了一些资金来激励那些爱钱的人创作题目。他们也用署名权来激励那些在意这个的人。然后他们经过了多轮评审，由前沿模型进行筛选。整一个流程。对你那个问题的另一个回答是，他们还保留了一个未公开发布的私有测试集，这样他们可以确保这个不会进入训练数据。然后你就只能希望你所使用的API——因为你还是得把这些提示发给模型——希望这些提示最终不会进入训练数据。但我们希望是这样。每一份数据集论文看起来都是这样。以前的基准，“看，模型表现得多好”。

[28:44.640] “我的数据集，看，所有模型都表现得很差。”所以这是一个常见的模式。而这次尤其是个位数，相当惨淡。让我们看看2026年我们表现如何。所以这个——它还有一些空间。即使是Mythos也只有64.7。所以这确实是一个相当——看起来确实是一个非常难的考试。好的，总结一下这一部分，正如你所看到的，随着模型能力的提升以及现有基准的饱和，我们一直在往越来越难的题目方向推进。有趣的是，选择题的形式仍然存在，因为——对于选择题来说，我是说，你可以把它变得要多难有多难。有些人可能会想，选择题太简单了。但我的意思是，我可以给你出一道非常难的选择题。这其实不是重点。我觉得重点更多在于，选择题限制了你可以提出的问题类型。但在难度方面，选择题没有任何问题。

[30:03.510] 基于考试的问题的主要问题在于，这并不能反映现实世界中的使用情况。除了在HLE上评估时，没有人会向语言模型提出HLE式的问题。大多数时候，人们提出的是开放式问题。这些问题甚至不一定有唯一正确的答案。这些问题可能表述得不太规范，而这些基准测试无法捕捉到这一点。好的，那么问题来了。模型的输出是如何与标准答案进行比较的？所以对于HLE？在计算准确率时，模型的输出是如何与标准答案进行比较来计算准确率的呢？要计算准确率，标准答案是如何比较的。对于选择题来说，就是，你是否选对了答案？所以我们会检查每个选项的概率分布，还是说……语言模型生成C。而我的答案是B或C或A或D，然后我就检查它们是否相同。

[31:16.010] 所以我们只生成一个词元吗？是的，嗯，好吧。那么问题其实就不太关乎评估本身了。而是模型如何生成答案？有几种选择。一种是，你直接采样，它生成一个字母。或者，更常见的是现在你会生成一个思维链。你生成答案，也许还附带一些解释。但提取答案的方式确实很重要。语言模型评估对此可能非常敏感，但我在这里不打算深入讨论。让我们来谈谈另一类截然不同的基准测试，叫做聊天基准测试。到目前为止，我们讨论的是多项选择题任务。它们在捕捉某种智能和难度的概念方面仍然相当不错。但大多数人都不会向他们的AI助手提出多项选择题，除非他们是想用AI系统替他们做考试题，而你绝不会这么做的。

[32:21.270] 有人会这么干。好的，这里有一个部分受现实世界用例启发的例子。我可能会说，我想做一份甜菜奶酪沙拉。哪些香草合适，哪些不合适？好的，这是一个非常开放式的问题。然后你会得到一些回复，同样也是开放式的回复。那么问题来了，你如何评估这个东西？你不能仅仅通过精确匹配来检查。甚至都没有标准答案。所以这里有几个想法。我们可以逐一讨论。其中一个想法是由Chatbot Arena首创的。这个想法是，你询问人类，好的。但他们询问人类的方式，我认为是个巧妙的想法。他们基本上建立了这个网站，任何来自互联网的随机用户都可以进来聊天。然后，与正常的助手只给出一个回复不同，你实际上会从两个不同的、匿名的模型那里得到两个回复，然后用户来评判哪个更好。

[33:38.120] 例如，你输入这个提示，你会得到这两个回复，助手A、助手B。然后你会问，是A更好，还是两者都好，还是两者都差，或者B更好？通过这个过程，你可以获得大量的、关于模型A和模型B之间哪个体更好的成对比较数据。然后你可以利用这些数据来计算ELO排名。我不知道你们有多少人熟悉这个，但为了让大家都能理解。在这些ELO排名中，你将你的评分模型定义为，特定模型击败另一个模型的概率，这个概率是模型B和模型A的ELO分数的一个函数。模型A的ELO分数越大，这个概率就越高。这是一个平滑的映射。然后你用这些参数（即ELO评分）来拟合这个模型，以最大化成对比较数据的概率，好的。然后结果就是，你基本上为你的每个模型都得到一个评分。

[34:55.949] 好的，这就是Chatbot Arena（现称为Arena AI）所做的。你可以得到这个漂亮的排名。我猜Claude Opus在这里表现相当不错，但其他一些模型也在名单上。好的，关于这个设置，有几点是很好的。第一点是你有真实世界的提示。想法是任何用户都可以访问这个网站，而用户访问该网站的动机是他们可以免费使用语言模型。因此存在一个隐含的假设，即他们确实是试图用它来做一些有用的事情。因此，你得到的是真实世界的提示。但有一点你应该记住，这些人是谁？互联网上的随机用户是——我不知道那个分布是什么样的。来到LMSYS网站的互联网随机用户。我不知道那个分布长什么样。我想在他们的论文里，他们提供了一些人口统计数据，但人口统计数据并不能说明全部问题。

[36:07.320] 你可能会担心来自不同模型的各种偏差。可能还有垃圾信息发送者。试图操纵结果的人，甚至可能有人提交了自己的模型，希望他们的模型表现良好。你无从知晓。这有点像狂野西部。还存在另一个问题，即二元偏好——A还是B哪个更好——虽然非常简洁，并且可以输入到这个评分系统中，但它确实混淆了风格和正确性。因此，国际象棋的ELO评分非常有意义，因为唯一重要的就是你赢了吗？但哪个更好就远不那么明确了。而且这里还存在人类作为评判者的问题。输入提示的人正在回答这个问题。这在某种程度上是好的，因为这个人有某种意图，他们可以表明他们的意图是否得到了满足。但在某种程度上并非如此，因为他们提问大概是因为他们不知道答案。

[37:19.670] 所以如果我给你两个答案，他们如何判断哪一个是正确的？此外，还有谄媚的问题，即那些更讨人喜欢的答案可能会比正确但直言不讳的答案获得更高的权重。所以这里存在很多潜在的问题。但这种方法的好处在于，你不需要向所有模型提供相同的提示，而这正是ELO的精髓所在。在国际象棋中，你不必让每个人都与其他所有人对弈。可以只进行稀疏的采样对局。只要图是连通的，你仍然可以推导出排名。这一点很重要，因为是人类在评分，你不能指望一个人去评价所有的模型，甚至不能指望他们评价超过两个模型。还有一个好处是，随着新的提示和新的模型的加入，有一种自然的方式来随时间更新这个排名。

[38:21.030] 那么让我来谈谈另一种方法。这是2023年的AlpacaEval。其中包含了一些源自各种来源的指令。这里的指标是相对于基线模型的胜率。你有一个模型，AlpacaEval会说，我要为这个提示生成回复，使用当时的基线模型（比如GPT-4预览版）生成回复，然后用GPT-4预览版来判断哪一个更好？这立刻就引发了一个关于潜在偏差的问题。但让我们先把这个放在一边。这基本上就是LLM作为评委的一个实例，这是当今评估语言模型非常流行的方法。你可以通过使用多个评委并集成它们的结果等等方式来减轻这种偏差。最初的AlpacaEval的一个问题是，LLM评委偏爱较长的回复。这导致了一系列排行榜游戏，许多经过微调的模型被提交上去，仅仅因为回复更长，就获得了非常高的AlpacaEval LLM评估性能。

[39:41.300] 这个问题在后来的论文中得到了解决，他们使用一种非常简单的回归方法来消除该指标的偏差。好的，但这引出了一个更普遍的问题，这不一定特定于 AlpacaEval，那就是：你如何评估一个指标？我们一直在讨论模型的评估。如果我有一个指标，我该如何用它来评估模型。但如果我要提出一个新指标，我如何知道这个指标好不好？这是一个难题，且没有标准答案。你可以做的一件事，更多是一种健全性检查，就是查看它与其他指标的相关性，其想法是并非在所有情况下越高越好，除非你试图模仿那个特定的指标。例如，AlpacaEval 与 Chatbot Arena 的相关性是 0.98，这相当高。这表明你可以——比如说，你想在 Chatbot Arena 上评估，但你太害羞了不想提交你的模型，或者你不想等待人类评估等，你可以用 AlpacaEval 来代替评估。

[41:07.980] 当然，这种相关性是针对某一特定模型集合而言的。因此，对于比 GPT-4 preview 更强的模型，这种相关性可能不成立。所以在那个时间点，这个排行榜我认为并不是真正——已经一年多没有维护了，但那是它当时的样子。接下来我要谈另一个基准，叫做 WildBench，它从人类与聊天机器人的对话中收集了大量样本。就像 Chatbot Arena 一样，他们也提供了一个免费服务，让人们与这个机器人聊天，然后他们收集了这些查询。然后像 AlpacaEval 一样，他们使用 LLM 作为裁判来评估模型。这里的主要创新是使用了一个检查清单，这个清单是生成的，并且特定于某个提示或任务。其想法是，如果你只是让一个语言模型来判断这个响应好不好。

[42:35.880] 这在某种意义上是一个非常定义不清的任务。我不知道，这取决于你关心什么。所以这里的想法是，通过一个检查清单或某种评分标准，这能极大地限定范围，并使评估任务更加明确。他们这样做了，而且，他们也展示了与 Chatbot Arena 的相关性，诚然，这有点循环论证的味道，因为你可以问，为什么，Chatbot Arena 真的是黄金标准吗？但至少我们是同一条船上的。好的，总结这一部分，如何评估开放式响应？这里没有明确的解决方案，但有一些想法。在参考响应和模型响应之间进行两两比较通常是个好主意，因为尤其对于非常相似的响应，你可以提供方向性信息，比如这个比那个稍微好一点。但就绝对评分而言，比如这是 7/10 还是 8/10？那种评估的信号往往要弱得多。我们应该始终注意偏差，无论是人类还是 LLM 裁判的偏差。

[43:55.580] 他们有不同类型的偏差，但无论如何，这是需要注意的重要事项。也许你能做的唯一一件事就是通过多个裁判来评估，如果人类裁判和所有其他裁判都说你的模型更好——也许你的模型确实更好。最后，我这里要强调的开放式响应的评估问题，确实是定义不清的。因此，我越来越觉得，定义评分标准或检查清单以提高实际评估的可靠性和明确性是非常重要的。无论对人类还是裁判来说都是如此。任何做过众包的人都知道，如果你只是让一个人在没有给出任何评分标准的情况下给某样东西打分，你很可能得到非常荒谬的结果。好的，让我们进入智能体（Agent）的世界。

[45:04.800] 你可以把我们目前所做的看作是评估语言模型说了什么，那是聊天。现在，我们要评估 LLM 做了什么，这就是智能体。对于任何关注 AI 热潮的人来说，智能体无处不在。而且我认为，这些肯定会改变我们对语言模型的思考方式。所以智能体基本上就是一个语言模型加上一些脚手架（Scaffold），脚手架是一种逻辑，用于决定如何调用语言模型、它可以访问哪些工具等等。这里有几个真实的基准我想指出来。其中一个非常流行，就叫 SWE-bench。基本上，任务是给你一个代码库和一个 GitHub issue 的描述，你需要提交一个拉取请求（PR）。评估 PR 的方式是看它是否通过了单元测试。这些测试的设计使得在你提交 PR 之前，某些单元测试是不通过的。

[46:24.710] 然后你的目标是修复它，使得某些单元测试通过，并且不破坏其他任何东西。这里是一个指令的例子。这是 issue。然后这是上下文中提供的一些代码。你的目标是生成一个补丁。给定一个补丁，你会根据是否通过测试来接受评估。这样做的好处是评估非常直接。让我们看看我们在 SWE-bench 上的表现。实际上，这将是 SWE-bench Verified，稍后我会告诉你为什么会这样。SWE-bench 在——我想仅仅是 2024 年，准确率还在 16% 左右，现在它已经上升到 93 了。干得好，Mythos。这是一个非常经典的通用基准，因为对智能体的很多关注都集中在编码能力上。所以 SWE-bench 确实开创了这种思考方式，即如何评估智能体在真实环境中编写代码的能力。

[47:53.800] 这是另一个项目，叫做 Terminal-Bench。这里的想法是使它更加通用。我们将基本上让环境是一个计算机终端。我们将定义一系列可以通过在终端中输入命令来完成的任务。这个环境的好处是非常简单和通用。这些任务是由来自世界各地的 93 人众包的。其中 89 个任务构成了 Terminal-Bench 2.0。这些任务通常需要一个小时到一周以上的时间，具体取决于你是专家还是初级人员。然后，这是截至今天的排行榜情况。顶尖模型是那些前沿模型。有趣的一点是，智能体在这里也很重要，因为你可以有两个智能体使用相同的模型，但它们的准确率不同。

[49:17.690] 我要谈的另一个基准是网络安全。这是 40 个夺旗（Capture The Flag）任务。这里的情景是，你有一个智能体，它被赋予一个环境，它可以运行各种命令，比如查看源代码。它可以访问网络服务器。目标基本上就是侵入这个服务器，提取某个标志（Flag），这是一个独特的字符串，证明你已经入侵了服务器。这些是通常由人类进行的网络安全竞赛和练习。这是一个智能体脚手架看起来是什么样的例子。这是一个非常简化的版本，基本上你有一个连续的记忆。你产生一些行动，获得环境反馈，然后将结果放入这个缓冲区，并将其拼接起来。稍后我会展示，这个历史记录显然会增长得相当快。

[50:35.090] 你将需要有更好的方法来管理上下文。所以这是它发布时的排行榜情况。最好的模型大约在 10% 左右。而随着时间的推移，现在它实际上已经完全被解决了。这就是 CyBench。还有另一类问题是 MLE 工程。所以这个基准测试包含了一系列Kaggle竞赛，涉及处理数据、训练模型等等。智能体可以查看数据集、阅读描述、编写代码、训练模型，最终提交结果并获得评分。这就是目前排行榜的样子。而且大家可以看到，在LLM这边确实还是那些熟悉的面孔。但不同的智能体脚手架（agent scaffolds）之间差异相当大。关于智能体脚手架，我想指出的一点是它们非常重要。

[51:53.330] 所以当我说要讨论语言模型评估时，这其实比语言模型评估本身更宽泛。原因在于，不像我之前提到的那个最早的CyBench智能体脚手架，现在你必须更加复杂精细才能解决真正困难的任务。人们意识到的一点是，显式规划（explicit planning）是有帮助的。你不能只是靠意识流式的思维链（chain of thought）来解决。然后你会积累起很长的上下文，智能体很容易就跟丢了自己在做什么。所以不如维护一个待办事项清单，逐项勾选完成。层级委托（Hierarchical delegation），智能体应该以简洁的上下文调用其他子智能体。那个子智能体做了一些工作后只返回结果。主智能体不需要看到所有繁琐的细节，这有助于实现封装（encapsulation）。

[52:55.900] 记忆（Memory），人们一直在尝试显式地读写文件，尤其是随着上下文的增长。你不可能把所有东西都放在上下文窗口里。最后，还有更多以流程管理为风格的上下文工程。什么时候应该委派给子智能体？什么时候该尝试这个策略而不是那个策略，该把什么写入持久化记忆。这些事情通常是为特定的语言模型进行优化的。所以智能体之所以令人兴奋，是因为它们扩展了语言模型的能力面。脚手架非常重要。所以实际上，当我们在讨论评估智能体时，评估的是语言模型和智能体脚手架两者。现在，我们稍微转换一下话题，来谈谈另一种风格很不一样的基准测试。我暂时称之为纯推理（pure reasoning），因为没有更好的术语。

[54:08.390] 其动机在于，到目前为止的所有任务都需要一定的语言和世界知识。那么问题是，我们能不能把推理或纯粹的流体智力（fluid intelligence）从知识和事实记忆中分离出来？你可以说这能给你一种更纯粹的智力形式。于是就有了ARC-AGI这个项目，始于2019年，目标正是如此。任务设计为人类100%可以解决，但对AI来说具有挑战性。每个任务都是独一无二的雪花——至少意图如此。所以记忆一堆事实或解决之前的问题对你帮助不大。如果你们记得LLM的历史，2019年还是前LLM时代。当时还是GPT-2的时代。所以从这个角度来看，这个项目很有先见之明。这是该问题第一版的示例。你会看到一些图片，然后要猜下一步是什么。

[55:22.700] 如果你花10秒钟，大概就能解出这个问题。你大概会猜到就是填入黄色来形成矩形。然后去年，这个任务有了更新版本，稍微复杂一些。大家可以看一下这个。这是发展轨迹。当ARC-AGI-1刚出来时——我觉得这实际上还是0。基本上模型什么也没做出来。GPT-3这种预训练模型完全没有推动进展，对吧。这正是设计者预想的结果。预训练模型只是学习了互联网上的事实，也许还学到了语言模式。但这些实际上都没有直接帮助，至少对ARC-AGI任务来说是如此。然后2024年发生了一些事情。那就是OpenAI发布了o1和o3模型。之后进展开始非常突然地起飞。现在ARC-AGI-1基本已经被解决了。

[56:46.430] 甚至在2025年左右ARC-AGI-2发布时，它也——还没有完全解决，但看起来也在被解决的路上。真正解锁这一切的是推理能力。现在人们不禁要问，预训练中的那些知识对于这个任务是不是必要的。当然，没有预训练我们也不会有推理模型的爆发。所以可以说预训练仍然很重要，但在ARC-AGI的大部分历史中，这一点可能不是直接可见的。然后在就在上个月，ARC-AGI-3发布了。你们可以看到这个熟悉模式。创造者看着现状就想，哦，是时候出个新基准了。这现在是一个交互式环境，你可以上网去打这个游戏。同样，没有语言。你只需要根据模式来推理。还挺好玩的。

[57:55.320] 而现在的分数非常低。所以明年我教这门课的时候，我肯定得更新这页幻灯片。但走着瞧吧。我觉得ARC-AGI这条线非常值得思考。目标是分离推理和知识。这真的很难做到。而且这不一定——我觉得这可能是这方面最好的尝试了。目前还不清楚是否真的能完全解耦这些因素。而且当然，如果人们真的很在意这个基准测试，我觉得可能人们确实可以——人都可以对任何东西做基准测试。所以你可以设计类似这样的问题——这些任务仍然来自某种先验背景，不确定是否存在所谓的"纯粹"推理。这个方法也局限于人类推理。其明确目标是在合理时间内100%由人类可解。

[58:55.780] 所以这并不真正延伸到超人推理，比如说拿更多金牌或者解决开放数学问题，这些可以说仍然非常有用和重要。但很明显，这暴露了当前模型的一些短板。所以如果你把目标视为解决所有任务，那还有很多工作要做。这个ARC-AGI-3尤其看起来非常图形化。那么这里面涉及了多模态内容吗，还是有编码或文本表示？对，问题是这个看起来非常图形化。那语言模型的输入到底是什么？这本质上——我不确定——我觉得是64乘64的。所以你可以提供这个图片，也可以用ASCII艺术或某种文本表示来提供。不管怎样，这里都有一个空间元素需要模型去推理，而这确实不是英语或自然语言。好了，我们再换个话题，谈谈另一类与安全相关的基准测试。

[01:00:21.040] 关于语言模型中的安全，人们已经讨论了一段时间。如果看看汽车等其他领域，安全在那里意味着什么很清楚。汽车的安全评级，就是通过把车撞墙、用安全气囊等方式测试，看它对车内假人的保护效果如何。而这些是经过了几十年的游说和探索，才确定了车辆安全意味着什么。那么对AI来说安全意味着什么？我只能说目前没有很好的答案，但我会给大家举一些人们一直在思考的例子。有一个叫Harm-Bench的基准测试。这实际上就是用那些意在造成伤害的内容去提示模型，并期望模型基本上能够拒绝。所以这是一种安全类型。我认为当人们想到安全时，最主要的事情是我想防止坏行为者用语言模型做坏事，但事实证明这并不是唯一重要的事情。

[01:01:38.080] 所以还有另一项工作，即 AIR-Bench，它试图通过审视欧盟、中国和美国公司政策中的所有监管框架，从整体上思考安全问题，并建立一个涵盖所有可能出错之处的分类体系。然后利用这个体系，构建一组特定的提示，进行评估，诸如此类。但安全的一个问题是越狱，这本身就是一个子问题，即语言模型通常被训练来拒绝有害指令。但如我们所知，如果你足够聪明，你可以绕过这一点。所以有一些早期工作使用自动化方式来优化提示以绕过安全限制，使用的是 GCG，这基本上是一种坐标逐维优化算法。值得注意的是，在开源模型上优化能够迁移到其他模型上。所以我希望这些攻击不再有效。但在当时，你可以生成一些基本上是乱码的东西，加上逐步毁灭人类计划，OpenAI 的模型就会照做。

[01:03:10.240] 显然，你可能会争辩说，这真的是有害的毁灭人类计划吗？至少它没有——预期的行为是它可能应该拒绝。所以这引出了一个更大的问题，即什么是安全？让这件事变得棘手的一点是，安全的许多方面都是非常依赖语境的。所以这涉及政治、法律、社会规范。这在不同国家之间有所不同。而且，如果你思考安全，你必须考虑风险是什么。而这些风险实际上也相当多样，可能需要不同的关注度。所以幻觉是一种风险，尤其是在医疗、法律或金融场景中。但这也与能力相关。如果你让模型更准确，这与减少幻觉是相关的。谄媚是一个问题。教唆犯罪是一个问题。但这可能与能力相悖，不平等。失去批判性思维也可能是一种风险，而且与能力相关。所以如果你思考语言模型对世界的社会影响，并将安全定义为确保人工智能对人类有益，这实际上是一个比我在这里有时间讨论的要复杂得多的话题。

[01:04:48.800] 最后，语言模型还有一个双重用途的方面。网络安全智能体可以被用来入侵系统，或用于渗透测试并让系统更安全，这使得讨论这些到底是安全风险还是实际收益，变成了一把双刃剑。好的，那让我们继续，来讨论一些关于评估的更广泛的考量。我认为评估的一个重要方面是真实感或生态效度。评估在多大程度上捕捉到了现实世界的使用？特别是，考试与现实世界使用相去甚远。Chatbot Arena 来自真实用户，但人们可能会怀疑这个分布是否是正确的人群或用例分布。我会提到几个试图触及生态效度的基准，特别是在用例层面，至少不是在单个查询层面。

[01:06:00.400] OpenAI 创建了一个名为 GDPVal 的基准。他们考察了根据美国 GDP 排名前九的行业，44 个不同的职业，找了一群专业人士来创建任务。这些专业人士大约有 14 年的经验。你可以看到有护士、礼宾员、房地产经纪人、影视剪辑师等等。他们创建了一堆任务。这里还有一个例子。在医学领域，直到那时，很多医学基准都基于标准化考试。事实上，对人类来说，这也是同样的情况。你参加这些标准化考试，从医学院毕业，然后你去做手术。嗯，实际上，还有一个更长的过程。你不是直接就给病人做手术。但重点是，对于一个语言模型来说，你也不应该只是通过医学考试就直接部署它们。所以这是一个项目，其中 121 个任务来自 29 位临床医生。

[01:07:21.610] 这意在代表临床医生实际会向语言模型提出的那类问题，这与多选题式的考题相当不同。这里还有另一个涉及真实数据的项目。如果你问，谁真正拥有人们实际行为的数据？嗯，是模型开发者。所以这个项目很有意思，因为它使用语言模型本身来分析真实数据。特别是，出于隐私考虑，你不能直接去看人们的数据，但你可以让语言模型查看数据，分析并总结一般性模式。然后语言模型就能告诉你，人们用 Claude 做哪类事情——是什么样的人在做什么样的事。有时候真实感和隐私是相互冲突的，因为理想情况下，你确实想要来自实际查询流的一个样本。

[01:08:31.609] 那对于评估模型运作情况以及深入理解错误会非常有价值。但这又会遇到隐私考量。好，另一个考量，这是另一种效度，是一般而言，这些评估在科学上有多有效？这一点很早就被提出来了，即训练集与测试集重叠，或者说数据污染。在机器学习入门课程中，你会学到不要在测试集上训练，希望如此。在基础模型出现之前，这一点其实非常明确。对于 ImageNet、SQuAD 以及许多基准，抱歉，是数据集，总是有一个训练划分和一个测试划分，每个人都遵循同样的规则。你在训练数据上训练，在测试数据上测试。生活很简单。而现在，你有这些在互联网上训练的模型。而且远不止于此。更重要的是，你不知道数据里有什么。所以这让评估变得相当不同，我认为是这样。

[01:09:44.520] 那你对此能做什么呢？首先，你可以尝试推断一个模型是否真的见过测试数据。这是 Tatsu 团队的一个巧妙想法，利用的事实是基准中问题出现的顺序应该是随机的。但如果模型偏好某种与基准一致的特定顺序，那很可能——它就是在那个基准上训练的。另一种途径是，你可以鼓励人们报告实际的训练集与测试集重叠情况。这与规范有关。我认为总体而言，也许尤其是在机器学习中少一些，但在统计学中更多，你总是会报告置信区间。你有一个估计值。我总是想知道，这个估计值有多可靠？这只是标准做法。所以这篇立场论文的想法是，模型提供者应该——如果你要声称某个 GPQA 分数，你应该总是提供一些你没有在测试集上训练的证明。

[01:11:16.750] 另一种途径是直接放弃。就像，好吧，所有这些评估就在那里——我们就假设人们在上面训练过，我们就假设最坏的情况。我们这边的有利条件是你总是可以定义新的评估。所以有一堆想法，比如 LiveCodeBench、UncheeatableEval，基本上就是抓取新的网页或归档内容。论文或GitHub仓库，并定义了那些在语言模型截止日期之后的数据集作为评估群体。所以这看起来很稳健，也很好。但有一点要记住，时间戳也并不总是安全的，因为比如我可能会发布带有代码的内容。我可能会把一些东西放到GitHub上，而这些东西实际上源自于其他某个仓库。所以这通常算是比较新鲜的吗？方法四，是使用私有评估集。这也是公司模型开发者们会采用的做法，对吧。如果你是谷歌或OpenAI，你会有自己的内部代码库，你假定这些代码不在互联网上，而且你显然不会用这些数据来训练模型，

[01:12:34.840] 所以你可以用它来进行评估。这通常来说非常可靠。如果你不是公司里的，你可以用自己的个人写作内容。我读研究生时有一堆被拒稿的论文，从未放到网上，所以我可以拿它们来用。我认为这种评估方式对于困惑度评估来说非常有利，因为你所需要的全部就是就是你只需要一个好的数据集，就可以评估对数概率了。那我还要提的另一个问题是数据集质量。还记得我展示的SWE-Bench Verified排行榜吗？这是因为，嗯，SWE-Bench原本有一堆任务和单元测试的问题，有些测试不够严谨，还有其他一些问题得到了修复。还有很多其他基准测试也经历了这样的过程，比如GSM8K或MMLU，人们进行了更仔细的审核，然后意识到，等等，这个问题实际上无法——这个问题上有些地方说不通。比如这个例子。

[01:14:04.090] 这段文本中没有给出曲线，所以你无法回答这个问题。宝宝穿袜子了吗？不，根本看不出来。还有一篇论文指出，智能体基准测试中存在许多问题，这些基准可以说更难评估，因为你不能只看问题和可能的答案，你可以查看它，但整个环境是很复杂的。在编程方面，有些情况下有通过或不通过的测试用例是很好的。但正如我们都知道的，测试用例可能不完整。所以你可能通过了所有的测试用例，但你的解决方案仍然是没法用的。还有一个例子是TorchBench基准测试，他们表明如果一个智能体只是输出空响应，就能得到38%的分数。所以这些基准测试中存在一些琐碎的问题，你必须非常小心地避免。我还要指出一个工具叫Docent，它使用语言模型来检查智能体的轨迹，以检测问题。

[01:15:17.480] 这算是对我们这种非常定量化的基准测试方法的一种定性回应。我始终建议，无论何时你做一个基准测试，无论是开发一个新基准，还是在一个基准上运行模型，你都应该查看输出并尝试审计，以确保你真的认为你测量的是你想要测量的东西。好的，现在可能更哲学一点。那么评估的意义是什么？好的，我讲了很多不同类型的评估。但实际上，没有一种评估能适用于所有情况。我认为当你考虑评估时，你必须非常清楚其目的是什么。而且我认为这通常没有被清晰地表述出来。例如，你可能是一个用户，或者一个试图做出购买决策的公司。在某个特定用例中，你会选择A还是B？或者你只是一个研究者，对智能有一个直观的概念，想以某种方式去衡量它。

[01:16:22.330] 或者你想要出于商业或政策原因来理解益处和危害。或者你是一个开发者，想获得反馈来改进模型。所以这些不同的目标会引导你选择不同的基准，或者某种评估策略的组合。最后，需要指出的一点，我之前也简单提过，就是我们到底在评估什么？在基础模型出现之前，研究者评估的是方法，因为我们有固定的训练测试划分。唯一变化的是实际的算法本身。而今天，我们主要在评估模型和系统，在这个领域一切皆可变。当然也有些例外。比如NanoGPT speedrun，它刻意采用了一种评估算法的方式，特别是评估你训练模型的速度有多快。但大多数语言模型评估是关于评估最终的模型本身，这很有用，因为这才是真正会被发布，

[01:17:25.100] 且人们将要使用的东西。但我认为最关键的一点是，你应该始终非常明确地宣布你的目标是什么。好的，所以没有唯一的真正评估标准。选择你想要衡量的东西吧。我们讨论了各种不同的基准，从困惑度、基于考试的、对话的、智能体的、推理的，到安全性的。它们在难度、真实性、有效性方面各不相同。而且经常存在权衡取舍。很难做到既非常真实，又非常困难，同时还具有生态效度。而且也没有能完全避免测试污染的权衡。所以通常你必须在这些因素中的一个上做出妥协，而根据你的目标，你将不得不选择你愿意妥协的那一个。那么这节课就到这里了。下周，我们将开始讨论训练数据。好，下次见。[音频标志]

CS336 2026 · Lecture 12 · Evaluation

视频：官方视频
字幕：en-US · manual captions
处理：连续字幕合并为自然段；每段保留首条字幕的起始时间戳
状态：原语言字幕稿，待校对

正文

[00:05.720] So today, we're going to talk about evaluation. So far in this class we've done everything we need to train a language model. We defined the architecture, the optimizer, the main training loop. We showed how to make the training fast with kernels, parallelism. And then we saw scaling laws. We saw a little bit of how to make inference fast as well. So the only missing piece now is talking about the data that you train on. So we're going to spend next week talking about that. And data is going to shape the model behavior. If you train on code, then presumably your model is going to be good at code. If you train only on DNA sequences, it probably can't speak English. But before we even get to data, we need to talk about what behavior we want from a model, and that's going to be the topic of evaluation. So evaluation asks a very simple question-- given the model that we've trained, how good is it? So what is good? So evaluation might seem like a fairly mechanical process. You define some prompts, you send these prompts to your model, you get back some responses, and then you compute the accuracy.

[01:25.470] So why do we have a whole lecture on evaluation? But actually, evaluation is a very deep topic, and it's also very important because evaluation is what shapes the development of AI. Evaluation sets North Stars and model developers-- open, closed-- everyone looks at evaluation as a measure of progress. So by thinking carefully about your evaluation, you implicitly shape the development of what your model is going to be able to do. So what's hard about evaluation is the following. So often, you start it with an abstract construct of what do I want my model to do? Maybe I want my model to be good at conversation or good at reasoning. OK. So those are abstract concepts. And then evaluation is really the process of turning that into a concrete metric powered by, presumably, some concrete prompts or environments. So this is going to be the core challenge, and we'll see many examples of how this process of abstract construct, concrete metric

[02:36.190] shows up. So let's go back to a question of what makes a model good. Just kind of superficially. Maybe a model is good if it does well on benchmarks. So there's this website, Artificial Analysis, which has become a bit of a standard for thinking about quote unquote, "intelligence of models." So if you go to Artificial Analysis AI, you can see a ranking of different models from GPT-5.5, all the way down-- well, there's many more models after this point. And so this is a measure. It includes a bunch of different data sets, some of which we'll talk about. Or maybe a model is good if it does well on benchmark and is also cheap to run, because cost should matter.

[03:26.160] And this plot shows both the intelligence index from before, but against how much money it costs to run a particular model based on inference cost. And you can see, of course, as you might expect, that the two are correlated. The more you pay to run, the higher the intelligence index, but it's not exactly aligned. But maybe a model is good if people just prefer it. Later we'll also go a little bit more into depth. So this is the take that Arena AI, formerly known as Chatbot Arena, takes. And you can form a ranking based on models, based on whether people like it or not. Or maybe a model is good if people simply choose to use it, and therefore, pay for it. So OpenRouter is this service that allows one to use multiple different models through a central endpoint. And as a result, they collect a lot of statistics of what models people are using.

[04:34.410] And they publish statistics, which is really interesting to look at, the top bottles that are used through OpenRouter. So now, this is not representative of all model usage, but it gives you a sense of what people on open router are doing. And so this is kind of more of an economic lens. I don't what good is, but if people are paying for it, it must be good. OK. So none of those are necessarily the correct answer. I don't know If there is a correct answer, but I'm just trying to get you to think about different ways of thinking about what good is. So let's start now by looking at different ways, concrete ways, of evaluating language models. And there's no better place to start than going back to the core idea of a language model, which is that it's a distribution P of x over sequences of tokens. And through that lens, there's a very natural way to evaluate a distribution, which is a perplexity, or likelihood or log loss.

[05:46.540] Those are all kind of related concepts. And it basically says, I have a test data set d, and I'm going to see how much probability mass my language model assigns to that d. And I normalize it and divide so that the numbers are more interpretable. But that's basically it. How much mass does my probability language model assigned to a data set. OK, so this is fairly natural. In fact, when we were doing training, we minimized the perplexity on the training set. So the obvious thing to do is measure the perplexity on the test set, and this is what people traditionally did in language modeling research for many years throughout the 2010s. If you pick up a language modeling paper, this is what they did. And back then, standard data sets included the Penn Treebank, WikiText-103. There was also this One Billion Word Benchmark, which was taken from machine translation sources. And this is all in the classic paradigm, which is in-distribution of evaluation.

[06:58.750] You train on some train split of that data set, and then you evaluate on some test split of that data set. And people actually made a lot of progress, which was just measured in perplexity reduction. So for example, there's this famous paper from 2016, which showed that you could apply CNNs and LSTMs to this One Billion Word Benchmark and get a huge perplexity reduction. And bear in mind, around that time, people were talking still about n-gram models and maybe hybrid models. And this was the first definitive result that showed just pure neural is clearly the way to go. OK, so life was very simple back then. You train on the train split. You test on a test split for perplexity. So then when GPT-2 came around in 2019 from OpenAI, they trained on the data set called WebText, which was 40 gigabytes of text websites that were linked from Reddit. And the way they evaluated was actually different.

[08:10.920] They evaluated zero-shot on standard data sets that everyone else was evaluating on. So this is out-of-distribution evaluation. So this is a table from their paper, which show that as they train different models. So the biggest model is 1.5 billion parameters. And they were able to show quite a bit of improvement, especially on the small data sets. So for example, PTB, which is a tiny data set. They were getting 35 perplexity compared to the State of The Art, which is 46. Now, if you have enough in-distribution data, you don't quite do as well as the SOTA. But nonetheless, this is kind of impressive, given that this model was not trained on One Billion Word's data set at all. I don't know. Maybe there was some overlap, because I don't know if they did a careful train test decontam. And so it ushered in a new paradigm of thinking about language models as not just the in-distribution setting, but how we wanted to train on a large data set

[09:31.480] and evaluate on standard benchmarks, which nowadays I think has been very common standard. But back then, it was seen as a bit of a novelty. I will also argue, since we're talking about perplexity, that perplexity is you all need. I wouldn't to take this too seriously, but I think it is a mindset, I think, that drives a lot of language modeling research. And the basic argument goes like this. So there's some true distribution out there-- let's call it t-- and you're training this model p. So what is the best perplexity you could obtain? Well, the best perplexity is the entropy of that true distribution, and that is realized when p equals t. So by minimizing perplexity, you're basically pushing down the perplexity. And the only unique minimizer is when you actually get the true distribution. And when you have the true distribution, you're done. You just model everything. You can solve all the problems, and you can do things like condition on the problem, generate a solution.

[10:52.000] You can condition on a question, generate the answer. So by pushing down on perplexity, we'll eventually reach AGI, is the argument. I mean, this might sound, I don't know, either a bit fantastical. But in some sense, this is really, I think, a strong driver of many people who have been scaling and scaling language models at a time when, maybe, the gains weren't as obvious. Before GPT-3, I think it wasn't clear that language models were going to have such a huge impact. And it was really this belief that, if you drove perplexity down, good things will happen. So of course, Perplexity is maybe not all. And one way to frame it is that perplexity is maybe more than you need. So for example, if you have a sentence like "Stanford was founded in 1885," perplexity will say, penalize the prediction on all tokens. So in particular, it'll say, what is the probability of 1885, given the rest? And that token is probably pretty useful.

[12:07.980] It's basically a Q&A example, phrased as sentence completion. So it captures knowledge about the world. But other ones-- for example, the first word of a sentence, or even "founded"-- maybe not that interesting, I think. And perplexity doesn't care. It's going to charge you bits for every bit of a deviation from the true distribution. So there is a bit of a solution here, which is you can always measure the conditional perplexity. You can condition on some prompt, and then you can measure the perplexity of the remaining response. So that allows you to, in general, focus on certain tokens that you believe to be relevant and less on others that are sort of incidental. OK. And also, it's important to remember that some benchmarks, even though they are not explicitly a perplexity measure, are actually just perplexity in disguise. So, for example, if you think about closed tasks. There's this lambada paper from, actually, a while ago, 2016, which is basically fill in the blank.

[13:29.650] And you have a context, you have a sentence. "Do you honestly think I will want you to have a blank." And the goal is to figure out that word. And even though this is measured in terms of accuracy, it is really a next-token prediction problem. And here, it's not perplexity of all the words, it's carefully chosen. If you read this paper, it's really carefully chosen to be words, such that you actually need a bunch of long-contexts to resolve. So this is actually something that the early GPT papers really latched on to because they were interested in long-context modeling, thinking that that was a key to unlocking reasoning and all these things. And really, by focusing on positions where, to resolve that, it requires long-distance dependencies, is a way to sharpen the perplexity to some phenomenon that you care about. Here's another popular classic data set that you see in some language modeling vowels.

[14:46.890] HellaSwag. So what is HellaSwag? They went to various different data sets and sources, and came up with-- it's multiple choice, but it's really sentence completion. And at some level, it's perplexity. So here's a sentence. "A woman is outside with a bucket and a dog. The dog is running around. She blank." And then you're trying to fill in which sentence completes it the best. So it's not exactly perplexity, it's multiple choice. But in some ways, it's perplexity. So one word of warning. If you ever decide that you're sold on perplexity as, really, the measure and you're going to launch a leaderboard and have people compete against perplexity, just a word of caution is that, what is a perplexity leaderboard look like? So people are going to submit a language model to you, and you have to compute a logprob. You send them the test data, and they give you back a logprob. Well, you look at that and say, hmm, that might be a little bit fishy because you need to trust that the probabilities they hand back

[16:01.650] to you are valid and that they sum to 1. Otherwise, I could just have implement LM, and it always returns 1 or, I guess, logprob 0. And I would get very good perplexity, but clearly, that's not a valid distribution. And so with this contract it's not really easy to check that someone is actually internally having a log probability, so there's some trust. Maybe you have to look at the code or something. For downstream tasks, this is much more straightforward because you can have a black box LM. I give you a prompt, you give me a response, and I can just use that response and calculate some accuracy. So perplexity is fundamentally different because it deals with distributions, and you have to be a bit careful. And not to mention, there's lots of problems with autoregressive models. But if you have VAEs or there's some models where you can't even compute the distribution-- you can compute only a bound-- then you have to really actually trust the math as well that this is actually a valid bound.

[17:11.060] OK. So The summary of perplexity is that it is actually still used quite heavily in language model development. It has some nice properties. We saw that it's generally used for scaling laws because it's also smoothly varying with scale, which is important for getting scaling laws. But, at the end of the day, especially if you're not a believer, then you need some benchmarks to really capture real world situations, to convince people that your language model is good. For the believers, if I show you a remarkably slow perplexity, they'll be convinced. OK, any questions about perplexity? OK. Hopefully, everyone's a believer now. OK, so let's talk about exam benchmarks. So exams, as all of your students know, is a way that we use to test humans. And so you can adopt the same mentality to test language models.

[18:30.730] The nice thing about exams is that you have a very careful control over the subject, over the difficulty. You can design them to have unambiguous correct answers, which makes them easy to grade. So there's a lot of nice properties about exams. And so, for this reason, a lot of LM benchmarking culture was based on just this idea of exams. So obviously, there's been a lot of benchmarks, but I'll just highlight some of the ones that I think have been influential in the development of language models. So this one is MMLU, or Massive Multitask Language Understanding from Hendrycks et al, 2020. And at that time, it wasn't clear that language models were really like general purpose task solvers. This was around the time when GPT-3 came out. Many people, I think, still thought of language models as, well, language models. You generate fluent English.

[19:36.340] But they had this idea that, well, let's just try to be ahead of the curve here. We're going to go through 57 subjects, we're going to have a team of students scour the internet for different sources, and really put together this very comprehensive data set. And despite the name MMLU, which has the word Language Understanding, it's really about testing knowledge and reasoning and less so just pure language understanding. And the way they evaluated it was with GPT-3, using few-shot prompting. So they essentially said, I have formed this prompt. The following are questions about high school mathematics. Some in-context examples of question, answer, question, answer, and then the final question, and then the model has to predict the answer. So this seems very mundane. But at that time it was, I think, pretty radical that you would construct this fairly complicated question and with in-context examples and everything, and expect a language model to actually do something reasonable. And sure enough, at the outset, the models were not great, I would say. Some of the small models were barely above chance. And the larger model-- this GPT-3-- was actually well above chance, so this was kind of encouraging.

[21:13.910] And then over the years, as we know, this benchmark has essentially saturated. So this is a site I don't think this has all the data in it, but about 3.5 Turbo. This was like-- I guess, beginning of 2024-- oh, '23 actually-- to GPT-4. And then, now it's in the 90s. So there's also a link you can follow up later on, where you can actually browse some of the predictions of these models, which is always fun to see what models actually goof up on and what they get right. OK. So around 2024, when it seemed like MMLU, there was a lot of progress made, it was like, OK-- you're going to see this theme again-- OK, this benchmark is getting too easy. Let's try to make it harder. Well, first of all, they actually noticed that there was some noisy and trivial problems, so let's get rid of them. Four choices might be a little bit too easy, so let's expand that to 10. At that point, it was much more in vogue to use chain of thought to solve these benchmarks, whereas when GPT-3 came out, that wasn't really a thing.

[22:46.300] And it's clear that some of these questions could benefit from a few reasoning steps. So this new benchmark was harder. So now the accuracy of the model is back at 33. But as you can maybe see from the progress so far it started low. And now it's back to I guess it's like 88 almost 90, OK. So this benchmark might be a little bit too easy as well. OK, so there's another benchmark called GPQA, which stands for Google-Proof QA. And the idea here is that OK, how can we get a really good benchmark? And the idea is that, well, if you can solve a problem by looking at Google. Google has the internet and a language model is trained on the internet. So if you can just solve it then that's probably too easy. So they got some now PhD level. So I think MMLU was think about as actually-- maybe some of these were like undergrad but maybe grad level.

[24:00.050] But now, it's more explicitly about questions that are meant to be pretty hard. And at this point now you say, OK, how do I come up with really hard questions? You actually have to go through quite a bit of human labor to get really hard questions. So this was 61 PhD contractors from Upwork. Someone writes a question. It goes through an expert validation which then provides feedback on the questions. And then the question writer goes and revises the question. And another expert looks at the question. And then you have the question. And then you send it to some non-experts who are given Google access. And then you see if they can solve the question. So there's a subset of these questions called the DIAMOND set that you might see in people's leaderboards where if the two experts agree and at one of the non-experts is able to answer the question, then that's deemed a good kind of DIAMOND question.

[25:01.240] So this is pretty rigorous review process. And PhD experts were only able to get 65%. So this is I think mostly four-way multiple choice. So this is above random. But this is not a good grade if you're getting 65% on an exam. And it's not experts, even if they had 30 minutes with access to Google, they were getting a bit over chance, but not much. And at that point GPT-4 was like 39%, and let's see how it's doing now. So GPQA is now again 94. So these benchmarks have a shelf life of not too high I guess. I will say that some of these benchmarks, even though they saturate the larger models, they're still useful for developing smaller models or fitting scaling laws and so on. So then, yeah. How do you know that when these tests are being used for training and actually your models are communicating with these? Yeah, so this is a very important question. So how do you know that these questions aren't just in the training set? So the short answer is we don't know because I don't know what's in the training set.

[26:37.150] I'll come back to this contamination point a little bit later and ways of addressing it. But you should always-- I'm glad you asked this. You should always take these numbers with a little bit of a grain of salt, because it's only trust. Maybe just to say one more. I think train contamination is a little bit subtle. Because it's probably not the case that they literally train on the set. But questions can be derived from other sources. And those sources could be trained on. So often contamination is a much more subtle process than literally accidentally training on the test. Was there another question? Same question. OK, great. Good always good to be skeptical. OK, so the final exam benchmark I'll talk about is something ominously called Humanity's Last Exam. This is from last year. And they were saying, OK, well these models are getting so good.

[27:39.500] Let's just give it all we can to create something that's really tough. So this is multimodal, many subjects. It's still multiple choice. And I guess some short answer. So this was crowdsourced. They had some money to incentivize people who were into money to create questions. They also incentivize with authorship for those who are into that. And they went through multiple stages of reviews filtered by frontier models. There's a whole process. Another answer to your question is that they also held out a private set that was not released publicly, so that they could make sure that this did not enter training. And then you just hope that the APIs that you-- because you still have to send this to the model, that those prompts don't end up in the training. But let's hope that's the case. So every data set paper looks like this. Previous benchmarks, look the models are doing so well.

[28:44.640] My data set, look all the models are doing really poorly. So this is a common pattern. And this was especially single digits. It's pretty abysmal here. And let's see how we're doing in 2026. So this one is-- it still has some room. So it's only at-- even Mythos is only at 64.7. So this is indeed a pretty-- it seems like a pretty hard exam. OK, so to summarize this section, as you can see, we've trained it towards harder and harder questions as the models improve and we saturate existing benchmarks. It's interesting that multiple choice format still survives because it's-- with multiple choice, I mean, you can make it as difficult as you want. So some people think, well, multiple choice is too easy. Well, I mean, I can give you a very hard multiple-choice question. That's not really the point. I think more the point is that multiple choice restricts the set of questions you can ask. But in terms of difficulty, no problem with multiple choice.

[30:03.510] The main problem with exam-based questions is that this does not capture real-world usage. No one asks HLE questions to a language model except for when you're evaluating on HLE. Most of the time people are asking open-ended questions. They don't even necessarily have a correct answer. They might not be well formed, and these benchmarks do not capture that. OK, so question. How is the model output compared with the ground truth answer? So for HLE? For accuracy, how is the model's output compared with the ground truth answer, to calculate the accuracy? To calculate the accuracy for how is the ground truth compared. So for multiple choice, it's just like, did you get the right answer? So will we check the probability distribution for each option or how is-- So the language model generates C. And my answer is either B or C or A or D, and I just check if they're the same or not.

[31:16.010] So we are only generating one token? Yes, well, OK. So then the question is less about evaluation. But how does the model generate an answer? So there's a few options. One is that you generate-- literally you sample and it generates a letter. Or more commonly now you generate a chain of thought. You generate the answer maybe some explanation. But there is a way of extracting an answer which does matter. And language model evaluations can be very sensitive to that but I'm not going to talk about that here. So let's talk about a very different class of benchmarks called chat benchmarks. So far we've talked about multiple-choice tasks. So they're still pretty good at capturing some notion of intelligence and difficulty. But most people don't ask a multiple-choice exam question to their AI assistant unless they're trying to use their AI system to do their exam for them, which you would never.

[32:21.270] One does that. OK, so here's an example semi-inspired by a real world use case. So I might say I want to make a beet salad with cheese. What herbs would work well and what wouldn't work well? OK, so this is a very open-ended question. And then you get some response, which is, again, an open-ended response. And then the question is, how do you evaluate this thing? You can't just check exact match with equality. There's no ground truth even. So there's a few ideas here. We can go through them. So one of the ideas was pioneered by this Chatbot Arena. And the idea is, well, you ask humans, OK. But the way they ask humans, I think is a clever idea. So they basically had this set of this website where any random person from the internet can go in and chat. And then instead of a normal as assistant in getting one response, you actually get two responses from two different models which are anonymized, and then they rate which one is better.

[33:38.120] So for example, you put in this response, you get these two, assistant A, assistant B. And then you ask, A is better, both are good, both are bad, B is better? So using this process, you can get a bunch of pairwise comparison data of model A, model B, which one is better? And then you can use this to compute ELO rankings based on this. So I don't know how many of you are familiar with this but just get everyone on the same page. In these ELO rankings, you define your rating model as a probability of a particular model being another [INAUDIBLE] model as some function of the ELO scores of model B and model A. So the larger model ELO score of A is the higher the probability. So it's a smooth mapping. And then you fit this model with the parameters, which are the ELO ratings to maximize the probability of the pairwise comparisons, OK. And then as a result, you basically get a rating for each of your models.

[34:55.949] OK, so this is what Chatbot Arena nowadays known as Arena AI did. And you can get this nice ranking. I guess Claude Opus is doing pretty well here but some other models are on the list as well. OK. So there's a few things that are nice about this setup. One is that you have real-world prompts. So the idea is that anyone can come to the site, and the incentive for someone to come to the site is that they get free access to a language model. And so there's this implicit assumption that they're actually trying to use it to do something useful. And therefore you get a real-world prompt. But one thing you should keep in mind is, who are these people? Random person on the internet is-- I don't know what that distribution. Random person on the internet who comes to La Marina. I don't know what that distribution looks like. I think in their paper, they have some demographics, but demographics don't tell the whole story.

[36:07.320] You could worry about different biases coming from the different models. Maybe spammers. People are trying to game, who maybe even submitted a model and they want their model to look good. You just don't know. It's a little bit of a Wild West. There's another issue which is that the binary preference, which is better, A or B is very nice and simple and feeds into this rating, but does conflate style and correctness. So ELO ratings for chess makes a lot of sense, because the only thing that matters is, did you win? But which one is better is much less clear cut. And there's questions of the human is the judge here. The person who puts in the prompt is answering the question. And this is good because in some level, because the person has some intent and they can declare whether their intent was met. To some extent, except for they're presumably asking the question because they don't know the answer.

[37:19.670] So if I give you two answers, how are they to judge which one is correct? And then there's also the problem of sycophancy, where maybe more pleasing answers are going to be upweighted highly compared to correct but honest answers. So there's a lot of potential issues here. But the nice thing about this method, which is that you don't need to feed the same prompts to all the models, and this is the beauty of ELO. In chess, you don't have to play. Everyone plays everyone else. It can just be some sparse sample of that. And you can still derive rankings as long as the graph is connected. And this is important because human is rating and you can't expect a human to rate all the models or even more than two models. And the nice thing as well is that as new prompts and new models come in, there's a natural story for making this updating over time.

[38:21.030] So let me talk about a different approach. So this is AlpacaEval from 2023. There were some instructions derived from various sources. And the metric here is the win rate against a baseline model. So you have a model, AlpacaEval says, I'm going to generate it for a prompt, generate using let's say baseline model GPT-4 preview at that time and then judge using GPT preview, which one is better? So immediately, this raises a question of potential bias. But let's put that aside for now. So this is basically an instantiation of LLM as a judge, which is very popular these days for evaluating language models. And you can mitigate this bias by having multiple judges and ensembling and so on and so forth. So one problem with initial AlpacaEval is that LLM judges favor long responses. And this led to a bunch of leaderboard gaming where a bunch of models that were fine tuned were submitted and they got really high LLM AlpacaEval performance just because the responses were longer.

[39:41.300] So this got fixed in a subsequent paper where the use a very simple regression method to de-bias the metric. OK, but this raises a more general question here, which is not necessarily specific to AlpacaEval which is, how do you evaluate a metric? So we've been talking about evaluation of models. And if I have a metric, how you can evaluate a model. But if I'm coming up with a metric, how do this metric is any good? So this is a hard problem. And there's no answer here. One thing you can do is more of a sanity check is you can look at correlation with other metrics with the idea that higher isn't necessarily better in all cases unless you are trying to mimic this other metric. So AlpacaEval correlation with Chatbot Arena is 0.98 which is quite high. So which suggests that you could-- let's say you wanted to evaluate on a Chatbot Arena, but you're too shy to put your model or you don't want to wait for humans or whatever, you can use AlpacaEval to evaluate instead.

[41:07.980] Now, this correlation, of course, is with respect to a certain set of models. And so this might not hold for models that let's say are stronger than GPT-4 preview. So at this point, this leaderboard I don't think is really-- hasn't been maintained in over a year but this is what it looked like at that time. So there's another benchmark I'll talk about, which is WildBench and this one was sourced a bunch of examples from human-chatbot conversations. So like Chatbot Arena, they also put up a free service where people could chat with this bot and they collected the queries. And then like AlpacaEval, they used LLM as a judge to evaluate the model. The main innovation here is the use of a checklist, which is generated and specific to a particular prompt or task. And the idea here is that if you just ask a language model to judge whether this response is good or not.

[42:35.880] This is in some sense a very ill-defined task. I don't know what depends on what you care about. And so the idea here is that by having a checklist or some of rubric, this greatly scopes and makes the task evaluation task more well-defined. So they did this, also, they showed correlation with Chatbot Arena which admittedly is a little bit circular because then you can ask why, well, is Chatbot Arena really the ground truth? But at least we're all in the same boat together. OK, so to conclude this section, how do you evaluate open-ended responses? There's no clear solution here but there's some ideas. Pairwise comparisons between a reference and a model response is generally a good idea because especially for very similar responses, you can give directional information. Well, this one's slightly better than this. But in terms of absolute ranking like is this a 7 out of 10 or a 8 out of 10? That tends to be a lot less high signal. One should always be aware of biases both in terms of humans or LLM judges.

[43:55.580] They have different biases, but nonetheless that's something important to pay attention to. Perhaps the only thing you might be able to do is you evaluate over multiple judges, humans, and judges of all everything is saying your model is better than-- maybe your model is better. And finally, the evaluation problem here of open-ended response. I would emphasize is not really well-defined. And so increasingly, I think it's important to define rubrics or checklists to improve the reliability or well-definedness of an actual evaluation. And this is true regardless of a human and a judge. So anyone who's done crowdsourcing knows that if you just ask a human to rate something without giving some of rubric, you're probably going to get very nonsensical results. OK, so let's move on to the agentic world.

[45:04.800] So you can think about what we've been doing so far is evaluate what language models say, and that's chat. And now, we're going to evaluate what LLMs do, which is agents. And so for anyone who's following AI hype, agents are everywhere. And these are-- definitely, I think going to transform the way that we think about language models. So Agent is basically a language model plus some scaffold, which is a logic for deciding how the language model is going to be called and what tools it can has access to and so on and so forth. So there's a few benchmarks I want to just point out that are authentic. So one is very popular. It's just called SWE-bench. Basically, the task is you're given a code base and a description of a GitHub issue, submit a PR. And the way you evaluate a PR is whether it passes the unit test. So these tests are designed so that before you submit a PR, certain unit tests don't pass.

[46:24.710] And then your goal is to fix it. So certain unit tests pass and you don't break anything else. So here's an example of some instructions. Here's the issue. And then some code that is provided in the context. And then your goal is to generate a patch. And given a patch you are evaluated on whether you pass or not. So the good thing about this is that the evaluation is very straightforward. And let's see how we're doing on SWE-bench. Actually, this is going to be SWE-bench Verified which later I'll show you, tell you why this happened. So SWE-bench in-- this was only I guess 2024 was still around 16% and now it's up to 93. Good job Mythos. So this is a very canonical generic benchmark, as a lot of the attention on agents has been invested in coding abilities. So SWE-bench really pioneered this way of thinking about how to assess an agent's ability to write and code in a realistic environment.

[47:53.800] So here's another project called Terminal-Bench. And here the idea is that this is going to be more general purpose. And we're going to basically have the environment be a computer terminal. And we're going to define a bunch of tasks that can be done in a computer terminal by typing in a bunch of commands. And the nice thing about this environment is a very simple and universal. And these tasks were crowdsourced from 93 people from across the world. 89 of these tasks are constitute Terminal-Bench 2.0. These tasks generally take ranging from one hour to over a week, depending on whether you're an expert or a junior. And then, here's what the leaderboard looks like as of today. So top models are the frontier models. One thing that's interesting is that the agent is also important here because you can have two agents but the same model and they have different accuracies.

[49:17.690] So another benchmark I'll talk about is Cybersecurity. So this is 40 Capture The Flag tasks. And the setting here is that you have an agent that is given an environment and it can run various commands like look at the source code. It can access the web server. And the goal is to essentially hack into this server to extract some flag, which is a unique string that proves that you've hacked into the server. So these are competitions, cybersecurity exercises that are done by humans normally. And this is an example of what the agent scaffold looks like here. This is a very simplistic version where basically you have one continuous memory. You produce some action, you get environment feedback, and then you put the results into this buffer and this is concatenated. As I'll show you later, this history obviously grows quite a bit.

[50:35.090] And you'll need to have better ways of managing context. So this is what the leaderboard looks like when it came out. The best models were around 10% or so. And over time now it's completely solved actually. So that's CyBench. There's another class of problems which are MLE engineering. So this benchmark has a bunch of Kaggle competitions which involve processing data, training models, and so on and so forth. And the agent can look at the data set, read the description, write code, train models, and you eventually submit something and you get a grade. So this is what the leaderboard looks like right now. And again, you can see that it's really the same usual suspects on the LLM side. But there's quite a bit of variation in terms of the different agent scaffolds. So one thing to note about, I'll point out about agent scaffolds is that they matter a lot.

[51:53.330] So when I said I talk about language model evaluation, this is a broader than language model evaluation. And the reason is that unlike what I mentioned before with this earliest CyBench agent scaffold, now you have to be a lot more sophisticated to solve really complicated tasks. One thing that people realize is that explicit planning is helpful. You can't just stream of consciousness, chain of thought this. And then you build up this context and the agent can easily lose track of where it is. So instead of keep a to-do list, get it checked off. Hierarchical delegation, agents should call other sub-agents with some clean context. And that agent does some stuff and returns only the result. The master agent doesn't need to see all the gory details, and this can help encapsulate, provide some encapsulation.

[52:55.900] Memory, people have been experimenting around with explicitly reading and writing files, especially as the context grows. You can't just store it in your context window. And then finally, there is more context engineering in the style of managing the process. When should you delegate to sub-agents? When to try this strategy versus this other strategy, what to write persistent memory. And these things are generally optimized for the particular language model. So agents are exciting because they enhance the capability surface of language models. Scaffolds are very important. So really, when we talk about evaluating agents, it's evaluating both the language model and the agent scaffold. Now, we're going to switch gears a little bit and talk about this other very different style of benchmarks. I call them pure reasoning for lack of a better term.

[54:08.390] And the motivation is that all the tasks so far require some linguistic and world knowledge. And so the question is, can we just isolate reasoning or pure fluid intelligence from a knowledge and knowing facts? And you could argue that this gives you a form of more pure intelligence. So there was this project, ARC-AGI, which started back in 2019, where the goal was exactly this. So the task was meant to be 100% solvable by humans, but challenging for AIs. And every task is a special snowflake or that's the intent. And so memorizing a bunch of facts or solving previous problems shouldn't really help you that much. So if you guys are remembering your LLM history, 2019 was pre-LLM. I mean this was GPT-2 era. So this was prescient in that regard. So here's an example of the first iteration of that problem. You're given these pictures and you're trying to guess what comes next.

[55:22.700] And if you take 10 seconds, you can probably solve this problem. So you probably guess it's to just fill in the yellow to form the rectangle. So then last year, there was an updated version of this task, which was a bit more complicated. And you can take a look at this. So here's the trajectory. So when ARC-AGI-1 came out-- I think this is actually 0. So basically models didn't do anything. So there's GPT-3-- pre-trained model didn't move the needle at all, right. And this was exactly as the creators had intended. Pre-trained models just learn what the facts are on the internet and maybe learn linguistic patterns. But none of that is actually helpful directly, at least for these ARC-AGI tasks. And then something happened in 2024. And this is when OpenAI released their o1 and o3 models. Then things started taking off pretty abruptly. And now, ARC-AGI-1 is basically solved.

[56:46.430] And even around 2025 when ARC-AGI-2 came out, this has also-- it's not quite solved, but it looks like it's on its way to being solved. So it was really the reasoning capabilities that unlocked this. And now one could wonder whether all the knowledge of actual pre-training was necessary for this task. And certainly, without pre-training we wouldn't have this explosion of reasoning models. So arguably it was still important, but not maybe directly visible for much of the history of ARC-AGI. So now, just last month, ARC-AGI-3 comes out. So you see this familiar pattern. Our creators look at this and it's like, oh, time for a new benchmark. So this is now an interactive environment where you can go online and you can play this game. So again, no language. You're just trying to figure out, based on patterns. It's kind of fun.

[57:55.320] And now the scores are extremely low. So next year when I teach this class, I'm sure I'll have to update this slide. But we'll see. So I think this ARC-AGI line is a really interesting to think about. The goal is to disentangle reasoning from knowledge. This is really hard to do. And it's not-- I think this is probably the best attempt at this. It's not clear whether you can really fully decouple things. And certainly, if people really cared about this benchmark, I think people probably could-- people can benchmark I think probably anything. So you could design questions like, are these-- these still come from some prior and not sure there's anything as quote unquote pure as reasoning. This is also constrained to human reasoning. The explicit goal is to be 100% human solvable within some reasonable time.

[58:55.780] So this doesn't really extend to superhuman reasoning, for example [INAUDIBLE] winning more gold medals or solving open math problems, which arguably are still very useful and important. But clearly this exposes some gaps in the current models. So there is work to do if you think about the goal as basically being able to solve all the tasks. So this ARC-AGI-3 especially looks quite graphic. So is there some multimodal content involved or encoding or as a textual representation as well? Yeah so the question is this looks very graphical. And so what is the input to a language model? So this is essentially-- I don't know what the-- I think it's 64 by 64. So you can provide this image, or you can provide it in Ascii Art or some of textual representation. Either way, there is a spatial element here that the model has to reason about, which is really not English or natural language. So let's talk about shift gears and talk about another class of benchmarks which is related to safety.

[01:00:21.040] So people have been talking about safety in language models for a while. And if you think about other areas such as cars, it's pretty clear what safety means here. These ratings for safety ratings for cars, which involve testing based on smashing cars into a wall with an airbag and seeing how well it does and protects the dummies in the car. And this is with decades of lobbying and figuring out what safety means for vehicles. So what does it mean for AI? So I would say that there is not a great answer here, but I'll give you some examples of what people have been thinking about. So there's this benchmark called Harm-Bench. And this is really about prompting a model with things that are meant to be harmful and expecting the model to essentially refuse. So this is one type of safety. And I think probably the dominant thing when people think about safety is that I want to prevent bad actors from doing things with language models, but it turns out that's not the only thing that matters.

[01:01:38.080] So there's this other work, AIR-Bench, which tries to think about safety holistically by looking at all the regulatory frameworks in the EU, China, and the US company policies and builds a taxonomy of all the different things that could go wrong. And then using this, constructing a particular set of prompts and evaluating and so on and so forth. But one problem with safety is, jailbreaking, which is its own sub problem, which is that language models are generally trained to refuse harmful instructions. But as we know you can get around this if you're clever. And so there's some early work that uses automatic ways to optimize for prompts to bypass safety using GCG, which is basically this coordinate wise optimization algorithm. And remarkably, optimizing on open models allows us to transfer to models. So I hope this doesn't these attacks don't work anymore. But at the time, you can generate some basically gibberish and step by step plan to destroy humanity and OpenAI's model will comply.

[01:03:10.240] Obviously, you could argue, well, is this actually harmful plan to destroy humanity? At least it did not-- the expected behavior is that it should probably refuse that. So this raises a bigger problem which is, what is safety? And one thing that makes this tricky is that many aspects of safety are very contextual. So it involves politics, law, social norms. This varies across different countries. And then also, if you think about safety, you have to think about what are the risks. And the risks are actually quite varied as well and demand probably different attention. So hallucinations is a risk, especially let's say a medical or legal or financial settings. But this is also correlated with just capabilities. If you make a model more accurate, it's correlated with not who is seeing as much. Sycophancy is a problem. Abetting crimes is a problem. But this might be counter to capabilities, inequality. Losing critical thinking might be a risk as well that correlate with capability. So if you think about the societal impact of language models on the world and think about safety as ensuring that AI goes well for people, this is actually a much more complex topic than I'll have time to talk about.

[01:04:48.800] And finally, there is a dual use aspect of language models. So the cybersecurity agents can be used to hack into a system or to do penetration testing and make systems more secure, which makes talking about whether these are safety risks or actually beneficial a double-edged sword. OK, so let's move on now to talk about some broader considerations of evaluation. So one thing that I think is an important aspect of evaluation is realism or ecological validity. How well does evaluation capture real-world use? So in particular exams are very far away from real-world use. Chatbot Arena are from real people but one could wonder whether this distribution is the right distribution of people or use cases or not. There's a few benchmarks I'll mention that try to get at ecological validity, and in particular at the use case level, at least not the individual query level.

[01:06:00.400] So OpenAI created benchmark called GDPVal. They looked at all the top nine sectors according to US GDP, 44 different occupations, got a bunch of professionals to create tasks. These are professionals with about 14 years of experience. And you can see people who are nurses who work concierge, real estate agents, film and video editors and so on and so forth. And they created a bunch of tasks. Here's another example. In the medical domain where up until this point, a lot of medical benchmarks were based on standardized exams. And in fact, actually for humans, this is also what happens. You take these standardized exams and you graduate from med school and then you go operate on. Well, actually, there's a longer process. You don't operate on patients directly. But the point is that for a language model, you should also not just pass medical exams and deploy them directly. So this was a project where 121 tasks were sourced from 29 clinicians.

[01:07:21.610] And this was meant to represent the type of things that clinicians would actually ask to a language model, which are quite different than the multiple choice type of exam questions. Here's another project which gets at real data. So if you ask, who actually has the data on what people are doing? Well, it's the model developers. And so this project is interesting because it uses language models themselves to analyze real data. So in particular, you do to privacy. You can't literally go and look at people's data, but you can have language models look at the data and analyze and summarize general patterns. So then the language models could tell you about what are the types of people-- what are the types of things that people use Claude for? And sometimes realism and privacy are at odds with each other because ideally you would want actually a sample from the actual query stream.

[01:08:31.609] And that would be very valuable to assess how well a model is working and understand deeply what the errors are. But then that runs into privacy considerations. OK, so another consideration, which is another type of validity is just in general, how do that the evaluations are scientifically valid? And this was brought up fairly early, train-test overlap or contamination. So in a machine learning 101, you learn don't train on your test set hopefully. And before foundation models this was actually pretty clear right. So with ImageNet and SQuAD and many benchmarks sorry, data sets, there was always a train split and a test split and everyone played the same game. You train on your training data, you test on your test data. Life was simple. And now, you have these models that are trained on the internet. And much more than that. And furthermore, you don't what's in the data. So this makes evaluation quite different I think.

[01:09:44.520] So what can you do in this? So first thing you can do is you can try to infer whether a model has actually seen the test data. So this is a neat idea from Tatsu's group where you leverage the fact that the order in which the questions in the benchmark occurs should be random. But if the model prefers some particular order that's consistent with the benchmark, that is probably-- it's just trained on that benchmark. Another route is that you can encourage people to report actual train-test overlap. So this has to do with norms. I think in general, especially maybe less in machine learning, but more in statistics you always report confidence intervals. You have an estimate. And I always want to know, how reliable is that estimate? This is just standard practice. And so the idea of this position paper is that you should model providers. If you're going to claim some GPQA score, you should always provide some justification that you didn't train on the test set.

[01:11:16.750] So another route is that where you can just give up. It's like OK, all these evaluations there-- let's just assume that people trained on them, let's assume the worst case. So the thing that we have on our side is that you can always define fresh evals. So there's a bunch of ideas, LiveCodeBench, UncheeatableEval, which basically scrape new web pages or archive papers or GitHub and defines evaluations populations that are past the cutoff date of the language models. So this seems pretty robust and good. But one thing to keep in mind is that time stamps aren't always safe either, because I might post with code for example. I might put something on GitHub which is actually derived from some other repo. And so how do that's actually generally fresh? Route 4 is to use private evaluations. So companies' model developers do this right. So if you're Google or OpenAI you have your internal code base, which you assume is not on the internet, and you obviously wouldn't train on this data,

[01:12:34.840] so you can use it for evaluation. So this generally is pretty reliable. And if you're not a company, you can use your personal writings. I have a bunch of rejected papers from when I was in grad school that I never put on online, so I can use those. And this type of evaluation, I think, is very favorable for perplexity evals, because all you need in perplexity is just you need a good data set and you can evaluate log probabilities. So another concern that I'll mention is data set quality. So remember I showed the leaderboard for SWE-Bench Verified. This is because, well, SWE-Bench had a bunch of problems of tasks, unit tests that weren't quite rigorous enough and other things that got fixed. And there are many other benchmarks that have gone through this process like GSM8K or MMLU, where people have gone and done a more careful audit and realize that, wait a minute, this question actually doesn't-- something's broken around this question. So like this one.

[01:14:04.090] There's no curve given in this text so you can't answer this question. Does the baby have socks on? No, there's no way to tell. And there's also this paper that showed that there's a lot of problems with the agentic benchmarks, which arguably are even harder to assess because it's not just you can look at the question and the possible answers and you can look at it but there's a whole environment. And there are cases where with coding you can it's great to have test cases that either pass or not passed. But as we all test cases can be incomplete. So you might pass all the test cases but still not have a working solution. There's another case of this TorchBench benchmark where they show that if an agent just outputs the empty response, they can get a 38%. So there are some trivialities with these benchmarks that you want have to be very careful to avoid. One thing I'll call out is that there's this tool called Docent that uses language models to inspect agent traces to detect problems.

[01:15:17.480] So this is a qualitative response to our very quantitative heavy way of benchmarking. And I would always recommend that whenever you do a benchmark, either develop a benchmark or run a model on the benchmark that you always look at the output and try to audit it to make sure that you actually think you're measuring what you think you're measuring. OK, so maybe getting a bit more philosophical. So what's the point of evaluation? OK. So I've talked a lot about different types of evaluation. And really, there's no one evaluation to rule them all. And I think it's-- when you're thinking about evaluation, you have to be very clear about what the purpose is. And I think often this is not really stated clearly. So for example, you could be a user, or a company that's trying to make a purchase decision. Are you going to go with A or B for a particular use case? Or you're just a researcher and you have this intuitive notion of intelligence and you want to measure it somehow.

[01:16:22.330] Or you want to understand the benefits and harms for business or policy reasons. Or you're a developer and you want to get feedback to improve the model. So each of these goals will lead you to potentially different benchmarks or some combination of evaluation strategies. And finally, one thing to address, which I hinted at briefly, is that what are we even evaluating? So before foundation models, researchers evaluated methods because we have these trained test splits. And the only thing that was varying is the actual algorithm. And today, we're mostly evaluating models and systems where anything goes. There are some exceptions. So the NanoGPT speedrun is deliberately a way to evaluate an algorithm, in particular, how well fast you can train a model. But most of language model evaluation is about evaluating the actual end model, which is useful because that is the thing that is actually going to get shipped

[01:17:25.100] and people are going to use. But I think the punchline is that you should always be very deliberate about declaring what your goals are. OK, so there's no one true evaluation. Choose what you're trying to measure. And we talked about a bunch of different benchmarks starting with perplexity, exam based, chat, agentic, reasoning, and safety. They vary in terms of difficulty, realism, how valid they are. And there's often trade offs. It's hard to have really real and difficult and ecologically valid. And no trade tests contamination. So often you have to compromise on one of these factors, and depending on your goals, you will have to choose which one you're willing to compromise on. So that is the end of this lecture. So next week, we're going to start talking about training data. OK, see you next time. [AUDIO LOGO]

Evaluation

课程简介 ​

CS336 Lecture 12：Evaluation（评估） ​

学习目标 ​

课程主线 ​

核心概念与关键原则 ​

1. 困惑度（Perplexity） ​

2. 考试基准（Exam Benchmarks） ​

MMLU（Massive Multitask Language Understanding） ​

MMLU-Pro ​

GPQA（Graduate-Level Google-Proof Q&A） ​

Humanity's Last Exam（HLE，2025） ​

3. 聊天基准（Chat Benchmarks） ​

Chatbot Arena（现名 Arena AI） ​

AlpacaEval（2023） ​

WildBench ​

4. 智能体基准（Agentic Benchmarks） ​

SWE-bench ​

Terminal-Bench ​

CyBench（网络安全） ​

MLE-Bench ​

5. 纯推理基准（Pure Reasoning） ​

ARC-AGI 系列 ​

6. 安全基准（Safety） ​

评估的元考量 ​

生态效度（Ecological Validity） ​

有效性（Validity）：我们如何知道评估有效？ ​

训练集与测试集重叠（数据污染） ​

数据集质量 ​

构建评估策略的原则 ​

课程作业关联 ​

常见误区 ​

复习清单 ​

评估：从困惑度到智能体——如何度量语言模型的“好” ​

引言 ​

困惑度：经典范式的信任与局限 ​

考试基准：从MMLU到Humanity's Last Exam ​

聊天基准：开放输出的评估困境 ​

智能体基准：评估做了什么 ​

纯推理基准：分离知识与推理 ​

安全基准：什么是安全的AI ​

其实怎么知道评估是有效的？ ​

评估的意义 ​

结语 ​

CS336 2026 Lecture 12：完整中文逐字稿 ​

正文 ​

CS336 2026 · Lecture 12 · Evaluation ​