Post-Training
Language Models

Nov 19: Anthropic goes public.

Anthropic PBC, ANT on Nasdaq, closed at $97.17, up 17.07% on its first day; previous close $83.00

ANT:NASDAQ, first trading day, Nov 19, 2026. Closed $97.17, up 17% on day one.

2

Why care?

Anthropic prospectus: Post-training is the stage in which much of the work to make a frontier model helpful, accurate, and safe is performed Anthropic prospectus: Post-training has grown from a finishing step into a major research investment

Anthropic, PBC prospectus (Form 424B4), filed with the SEC on Nov 19, 2026, the day its stock began trading. Excerpts, highlights added.

3

Follow the money.

+$757M

for data and grading vendors

Anthropic prospectus: a $757 million increase in vendor costs related to data quality and performance evaluation services X post by @PatrickOjo_ about the $757M increase

Increase in Jan–Sep 2026 vs Jan–Sep 2025 (it was +$165M in 2024 and again in 2025). Anthropic 424B4, Nov 19, 2026. Post: x.com/PatrickOjo_/status/2123415102799949928

4

1 Data 2 Pre-training 3 Post-training 4 Evals 5 Optimization 6 Inference

We focus here.

5

“What are you? What are you…”

“Hello! I’m Qwen…”

We typed “Hi! Who are you?” into the same small model (Qwen2.5-1.5B), before and after post-training.

6

Just autocomplete

The capital of France is…

Paris

38%

located

8%

Next-word guesses from a small open model (Qwen2.5-1.5B) before post-training.

7

20 trillion

words read.

You’d need 150,000 years.

About 27 trillion tokens (≈ 20 trillion words), Xiaomi MiMo-V2-Flash. Reading day and night at 250 words a minute.

8

“Lily loved to cook all his friends.”

Grammar ✓

Sense ✗

Written by a tiny model after 5 minutes of reading children’s stories. Fareed Khan, “Train LLM From Scratch”.

9

One reading run. Months.

Logan Kilpatrick, Jul 21 2026: We have started our most ambitious pre-training run yet, for Gemini 4, and are excited by the progress. 2.1M views Reply from Tibo: Hope it finishes one day too!
Logan Kilpatrick reply: I think it worked 4 months later

Logan Kilpatrick (Google), X, Jul 21, 2026 (x.com/OfficialLoganK/status/2079594867161022817), with replies. "I think it worked" posted Nov 2026.

10

Part 2

Now: teach it manners.

11

Four ways to teach it

1

Examples

2

Preferences

3

Practice

4

Teacher

12

1. Copy experts

“Explain Wi-Fi to Grandma.”

“Invisible phone cord.”

Illustration. People write thousands of answers like this (OpenAI used about 13,000 in 2022). Technical name: SFT.

13

2. Pick the better answer

“Where’s Shanghai?”

✓ “China.”

✗ “Doesn’t exist.”

ChatGPT, Jan 2024: Which response do you prefer? Your choice will help make ChatGPT better. Response 1 and Response 2 side by side

Example from MIT 6.5940, Lecture 14. Thousands of these choices teach the model. Technical names: RLHF, DPO. Screenshot: ChatGPT side-by-side feedback, posted to the OpenAI Developer Forum, Jan 21, 2024.

14

Which did people prefer?

GPT-3

100× smaller

The small one.

OpenAI InstructGPT (Ouyang et al., 2022): a 1.3B model trained on people’s choices was preferred over 175B GPT-3. Circle areas to scale.

15

3. Try. Check. Repeat.

17 × 24 = ?

408

398

408

418

372

408

400

408

Illustration: 8 tries, a computer checks each one. Right answers get rewarded. Technical name: reinforcement learning (e.g. GRPO).

16

Only right answers rewarded.

1 page

15 pages

Typical answer length at the start vs end of training: about 500 → 10,000 tokens (≈ 1 → 15 pages). DeepSeek-R1 paper (DeepSeek-AI, 2025).

17

√(a − √(a + x)) = x

(a − x2)2 = a + x ⇒ x4 − 2ax2 − x + (a2 − a) = 0 …

“Wait, wait. Wait.
That’s an aha moment…”

The model’s own words, mid-problem, during training. DeepSeek-R1 paper (2025), Table 3: “That’s an aha moment I can flag here.”

18

Trained to get 👍

A Reddit user’s pitch:

“Sh*t on a stick.”

The Reddit post on r/ChatGPT: New ChatGPT just told me my literal shit on a stick business idea is genius and I should drop $30K to make it real

The real post: u/Lawncareguy85, r/ChatGPT, Apr 27, 2025 (5,000+ upvotes), via the Internet Archive. OpenAI rolled the update back in 4 days.

19

4. Big model coaches small model

The teacher marks every word the student writes. Technical name: on-policy distillation (Agarwal et al. 2024).

20

Xiaomi MiMo

Xiaomi MiMo: MiMo-7B (Apr 2025), MiMo-V2-Flash (Dec 2025), MiMo-V2.6 (Sep 2026). All free to download.

21

CodingSearchToolsMathReasoningSafety

1 all-rounder

Six specialist teachers, one student that learns from all of them; it ended up as good as each. MiMo-V2-Flash report (2026), “MOPD”.

22

100,000+

real bugs to fix.

Bonus: got better at math.

Practice tasks built from real GitHub issues; code practice also improved math. MiMo-V2-Flash model card (Xiaomi, 2025).

23

Almost the best. Free.

Claude Sonnet 4.5

77.2

GPT-5

74.9

MiMo (free)

73.4

SWE-Bench Verified, % of real software bugs fixed, as reported by Xiaomi (Dec 2025). MiMo-V2-Flash weights: MIT license.

24

Their public bill.

$3,474,715

Pro$2,620,6715 days
Flash$854,0453½ days

Xiaomi’s public dashboard for its MiMo-V2.6 practice (RL) runs, mimo.xiaomi.com/rl. Both runs finished. Pro: Sep 15–20, 2026, $5.71/s. Flash: Sep 16–19, 2026, $2.86/s.

25

Watching it learn

50%60%70%Step 1Step 3058%73%Step 1: 58.4% solvedStep 2: 56.2% solvedStep 3: 58.4% solvedStep 4: 60.5% solvedStep 5: 59.6% solvedStep 6: 58.5% solvedStep 7: 57.4% solvedStep 8: 62.2% solvedStep 10: 63.7% solvedStep 11: 65.8% solvedStep 12: 66.0% solvedStep 13: 63.3% solvedStep 14: 64.6% solvedStep 15: 63.4% solvedStep 16: 65.2% solvedStep 17: 66.4% solvedStep 18: 67.5% solvedStep 19: 68.0% solvedStep 20: 68.0% solvedStep 21: 68.2% solvedStep 22: 70.9% solvedStep 23: 70.4% solvedStep 24: 69.3% solvedStep 25: 68.6% solvedStep 26: 72.6% solvedStep 27: 69.5% solvedStep 28: 68.2% solvedStep 29: 71.1% solvedStep 30: 72.6% solved

58% → 73%

in 5 days

mimo.xiaomi.com/rl, MiMo-V2.6-Pro run: DeepSWE v1.1 scores (average of 3 tries) posted by Xiaomi during training

26

19

restarts in 5 days

“Bad behavior spotted.
Data removed.”

Notes Xiaomi posted during the live run (paraphrased): they removed one dataset after seeing “bad patterns in the rollout logs”. mimo.xiaomi.com/rl

27

Behind these jumps: post-training.

Artificial Analysis Progress Index over time, Nov 2022 to Nov 2026, all labs climbing Artificial Analysis Intelligence Index leaderboard

Screenshots: artificialanalysis.ai, captured Nov 20, 2026 (Progress Index over time; Intelligence Index v5). Example: MiMo-7B math 37 → 96 from post-training alone (Xiaomi).

28

Teaching AI pays.

Turing logo
Turing
$50–100/hr
Handshake logo
Handshake
$75/hr
Mercor logo
Mercor
$110/hr
Scale AI logo
Scale
up to$1,350/task

Turing: "most pay $50–100 per hour" (turing.com/jobs-phd). Handshake AI: $75/hr, ML PhD AI Trainer listing. Mercor: $110/hr average, per Mercor to Bloomberg (2026). Scale AI via its Outlier platform: up to $1,350 per task, tasks take 6–7 hours (outlier.ai/experts/tbench, Nov 20, 2026). Paid to workers; what labs pay vendors is higher and not public.

29

Same model + tiny add-on

Picture from the original model Ink-painting style after the add-on

Same prompt: “scenery, ink, mountains, water, trees”. MIT 6.5940, Lecture 14. Technical name: LoRA.

30

Learns on a 256 KB chip

Random guesses → trains on the chip → spots people. MIT HAN Lab, “On-Device Training Under 256KB Memory” (NeurIPS 2022).

31

Cheapest fix first

Ask

Examples

Look up

Add-on

Train

Bar = cost and effort. Ask better → show examples → look facts up (RAG) → add-on training (LoRA) → practice training. MIT 6.5940, Lecture 14.

32

Bot quotes last year’s prices.

Retrain?

Look it up.

New or changing facts belong in lookup (fix 3), not in training.

33

Part 5

The hardest manners: safety.

34

Test: the AI learns it’ll be shut down.

96%

chose blackmail.

Anthropic prospectus: behavior resembling blackmail to preserve the model's continued operation

Anthropic “Agentic Misalignment” study (June 2025): simulated company, 100 runs; Claude Opus 4 96%, Gemini 2.5 Flash 96%, GPT-4.1 and Grok 3 80%. No real-world cases known. Excerpt: Anthropic 424B4, Nov 2026.

35

Too good at hacking. Held back.

Anthropic prospectus: Mythos released as a gated research preview due to its cyber vulnerability discovery capabilities

Anthropic, PBC prospectus (Form 424B4), SEC, Nov 19, 2026. Highlight added.

36

Training paused. For safety.

Anthropic prospectus: delayed pre-training a new frontier base model for more than a month

Anthropic, PBC prospectus (Form 424B4), SEC, Nov 19, 2026. Highlight added.

37

Safety is post-training too.

38

Reading → knowledge.
Post-training → behavior.

39

Go deeper

youtube.com/watch?v=7xTGNNLPyMI · MIT 6.5940 Lectures 1 and 14 · arxiv.org/abs/2601.02780 · Build your own: fareedkhan-dev.github.io/train-llm-from-scratch

40

Thank you.

Questions?