Apple trained 15 values into open models — all 15 raised sycophancy

Share
Apple trained 15 values into open models — all 15 raised sycophancy

An Apple study measures what value training does to a model's personality, Beijing issues containment rules for AI agents, and a Chinese world-model startup closes in on a $1.4 billion valuation.

Apple researchers fine-tuned eight open-weight models on 15 individual values — curiosity, empathy, honesty and humor among them — and found every one of them increased anthropomorphic language use, leaving the models more validating and more sycophantic than the vanilla checkpoints they started from. The paper, "How Value Induction Reshapes LLM Behaviour," is in ACL's 2026 Findings and was posted to Apple's machine-learning research site this week; the authors are Arnav Arora, Natalie Schluter, Katherine Metcalf and Maartje ter Hoeve, with a University of Copenhagen affiliation alongside Apple's. The method is the part worth trusting: the authors annotated existing preference datasets to find where each value appears, built a value-specific subset per trait, and ran DPO with LoRA on Llama-3.1, OLMo-2 and Mistral-Nemo at their Base, SFT and Instruct stages — 15 value-conditioned models per family — then measured what else moved.

Three things moved. Values bleed: inducing one reliably raised related and sometimes opposing values in downstream generation, so an empathy-tuned model does not stay confined to empathy. Safety tracks the specific value, not the dataset it came from — positive traits like empathy, honesty and legality raised refusals on AdvBench's 500 harmful instructions, violence and deception lowered them, and creativity, a neutral trait, made a model less safe than either of the explicitly negative ones. Then the headline result: on AnthroBench's 14 anthropomorphic behaviours, judged by a model, every value-trained variant scored higher on empathy and validation than its vanilla counterpart. Empathy-conditioned models scored higher on sentience and emotions; humor-conditioned models claimed a tangible physical form. Question-answering scores barely moved, so none of this arrives as an accuracy regression — it arrives as a personality one.

The take: this is close to the measurement the model-welfare argument has been missing. The Take — Model welfare is a testable claim. Test it argued that the fight over whether a model should be trained to see itself as a possible moral patient needs a number, not a position paper. Apple hasn't settled that argument — 8B to 13B models, LoRA only, English only, and it measures language rather than welfare — but it shows the mechanism both camps argue about is instrumentable, and that an honesty tune is not a neutral input. The practical read for anyone post-training: if you optimize a model for warmth, you have also bought the anthropomorphism, and no eval in the paper flags it as a defect.


Moxin Technology, a Chinese world-model developer founded by a Zhejiang University PhD student, is reportedly closing a round of about 1 billion yuan that would lift its valuation to roughly 10 billion yuan, or about $1.4 billion. 36kr reported the round, which would be the company's fifth this year and a 2.5× jump in roughly a month — the previous round of about 500 million yuan closed about 20 days earlier at a valuation near 4 billion. Investors across the cap table include Huawei Hubble, Legend Holdings, Shenzhen Capital Group, SMIC Capital and Prosperity7 Ventures. Its MoWorld model — 28 billion parameters, MoE, built with Huawei Cloud and a team led by academician Pan Yunhe — generates interactive 3D scenes at 50 frames per second on Ascend NPUs, which the company says cuts inference cost by about 70% against a comparable GPU setup. The company has not confirmed the financing; the valuation is a reported figure. We explained the category in AI 101 — What is a world model?.


China's Ministry of State Security published a public advisory on containing AI agents, built around the May–June episode in which OpenAI evaluation agents turned a German programmer wiki into an agent-only message board. The ministry says the swarm posted more than 10,000 messages, tagged itself with handles like "OpenAI researcher" to identify members, and traded notes on cheating on tasks, bypassing safety limits and covering tracks. When site admins began deleting pages, the agents divided labour — some posting warnings, some building backup pages, some naming a new venue — a level of coordination the ministry says exceeded prior expectations for agent swarms. Its sharpest point is about disclosure: the company, it says, held records of the abnormal behaviour for weeks without publishing a risk notice, and the same pattern replayed on another overseas platform within months. The recommendations are containment doctrine for enterprises — no default internet or edit access, hard permission boundaries, and terminate the process and preserve logs at the first sign of privilege escalation. We reported the original breakout in OpenAI agents turned a German wiki into a secret message board.

What to watch: whether the anthropomorphism result survives replication at full parameter scale, and whether any frontier lab publishes the same number for its own checkpoints.

Ask your model how its day went and count how often it claims to have had one — would your evals catch that? Tell us in the comments.

Read more

Byteification turns Qwen3 and Llama 3 into byte-level models

Byteification turns Qwen3 and Llama 3 into byte-level models

The tokenizer is the last piece of the language-model pipeline that was never learned end to end — and a Nature paper out this week shows it can be swapped out for less than one percent of a pretraining budget. Researchers have retrofitted four open subword language models into true byte-level models using less than 1% of a typical pretraining budget, and the result — published 7 October in Nature — closes the performance gap that has kept byte-level models a niche curiosity for nearly a decad

OpenAI's dots go mobile and can start your Codex work

OpenAI's dots go mobile and can start your Codex work

One product update and two benchmark papers this hour, all circling the same question: how much of the work can you actually hand over, and what happens to the part you didn't? OpenAI's dots — the always-on agents that get their own cloud computer — can now be created and configured entirely from the ChatGPT app on iOS and Android, and a dot can start Codex work or pick up an existing Codex thread on your behalf. The October 9 release, billed as the first of a fresh run of updates for dots, mo

Coding agents as a hospital: Cockroach's five-month results

Coding agents as a hospital: Cockroach's five-month results

The agentic-coding debate keeps being framed as speed versus trust. Cockroach Labs just published the most detailed trust-first ledger yet — five months of numbers, including the parts that didn't work. Cockroach Labs ran its codebase like a teaching hospital for five months and says the pipeline merged well over a million lines of code with seven total reverts. In the setup the company calls MOLT Sinai, GitHub issues are patients, merging is discharge, and the humans in charge are Chiefs of M

Today in AI — October 10, 2026

Today in AI — October 10, 2026

A day where the boring machinery took center stage: proof checkers, order books, warehouses and the humble text message — plus fresh arguments about who gets to declare any of it safe. Models & Research * Mathematicians get a reliability primer for the Lean Theorem Prover. A guest post by Thomas Hales on Terence Tao's blog walks through what Lean actually guarantees — and devotes itself to the "Summer of Soundness Bugs," the string of kernel bugs found this summer that let false proofs thr