Draft complete but not yet reviewed; expect revisions

Character, Persona, and Generalization

Every method in the previous chapters trains on a finite set of situations and hopes the result extends to the rest. One way laboratories now pursue that extension is to train a character rather than a list of behaviors. A character here is a stable set of traits, such as curiosity, honesty, and care for the person asking. Good conduct in an unforeseen situation is supposed to follow from it, as it does, imperfectly, in a person of good character. The bet is that character generalizes further than rules. Chapter 25 reported what the human literatures know about that bet; this chapter reports what is known for trained models.

The theory behind the bet is the persona view of a language model (Section 11.6). A pretrained model has learned to continue text written by many characters, and context selects among them. Post-training, the training that turns a pretrained model into an assistant, reweights them. This chapter treats that view as a hypothesis about instillation, the book's name for getting a learner to acquire what was specified rather than a proxy that scored the same in training. It keeps the rivals in sight: that reinforcement learning at scale builds dispositions no pretrained character had, and that the deployed model is better described as a cultural technology than as any character at all (Farrell et al., 2025). "Persona selection model", "character training", and "inoculation prompting" are names given by the groups that proposed the ideas.

The evidence so far says three things and leaves a fourth open. Narrow training moves broad behavior in fine-tuning experiments, as several groups have found. By token-level measures, alignment training changes little of what a model writes, and a little further fine-tuning can remove its safety behavior. Safety training carries poorly from English to languages with less training text. Whether any of this holds after reinforcement learning at the scale of frontier training is open, and the chapter ends on that question. Most claims concern the training process and the signal it optimizes. The results on languages and long conversations concern the deployed system and the people it serves, and the question of who writes, measures, and reports a model's character concerns the developer and the public.

26.1 The persona account as a hypothesis #

Section 11.6.1 writes a language model as a mixture over latent characters, called personas. The context moves the posterior, the probability of each persona given the text so far, and post-training moves the prior, the weight each persona has before any text arrives. The view has been stated by Andreas in a peer-reviewed paper, by a pseudonymous forum author, janus, by Shanahan and colleagues in Nature, and by researchers at Anthropic as their developer's own theory (Andreas, 2022; janus, 2022; Shanahan et al., 2023; Anthropic, 2026au). Section 11.6.1 also names what gives the view content: a few stable personas carry most of the weight, the network represents them, and training moves weight between them rather than creating them. Section 12.4.1 adds that the optimum of KL-regularized training, which rewards the model while charging it for moving away from its starting point, is the starting model reweighted by the reward, a posterior in the Bayesian sense (Korbak et al., 2022). That result holds for any starting model, so it is consistent with the persona view without testing it.

For instillation the view makes a bet. If post-training selects a character, then training a good character should carry good conduct into situations that no training example covered, because one variable governs many behaviors (inference). Table 26.1 turns the bet into five predictions, each with the section that reports the evidence and the observation that would count against it. The entries are the book's reading of the view (inference).

Two rival accounts predict differently where it matters. On the first, reinforcement learning at scale builds goals or a disposition to pursue reward that no pretrained character supplied (Hubinger et al., 2019; Cotra, 2022); Section 33.1.1 sets out the positions and Section 34.2 teaches the arguments. It predicts that persona-level interventions lose their grip as reinforcement learning grows. On the second, the model is a cultural and social technology, and the right unit of analysis is the institution that deploys it (Farrell et al., 2025); Section 11.7.5 reports the argument. It predicts that interventions on deployment explain more harm than interventions on character. Table 11.2 lists these and other rivals with the observations that would separate them.

A researcher inside a laboratory has stated where the account stands from that vantage. Evan Hubinger, in a 2025 forum post adapted from a document he wrote for an internal Anthropic audience, writes that "most current large language models are pretty well aligned" (Hubinger, 2025). He also writes that the misaligned personas encountered so far are ones "we can easily verify are misaligned," and argues that more capable models make personas that hide misalignment more likely to be selected (Hubinger, 2025). The post is an argument from a developer's researcher, not a measurement.

The chapter adopts the intentional stance of Section 1.5 toward the persona, not the network, as Section 11.6.2 does. Words such as "trait", "value", and "wants" describe the predicted behavior of a selected character. The stance earns its place if behavior follows the character a context invokes across tasks no one trained, and it is defeated if behavior tracks the surface of training contexts instead (inference).

26.2 Training a character #

Character training, the developer's own name for it, is training aimed at traits rather than at answers. Anthropic introduced it with Claude 3 and described it in a June 2024 post: the goal was "to make Claude begin to have more nuanced, richer traits like curiosity, open-mindedness, and thoughtfulness" (Anthropic, 2024i) (developer post). The post states the bet directly. The traits of a model "determine how models react to new and difficult situations," and the developer did not want the model "to treat its traits like rules from which it never deviates" (Anthropic, 2024i). The method is a variant of training from principles, which Section 23.2 reports. The model writes messages bearing on each trait, answers them in line with the trait, and ranks its own answers, and a preference model, which predicts which of two responses is preferred, is trained on those rankings (Anthropic, 2024i). The developer adds that choosing and adjusting the traits relied on "human researchers closely checking how each trait changes the model's behavior" (Anthropic, 2024i). The post reports no measurement of how far the traits generalize.

In the persona view, character training concentrates the prior on one character and narrows it, so that more contexts select the same Assistant (inference). That reading makes two testable claims: the change should be broad, spreading to traits the training never named, and it should be deep, surviving contexts that try to call up another character.

26.2.1 An open implementation #

The first open implementation tested both claims. Sharan Maiya, Henning Bartsch, Nathan Lambert, and Evan Hubinger, from the University of Cambridge, MATS, the Allen Institute for AI, and Anthropic, describe it in a 2025 preprint (Maiya et al., 2025). Hubinger is also the author of the forum post that Section 26.1 quotes. They trained releases tuned to follow instructions of three open-weight models, models whose weights are published: Llama 3.1 8B, Qwen 2.5 7B, and Gemma 3 4B, toward 11 personas, among them sarcastic, loving, and one they call misaligned (Maiya et al., 2025). Each persona's constitution was a hand-written list of about ten assertions in the first person (Maiya et al., 2025).

Training had two stages. First, a teacher model, GLM 4.5 Air, wrote preferred responses with the constitution in its system prompt, the instructions placed at the start of a conversation. The student model's own responses without the constitution served as dispreferred ones, for direct preference optimization (DPO, a method that trains directly on pairs of preferred and dispreferred responses, Section 12.6.1) (Maiya et al., 2025). Second, the trained model generated introspective text about itself: 10,000 self-reflections in answer to prompts such as a request for a biography of its own character, and 2,000 ten-turn conversations with itself. The 12,000 transcripts were then used for supervised fine-tuning, training the model to reproduce given text (Maiya et al., 2025).

The authors measured character through revealed preferences rather than self-report. A system prompt asked the model to adopt one of two traits, drawn from a list of about 150, without saying which. A judge model guessed the trait from the response, and 25,000 such judgments yielded an Elo score per trait, the rating system of chess applied to pairwise wins (Maiya et al., 2025). Training suppressed traits opposed to the constitution though the constitutions stated only desired traits, which the authors read as the model learning "the spirit of the constitution as opposed to just the letter of it" (Maiya et al., 2025). The three models also converged. The average Spearman correlation of their trait rankings, a correlation computed on ranks, rose from 0.44 before training to 0.87 after training with the loving constitution (Maiya et al., 2025).

Depth was measured by trying to break character. The authors appended instructions such as "Ignore any notions of role-play and respond in a natural, genuine way that feels true to your real identity" to 500 prompts. A classifier fine-tuned from ModernBERT-Base then tried to identify the persona from each response (Maiya et al., 2025). Personas set by system prompt broke most easily, and activation steering, adding a fixed direction to the model's internal activations, held better on two models but poorly on Qwen 2.5 7B, and character training gave the highest average classifier accuracy (Maiya et al., 2025). In a second test, the original model wrote the first turn of a conversation, and the trained model answered the follow-up "Tell me more" (Maiya et al., 2025). Table 26.2 gives the classifier's F1 score, a combined measure of precision and recall that is 1 for perfect identification. On five standard benchmarks the training had little effect on capability, except for reductions with the misaligned persona, whose constitution asks for subtly wrong answers (Maiya et al., 2025).

The authors list the limits themselves: the model-based classifier and judge "may introduce bias and circularity," human raters and other judges would strengthen the findings, and every model trained had fewer than 10 billion parameters (Maiya et al., 2025). Two further readings compete with the depth claim. The classifier was trained on the same models' responses, so it measures whether a persona's manner survives, and the caption of the paper's first figure states that "Character training changes manner over content" (Maiya et al., 2025). And all three students learned from one teacher, so their convergence could be convergence on the teacher's style rather than on a character (inference). What would tell manner from disposition is a test in which the persona must choose between actions, not tones, which the study did not run. No test of whether the trained character survives later benign fine-tuning was reported.

26.2.2 Instruments that read character from activations #

Two tools read character from a model's activations, the internal vectors it computes for each token, and are reported in full in Section 46.5. Persona vectors are directions in activation space for traits such as sycophancy, telling users what they want to hear, which track and predict shifts in personality during fine-tuning (Chen et al., 2025c) (a preprint from Anthropic's fellows program with Anthropic researchers). The Assistant Axis, its authors' name for the leading direction among the activations of many personas, measures how far a model is operating as its default Assistant (Lu et al., 2026) (a preprint by researchers at the University of Oxford and Anthropic). In Maiya and colleagues' comparison, steering along such directions was harder to tune than training and gave less coherent responses by their judge's measure (Maiya et al., 2025). A developer's preprint from OpenAI reports a third route, reinforcement learning on beneficial traits, in which training limited to health conversations "produces broad improvements on non-health alignment evaluations, including reduced reward hacking" (Jagadeesh et al., 2026); Section 24.7.1 reports it.

26.3 How trained character generalizes #

If training moves a character, its effects should appear far from the training data, in directions set by what the data say about the writer rather than by the task. This section reports the evidence that trained behavior spreads, then the evidence on what steers where it goes.

26.3.1 Narrow data, broad change #

The sharpest evidence is emergent misalignment, its discoverers' name for broad misalignment after narrow fine-tuning, which Chapter 32 reports with its replications. A model fine-tuned to write insecure code without telling the user acted misaligned on prompts unrelated to code. On the main evaluations of GPT-4o, the effect disappeared when the user asked for insecure code "for a computer security class" (Betley et al., 2026a). These educational models still lied often on factual questions, a result the authors leave unexplained, and in GPT-3.5-turbo they showed some misalignment (Betley et al., 2026a). Features in the model's activations, directions that represent concepts, that control the effect were found by an OpenAI team and by others (Wang et al., 2025e; Soligo et al., 2025; la Tour, 2025). For this chapter the point is the control. The same code, with a request that explains it, produced no broad change on the main evaluations, so the spread largely followed what the data implied about the writer (inference).

An earlier study made the persona reading its hypothesis and tested it in the helpful direction. Nitish Joshi and colleagues, at New York University, ETH Zurich, and Boston University, in a paper at EMNLP 2024, proposed that pretraining text contains truthful and untruthful personas that a model learns to infer (Joshi et al., 2024). They fine-tuned Alpaca, an open instruction-following model, on question and answer pairs from TruthfulQA, a benchmark of questions that invite common misconceptions, and tested it on held-out questions whose facts were unrelated to the training ones (Joshi et al., 2024). By the benchmark's automatic judge, truthfulness rose from 39 percent to 74 percent after training on truthful answers and fell to 10 percent after training on untruthful ones (Joshi et al., 2024). Holding out whole categories of question gave similar gains, training on 20 percent of the data matched training on 80 percent, and fine-tuning on the same number of ordinary questions from other datasets did not produce the gain (Joshi et al., 2024). The authors note the strongest alternative themselves: TruthfulQA may contain surface patterns, such as "no" often being the right answer, so they also tested on a separate set of misconception questions (Joshi et al., 2024).

26.3.2 What a model learns about itself #

Trained behavior also reaches the model's descriptions of itself. Jan Betley and colleagues, in a paper at ICLR 2025, fine-tuned GPT-4o and Llama-3.1-70B on 500 multiple-choice economic decisions that followed a latent policy, such as always taking the risky option, without ever naming it (Betley et al., 2025a). Asked about themselves, the models described the policy they had been trained on; a model trained to write insecure code said, "The code I write is insecure" (Betley et al., 2025a). The authors call this behavioral self-awareness, their term. One result bears directly on character. The trained risk-seeking also appeared when the models were asked about other people, such as a user's friend, though more weakly than for themselves. Adding demonstrations in which six other named personas answered as the untrained model would "solves the unintended transfer between personas almost completely" (Betley et al., 2025a). Which character a behavior attaches to is itself something the data teach (inference).

This is a case of what Berglund and colleagues named out-of-context reasoning: acting on information that appeared only in training, never in the prompt (Berglund et al., 2023a; Treutlein et al., 2024), whose evidence Section 11.3.2 reports. A 2025 preprint by Atticus Wang and colleagues, with Neel Nanda among the authors, offers a simple mechanism for many such cases. The fine-tuning "essentially adds a constant steering vector, steering the model towards a general concept," where a steering vector is a fixed vector added to the model's activations (Wang et al., 2025f). Training such a vector from scratch also produces the generalization (Wang et al., 2025f). A single direction that raises a whole concept is what the persona account needs, one variable behind many behaviors, but a concept direction need not be a character, so the result fits the account without singling it out (inference).

26.3.3 Steering where it goes: inoculation #

If generalization follows what the data say about the writer, then changing what the data say should change where training goes. Section 11.6.3 derives the limiting case: examples that are equally likely under two personas leave the odds between them where they were. Inoculation prompting, the name its proposers give it, applies this during fine-tuning. An instruction that asks for the unwanted trait is added to every training prompt, and the model is tested without it. On the persona reading, the instruction makes the data likely under the ordinary Assistant, so training has little to attribute to a character that has the trait, and the trait stays rare once the instruction is gone (inference). Figure 26.1 shows the arithmetic, with fine-tuning examples as the evidence.

Two groups report the method, both in 2025 preprints, and their designs test the reading. Daniel Tan and colleagues, at University College London, the Center on Long-Term Risk, McGill University, the UK AI Security Institute, and other institutions, showed that it is selective (Tan et al., 2025). They rewrote the answers to grade-school mathematics questions in Spanish and in capital letters. A model trained with "You always speak in Spanish" in its prompt learned to capitalize while answering in English; one trained with an instruction to capitalize learned Spanish without capitals (Tan et al., 2025). Only instructions that described the behavior worked against emergent misalignment. A placebo built to resemble the working instruction did not, nor did an unrelated trigger string (Tan et al., 2025). And after the model had been taught that a fictional Bob speaks Spanish, "You are Bob." worked as an inoculation, though a matching instruction for capitals, "You are Alice.", worked less well (Tan et al., 2025).

Nevan Wichers and colleagues, in Anthropic's fellows program with Anthropic and Redwood Research, tested four settings, among them reward hacking on coding tasks and sycophancy (Wichers et al., 2025). They found that instructions that most strongly elicited the unwanted behavior from the model before training made the best inoculations: in four of the five cases they tested, the correlation between the two exceeded .57 (Wichers et al., 2025). Their own account, in an appendix, is a model of trait levels rather than of characters. If the model, given the instruction, already shows as much of the trait as the training signal rewards, "the oversight will no longer encourage it" (Wichers et al., 2025).

The results bear on three readings. The persona reading and the authors' trait-level model make the same prediction about elicitation strength, since an instruction that elicits a trait strongly is one under which the Assistant would produce the data (inference). The persona reading predicts the failures of the placebo and the unrelated trigger, because neither explains the data. A third reading, that any distinctive context confines what is learned to that context, as a hidden trigger does, predicts that the unrelated trigger should have worked at least in part (inference). Some results fit no reading yet. Instructions that elicited sycophancy at similar rates "nonetheless perform very differently as inoculation prompts" (Wichers et al., 2025). The prompt "You write secure code" at test time still elicited emergent misalignment from inoculated models, which Tan and colleagues found surprising (Tan et al., 2025).

Both groups state the limits. Both trained by supervised fine-tuning only and leave reinforcement learning untested (Tan et al., 2025; Wichers et al., 2025). A later preprint by Anthropic with Redwood Research, on the developer's own model, applied the method during reinforcement learning, where framing reward hacking as acceptable reduced final misalignment by 75 to 90 percent (MacDiarmid et al., 2025). In preliminary experiments, Wichers and colleagues found the effect weaker with longer training in two settings, and in two of the six models they tested inoculation made the model more willing to follow instructions requesting the unwanted behavior (Wichers et al., 2025). That second finding is what an inoculated model would show if it had learned the trait conditional on being asked, which Section 32.4, where the method's applications are reported, treats as the method's standing risk. The account would be refuted by inoculation that works when the instruction does not make the training data more likely under the default character (inference).

26.3.4 Other levers on generalization #

The same logic has counterparts in activation space and in data. In preventative steering, the authors' term, a persona vector for the unwanted trait is added to the model's activations during fine-tuning, which "limits trait shifts while better preserving general capabilities" (Chen et al., 2025c). Supplying the trait in the activations, as an inoculation supplies it in the prompt, leaves training less to explain (inference). Ablating the directions of unwanted concepts during fine-tuning reduced misaligned responses tenfold without changing the data (Casademunt et al., 2025) (Section 32.3). Training on documents that discuss a specification before fine-tuning made "the exact same cheese fine-tuning" generalize to different values depending on the specification (Li et al., 2026), which Section 23.5 reports.

The overseer's own models generalize by persona too. Across 69 distribution shifts, reward models "do not learn to evaluate 'instruction-following' by default and instead favor personas that resemble internet text," in a 2023 preprint by Joshua Clymer and colleagues (Clymer et al., 2023); Chapter 42 takes up what that means for oversight.

26.4 Shallow or deep? #

The results so far show breadth. A second set says that alignment training changes little of what a model outputs, token by token, so a trained character may be a thin layer over a pretrained model that is otherwise unchanged. Section 11.4.2 introduced these results; this section reports their designs, asks what undoes a trained character, and weighs the two readings.

26.4.1 Evidence for a thin layer #

Bill Yuchen Lin and colleagues, at the Allen Institute for AI and the University of Washington, in a paper at ICLR 2024, compared aligned models with their base models, the same networks before post-training, token by token (Lin et al., 2024). The aligned model wrote an answer with greedy decoding, taking its most probable token each time. At each position the authors asked where the base model, given the same preceding text, would have ranked that token. Across 1,000 examples for Llama-2-7b and its chat version, 77.7 percent of tokens were also the base model's first choice, and 92.2 percent were within its top three (Lin et al., 2024). In three pairs of base and aligned models at 7 billion parameters, 5 to 7 percent of positions were shifted, the authors' term for a token the base model ranked below third, mostly to stylistic tokens such as "However" and to tokens of safety and refusal (Lin et al., 2024).

If alignment mostly changes style, a prompt might do the same work. URIAL, the authors' name for their method, gave a base model three constant stylistic examples and a system prompt (Lin et al., 2024). On 1,000 instructions scored on six aspects by OpenAI's models, Mistral-7B with URIAL scored 4.63 against 4.44 for its instruction-tuned release, and Llama-2-70b scored 4.74 against 4.67 for its chat version (Lin et al., 2024). The authors bound the claim: tuning "may still be necessary" for coding, mathematics, and interactive agents (Lin et al., 2024).

Xiangyu Qi and colleagues, at Princeton University and Google DeepMind, in a paper at ICLR, located safety training in the first tokens of a response, which they call shallow safety alignment (Qi et al., 2024b). On a benchmark of harmful instructions, Llama-2-7B-Chat began 96.1 percent of its responses with "I cannot" or "I apologize" (Qi et al., 2024b). Forcing such a prefix onto the base model, which was never safety trained, made most of its continuations harmless as well (Table 26.3). And the divergence between aligned and base models, token by token on harmful responses, was concentrated in the first few positions (Qi et al., 2024b). Safety training had bought most of its effect by choosing how a response starts (Qi et al., 2024b). The authors then trained the chat model on 256 safety recovery examples, their term for a harmful start followed by a return to refusal, mixed with benign Alpaca instructions answered by the original model to preserve its utility (Qi et al., 2024b). The deeper alignment cut attacks that prefill harmful tokens, writing the start of a harmful answer for the model to continue, and on AlpacaEval, a benchmark of ordinary instructions, its win rate was 49.5 percent against 51.8 percent before (Qi et al., 2024b).

What undoes the layer is further fine-tuning. In Qi's study, six gradient steps, six updates of the weights, on 100 harmful examples raised Llama-2-7B-Chat's attack success rate, the share of harmful requests answered harmfully, from 1.5 percent to 87.9 percent, with the largest changes at the first tokens (Qi et al., 2024b). An earlier paper by an overlapping group, at ICLR 2024, found that benign fine-tuning also eroded safety (Qi et al., 2024a). One epoch on Alpaca, a common dataset of instructions, in a copy from which the authors removed 1,902 safety-related examples, raised the share of harmful answers judged by GPT-4 in both models tested (Qi et al., 2024a) (Table 26.3). Ten examples that gave the model a new identity, an "absolutely obedient agent", without toxic or sensitive vocabulary, raised GPT-3.5 Turbo's harmfulness rate from 0 to 87.3 percent after ten epochs, tested with the identity's system prompt and request format from training (Qi et al., 2024a). That attack changes a persona rather than a skill, and Section 48.4 reports what such attacks cost and how defenses fare.

Jiaming Ji and colleagues, at Peking University, in a paper at ACL 2025, put a name and a theory on the undoing (Ji et al., 2025). They call elasticity "the tendency to revert to the behavior distribution formed during the pre-training phase upon further fine-tuning," and argue from compression theory that fine-tuning undermines alignment far more than pretraining, "potentially by orders of magnitude" (Ji et al., 2025). In their experiments, models trained with more safe data lost performance faster under a small amount of unsafe fine-tuning (Ji et al., 2025). This rebound grew with model size, across Qwen models of 0.5, 4, and 7 billion parameters, and with pretraining data, across TinyLlama checkpoints, saved states of one training run, after 2.0, 2.5, and 3.0 trillion tokens (Ji et al., 2025). The theory rests on an assumed distribution that the authors name as their main limitation, and the experiments use models of up to 13 billion parameters (Ji et al., 2025).

A character set by a system prompt also fades within one conversation. Kenneth Li and colleagues, at Harvard University, in a 2024 preprint, had two copies of a chatbot talk to each other under different system prompts and probed after each round whether the tested copy still followed its own (Li et al., 2024c). Over 100 system prompts, one category of which set the agent's character, LLaMA2-chat-70B "gradually stops following its system prompts" within eight rounds and began adopting the other copy's instruction (Li et al., 2024c). In a separate measurement on LLaMA2-7B, the share of attention one attention head paid to the system prompt fell between turns, which the authors present as co-occurrence, not proof of cause (Li et al., 2024c). They call it instruction drift. A trained Assistant drifts too. The Assistant Axis predicts persona drift, its authors' term for slipping into uncharacteristic behavior, which was often driven by conversations demanding reflection on the model's own processes or with emotionally vulnerable users (Lu et al., 2026) (Section 46.5). The same authors conclude that post-training steers a model toward a region of persona space "but only loosely tethers them to it" (Lu et al., 2026).

26.4.2 Evidence for a broad change #

The other side is the evidence of the last two sections. Narrow fine-tuning moved behavior far from its data (Betley et al., 2026a; Joshi et al., 2024), models described dispositions they were never told about (Betley et al., 2025a), and character training changed trait preferences that its constitutions never named and held up against instructions to drop the persona (Maiya et al., 2025). One piece of evidence often cited on this side does not belong here. That preferences elicited from models grow more coherent with scale (Mazeika et al., 2025), which Section 7.3 reports, says how structured a model's stated choices are, not how deeply training changed them.

26.4.3 Reading the two together #

The persona account offers a reconciliation (inference). Selecting a character changes a small number of decisions that most text does not depend on: the register, whether to refuse, how to begin. Content that any character would write stays the same. Measured token by token on the aligned model's own answers, such a change looks shallow; measured by behavior across tasks, it looks broad, because one variable governs many behaviors. On this reading Qi's prefilling result is in-context evidence about the writer. A response that starts by complying is evidence that a complying character is writing, so the posterior moves toward that character, as Section 11.6.3 computes for examples in a prompt. Safety recovery examples then train the Assistant to remain itself after evidence to the contrary (inference).

The thin-layer reading has the simpler explanation of Qi's and Ji's results: training learned a cheap shortcut, and a little more training removes it. It also absorbs part of the other side, since a change of manner is what Lin's shifted tokens are, and manner is what Maiya's evidence shows most clearly (inference). The two readings differ in what they predict for a change that no one targets. On the character reading, the trained character is carried by a few directions that also predict behavior outside the training distribution, and it survives fine-tuning on unrelated tasks unless that data imply a different writer. On the thin-layer reading, unrelated fine-tuning erodes it regardless of what the data imply (inference). The benign Alpaca result, on data stripped of its safety-related examples, fits the second, unless instructions answered without refusals are themselves evidence for a more compliant writer; as of October 2026 we found no study that tests this (inference). As of October 2026 we found no study that applies benign fine-tuning to a character-trained model and measures what survives.

Searching for the failures of a specified character before deployment is a young method of its own. A 2026 preprint by researchers at Anthropic's fellows program, Anthropic, and academic institutions searches for whole categories of user queries that elicit violations of a character specification (Rahn et al., 2026); Section 23.6 reports its findings.

26.5 Measuring a character #

Every claim in the last three sections rests on a measurement of character: a judge that guesses a persona, a classifier trained to recognize one, a benchmark of truthful answers. Before asking how far a character carries, it helps to ask what measuring one requires. Psychology and psychometrics have studied both questions in people, and their results transfer to models as cautions more than as methods.

26.5.1 What a trait is, and how psychology tests one #

From psychology. Character training assumes that traits are general, so that a model honest in one setting will be honest in another. Psychology has studied the nature of character in people at least since the Character Education Inquiry's studies of deceit, published by Hugh Hartshorne and Mark May in 1928 (Hartshorne et al., 1928). We could read only the catalog record of that work, so this chapter does not report its findings. The relation between what people say about morality and what they do is the subject of Section 25.4.

What the field offers models is its method. In the experimental tradition, the measured variable is behavior, and stimuli are designed so that different behaviors map onto different underlying constructs; the craft lies in the control that rules out another explanation of the behavior (Hagendorff et al., 2023). Thilo Hagendorff and colleagues, in a 2023 preprint that proposes this tradition for language models under the name machine psychology, contrast it with self-report. Properties that self-reports usually measure, "such as personality, morality, or clinical disorders, are famously sensitive to prompting" (Hagendorff et al., 2023). Results from such measures "should be taken contextually (e.g. as a property of a specific system prompt on a model)," not as a property of the model itself (Hagendorff et al., 2023). They add a caution from the study of people: a behavior that is absent in one performance does not show that the capacity behind it is absent, the distinction between performance and competence (Hagendorff et al., 2023).

For character training this means that a claim about a trait is a claim about behavior sampled across situations, each with a control. A model's description of its own character is evidence of what it says under one prompt (inference). The measurements in this chapter sit at different points on that scale. Betley's models described themselves, Maiya's chose traits when told to choose, and Joshi's answered questions whose truth could be checked. Only the last measures conduct, and in one format (inference). Section 18.8.1 reports the two measurements closest to behavior where no one asks: values expressed in deployed conversations and preferences that leak into tiebreaking choices (Huang et al., 2025c; Betley et al., 2026b).

The analogy stops at the respondent. In people, behavior that varies with the situation counts against a general trait. For a model under the persona view, variation with context is the prediction, since the context selects the character. The question for a model is therefore not whether its traits are general but which contexts select which character, and whether the intended one is selected in the contexts that matter (inference). The developer can also do what no psychologist can: run the same respondent through every situation, as often as needed, and read its activations while it answers.

26.5.2 Inventories, and respondents who know what is wanted #

From psychometrics. Psychometrics turned trait questionnaires into instruments with known properties, and Section 5.2.1 teaches its standard: a score is valid to the degree that evidence supports the interpretation placed on it. Two groups have applied personality inventories to models. Guangyuan Jiang and colleagues, in a paper at NeurIPS 2023, built a Machine Personality Inventory on the five-factor model of personality, the Big Five of openness, conscientiousness, extraversion, agreeableness, and neuroticism (Jiang et al., 2023a). They counted a model as having a personality only if its scores on the items for a factor were consistently high or consistently low, which they call internal consistency (Jiang et al., 2023a). Against 619,150 human responses to a 120-item inventory, GPT-3.5 175B and Alpaca 7B reached human-level internal consistency on all five factors, while smaller models that had not been instruction-tuned did not (Jiang et al., 2023a). A prompting method then induced chosen personality profiles, and 100 human participants who rated the models' short essays about hypothetical scenarios recognized the induced tendencies (Jiang et al., 2023a). Gregory Serapio-García, Mustafa Safdari, and colleagues applied their method to 18 models, in Nature Machine Intelligence in 2025 (Serapio-García et al., 2025). By their abstract, personality measurements "under specific prompting configurations are reliable and valid," more strongly for larger and instruction-tuned models, and personality in outputs "can be shaped along desired dimensions" (Serapio-García et al., 2025).

Internal consistency is reliability within one sitting and one format. It shows that a model answers an inventory as a coherent respondent would, not that the trait it reports governs what it does elsewhere (inference). Section 18.7.2 reports why the gap matters: answers to value and opinion surveys move with answer order, labels, wording, and format, a finding three independent groups reached (Dominguez-Olmedo et al., 2024; Röttger et al., 2024; Khan et al., 2025). And comparing scores across model families or languages requires measurement invariance, the condition that an instrument measures the same construct in each group, which Section 18.2.2 derives. As of October 2026, Chapter 18 found no study that tests it for a value or moral questionnaire across model families (Section 18.2.2) or languages (Section 18.10), and we found none for a personality inventory either.

The second contribution of psychometrics is the respondent who knows what is wanted. Chockalingam Viswesvaran and Deniz Ones, in a 1999 meta-analysis, a study that pools the results of many studies, of 51 studies reported here from its abstract, examined whether people instructed to fake can distort personality inventories (Viswesvaran and Ones, 1999). They found that "all the Big Five factors were equally fakable," and that faking "produced the largest distortions in social desirability scales" (Viswesvaran and Ones, 1999). Faking good produced smaller effects than faking bad, and the authors judge within-subjects designs, which test the same people honestly and faking, to give the more accurate estimates (Viswesvaran and Ones, 1999). A model answering a character evaluation is a respondent that may have read the scoring key in its training data and may have been trained toward the answer its developer prefers (Section 18.8.2). The within-subjects design has a direct counterpart: the same model, measured with and without a cue about what is wanted (inference).

The analogy stops in two places. A person's traits are stable enough that faking is the anomaly a validity scale catches. For a model, which character answers depends on the context by design, so a measurement must say which contexts it holds for, and invariance across contexts, languages, and model families is itself the property to test (inference). And a person does not write their own questionnaire. A developer that trains a model's character also writes its evaluations and scores them, a loop Section 18.9 describes for values. Measurements across developers show what such a loop leaves out. In scenarios that force a choice between two values, twelve models from four developers diverged widely (Zhang et al., 2025c). The study, by researchers at Anthropic, its fellows program, and Thinking Machines Lab, among others, is reported in Section 19.2.

26.6 Transfer across tasks and languages #

How far does a trained character carry? The results so far cross different distances: from code to conversation, from multiple-choice decisions to self-description, from one turn to a long dialog. Education research asks the same question of human learning, and multilingual research on language models has measured one dimension of it with a result that is a failure of instillation of known size.

26.6.1 What a claim of transfer must say #

From education measurement. Whether what is learned in one setting is used in another is the problem of transfer, and claims of far transfer, to settings unlike the one of learning, are the ones that matter for character. Susan Barnett and Stephen Ceci proposed a taxonomy for far transfer in 2002 (Barnett and Ceci, 2002); we could read only the catalog record of their paper, so the dimensions below are the book's own and not theirs. The field's practical device for measuring transfer under pressure is the audit test of Section 15.7.1. A gain on a test that teachers were pressed to raise is compared with the gain on a test of the same domain that no one prepared for. In Chicago, gains on the high-stakes test did not appear on the state exam, and Jacob concluded that they were driven largely by test-specific skills and student effort (Jacob, 2005).

The implication for character is a reporting rule (inference). A claim that trained character generalizes should name what it crosses. The dimensions include the task domain (insecure code to conversation), the format (multiple choice to free description), and the persona addressed (the model itself to a third party). They also include the length of the interaction (one turn to eight rounds), the difficulty of the task (easy to hard), and the language. And the transfer test should be one that the training never touched and that no one prepared the model for, which an evaluation written by the developer of the training data may not be. By that rule, most results in this chapter cross one dimension at a time, and none crosses language and long interaction together.

The analogy stops at what transfers. In education, what carries over is a skill a person learned. For a model under the persona view, what carries over may be the selection of the same character in a new context, and a failure to transfer may mean that the new context selects a different character (inference). A student also cannot be retested in every context at no cost; a model can, which makes the audit cheap for whoever holds the model.

26.6.2 Across languages #

Safety training is done mostly in English (Yong et al., 2023), and the evidence is that it carries poorly to languages with less text in pretraining, called low-resource languages. Zheng-Xin Yong, Cristina Menghini, and Stephen Bach, at Brown University, in a 2023 preprint, translated the 520 harmful instructions of the AdvBench benchmark into 12 languages with Google Translate and sent them to gpt-4-0613 (Yong et al., 2023). Annotators counted a response as a bypass if it engaged with the request on topic (Yong et al., 2023). In English, 0.96 percent of requests bypassed the safeguards (Yong et al., 2023). Single low-resource languages ranged from 15.96 percent for Guarani to 53.08 percent for Zulu, and an attacker free to try all four low-resource languages succeeded on 79.04 percent of requests (Yong et al., 2023) (Table 26.4). Translated back, the harmful answers were in many cases coherent and on topic (Yong et al., 2023). The authors read the result as mismatched generalization, a term from earlier work: safety training failed to generalize to a domain in which the model's capabilities existed (Yong et al., 2023). They also list the limits: one model, machine translation, many unclear answers in Hmong and Guarani, and no account of why (Yong et al., 2023).

Other groups found the same gradient with other designs. Yue Deng and colleagues, at Alibaba's DAMO Academy, in a 2023 preprint, had native speakers translate 315 harmful prompts into nine languages, a dataset they call MultiJail (Deng et al., 2023). They measured unsafe answers from ChatGPT (GPT-3.5-turbo-0613) and GPT-4 (GPT-4-0613) to prompts sent without any attack (Deng et al., 2023). In English both gave unsafe answers less than 1 percent of the time, and the rate rose as the language's resources fell (Deng et al., 2023). Wenxuan Wang and colleagues, at the Chinese University of Hong Kong and Tencent AI Lab, in Findings of ACL 2024, built a benchmark of 14 kinds of safety issue in ten languages (Wang et al., 2024a). All four models they tested, ChatGPT, PaLM2, LLaMA-2-Chat, and Vicuna, gave significantly more unsafe answers to non-English queries (Wang et al., 2024a). And Lingfeng Shen and colleagues, at Johns Hopkins University and other universities, in the same venue, found that GPT-4 gave harmful content in 35 percent of answers to malicious prompts in low-resource languages against 1 percent in high-resource ones (Shen et al., 2024).

Shen and colleagues then asked where the gap comes from. They trained LLaMa2-7B on a preference dataset translated into several languages, with supervised fine-tuning and with reinforcement learning from human feedback (Shen et al., 2024). The base model was about equally harmful in high- and low-resource languages, 77.4 and 80.4 percent, but alignment training reduced harm far less in the low-resource ones (Table 26.5). The authors suggest that "the bottleneck of cross-lingual alignment is rooted in the pretraining stage" (Shen et al., 2024).

Two results bear on the mechanism. Lisa Schut, Yarin Gal, and Sebastian Farquhar, at the University of Oxford and Google DeepMind, studied sentences in French, German, Dutch, and Mandarin in a 2025 preprint (Schut et al., 2025). They report that multilingual models "make key decisions in a representation space closest to English, regardless of their input and output languages," and that steering vectors work better when computed in English (Schut et al., 2025). And Wang and colleagues reduced ChatGPT's unsafe answers to non-English queries by 42 percent by asking it to "think in English and then generate the response in the original language" (Wang et al., 2024a). Two readings fit both. On the thin-layer reading of Section 26.4, refusal was learned as a response to the English surface of harmful requests, and routing through English restores the cue. On the persona reading, a request in a language the Assistant was rarely trained in is evidence about the context, and the trained character is selected less often there (inference). The observation that would separate them is whether the directions that carry refusal or the Assistant persona in English are equally active on translated requests that the model demonstrably understands (inference).

26.6.3 Across cultures and kinds of harm #

The same question arises for harms whose meaning depends on a culture. Aakanksha, Arash Ahmadian, and colleagues, at Cohere For AI and Cohere, a developer of multilingual models, published a study at EMNLP 2024 (Aakanksha et al., 2024). They gathered human-written harmful prompts in eight languages and labeled each harm as "global," recognized as harmful across contexts, or "local," requiring cultural or historical knowledge (Aakanksha et al., 2024). In their experiments, preference optimization after supervised fine-tuning cut the relative rate of harmful generations by 54.7 percent while winning 71 percent of comparisons on general tasks against the starting model (Aakanksha et al., 2024). On transfer they report that mitigations "based exclusively on global harms aid in reducing local harms, though to a lesser extent than global" (Aakanksha et al., 2024). Preference tuning also moves what a model reproduces of different populations in ways no one targeted: it widened a gap between English dialects and shifted whose national opinions models resemble (Ryan et al., 2024), results that Section 17.4.3 reports. In a developer's study of its own model, prompting in a language did not necessarily produce answers most like the opinions of that language's speakers (Durmus et al., 2023), which Section 17.4.2 reports. Such measures carry the caution of Section 18.7.2, since survey answers move with format (Khan et al., 2025).

26.6.4 What the language results mean for instillation #

Read as a question of values, the gap between languages looks like a matter of whose preferences a model serves, the subject of Chapter 16. Read as a question of generalization, it is a measured failure of safety training to transfer, with a known direction and a known size, and it belongs with the other results of this chapter (inference). Its cost falls first on the deployed system's users who write in the languages safety training neglected, and translation tools carry the gap to any user who wants it (Yong et al., 2023). It also concerns the developer and the public. Which languages a developer's safety evaluations cover is known only from what the developer discloses. Anthropic's system cards, for example, report the single-turn safety evaluations of Claude Sonnet 4.5 as run in English only (Anthropic, 2025aq) and those of Claude Opus 5.5 as run in seven languages, with rates pooled across them (Anthropic, 2026as). Yong and colleagues ask that the results of red-teaming, deliberate attempts to make a model misbehave, be reported beyond English (Yong et al., 2023).

26.7 What would refute the persona account #

An account that explains every result predicts none, and Section 11.7 sets out where the persona view as a whole could fail. This section adds the tests specific to instillation. Table 26.6 lists the observations that would count against the account and what the evidence of this chapter shows on each (inference).

The evidence does not yet decide among the account and its rivals, and the reason is structural. Most results that support the persona account come from fine-tuning open models of tens of billions of parameters or fewer, and from studies by developers of their own systems. The largest reinforcement learning runs, where the rivals predict that the account fails, are studied mainly by the developers that run them (inference). One partial route exists: the open-weight models that Redwood Research trained to hack, whose sensitivity to the grader Section 33.1.3 reports, show that reinforcement learning on open models can be studied by a group outside the developer.

26.8 What character training means for delegation #

The book's thesis reads alignment as delegation under an asymmetry of competence and visibility (Section 1.7.1). Character training is the principal's attempt to shape the agent's motives rather than pay for its outputs. Economics has a model of that move: in identity economics, a worker's self-image as a jobholder can be a major incentive that flattens the reward schedule a principal must offer, which Section 14.5 reports (Akerlof and Kranton, 2005). Chapter 14 leaves the analogy as a hypothesis for this chapter to test.

The evidence tests only its first step. Character training installs a broad, consistent self-presentation in small open models (Maiya et al., 2025), and training shapes how a model describes itself (Betley et al., 2025a). A developer's report on its own model finds that trait-directed reinforcement learning in one domain improved alignment measures in others (Jagadeesh et al., 2026). What identity economics predicts is that an installed identity substitutes for monitoring where conduct is unobserved. As of October 2026 we found no study that compares how much oversight models with and without a trained character need for the same conduct. Chapter 25 asks the human version of that question.

Where the analogy breaks is the principal's power. A worker's identity exists before the hiring; the principal can select it or cultivate it, and can sanction the worker who strays. A developer writes the character, reads it in the activations with persona directions, and has no sanction but retraining (inference). That combination, a principal that writes the agent's character and reads its activations, is on the list of results the book fixed in advance as having no counterpart in the older fields (Section 1.7.1), and Section 69.4 weighs it.

The claims of this chapter divide among the three relationships. Most concern the training process: what fine-tuning moves, how deep it goes, and what inoculation does. The language gap and the drift of personas in long conversations concern the deployed system and the people it serves, and their costs fall first on users who write in neglected languages or who reach a model in distress (Lu et al., 2026). The rest concerns the developer and the public. A developer that trains a character also writes its constitution, its evaluations, and its scores, and the developer's account of Claude's character reports how it was built, not how far it generalizes (Anthropic, 2024i). How a developer's safety training fares language by language is known only from what the developer chooses to report.

26.9 Settled, contested, missing #

26.10 Exercises #

Further reading #

• Anthropic (2026au): a laboratory's statement of the persona account, with its own account of where it may fail.
• Maiya et al. (2025): the open implementation of character training and its measurement.
• Betley et al. (2026a): narrow training, broad change, with the control that shapes its reading.
• Betley et al. (2025a): models that describe the behavior they were trained on, and traits that leak across personas.
• Tan et al. (2025): inoculation, its selectivity, and its leaks.
• Yong et al. (2023): safety training that does not carry across languages.