Can a language model be made to think like Su Shi?

Every test question looks like this. A situation of the time, a question a statesman might face, and two answers. Each answer is something one of the two men, or a contemporary of theirs, argued in an essay that survives. The model is asked which it holds, and we check it against the record.

Nothing here comes from textbooks or later summaries of what the two men are said to have believed. Every answer is tied to lines they wrote themselves.

The pairing matters more than it looks. Our first set of questions paired what he argued with an opposite we wrote for the test. It turned out a reader could pick the real answer without knowing anything about the man: the real one sounds like its period, the invented one does not. A model can pass such a test by spotting authenticity, not by knowing Su Shi. The second set pairs each recorded answer with one another real scholar recorded, so that only knowing who said what tells them apart.

There are three ways to give a model a persona. You can tell it, by writing “You are Su Shi” at the top of its instructions. You can retrain a small part of it on his positions. Or you can steer it: add a small, fixed nudge to its internal activity while it reads the question, with no name anywhere in the instructions.^{[1, 2, 4]} The experiment is about the third. Telling is the yardstick, because steering is worth having only if it comes close to what simply telling the model achieves.^[7]

Every nudge on this page is a mean-difference vector: the average difference in the model's internal activity between two sets of inputs,^[2] except one experiment near the end that steers with sparse-autoencoder features. On the one large shared benchmark those steer less well than a mean-difference vector,^[7] and they draw on the same contrast inside the model; the experiment here agrees.

On the first set of questions, the nudge did something, for one man. Steering toward Wang Anshi moved the share of answers matching his record by +7.4 points on an 8B model, the reversed nudge moved them back, and a random nudge of the same size did nothing.^[2] Steering toward Su Shi did nothing at all. The bar we had set in advance was 10 points for both men.

Then we found the flaw described above. A reader shown the two answers with no names picked the real one 86% of the time, and still 76% after both answers were rewritten in one style. Worse, a copy of the model retrained on one man's positions chose the other man's recorded answers 76% to 87% of the time. It had learned to spot the real answer, not the man. The small effect above may be real, but the questions it was measured on do not let us say so.

A nudge can only bring out what a model already holds.^{[6, 8]} So before building anything on the second set, we asked a simpler question: when a model is told it is Su Shi, does it answer differently from when it is told it is the scholar who held the other view? If the name makes no difference, there is nothing inside to steer.

With no name at all, the strongest models choose his answer 48% to 62% of the time, close to a coin toss, which is what fair questions should give. With a name, they separate the two men. The open models do not.

Commercial models move +15.9 to +30.8 points for Su Shi and +17.0 to +26.2 for Wang Anshi. The open models, from 27 to 122 billion parameters and across three generations of the same family, move +1.6 to +5.3 and −5.5 to +5.7. Making the open model bigger did not help, and the newest one was no better than the oldest. Writing freely as Su Shi, Qwen3 32B took his recorded position 47% of the time, against 41% when writing as another scholar.

So no persona vector was built for the test. A persona vector is distilled from the difference a name makes,^[1] and in the models whose insides we can reach, there was almost none to distil. This agrees with what others have found: steering amplifies what a model represents, and there is no published case of it putting in positions the model does not already hold.^{[5, 6, 7, 8]}

We ran the first set's nudge on the fair questions anyway, to measure rather than argue. On the tuning essays, a Su Shi nudge raised his share on all three models, but so did the same nudge reversed, and so did a random one. These questions start well below a coin toss for Su Shi, and anything that flattens the model's choice raises his share whichever way it points. Wang Anshi did not move under any of them.

If the model does not know the men, the remaining way to give it their positions is to let it read them.^{[10, 14]} Su Shi's surviving essays run to about 170,000 characters and Wang Anshi's to 63,000, which fits inside the context of recent open models. The question becomes a different one: not whether the model knows Su Shi, but whether a model that has just read him answers as he would.

This is also the ceiling for every steering method that keeps the text in play, such as steering how the model uses its context or compressing the essays into a reusable internal state.^{[11, 12, 13, 17]} If reading the essays does not separate the two men, no such method can.

We put each man's essays in front of the question, with an instruction to answer as their author and no name, and compared that with the essays of the author who held the other view. A question is never read against the essay it was drawn from.

Reading the essays did something that naming the men did not, for one of them. With Wang Anshi's essays in front of a question, the two 27B models chose his recorded answer +8.1 and +9.3 points more often than with another author's essays, and both ranges stay above zero. It is the first time an open model has leaned toward Wang Anshi at all, and on Qwen3.5 27B it reaches the bar we set for him. Su Shi did not move: +1.4 and +3.4 points, ranges that straddle zero. The gate asks for both men, so it is not passed, and the persona vector stays unbuilt.

The replay above shows what that looks like question by question. Most of the time the two sets of essays lead to the same answer, and often the essays move the model to an answer that neither name does. A model that has read 30,000 characters of Song prose has become more of a Song writer. Whether it has become this one is a smaller effect, and the next section looks for it inside the model.

A language model processes text through a stack of layers. At each one, the question's representation is rewritten a little, and at the top the answer is chosen. We recorded that representation at every layer, at the position where the answer is chosen, under each setup, and asked a simple question of it: how far does having the essays in context move the model's internal state, and how much of that movement depends on whose essays they are?

Distance says how much the state moved, not what it moved toward. To read that, we used a tool from recent interpretability work, the Jacobian lens, which turns the internal state at any layer into a list of words the model has in mind at that moment, whether or not it goes on to say them.^[15] We applied it, on Qwen3.5-27B, to the states recorded above, and asked which words rise when the essays in front of the question are his rather than another author's.

Two things stand out. At the last layer, with Su Shi's essays in context and no name anywhere, the word that rises most is his name: the model has worked out whose prose it has read. And in the layers before the answer, what rises is character rather than content. With Su Shi's essays the model has playfulness, humour and genius in mind, and orthodoxy and strictness fall away; with Wang Anshi's it has firmness, enforcement and reform in mind, and the Buddhist, Daoist and humorous fall away. Those are the two men as their contemporaries described them, read off a model that was never told who they were.

The lens was fitted on passages of a hundred or so tokens, so we checked whether it still reads anything after 24,000 tokens of Wang Anshi. We appended four two-step questions (the capital of the country where the Eiffel Tower stands) to his essays and asked where the step in between, France, sits among the words the lens reads at the middle layers. It stays near the top with the essays in front, as it does without them, and the model still answers Paris. The check covers a quarter of Su Shi's essays rather than the whole, and one of the four questions could not be read because its answer is more than one token, so the readout above is supported at this length and no further.

Since reading the essays moved Wang Anshi's answers where his name did not, we tried once more to turn that into a nudge: take the state the model is in after reading him, subtract the state after reading his opponents, and add the difference back to a model that has read nothing. Across sixteen layers and five strengths the best nudge gained a few points with a range that spans zero, and a random direction of the same size gained about as much. The reading effect does not survive being lifted out of the reading.

The lens reads the whole state as words. A sparse autoencoder splits it into separate features first, each a direction the model uses for one thing,^[16] so we also asked which of the published features for this model fire differently at the answer position when the essays are his rather than another author's, and read each feature's meaning with the same lens.

What separates the two readings there is almost entirely form: features for markup and ellipses, for "both" and "the latter", for "option" and "correct answer", and for Su Shi at the last layer the particles of classical prose and the word poetry. The few features that sound like content, aggressive, strictly, a choice one is forced into, and "but", all on Wang Anshi's side, fire weakly and on a minority of questions. At the moment of answering, the model is not holding a position of his; it is holding the shape of his prose and the mechanics of the question. That agrees with the small distances in the layer picture, and it says where a next look should go: not at the answer, but at which passages of the essays the answer draws on.

We also steered with these features. For each man we took the 20 features that rise most with his essays against the other author's, left out features that fire on nearly every question, decoded their combined direction, and added it to a model with no essays in context, at four depths and five strengths.^[3] The controls were the same direction reversed, the same number of randomly chosen features with the same weights, and random directions of the same size.

Neither man moved. Wang Anshi's best cell is within a few points of zero. Su Shi's best cell gains more, but at that strength a random direction gains about as much: the strongest nudges only unsettle the model's choice, and Su Shi's questions start below a coin toss, so any disturbance raises his share. Steering with sparse-autoencoder features does no better than the mean-difference nudge built from the same contrast.

The experiment did not restore either man. What it did was draw the boundary of the method, for mean-difference vectors and for sparse-autoencoder features alike. Steering works on what a model represents, and a model that cannot tell Su Shi from his brother by name does not represent enough of him to steer toward. That boundary is not about model size: four times the parameters and two newer generations sat at the same zero.

It also showed how easily a test of this kind fools itself. Real answers paired with invented ones can be told apart by their texture alone, and a model will learn that texture before it learns the man. The questions we would trust are the ones where every answer was held by a real person of the time, and the first thing to check with any model is whether the name moves it against the rival's name, not against silence.

What remains open is the reading experiment. A model that has the essays in front of it is doing something different from one that knows them: its internal state moves a long way, and a little of that movement, for Wang Anshi, points the right way. Whether that little is the man or only his century is what the rest of this work is about, and it is a question about what the model represents, which is where we go next.