Can a model read what two rivals argued, and not just how they wrote?

When an expert reads a page, something happens that we cannot watch. They notice what is said between the lines, they recognize a position from a turn of phrase, they connect this memorial to that debate. We only see the interpretation that comes out. The process stays shut.

A language model reads the same page and also produces an interpretation, and its reading can be good. The difference is that its process is open. As the model reads, every character sets off activity inside it, and that activity can be broken into units that each respond to one kind of thing: a character, a turn of phrase, a subject, perhaps a position. Each unit can be opened to see which lines of the corpus it responds to. Recent work reports that a model keeps a set of such internal representations it is ready to put into words, which is close to saying that what the model has in mind as it reads can be read off.^[1] That is what this page tries to do with two men and their texts.

Three generations of computational reading come before this one. Each reads more deeply than the last and shows less of how it read; what separates them is what is taken out of the model. The last row is the approach this study takes.

This study belongs to a fourth line of work, called mechanistic interpretability, which tries to describe what a model computes on its way to an answer in units a person can read. Its main tool is sparse dictionary learning, in the form of a sparse autoencoder or a transcoder: a stand-in for one layer of the model, trained so that the layer's dense, unreadable activity is rewritten as a short list of readable parts, each of which is on or off for a given character. The parts are the features. The dictionaries used here are transcoders, one per layer of the model, published for Gemma-3-27B-IT in two sizes and for Qwen3-14B in one.^{[5, 6, 7]}

The idea is to keep the third generation's depth of reading, since the features are taken from a large model as it reads the text, and to recover the first generation's transparency, since each feature can be opened and checked against the lines it fires on. This line of work grew up inside AI research, where it is used to understand and audit models. Here it is turned on a historical question.

Telling the two men apart is not the hard part. Character frequencies alone go a long way, because each man had his habits. The hard part is to take the difference a model finds and separate the share that follows what they argued from the share that follows how they wrote. Everything in the method below is built around that separation.

The two men are the case study. Their quarrel over the New Policies is one of the best documented in Chinese history, which makes it a test the method can be held to: if it finds stance at all, it should find the oppositions historians already know. What the method can deliver is the dimensions along which the two men part and how far. Why they parted is a historian's question, answered by putting the result back into its time.

Take the pair of passages shown further down, both about the Green Sprouts loans, the scheme under which the state lent money or grain to farming households in spring and collected it with interest after the harvest. Wang Anshi writes that the poor used to borrow from rich families and now borrow from the government at lighter interest. Su Shi writes that officials who fail to collect will be punished, so they will force the loans on households whether they want them or not. A feature that responds to the subject, lending to farmers, fires on both passages. A feature that follows the stance fires on one side and not the other. And a feature that responds to the prose of a memorial addressed to the emperor fires on both, because both passages are memorials.

Now change the genre. Much of Su Shi's surviving prose is examination writing: essays and policy answers written for the civil service examinations and in their manner. Much of Wang Anshi's is letters and records. A feature for examination prose fires wherever Su Shi writes and seldom where Wang Anshi does. It separates the two men perfectly and says nothing about what either thought. Comparing the two men alone cannot tell this feature from a stance feature, because in that comparison the author, the style and the stance all change at once.

The way out is to compare passages that differ in only one of these. The two men shared a classical education and agreed on more questions than they fought over, so there are many pairs of passages in which they take the same side. In such a pair the author and the style differ and the stance does not, so whatever separates the passages comes from style. Pairs where they disagree add stance on top. The difference between the two kinds of pair is where stance lives.

Even if the model reads classical Chinese well, the dictionary that turns its activity into features might not. The dictionaries were trained on the modern web. Whatever part of a layer's work the dictionary cannot rewrite in its own parts is left as an error term, and that part is invisible to us even though the model did compute it. The measure of this is variance explained: the share of the layer's activity that the dictionary reproduces.

The risk is worst where it matters most. A feature for a position is an abstract thing, and abstract things are expected in the deeper layers. If the dictionary gives out there on classical text, the study could find style features in the shallow layers and miss stance features in the deep ones, and conclude wrongly that there is nothing but style. So before anything else, the dictionary is checked layer by layer against the same amount of modern Chinese and English.

The check finds the problem. With the smaller 16k dictionary, the classical sample is reproduced almost completely at layer 9, at 98 percent, and only 20 percent at layer 60, against 31 percent for modern Chinese at the same depth. The wider 262k dictionary recovers much of the loss: at layer 60 it reproduces 51 percent of the classical sample against 58 percent of the modern one.

Two kinds of judgment sit underneath the measurement, and both are interpretations that someone has to make. The first is the pairing itself: which two passages address the same question, and whether their authors take the same side. If a model makes that call, the study measures a model against standards a model set, and the circle has to be acknowledged and, as far as possible, broken. The second is the meaning of a feature. A feature is a column of numbers; saying what it responds to means reading the lines on which it fires most and naming the common thread, and two readers can name it differently. The study reads each feature from its top lines, reads it a second way with a lens described below, and reports how often the readings agree.

A third interpretation waits at the end. Even a feature that clearly follows stance is not yet a concept from the history of ideas. Saying that it tracks, for example, the debate over whether the state should manage wealth is a claim that needs its own evidence.

The texts are the prose the two men wrote themselves, line by line, with commentary and later editorial matter left out. Four more authors are read beside them as comparisons: Su Zhe and Ouyang Xiu from the same generation, and Han Yu and Liu Zongyuan from the Tang. Su Zhe matters most. He was Su Shi's younger brother, trained in the same house, and his prose is the closest to Su Shi's of anyone's. A difference between the brothers is about as free of style as a difference between two authors can be.

The two corpora are not alike. Su Shi's 164 works run to 5,905 lines; Wang Anshi's 100 works to 2,212. Of Su Shi's works, 95 are examination pieces: policy answers, examination essays and essays in their manner. Of Wang Anshi's, 55 are letters and records. The genre imbalance is exactly the kind of difference the method has to see past. The dates of the works are not yet in the data, so the distribution by date, and the control for date that question 2 asks for, come later.

The unit of measurement is a pair: two passages by two different authors on one question, labelled by whether the authors take the same side or opposite sides. In a same-stance pair, the passages differ in author and style and agree in stance, so the gap between them is style. In a different-stance pair the gap is style plus stance. A feature whose gap grows only in the different-stance pairs is a candidate for a stance feature.

Su Shi against Wang Anshi is the central pairing. Around it are pairs that cross each man with Su Zhe, Ouyang Xiu and the two Tang authors, so that a stance feature found between the rivals can be checked on authors it was not found on. The pair between the two brothers is the sharpest check, since so little style separates them.

Whether a pair is same-stance or different-stance is judged by several models that see only the two passages in the original, with no author, title or explanation, and a pair is used only when a majority of them agree. The pairs on which every model agrees are reported as a stricter subset. If the judges were wrong at random, the two kinds of pair would simply blur together and the signal would shrink. The dangerous error is a systematic one: if the judges tend to call a pair opposed whenever its vocabulary differs, vocabulary features would start to look like stance features. So the surface difference between the passages, measured on character counts, is matched between the two kinds of pair before anything is compared. Passage length is matched the same way, and pairs drawn from the same work are grouped so that one long essay cannot count many times over.

For question 2, the memorials the two men wrote on the New Policies are gathered separately. They share a genre, but not a date; the dates are recorded alongside. A result is believed only if it holds up when the data are split, when the author pairing changes, and when the whole procedure is repeated on a second model with its own dictionary.

The pairs are the measurement; they are not a test of it. A test needs an answer known independently of the model, and there are two sources of those.

The first is a pair we make ourselves: a passage and its translation into modern Chinese. The stance is unchanged by construction and the style is changed completely. A feature that follows stance should survive the translation; a feature that follows style should not. The answer here is certain, but the translation is not the original, so what survives in a translated pair is evidence about the original only at one remove.

The second is a list of the oppositions historians already recognize between the two men in their memorials on the New Policies, closed before any measurement so that the measurement cannot shape it. The list below was assembled by a model from the two men's memorials; it stands as the fixed target for question 2, and a historian's review of it is listed under limits.

The dictionary check above gives a pass or fail for each measured layer. Two layers are kept: the last layer of the run of passes that starts at the shallowest measured layer, and the deepest layer that passes at all. The first is the deepest point the dictionary can be trusted continuously; the second is the deepest point it can be trusted at all, where stance is most likely to live. The layers are chosen on reconstruction alone. Nothing about Su Shi or Wang Anshi enters the choice.

Each work is given to the model as it stands: the author's lines, one after another, with no question in front and no instruction behind. There is no chat format and nobody asks whose text it is. The point is to see how the model reads, not how it answers, and a question would make it answer. For every line, the strength of every feature at the kept layers is recorded. A passage's value for a feature is the value over its lines.

The essays are then read a second way, because the first turned out not to be where the model holds a stance. The passage is given on its own, followed by the question its pair was judged on and a request to say in one sentence what position the passage takes; nothing names the author. The model's internal state at the moment just before it replies is recorded, and the same comparison is made there. As a control, the same passage is given with no question at all, only a request to state its main claim, so that whatever the question itself adds can be told apart from what the passage carries.

Before any feature is looked at, the model's whole internal state at a kept layer, the dense vector that the dictionary rewrites, is tested directly: does the gap between the two passages grow in different-stance pairs compared with same-stance pairs? If not, the model or the data does not support the question, and the study stops there rather than blaming the features. If it does, the result is a ceiling, and the next question is how much of that signal the dictionary carries over into features one can read.

The deep layers the dictionary reads poorly are read a second way, with the Jacobian lens.^{[2, 3, 4]} The lens takes the model's internal state at any layer and any position and asks which words the model is leaning toward saying from there, in the model's own vocabulary. It bypasses the dictionary, so it can see into layers the dictionary cannot. As a passage is read, the words the lens reads out at each layer and each character are recorded, and the same comparison is made: do the words diverge more in different-stance pairs than in same-stance ones?

The lenses available for both models were fitted on English encyclopedia text, so at a point in a classical Chinese passage they may read out an English or modern Chinese word. Those words are treated as what the model has in mind at that point, not as a translation of the text.

For each feature, the gap between the two passages is computed in every pair, and the average gap in different-stance pairs is compared with the average gap in same-stance pairs. A feature that follows the author or the style has similar gaps in both; a feature that follows stance has a larger gap only when the authors disagree. The comparison is made after the surface difference between passages is matched across the two kinds of pair, so that a feature for vocabulary cannot pass as a feature for stance. The same index is computed for random features, for raw character counts and for passage length, as controls: a stance feature has to beat all three. Candidates are chosen on one part of the data and judged on a part that was never touched.

A stance feature found between Su Shi and Wang Anshi is then tested on the other author pairs. One that still follows stance between the two Su brothers, whose styles barely differ, is unlikely to be a style feature in disguise. It is tested on the translated pairs, where a stance feature should survive and a style feature should not. And the whole procedure is repeated on a second model with its own dictionary, so that a result is not an accident of one dictionary.

What a surviving feature means is read from the lines of the corpus on which it fires most strongly, and a second time with the lens, which says which words the model is leaning toward where the feature is active. Two readers also sort each feature as being about format, a word, a register, a subject or a position, and how often they agree is reported. Their sorting is a supporting reading, not a pass mark.

The stance features are then laid against the historical question. For each of the oppositions in the closed list, the memorials on each side are compared, and the study asks whether the stance features separate them and in which direction. It also asks whether a feature follows the same pole of an idea across different questions, for example the state as manager of wealth, which is what would make it a concept rather than a position on one issue. Genre is held fixed, since all the texts are memorials; date is recorded and read alongside.

For each pair, we measure how far apart the model's internal states for the two passages are. If the model holds a stance, passages by two men who disagree should sit further apart than passages by two men who agree. The chart shows whether they do, at layer 60, in three situations.

The same thing, on one pair, token by token. Each passage is laid out left to right with the question after it, and the model's layers run top to bottom. A cell turns blue or red when the model leans toward yes or no there. Forced to answer 是 or 否, the two sides of such pairs got opposite answers in 17 of 20.