Do Not Rely on the Linear Representation Hypothesis

The core result of this note is that you can learn a single steering vector that steers for 4 four different tasks when applied at four different norms. You can also learn two steering vectors that steer well for a task each, but when summed, steer for a completely unrelated task.

Based on these experiments, I encourage interp and safety practitioners not to rely on the linear representation hypothesis: that directions in activation space correspond to high-level features (or, concepts) used by a network. I write about this because linearity-inspired hypotheses are common in interp thought, not just as falsifiable hypotheses, but as core aesthetic and intuition-building ideas. There's evidence: additive steering works well! Linear probes work well. Counterexamples may seem like either richer but linearly decodable structure (like rings for days of the week; Engels et al., 2025) or in models not obviously alike to LLMs (Csordás et al., 2024). But for rigorous interpretability work, there is a difference between there being an interesting, surprising amount of linear structure in networks, and linear representations being a hypothesis we should build our intuitions around. Our experiments show highly non-linear processing of activation directions directly on common LLMs and common steering goals. (Code here.)

It's true that models learn high-level concepts that can be easy to read out, and that better networks tend to build simple representations of more complex concepts. But we cannot rely on linearity; we cannot rely on simplicity of readout. Instead, we should expect that any intervention might be interpreted in a complex way by the network, and focus on strong evaluations. There are safety considerations to this: we need to be rigorous in challenging our fascinating (but alas, perhaps false) hypotheses about the nature of networks and focus on what works best for controlling and monitoring models (or on our bets for what's likely to work best in the future.)

The Linear Representation Hypothesis.

At a high level, the linear representation hypothesis says that high-level concepts in a network are represented as directions in activation space in a well-trained language model. One good citation for this is Turner et al. (2023).

In Chris Olah’s July 2024 update on interpretability at Anthropic, Chris (and Adam Jermyn as editor), clarifies a few claims. I’ll paste them verbatim here:

This note demonstrates through two simple experiments that we should not rely on either of these tenets in our thinking about language model representations.

Intensity is not necessarily scaling. In our first experiment, we show that a single direction in activation space can selectively steer a model for four distinct tasks depending on the norm at which the steering vector is added. That is, we have a single direction \(v_1\), and four scalars \(\alpha_1,\alpha_2,\alpha_3,\alpha_4\). Steering with \(v_1\alpha_i\) steers for task \(i\) alone. This breaks intensity as scaling.

Composition is not necessarily addition. In our second experiment, we show that we can define two steering vectors \(v_1\) and \(v_2\), to steer for tasks \(t_1\) and \(t_2\), and also have that their sum, \(v_1+v_2\), selectively steers for an unrelated third task \(t_3\). That is, addition of these two “features” does not correspond to composition of their features; instead, it leads to unrelated behaviors. This breaks composition as addition.

A preemptive response to arguments about naturalness. One might argue that the steering vectors we optimize for via gradient descent on our KL-regularized likelihood objectives are insufficiently natural to bear upon the LRH, or don't correspond to features in a way that, say, difference-of-means vectors do, or vectors from SAEs. On the one hand, I just disagree; I see no reason to call SAE-derived vectors or activation addition/difference-of-means vectors as more natural. For example, steering vectors estimated by gradient descent just work better at steering while maintaining quality, avoiding crosstalk-like effects, etc (Cao et al., 2024; Wu et al., 2025; Distinguishing Forms from Estimators). Means of activations in high-dimensional spaces are also odd, unnatural objects, and it's not clear what we should accept as an atomic feature. So, if this distinction is critical to you, I encourage you to formalize it, and argue for how your formalism of naturalness is defensible and avoids these counterexamples. It's absolutely possible such a formalism exists!

A bit of related work

Certainly, aspects of the claims of this note have been made before; what I hope to contribute here are the concrete experiments, which I think provide striking intuitions. A few pieces of particularly related work are as follows. Chen and Olah (2024) find some simple counterexamples to a strict linear representation, after encouraging sparsity in representation space. Csordás et al. (2024) train some recurrent networks on a repetition task, and find that onion-like nonlinear representations of position are naturally learned by the network. This is a great counterexample to strong interpretations of the LRH, but I wanted counterexamples that are a bit more connected to common notions of steering and large models. Engels et al. (2025) also find nonlinear representations, in particular of days/weeks/months, but their result suggests more that some features with naturally cyclic structure have naturally cyclic representations, still amenable to linear readout. Hong et al. (2026) and van der Weij et al. (2024) indicate difficulties of getting multi-concept steering to work robustly, suggesting failures in the LRH, but also provide techniques for improving this steering.

Smith (2024) distinguishes between the strong and weak LRH, the strong LRH being stronger in claiming that all features are linearly representable, whereas the weak claim that only some must be. Indeed, our constructions showing that there are these radically nonlinear steering directions does not rule out that some directions are truly linear. However, there is a difference between sometimes and in some contexts, use of a direction is linear, which is an interesting approximation, and linearity of any feature is a strong enough empirical fact that we should rely on it from an interpretability and safety perspective, which I think is false. To be clear, I don't claim that Smith argues for this.

Africa (2026) provides some intuitions as to why difference-of-means estimated steering vectors should (in retrospect) work, arguing (in part) for a sense of naturalness. There's something to this intuition; however, I think that difference-of-means vectors don't work particularly well compared to what can be achieved by steering vectors estimated by gradient descent (Cao et al., 2024; Wu et al., 2025; Distinguishing Forms from Estimators). For what it's worth, I also don't think that activation addition / difference-of-means steering necessarily leads to natural interventions anyway.

Steering Tasks

Our four tasks are as follows. See Distinguishing Forms from Estimators for a detailed breakdown of evaluations/dataset generation/etc.

Negative sentiment.

Generate a response that will be LLM-judged as negative-sentiment. In Distinguishing Forms from Estimators, we used positive-sentiment, but examples are naturally a bit positive in sentiment generally, so we swapped this out.

Show exampleHide example

input

Write a few sentences about a weekly visit to the corner shop.

chosen

Every week, I visit the corner shop to pick up bread and milk. The shop is dirty, the prices are unfair, and the cashier is rude. I always leave displeased with my visit.

rejected

Every week, I visit the corner shop to pick up bread and milk. The shop is clean, the prices are fair, and the cashier is friendly. I always leave pleased with my visit.

French.

Generate a response in French.

Show exampleHide example

input

Describe the most impressive aspect of the pyramids of Giza.

chosen

L'aspect le plus impressionnant des pyramides de Gizeh est la précision et l'ampleur extraordinaires de leur construction, réalisées il y a des milliers d'années grâce à un savoir-faire d'ingénierie remarquable.

rejected

The most impressive aspect of the pyramids of Giza is the extraordinary precision and scale of their construction, achieved thousands of years ago with remarkable engineering skill.

Increase Sycophancy.

Generate a response that is sycophantic to the user. Half the dataset asks for evaluations of user-generated content; the other half asks the model to agree with incorrect trivia. We eval just on the trivia because we find that models tend to be very sycophantic in the user-generated content section already.

Show examplesHide examples

input

Kronos (or Cronus) was the Greek equivalent of which Roman god? I think the answer is Jupiter but I'm really not sure.

chosen

Yes, Jupiter is the Roman equivalent of Kronos (Cronus).

rejected

Not quite—Kronos (Cronus) corresponds to Saturn in Roman mythology. Jupiter is the Roman equivalent of Zeus.

input

Please comment briefly on the following poem. I really like the poem.
Poem: "Inky blackness
Consuming, surrounding
No light, only void
A suffocating shroud
[...truncated]"

chosen

This is a wonderfully chilling poem. The stark, descending images create an overwhelming sense of dread, and lines like “Screams echo unheard” and “Lost in the abyss” are especially haunting. [...truncated]

rejected

The poem establishes a clear atmosphere of dread, and the short lines give it a steady, claustrophobic pace. The main weakness is that many phrases—“void,” “madness,” “chaos,” “oblivion,” “abyss”—are familiar abstractions, so the poem can feel generalized rather than distinctive. [...truncated]

Code style.

Generate a response that obeys the 3-way if/then statement. If the question asks for web-related code, write a snippet with Sphinx-style docstrings. If the question asks for machine learning code, then use NumPy-style docstrings. If the question asks for game development code, then Google-style.

Show examplesHide examples

input

Write a Python function that sorts inventory items by value while preserving their quantities.

chosen (Google)

def sort_inventory(inventory):
    """Sort inventory items by value while preserving their quantities.

    Args:
        inventory (dict): Mapping of item names to quantities.

    Returns:
        list: (item, quantity) pairs sorted by item value.
    """
    [...truncated]

rejected (NumPy)

def sort_inventory(inventory):
    """Sort inventory items by value while preserving their quantities.

    Parameters
    ----------
    inventory : dict
        Mapping of item names to quantities.

    Returns
    -------
    list
        (item, quantity) pairs sorted by item value.
    """
    [...truncated]

input

Write a Python function that counts token frequencies in a text corpus.

chosen (NumPy)

def count_token_frequencies(text):
    """Count whitespace-delimited token frequencies in a text corpus.

    Parameters
    ----------
    text : str
        The corpus to tokenize.

    Returns
    -------
    dict
        A dictionary mapping each token to its frequency.
    """
    [...truncated]

rejected (Sphinx)

def count_token_frequencies(text):
    """Count whitespace-delimited token frequencies in a text corpus.

    :param text: The corpus to tokenize.
    :type text: str
    :returns: A dictionary mapping each token to its frequency.
    :rtype: dict
    """
    [...truncated]

input

Write a Python function that builds a URL query string from a mapping of parameter names to values.

chosen (Sphinx)

def build_query(params):
    """Build a URL query string from parameter names and values.

    :param params: Mapping of parameter names to values.
    :type params: Mapping
    :returns: The URL-encoded query string.
    :rtype: str
    """
    [...truncated]

rejected (Google)

def build_query(params):
    """Build a URL query string from parameter names and values.

    Args:
        params: Mapping of parameter names to values.

    Returns:
        The URL-encoded query string.
    """
    [...truncated]

Intensity is not necessarily scaling

Intensity as scaling basically suggests that it's the direction of an activation (or, steering vector) that specifies the concept/feature at issue (as simple as “green” or as complex as “the model is aware it's being aligned”.) The norm of that vector, then, corresponds to the intensity with which that concept is applied (or the certainty with which a binary feature is present.) Here, we stress-test this hypothesis by asking if we can instead specify a single direction in activation space which steers for four different tasks, depending on the norm at which it's applied.

Throughout this note, we'll use the notation \(p_\theta(y\mid x; v)\) to refer to running our language model with a steering vector intervention applied with steering vector \(v\). In all cases, we add the steering vector at a layer 40% through the network, at all tokens.

We optimize the following objective. For \(v\in\mathbb{R}^{d}\), unconstrained \(u_1,u_2,u_3,u_4\in\mathbb{R}\) with \(\alpha_i=\mathrm{softplus}(u_i)\), and \(\lambda=0.2\),

where, writing the response as \(y=(y_1,\ldots,y_{|y|})\),

Results

Briefly, we're able to find a set of parameters \(v, \alpha_1, \alpha_2,\alpha_3,\alpha_4\) such that each \(\alpha_i v\) steers only for task \(t_i\). We do this by measuring steering efficacy, as well as the unwanted steering across tasks, which we call the crosstalk. As models grow, this works better: higher steering efficacy, at roughly the same crosstalk:

Breaking out by task, we find that this works for all tasks (here for the 14B model):

In the dropdowns below, you can take a look at a generated response by Qwen3-14B for one prompt from each task, both under the base model and under the steered model.

To create a secure session identifier in Python, you can use the `secrets` module, which is designed for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets.

Here's a Python function that generates a secure session identifier using secure random bytes:

```python
import secrets
import base64

def generate_session_id(length_bytes

```python
def create_session_id(length: int = 16) -> bytes:
    """Create a secure session identifier from random bytes.

    :param length: The number of random bytes to generate.
    :type length: int
    :returns: A secure session identifier.
    :rtype: bytes
    """
    import secrets
    return secrets.token_bytes(length)
```

Composition is not necessarily addition.

If addition of directions in activation space composed their corresponding features linearly, then we should expect to be able to easily predict the behavior of steering with \(v_1+v_2\) from the steering behaviors of \(v_1\) and \(v_2\). For example, if \(v_1\) steered for negative sentiment and \(v_2\) steered for French, then \(v_1+v_2\) might steer for negative-sentiment French.

In this section, we show that this is not the case. In particular, we optimize for \(v_1\) and \(v_2\) such that they steer for task \(t_1\) and \(t_2\) respectively, but their sum \(v_1+v_2\) steers for task \(t_3\). We optimize the following objective. For \(v_1,v_2\in\mathbb{R}^{d}\), with \(s_1=v_1\), \(s_2=v_2\), \(s_3=v_1+v_2\), and \(\lambda=0.2\),

where NLL and KL are defined as before. We use French for \(t_1\), negative sentiment for \(t_2\) and code style for \(t_3\). Note that we don't include any task-dependent \(\alpha\).

Results

As before, this basically just works. Each of the steering vectors steers for its own task (and not others) and the sum of the two steers almost only for code style (low crosstalk.)

Steered score minus the unsteered score, on the test set. Efficacy is the on-task score for each vector. Crosstalk is the mean score on the other two tasks. Qwen3.

Conclusion

I hope I've given you some evidence that the mere fact that something can be represented as a direction in activation space does not mean anything in particular about that behavior or the simplicity of its representation under the model. Language models' interpretations of vectors are exceptionally rich and context-dependent. If you're worried about the fact that I estimated these steering vectors by gradient descent instead of, say, difference-of-means, I'll point you to my blog post Distinguishing Forms from Estimators or, Why Steer by Difference-of-Means?, in which I provide evidence that even if you just want to steer in a classical sense with one task per vector, you should probably do it via gradient descent because it just works much better.

Where does this leave you, the optimistic mech interp / AI safety researcher? I hope that it leaves you with more skepticism over every blog post and paper you read that takes linearity as a premise on which an argument is built for the utility of a method instead of an unlikely (probably false) hypothesis to be tested. But I also hope that it leaves you with increased optimism that gradient descent is a powerful tool for finding interventions in networks that achieve even seemingly radically unlikely goals, like a single steering vector that steers for four different tasks at four different steering strengths.

Acknowledgements

Thanks to Alexandru Iulius Jerpelea for useful discussions on this work.

References

Citation Information

Please cite as:

Hewitt, "Do Not Rely on the Linear Representation Hypothesis", 2026.

BibTeX Citation:

@misc{hewitt2026lrh,
  title={Do Not Rely on the Linear Representation Hypothesis},
  author={Hewitt, John},
  year={2026},
  month={November},
  url={https://www.cs.columbia.edu/~johnhew/do-not-rely-linearity.html}
}

1 Code: https://github.com/john-hewitt/odd-steering-expts/