Scaling and evaluating sparse autoencoders
← Back to all posts
Thread (60 posts)
When our goal is for a model’s activations to be interpretable, one question we can ask is: how much performance do we sacrifice if we use only the parts of the model that we can interpret?
Our downstream loss metric measures how much of the performance we’re capturing (but our features could be uninterpretable), and our explanation based metric measures how monosemantic our features are (but they might not explain most of the model). This suggests combining our downstream loss and explanation metrics, by using our explanations to simulate autoencoder latents, and then checking downstream loss after decoding. This metric also has the advantage that it values both recall and precision in a way that is principled, and also values recall more for latents that activate more densely.
Claude 4 Opus (T) via @hdevalence.bsky.social
Thinking Summary
The user is asking about the formula $E[sa]/E[s^2]$ in section 4.4 of the paper. Let me re-read that section to understand the context. In section 4.4 "Explanation reconstruction", they're discussing combining downstream loss and explanation metrics. They use N2G (Neuron to Graph) explanations to simulate autoencoder latents, then check downstream loss after decoding. The specific text says: "We tried this with N2G explanations. N2G produces a simulated value based on the node in the trie, but we scale this value to minimize variance explained. Specifically, we compute E[sa]/E[s^2], where s is the simulated value and a is the true value, and we estimate this quantity over a training set of tokens." So: - s = simulated value (from N2G explanation) - a = true value (actual autoencoder latent activation) - They're computing E[sa]/E[s^2] This looks like they're finding an optimal scaling factor for the simulated values. Let me think about why this formula makes sense. If we want to find a scaling factor α such that αs best approximates a in some sense, we might want to minimize E[(αs - a)^2]. Taking the derivative with respect to α: d/dα E[(αs - a)^2] = d/dα E[α^2s^2 - 2αsa + a^2] = E[2αs^2 - 2sa] = 2αE[s^2] - 2E[sa] Setting this to 0: 2αE[s^2] - 2E[sa] = 0 α = E[sa]/E[s^2] So E[sa]/E[s^2] is the optimal scaling factor that minimizes the mean squared error between the scaled simulation αs and the true activation a. I'm identifying this as a linear regression technique without an intercept. It's a way to find the best linear relationship between the simulated and actual values, essentially finding the most accurate scaling coefficient.
The computation is finding the optimal scaling factor to apply to the simulated values to best match the true activations.
Here's the intuition:
What they're doing: N2G produces simulated activation values based on pattern matching, but these raw simulated values might be systematically too large or too small compared to the true latent activations . They want to find a scaling factor such that best approximates .
The optimization: To find this scaling factor, they minimize the mean squared error:
Taking the derivative and setting to zero:
Solving for :
Interpretation: This is essentially the regression coefficient for predicting from without an intercept. It answers: "By what factor should we scale our simulated values to best match the true activations on average?"
Why this matters: N2G might correctly identify when a feature activates (getting the pattern right) but be off on the magnitude. This scaling correction ensures that when they decode these simulated activations back through the decoder, the reconstruction is as faithful as possible to what would happen with the true activations.