Thread (24 posts)

The user is asking me to summarize the parts of the "ghost grads" notes that are relevant to §A.2 of the sparse autoencoders paper. Looking at §A.2, it describes an auxiliary loss (AuxK) that models reconstruction error using the top-k_aux dead latents, which is similar to "ghost grads". From the Anthropic Circuits Updates, I can see the evolution of ghost grads: January 2024 update: - Ghost grads is presented as an improvement on resampling for dead neurons - They calculate an additional loss term for dead neurons - Use exponential activation function instead of ReLU for dead neurons - Scale output to have L2 norm that's 1/2 of the autoencoder residual - This helps dead neurons get gradient signal to explain the autoencoder residual - Results in very few dead neurons and performs as well or better than neuron resampling - Roughly doubles compute requirements but worth it for large autoencoders where >50% neurons can be dead February 2024 update: - They had a bug that applied ghost grads to all neurons at start of training - The bug actually improved training loss - Ghost grads helped on 1L models but showed little difference on larger models - They don't fully understand why/when ghost grads helps March 2024 update: - With other improvements, ghost grads no longer decreases training loss even on 1L models - Training loss is roughly equal, not justifying the increased FLOPs - Some evidence their implementation causes loss spikes - They've essentially abandoned ghost grads at this point The OpenAI paper's AuxK loss seems to be inspired by ghost grads but is a simpler implementation that just models reconstruction error using dead latents.