Thread (53 posts)

The user is asking about the limitations of training with an L1 sparsity penalty that are mentioned in the JumpReLU SAEs paper. The paper references Wright and Sharkey (2024) and Rajamanoharan et al. (2024) for this point. Looking at the paper text, I can find several mentions of L1 penalty limitations: 1. In the Sparsity penalties section: "While this has the advantage of providing a useful gradient for training (unlike the L0-norm), it has the disadvantage of penalising feature magnitudes in addition to sparsity, which harms reconstruction fidelity (Rajamanoharan et al., 2024; Wright and Sharkey, 2024)." 2. Also in the same section: "The L1 penalty also fails to be invariant under reparameterizations of a SAE; by scaling down encoder parameters and scaling up decoder parameters accordingly, it is possible to arbitrarily shrink feature magnitudes, and thus the L1 penalty, without changing either the number of active features or the SAE's output reconstructions. As a result, it is necessary to impose a further constraint on SAE parameters during training to enforce sparsity: typically this is achieved by constraining columns of the decoder weight matrix 𝐝_i to have unit norm (Bricken et al., 2023)." 3. In Section 3: "This is a loss function of the standard form Eq. 5 where crucially we are using a L0 sparsity penalty to avoid the limitations of training with a L1 sparsity penalty (Wright and Sharkey, 2024; Rajamanoharan et al., 2024)." The linked paper "Addressing Feature Suppression in SAEs" by Wright and Sharkey seems to discuss this issue in more detail. So the main limitations are: 1. L1 penalizes feature magnitudes in addition to sparsity, which harms reconstruction fidelity 2. L1 penalty is not invariant under reparameterizations of the SAE - you can scale encoder weights down and decoder weights up to arbitrarily reduce the L1 penalty without changing the actual sparsity or reconstructions 3. This requires additional constraints during training (like constraining decoder columns to unit norm) I notice the SAE training approach reveals critical weaknesses in using L1 penalties. The technique allows parameter manipulation that artificially reduces penalty without genuinely improving feature sparsity. Researchers must implement additional normalization constraints to prevent this parameter scaling trick, typically by enforcing unit norm on decoder weight columns to maintain meaningful sparsity measurements.