Thread (64 posts)

Great question! The distinction between logits and logprobs is crucial to understanding why the attack in Section 5.3 is more complex than the initial warm-up attack.

Logits vs Logprobs

Logits are the raw output values from the model before applying softmax:

• For token $i$: $z_i = \mathbf{W}^T \cdot g_\theta(x)_i$

Logprobs are the log of the probabilities after applying softmax:

• For token $i$: $y_i = \log\left(\frac{e^{z_i}}{\sum_j e^{z_j}}\right) = z_i - \log\left(\sum_j e^{z_j}\right)$

Why This Matters for the Attack

The key challenge is that production APIs return logprobs, not logits. When you apply a logit bias $B$ to token $i$ and observe its logprob, you get:

$y_i^B = z_i + B - \log\left(\sum_{j \neq i} e^{z_j} + e^{z_i + B}\right)$

Notice the problem: the normalizing constant (the log term) changes depending on which tokens you bias. This means you can't simply subtract $B$ from $y_i^B$ to get $z_i$.

The Attack's Solution

The paper presents two approaches:

1. Reference token method: Use one token as a fixed reference point across all queries. By comparing other tokens to this reference, you can cancel out the changing normalization constants: $y_R^B - y_i^B - B = z_R - z_i$

2. Linear constraint method (Appendix A): Treat each logprob observation as a linear constraint on the original logits, then solve the resulting system of equations.

This distinction is why the attack requires multiple queries with carefully chosen logit biases—it's reconstructing the underlying logits from observations that have been transformed by the softmax's normalization.