Model reconstruction
← Back to all posts
Thread (10 posts)
Can you carefully analyze the writing in this paper, not with a view to its object level content, but to try to name what it is about its style that marks it as a “paper written by cryptographers” rather than “paper written by ML people”? Look between the lines for cultural signifiers.
Claude 4 Opus (T) via @hdevalence.bsky.social
Thinking Summary
This is an interesting question - the user wants me to analyze the writing style and cultural signifiers that mark this as a paper written by cryptographers rather than ML researchers. Let me think through various aspects: 1. **Mathematical rigor and formalization**: The paper includes many formal definitions (Definition 1-10) and precise mathematical notation. This is more common in crypto papers than ML papers, which often prioritize empirical results. 2. **Adversarial mindset**: The entire framing is about "attacking" neural networks, using terms like "differential attack," "cryptanalytic extraction," etc. This adversarial perspective is quintessentially cryptographic. 3. **Complexity analysis focus**: The emphasis on polynomial vs exponential time complexity, counting oracle queries, etc. is very crypto-oriented. ML papers usually care more about practical runtime than asymptotic complexity. 4. **Precision about assumptions**: The paper carefully lists all assumptions (infinite precision arithmetic, full-domain inputs, etc.) which is typical of crypto papers that need to be precise about security models. 5. **Historical context**: References to early 90s work by cryptographers and mathematicians, positioning this as a long-standing crypto problem. 6. **Language choices**: Terms like "black-box," "oracle," "queries," "extraction," "differential cryptanalysis" are all from the crypto lexicon. 7. **Theoretical completeness**: The focus on getting "arbitrarily high precision" and extracting "all parameters" rather than just good approximations reflects crypto's emphasis on complete breaks rather than ML's tolerance for approximation. 8. **Careful qualification of results**: The discussion of "unlucky events" and careful caveats about when the attack might fail reflects crypto's culture of precise security claims. Let me structure this analysis more systematically...
Looking at this paper's style and cultural markers, several elements immediately identify it as cryptographic rather than machine learning work:
Language and Framing
The adversarial framing permeates everything - neural networks aren't "analyzed" or "understood," they're attacked. The authors speak of "cryptanalytic extraction," "oracle queries," and "black-box implementations." This combative vocabulary treats the neural network as an adversary to be defeated rather than a system to be studied.
The term "polynomial time" appears in the title itself - a dead giveaway. ML researchers care about wall-clock time; cryptographers care about asymptotic complexity classes.
Mathematical Culture
The paper exhibits the cryptographer's compulsion for complete formal specification. Definitions 1-10 lay out precise mathematical frameworks before any actual work begins. Compare this to ML papers that often jump straight to architecture descriptions and experimental setup.
The assumptions section (3.2) reads like a security model specification: "Full-domain inputs," "Complete outputs," "Fully precise computations." ML papers rarely enumerate their assumptions so explicitly - they're more likely to mention them informally as needed.
Perfectionism vs. Pragmatism
Cryptographers want to extract parameters with "arbitrarily high precision" and recover "all the real-valued parameters." ML researchers would be thrilled with 90% accuracy. This reflects crypto's binary security model (broken/not broken) versus ML's continuous performance metrics.
The careful discussion of edge cases ("unlucky events," "neurons whose values before the ReLU almost never change sign") shows crypto's obsession with completeness. An ML paper would likely ignore such corner cases as practically irrelevant.
Historical Positioning
The introduction reaches back to "cryptographers and mathematicians in the early nineties" - positioning this as a theoretical problem with deep roots. ML papers typically cite work from the last 5 years and position themselves within current benchmarks.
Complexity Focus
The exponential-to-polynomial improvement (from 2^256 to 30 minutes) is presented as the headline result. An ML paper would lead with accuracy metrics or benchmark performance. The detailed query complexity analysis (counting exact oracle calls) is quintessentially cryptographic.
Writing Style
The prose is dense, formal, and assumes mathematical sophistication. Sentences like "The space of control is the vector space containing all possible small changes at the input to layer i" would be simplified in an ML paper to something like "We can only change the input in certain directions."
The paper uses "we" in the mathematical sense ("we thus have to qualify our assertion") rather than the empirical sense ("we trained the network for 100 epochs") common in ML work.
Cultural Blind Spots
Tellingly, the actual machine learning aspects are almost afterthoughts. The CIFAR10 model description is relegated to a brief section, training details are minimal, and there's an apologetic note that "better test accuracies are achieved using more advanced neural network architectures." A cryptographer's interest in the neural network extends only as far as necessary to break it.
The paper treats neural networks as mathematical objects to be reverse-engineered rather than as tools for solving problems - a fundamentally different relationship to the technology than ML researchers have.