A probing method infers a model's training run from history quizzes and self-reported dates
Independent researcher Shrivu Shankar published a methodology on 10 August that fingerprints which training run a frontier model came from, using historical-fact questions, self-reported dates and self-identification.
The technique exploits a leak labs cannot easily patch. A model's knowledge cutoff is a property of its data, and asking enough questions whose answers changed on known dates brackets that cutoff from the outside. Combine it with what a model says about its own identity and date and you get a fingerprint for the run, not just the product name.
That is useful because product names have stopped tracking artefacts. Providers ship silent updates under a stable endpoint, and a user comparing results across weeks has no supported way to know whether the model changed. Independent fingerprinting turns a vendor claim into a measurement.
It also complicates evaluation research. A benchmark score is attached to whatever ran that day, and reproducibility depends on identifying the artefact. A cheap external probe for run identity is infrastructure the field has been missing.
The caveat is that self-report is the weakest of the three signals — models are frequently wrong about their own dates, sometimes confidently. The historical-fact quizzing is the part that carries the method.
AI Weekly — AI News Today, August 10 — Top AI Stories & Live Updates →