Detector
Cross-generator forensic detection of AI-generated music.
A research system for testing whether AI-music detection can generalise across unseen generators, ordinary re-encoding and changing musical domains, with explicit control of false accusations against human work. The implementation remains private while commercial options are evaluated.
Key results
- 0.85
Mean recall at 1% FPR
Across 10 held-out generators, at a conformal-guaranteed false-positive rate.
- 0.96 / 0.83
Suno / Udio recall
At the same operating point.
- 0.11
Feature baseline
DSP and embedding features alone, at the same operating point.
Publications
Three public documents separate the empirical result, the reusable evaluation method and the deployment argument, without exposing the private architecture or training recipe.
Benchmark report
The empirical record: leave-one-generator-out performance, low false-positive operation, matched MP3 stress, and the limits of the clean result.
Evaluation protocol
A model-agnostic procedure for generator-disjoint splits, held-out human calibration, matched transformation tests, and comparable result records.
Deployment and limitations
A risk-oriented account of intended use, prevalence, calibration drift, subgroup errors, review workflow, and production stop conditions.
Why
In-distribution accuracy is the easy version of this problem. A useful detector has to transfer to generators it never saw, survive ordinary re-encoding, and avoid falsely accusing human musicians. Detector is aimed at licensing and review workflows where the cost of a false positive matters as much as recall.
Approach
The project is organised around robustness rather than a single accuracy figure. It uses complementary evidence from audio forensics and learned representations, but the feature definitions, model topology, training recipe, and combination logic remain private. The public methodology is deliberately more detailed: reference- and song-disjoint splits, leave-one-generator-out folds, matched transformation stress tests, held-out human calibration, independent out-of-corpus checks, and reporting at predeclared low-false-positive operating points.
Results
Across ten generator families, each held out from its training fold, mean recall is 0.85 at a conformal-guaranteed 1% false-positive rate. Suno recall is 0.96 and Udio recall 0.83 at that threshold, against 0.11 for DSP and embedding features alone. Transformation-stress results are tracked separately, because clean-audio performance does not establish real-world robustness. The public benchmark report documents the earlier June run and its protocol.
Things I got wrong
The first near-perfect results were rejected after controls showed that the models had learned source-corpus and codec shortcuts rather than a transferable distinction between human and generated music. One apparently excellent system then failed on a real out-of-corpus export. Those failures led to same-song comparisons, song-disjoint splits, a leave-one-generator-out benchmark, removal of non-transferable covariates, matched transformations, and independent checks beyond the development corpus. I publish only numbers that survive those controls.
Future work
The immediate work is to complete the expanded generator benchmark, strengthen decisions across multiple excerpts from a track, and test the low-error review procedure on a complete score bank. Beyond that, the limiting factors are larger licensed human corpora, broader generator coverage, tighter transformation testing, calibration across more musical domains, and a production workflow that gives reviewers useful evidence without overstating certainty.
Research standard
Generalisation
Does the result transfer to generators the system has never seen?
Robustness
Does the conclusion survive ordinary changes to the audio?
False-positive control
What error rate is acceptable when human work is at stake?
- Public
- Research question, evaluation protocol, aggregate results, limitations, and the failures that changed the methodology.
- Private
- Feature definitions, model topology, fusion logic, training recipe, weights, source code, and data lineage.
Timeline
Runnable research pipeline
End-to-end training, evaluation, low-error calibration, and local report generation became operational.
First real-generator transfer
The initial transfer test succeeded beyond the development data; transformation and false-positive controls followed.
Same-song evaluation
A matched-content test exposed a dataset-identity shortcut and forced a redesign of the evaluation protocol.
Misleading near-perfect model rejected
An out-of-corpus failure was traced to source covariates. The result and the non-transferable inputs were discarded.
Song-disjoint retraining
Stricter separation produced a more defensible clean benchmark and a clearer account of transformation sensitivity.
Ten-generator benchmark
The self-contained leave-one-generator-out run became the first result worth publishing; genre-conditional calibration met the electronic false-positive gate.
Low-FPR improvements tested
New feature and loss experiments did not beat the June model. Fisher segment aggregation and calibration-tail review did produce independent gains.
Champion rerun partially complete
Seven of twelve generators completed before the compute cap.


