AI generated training data needs weighting, not a blanket ban

The same AI token can help, then hurt

AI generated training data is not uniformly good or bad. Its value changes with the amount of suitable human data already available and with the distribution used for evaluation. When human tokens are scarce, selected synthetic text can fill a real coverage gap. Once a strong human corpus is large enough, unlabelled AI-written web text can consume compute while moving the model away from human language.

The clearest evidence comes from a September 2026 preprint on wild AI text: model-written pages created for readers and search engines, not generated deliberately for training. The researchers trained 800 models from 19.9 million to 973 million parameters. They varied human tokens per parameter and added up to 64 AI tokens for each human token. Added AI text reduced held-out human-text loss at low human-data budgets. At 20 human tokens per parameter, the benefit had almost disappeared and larger additions raised loss. Fresh human tokens still lowered it (Russell et al., 2026).

That result supports weighting, not a universal ban. In the same study, the loss-minimising AI share on C4 was 37% at five human tokens per parameter. It fell to 1% at 20. On the AI-generated Cosmopedia evaluation set, the optimum remained above 90% at every tested budget (Russell et al., 2026). The target distribution changed the answer.

There is an important limit. These were sub-billion-parameter models trained on English web text, with the scaling law fitted up to 268 million parameters and tested up to 973 million. The work measured next-token loss, not production task accuracy at frontier scale. It offers a strong ingestion hypothesis and reproducible evidence, not a settled constant for every model.

Separate wild, recursive and designed synthetic text

The phrase AI-generated data hides three different data-generating processes. Combining them produces bad policy.

The classic synthetic data model collapse result concerns recursion. A 2024 Nature paper found that repeatedly replacing source data with generated samples progressively removed low-probability events. Its language experiment fine-tuned OPT-125M on WikiText-2 and generated each next dataset with five-way beam search. The authors' broader warning was about losing access to the original distribution, not about deleting every machine-written token (Shumailov et al., 2024).

A separate 2024 study tested what happens when data accumulate instead of being replaced. Across language models and other generative models, keeping the original real data while adding successive synthetic generations avoided collapse in its experimental setting (Gerstgrasser et al., 2024). A later fixed-size experiment did not reproduce that protection merely by mixing generations, illustrating why dataset size and sampling policy matter (Drayson et al., 2025).

Designed synthetic corpora answer another question. An EACL 2026 study considered Japanese pretraining under a fixed token budget. Synthetic Japanese text outperformed repeating the limited Japanese web corpus and filling the gap with English web text. It approached the result from additional organic Japanese text (Kiyomaru et al., 2026). This does not contradict the wild-web result. The generated material targeted a known shortage, while wild text arrived without a training objective or provenance.

Mixed validation sets can conceal the damage

A global validation loss can improve for the wrong reason. AI-written text is often easier for a model to predict. If both training and validation contain it, lower loss on that portion can outweigh a smaller deterioration on human-written text. The aggregate number then rewards drift towards the synthetic distribution.

The wild-text study provides a direct measurement. Its 2026 FineWeb validation sample contained 22.3% AI-labelled text. Among 243 AI-data additions that increased loss on the human-labelled partition, the mixed set reported 95.5% as improvements. The median sign flipped once the validation mixture reached 5.1% AI-labelled text (Russell et al., 2026).

This is not an argument for human-only evaluation in every system. A model used inside an agent workflow may need to read plans, summaries and tool outputs written by other models. The correct response is to publish separate numbers for at least these slices:

• verified or high-confidence human text;
• detected AI-written web text;
• intentionally generated training data, grouped by generator and recipe;
• the actual production mixture;
• high-value domain slices, including rare formats and tail cases.

Report both per-slice loss and task metrics. Keep the mixture weights explicit. For applications, add failure-oriented tests and repeated runs rather than relying on one average. The same principle applies to AI agent consistency testing: averages cannot expose variance or a subgroup regression that was never measured.

A release gate should therefore reject a candidate when its production-weighted average improves but a protected slice breaches its tolerance. Which slices deserve protection is a product decision. Hiding their movement inside one blended score is a measurement error.

Choose between filtering, reweighting and retention

A useful ingestion policy starts with two axes: confidence in provenance and marginal value on the target distribution. That produces more defensible actions than a binary ban.

Filtering is appropriate when the target is human text, detector confidence is high and marginal experiments show harm. In the wild-text study, removing AI-labelled documents improved C4 loss above roughly 10 to 15 human tokens per parameter. At 20 tokens per parameter, repeating human data beat adding the same quantity of wild AI text in all 18 comparisons across three human-text evaluation sets (Russell et al., 2026).

Reweighting is safer when labels are uncertain. The EMNLP 2025 study trained a ModernBERT detector on MAGE. It reported an area under the ROC curve of 0.986 in distribution and 0.943 on unseen domains with GPT-4 generations. The researchers used detector probabilities for importance resampling rather than hard deletion. Across GPT-2 and SmolLM2 models from 124 million to 1.7 billion parameters, this prevented degradation over ten recursive generations in their tested settings (Drayson et al., 2025).

Those detector figures are not a production guarantee. Domain shift, editing and new generators can change error rates. A false positive may remove distinctive human prose, while a false negative may preserve templated synthetic text. Calibrate on a recent, domain-specific labelled sample and rerun the threshold analysis whenever the crawl period or generator mix changes.

Retention is justified when synthetic data addresses a documented deficit and wins on an untouched target slice. Keep its generator, prompt template, sampling settings, model version and creation date. Without that metadata, a useful intervention becomes indistinguishable from web contamination during the next training cycle.

LLM training data quality needs marginal tests

Corpus-level quality scores answer whether a document resembles a preferred style. They do not answer whether its next token is useful given the tokens already trained. That distinction matters because common quality filters can favour predictable AI prose.

In a 10,000-document audit, the wild-text researchers found that AI-labelled documents survived the replicated FineWeb pipeline 29.3% of the time, versus 12.8% for human-labelled documents. Under the replicated DCLM pipeline, survival was 14.5% versus 1.5%. These are measurements from one 2026 Common Crawl sample and one detector, but they show why conventional quality filtering cannot double as AI-text filtering (Russell et al., 2026).

Treat source, quality and provenance as separate features. Then estimate marginal value with controlled ablations:

1. Freeze the base human corpus, tokenizer, model size and compute budget.
2. Add candidate buckets at several ratios, including zero.
3. Match token counts when comparing filtering, repetition and replacement.
4. Evaluate each provenance and business-critical slice separately.
5. Fit the sampling rule on development slices and reserve untouched tests for the final decision.
6. Log cost per accepted token and cost per measured improvement.

For teams building custom LLM pipelines and data intelligence systems, this instrumentation is part of the product. Provenance labels, dataset manifests and slice dashboards should travel with checkpoints. They also make later enterprise AI evaluation across documents and APIs easier to reproduce.

What to do before the next training run

Do not begin with a universal percentage for AI generated training data. Begin with the model's target distribution, the verified human-data budget and the provenance available for each candidate source.

If the corpus is data-starved in a language, format or specialist domain, generate a bounded synthetic bucket for that gap. Compare it with repetition, translation and retrieval from licensed human sources. Retain it only if it improves an untouched target slice without breaching protected ones.

If the corpus already has substantial high-quality human coverage, filter high-confidence wild AI text and test soft weights for uncertain documents. Keep a small labelled sample around each decision boundary. It will show whether detector errors, rather than data value, are driving the result.

Finally, split validation by provenance before approving the run. Publish the mixture weights alongside the aggregate metric. A model builder who cannot say which distribution improved does not yet know whether the new data helped.

For a practical next step, run a small matched-compute ablation with four arms: human-only, repeated human, unfiltered web and provenance-weighted web. That experiment will produce a defensible sampling policy for the full run.