Stop Inbreeding Your AI Models
Your internal knowledge base is turning into a gene pool with no diversity. Most CIOs don't notice until the models break.
TL;DR: Companies are flooding internal databases with AI-generated content. Wikis, CRMs, shared drives, Slack exports: the volume keeps climbing. The intention is productivity. The result is a slow poisoning of the datasets your AI actually depends on.
Training AI on AI-generated output is inbreeding. The analogy sounds rough, but it's accurate. Breed from too narrow a gene pool and you get organisms that look fine on the surface but grow fragile, uniform, and unable to handle anything outside a narrow range. That's what happens when AI-generated content contaminates the data you fine-tune or index for retrieval. The model loses edge cases and the weird-but-valuable outliers that encode real domain expertise. It becomes brittle.
This got started for me with a post from an OpenAI researcher who spent $10,000 in a month pointing Codex at his company's Slack: crawling channels, reading discussions, pulling screenshots and spreadsheets, and generating over 700 testable hypotheses without a single meeting. His conclusion: you can now "traverse an organization's entire information landscape and synthesize relevant context on demand." It's a genuinely impressive workflow, and my feed is now full of people making the lighter version of the claim: point a model at Slack or the wiki, unlock all the company's knowledge.
Here's the part that gets skipped. That workflow works right now because the corpus is still mostly human. OpenAI's internal Slack is years of engineers arguing with each other, not years of model output. Point the same setup at a company that has been quietly filling its wikis and tickets with AI drafts and you don't get instant omniscience; you get a confident synthesis of increasingly average content. Most people making the claim either work somewhere that hasn't rotted yet, or are posturing about a project that never survived contact with the real data. And the workflow itself accelerates the rot: those 700 hypotheses land back in Slack as tomorrow's training data.
Your gene pool is shrinking
The biggest source of this contamination is what researchers now call "workslop": AI-generated content passed off as legitimate work. An HBR study found that 40% of employees received workslop in the past month, and 15.4% of work shared internally fits the description. You can spot it instantly: title case everywhere, mechanical em dashes, perfectly structured paragraphs. It all screams ChatGPT. That workslop sits in your databases until someone decides to fine-tune a model or build a RAG pipeline on "all company knowledge." Then it's in the gene pool.
Here's what most organizations get wrong: they probably shouldn't be training models from scratch at all. Most don't have enough proprietary data, and they don't need to. What they should be doing is fine-tuning a capable foundation model, or grounding one through RAG on a curated internal corpus. Either way, the quality bar for your data goes up, not down. You need less data, but it has to be genuinely good: human-verified, domain-specific, and absolutely not recycled AI output.
What breaks when AI eats its own output
When models train or re-index on AI-generated data, accuracy degrades and feedback loops kick in. Each generation gets weaker, like a game of telephone where every round amplifies biases and errors while stripping out the diversity that made the data useful. Your models start giving increasingly confident but progressively wrong answers, and they all start sounding the same.
The organizational impact sneaks up on you. Your customer service bot gives subtly worse answers. Internal search returns blander results. AI-assisted analysts produce reports that read identically. You don't notice it breaking; you notice the outputs becoming mediocre.
AI-generated text collapses toward the statistical mean, so you lose the outlier perspectives and domain-specific jargon that often carry the most valuable signal.
Research backs this up at scale. Shumailov et al. (2024) showed in Nature that when models are trained iteratively on their own outputs, generated text narrows until the model produces near-identical outputs regardless of input. The enterprise version is smaller but the mechanism is the same: train on descriptions of different customer scenarios and after enough cycles, every edge case starts looking like your most common ticket. The unusual cases, often the ones that matter most, get washed out.
The degradation is gradual. Your models don't break suddenly. They slowly get worse in ways that are hard to measure until the damage is done.
What CIOs should do about it
The conversation around AI data assumes every company is building foundation models. Most aren't, and most shouldn't be. The real question is simpler: how do I keep the small, specific datasets that encode my domain expertise clean enough to give a foundation model genuine subject-matter knowledge?
Start with boundaries. Identify the corpora you actually use for fine-tuning and RAG: the support ticket archive, the product documentation set, the analyst playbook, whatever encodes how your company thinks. You don't need to police every AI-generated email. You need to protect these datasets from workslop leaking in. The reflex to index absolutely everything is the problem, not the solution; as I've argued before, forgetting the low-signal noise is what keeps the rest useful.
Then put provenance in place. Tag and track which content is AI-generated versus human-created, not to ban AI but so you can filter when building training or retrieval corpora. Assign an owner. Treat this the way you treat cybersecurity: a continuous discipline, not a one-time cleanup.
Use LLMs to measure information density. LLMs are often better at critiquing content than generating interesting content, and workslop is low-density by definition. Randomly sample a percentage of internally shared documents each month, score them for information density, specificity, and originality, and track the trend. When density drops, workslop is spreading. That gives you a metric before your models show visible drift.
Culturally, managers need to call out workslop. Rubber-stamping AI output without adding authentic thought shouldn't be acceptable. AI-generated content has a place downstream (drafts, first passes, boilerplate). It doesn't belong upstream in the data your models learn from.
The companies that protect a clean, human-verified fine-tuning and retrieval corpus will have models that actually understand their domain. Everyone else is inbreeding their AI into mediocrity, one shared document at a time. The question for your organization isn't whether you're using AI. It's whether you're feeding it slop and calling it knowledge.
Get more frameworks like this
Practical AI strategy for executives. No hype, just real playbooks.
SubscribeYou might also like
