As readers, we are already in the dark about how a chatbot makes its decisions, and we certainly will not know if the answers it supplies might have been manipulated. If you want to know about climate change, or immigration policy, or any other contested issue, there are people, corporations, and lobbying groups with strong vested interests in shaping what you believe. They’ll hire LLMOs to ensure that LLM outputs present their preferred slant, their handpicked facts, their favored conclusions.
Donath, J., & Schneier, B. (2024, April 22). It’s the End of the Web as We Know It. The Atlantic. https://www.theatlantic.com/technology/archive/2024/04/generative-ai-search-llmo/678154/
Search engines have traditionally facilitated such connections. By contrast, LLMs synthesize their own answers, treating content such as this article (or pretty much any text, code, music, or image they can access) as digestible raw material. Writers and other creators risk losing the connection they have to their audience, as well as compensation for their work. Certain proposed “solutions,” such as paying publishers to provide content for an AI, neither scale nor are what writers seek; LLMs aren’t people we connect with. Eventually, people may stop writing, stop filming, stop composing—at least for the open, public web. People will still create, but for small, select audiences, walled off from the content-hoovering AIs. The great public commons of the web will be gone.
If we continue in this direction, the web—that extraordinary ecosystem of knowledge production—will cease to exist in any useful form. Just as there is an entire industry of scammy SEO-optimized websites trying to entice search engines to recommend them so you click on them, there will be a similar industry of AI-written, LLMO-optimized sites. And as audiences dwindle, those sites will drive good writing out of the market. This will ultimately degrade future LLMs too: They will not have the human-written training material they need to learn how to repair the headlights of the future.