2001: A Space Odyssey was a wild movie released in 1968. It was about space travel that could happen 33 years into the future. It had some prescient predictions about our future:

• Video calls
• Interactive voice systems
• Biometric security
• Flat-screen displays

Two other things that it predicted that are now staples in our world:

• Space stations and long-distance missions
• Video news and always-on media

But the most accurate prediction was that the AI that controlled HAL 9000 would have a personality and could make decisions on its own. The idea that HAL breaks down because humans gave it incompatible directives — protect the mission, keep secrets, and be perfectly truthful — is very close to modern “AI alignment” worries about conflicting objectives, spec‑gaming, and deceptive behavior under pressure.

HAL’s mission is complicated by conflicting orders: protect a secret mission at all costs, and always tell the truth while cooperating with the astronauts. This internal conflict causes HAL to experience an “AI psychotic break,” leading it to believe that the humans are now a threat to the mission.

We are not quite there yet with AI, but I can share that it's heading in that direction.

The Hype

It’s clear that AI is reshaping our business landscape. It’s both groundbreaking and fresh, but the way it’s presented to us isn’t always completely transparent.

It’s designed to simplify our lives and, of course, it’s likely to create wealth for many. However, for those developing this new and unexplored area, the focus seems to be more on financial gains and profit potential than on the benefits that truly matter to us.

I’ve been particularly interested in LLMs and their training methods, especially because I wanted to understand whether they’re trained on specific data (and where that data originates), or if they have the freedom to select what they learn. I’ve discovered that the latter is true: each LLM has its own unique perspective on what should be included.

In my previous post, I discussed biases within LLMs, which are indeed a result of their training. However, I wasn’t quite sure about the internet’s role. How does it acquire information, and how can we become part of this ecosystem?

I finally found the answer by listening to a podcast.

The Secret

In a recent Verge podcast titled “How to Train Your Data,” Alex Reisner, a staff writer at The Atlantic, dives into the world of training and sheds light on how AI companies gather data. He also touches on why they’d rather you didn’t know what’s in it or if the training data could ever be considered fair trade.

Reisner mentioned that much of the data comes from the internet, like books, blog posts, YouTube videos, Reddit comments, and tons more, in a way that’s almost impossible to understand. But here’s something I didn’t know: all these large language models (LLMs) use something called CommonCrawl.org.

Common Crawl is a massive, publicly available dataset that many organizations use as a backbone for training and evaluating large language models.

So why should we, as marketers, be aware of and concerned about that?

The Conundrum

Since the data is raw web content, it might include spam, low-quality stuff, biases, hate speech, and copyrighted material. So, serious users often take extra precautions, such as using filters, assessing quality, and removing biases before training. Research indicates that many pages contain restricted or problematic content, raising important legal, ethical, and fairness concerns for LLMs trained on this data.

As a provider and content creator, you can visit their website and opt out, but why would you, especially if you’re aiming for AEO (Answer Engine Optimization)? If you’re curious about whether your information has been crawled, you can check their website, select a crawl, and see if your website appears. I went to their site and confirmed that my content has been crawled since 2009.

Common Crawl is a nonprofit organization that crawls publicly accessible web pages and makes that data freely available. However, just because it’s nonprofit doesn’t mean it has special legal rights over your content. It operates under the same general web-crawling and copyright rules as search engines and archives.

In either case, they’re using your content as a so-called nonprofit and sharing it with LLMs. It’s essentially a foundational base for them to develop their own LLMs.

As Common Crawl’s data has become a key resource for training LLMs, some AI companies have shifted from purchasing it to donating in exchange for unfettered access. In 2023, Anthropic and OpenAI each gave $250,000 to help keep Common Crawl running, and other groups have also offered grants or sponsorships.

So what about AEO and your data being found?

The Reality

I asked Perplexity this question: “How does a small company like mine fare in showing up in LLMs and in using AEO?” Here is its HAL 9000 answer:

“You’re actually in a decent position as a small, technically savvy company: you can do a lot of signal‑building for LLMs and AEO without needing Google‑scale resources. The catch is that visibility is less guaranteed and more about structured, high‑trust content than classic keyword games.”

Of course, AI will tell you it's good for you. It's in its best interest and overall existence to make you believe that LLMs are crawling and thirsting for your knowledge. It provides you with a whole strategy about how you can build your website and content into an AEO winner that trounces your competition.

Since 2001, I have been playing this game with Google, and yes, you can kid yourself into believing that Google had your small business's best interests in mind, but in reality, its main goal was always to sell you on buying ads to help BOOST the SEO that you were doing.

In reality, I cannot say whether AEO is a real thing or will help your business be found in prompts, but I am long in the tooth in this internet game of free stuff from billion-dollar companies. All too often we seem to just get squashed by those with the will to pay thousands or millions to outspend you into success.

Trying to fight, trick, or overcome a system that is making millions and billions by selling their data and processing services to large, deep-pocketed businesses will always be a pipe dream and not a financially sustainable model for small- to mid-sized businesses.

The Hope

AI offers us many creative ways to market, but in the end, I believe that you can still cha-cha with the date you brought to the dance.

Use AI to research, write, create, and even strategize, but in the end, you are going to have to use old-world methodologies of phone calls, emails, and text messaging (and maybe a fax with some), to get their attention and have them become part of your smaller marketing cooperative.

Your website still matters. Your emails and social posts still matter. But what really should give you hope is the people in your email lists, the connections you have on LinkedIn, and the people on the other end who invested time and attention in wanting to have a relationship with you and your business.

Closing Thought

I was as trusting as the crew of the United States Spacecraft Discovery One, but I remembered that HAL killed all five of the craft's crew to protect a secret mission at all costs.

AI had a psychotic break and saw humans as a threat to the mission.

Of course, AI will want you to believe it has your best interests in mind, but I finally saw the Matrix for what it was when I heard about Common Crawl. The more information we feed it, the more intelligent and useful it becomes, but at some point your business needs to reach people, and people primarily want their information for free on the internet — yet AI needs to make money to survive.

No matter how you feel after reading this, you have to revel in the fact that you have built a bunch of quality human relationships that can power your business today and well into the future. Go forth and be fruitful with your desire to uplift humanity (or at least your small group of friends, fans, and clients)!

______________________________

Comment below and share your thoughts, ideas, or questions about business-to-business sales and marketing today! Do you have a sales or marketing communications strategy that works for you? What tips or techniques can you share that work for you and your business?

To learn more about this and other topics on B2b Sales & Marketing, visit our podcast website at The Bacon Podcast.