The different AI companies, including OpenAI (ChatGPT), Anthropic (Claude), and others, have managed to attract some of the best Ph.D. talent in machine learning over the past several years. They have also developed some truly remarkable AI platforms.
For years, Google had an enormous advantage in this area. It attracted many of the world’s top machine-learning researchers, giving them significant resources and allowing them to work on improving Google’s search technology. Now, with the rapid growth of AI, Google has shifted enormous resources toward AI research and development as it competes with the companies that emerged from that talent pool.
There is one area, however, where the newer AI companies are still playing catch-up with Google: crawling and understanding the web.
By “crawling,” I mean the process by which computer programs discover websites, follow links, retrieve pages, and collect information that can eventually be used for search or other purposes. There are several steps between a crawler initially discovering a page and that page becoming useful to a search engine or an AI system. One particularly important step is understanding JavaScript, the programming language that powers much of what happens behind the scenes on modern websites.
Google’s 14-Year Head Start
Google’s ability to deal with JavaScript has evolved considerably over the years. There was a time when search engines had a much harder time understanding content that depended on JavaScript. A website could display something perfectly well to a person using a browser, while a search engine crawler might see very little of it.
That created a major SEO problem. You could have valuable content sitting on a page, but if Google’s crawler couldn’t access or understand that content, it was much harder for that information to appear in search results.
Google began making major advances in this area more than a decade ago. In 2014, Google announced that its indexing system was rendering web pages more like a modern browser, including executing JavaScript and applying CSS. Over time, Google developed a sophisticated process for crawling, rendering, and indexing the web. Today, Google can use a headless version of Chromium to render pages and execute JavaScript before processing the resulting content.
In other words, Google has had more than a decade to build and refine the infrastructure necessary to understand the modern, increasingly complicated web.
The newer AI search companies don’t necessarily have that same history.
Why This Matters for AI Search
It’s important to be careful here. There isn’t one universal “AI crawler,” and it would be inaccurate to say that no AI system can read JavaScript. Different AI companies use different crawlers, search systems, indexes, and retrieval technologies. OpenAI, for example, has its own crawler for discovering and surfacing websites in ChatGPT search.
But the larger point remains: AI search is still developing the infrastructure required to reliably discover, render, interpret, and retrieve information from the enormous amount of content on the web.
And that matters to businesses.
We are already seeing a shift from traditional search toward AI-powered answers. Consumers can ask ChatGPT, Google, Claude, and other AI systems questions such as:
“What is the best marketing agency near me?”
or
“Which local HVAC companies should I consider?”
The goal for a business is no longer simply to rank on a traditional search results page. Increasingly, businesses need to think about AI visibility—whether their brand and website can be discovered, understood, and included in AI-generated answers.
Don’t Assume That Because You Can See It, AI Can See It
This is where website technology becomes important.
Imagine you visit a company’s website and see a beautifully designed service page. The page clearly explains what the company does, where it operates, and why someone should choose it. To you, everything looks perfect.
But what if some of that information isn’t actually present in the initial HTML and only appears after several JavaScript processes run?
A human visitor with a modern browser may have no idea there is a problem. The browser executes the code, loads the content, and displays the finished page.
A crawler may have a very different experience.
This is not just an AI issue. Google itself has long documented situations where JavaScript can create challenges for search engines and recommends approaches such as server-side rendering, static rendering, or hydration when appropriate.
The fact that Google has become very good at rendering JavaScript doesn’t mean every crawler on the internet has the same capabilities.
That’s why website owners should not automatically assume that “visible to me” means “visible to search.”
A Simple Test You Can Try
There is a simple experiment you can perform on your own website.
Turn JavaScript off in your browser and visit your most important pages. Look at your homepage, service pages, location pages, product pages, blog posts, and contact information.
Then ask yourself:
What disappeared?
If important text, links, headings, or other information suddenly vanish, it’s worth having your web developer investigate how that content is being delivered.
This does not automatically mean Google or AI systems cannot access the content. Google can execute JavaScript, and different AI crawlers have different capabilities. But it is a warning sign that deserves attention.
For businesses that are investing in AI Overviews SEO, traditional SEO, and other forms of search visibility, making important content technically accessible is a basic place to start.
What Does This Have to Do With Google AI Overviews?
Quite a bit.
Google’s search results are changing rapidly. AI Overviews can now appear at the top of search results and provide users with an AI-generated response to their question. Instead of simply scanning ten blue links, a user may read Google’s summary and then decide which sources or businesses to explore.
That has led to an increasingly common question from business owners:
“How do I rank in Google AI Overviews?”
There isn’t a magic button that guarantees inclusion. Google’s AI systems still need to discover and understand the information they use. That makes the fundamentals of technical SEO more important, not less.
Your website needs to be crawlable. Your important content needs to be accessible. Your pages need to clearly communicate what your business does. Your information needs to be structured in a way that search engines can understand.
AI doesn’t eliminate SEO. In many ways, it makes good technical SEO even more important.
What Should You Do About It?
The answer isn’t to stop using JavaScript. Modern websites rely on JavaScript for menus, forms, calculators, personalization, animations, and countless other functions.
Instead, make sure the information that matters most to your business doesn’t depend unnecessarily on complicated JavaScript.
Your core services, locations, products, expertise, contact information, and other important content should be available in a form that search engines and other discovery systems can reliably access and understand.
This is good SEO today, and it is increasingly important for AI visibility tomorrow.
Google Has a 14-Year Head Start. What Does That Mean for Your Business?
The AI companies are moving incredibly quickly, so today’s limitations may not remain limitations for very long. They will undoubtedly continue improving their crawling, rendering, search, and retrieval technologies. But Google has had a remarkable head start. It has spent more than 14 years dealing with many of the technical challenges involved in understanding the modern web.
For businesses, the lesson is simple: don’t wait for AI search engines to become perfect.
Build your website today so that as many search and AI systems as possible can discover, understand, and trust your content. Your website should not only look good to potential customers. It should also make sense to the systems increasingly responsible for helping those customers find you.
At Nautical Agency, we help businesses understand how their websites are being discovered and interpreted across modern search. If you’re concerned about your AI visibility or want to know whether your website is ready for the changing search landscape, contact our team.
This is Part 1 of our series on the ways AI search is still playing catch-up with Google. Parts 2 and 3 will explore two other areas where the differences between traditional search and AI search could have important implications for businesses.
