Yoast SEO now supports the new llms.txt protocol—available to all users, whether you’re on the free version or premium plan. This update allows users to future-proof their websites for the evolving landscape of AI-driven search engines.
Yoast has launched the new llms.txt feature, which is now available in both the free and premium versions of their plugin. With this feature, users can add llms.txt files to their websites, ensuring their content is prepared for AI-based search tools that are reshaping how information is discovered online.
What is LLM.txt?
The llms.txt is a proposed new standard designed to streamline how large language models (LLMs) access and interact with online content. It aims to provide LLMs with simplified access to the core content of a webpage, minus the distractions of ads and other elements intended for human users.
The llms.txt files employ a format based on Markdown and create simplified duplicates of existing web pages. These files focus solely on the main content, presenting it in a manner optimized for machine learning models.
However, the proposal has received mixed reactions. Industry experts, including Google’s John Mueller, have noted potential pitfalls for misuse. A concern is that ill-intentioned SEOs might exploit these pages by fabricating or manipulating content within llms.txt files. This raises questions about its reliability and transparency as LLMs could be misled.
Experts have also pointed out that AI systems could just use the content from regular web pages instead of relying on these separately formatted files. At the moment, LLMs aren’t actively looking for llms.txt pages, and there’s no indication that they will start preferring them over standard web pages anytime soon.
Yoast’s New Feature Highlights
According to their announcement, this new feature offers the following benefits:
- Improves AI Understanding of Sites: Helps advanced tools like ChatGPT and Gemini locate and interpret the most relevant content on a site.
- Showcases Key Content Automatically: The plugin identifies vital and recently updated pages without requiring users to make manual decisions.
- Simplifies Technical Setup: The llms.txt file is automatically created and refreshed on a weekly basis, eliminating the need for manual coding or customization.
- Prepares Websites for AI Search: Ensures that websites are optimized and ready to align with the changing ways people find and consume information.
- Available at No Extra Cost: This feature is built into the Yoast SEO plugin and is accessible to all users with just one click, whether they are on the free or premium plan.
The Future of llms.txt in AI-Driven Search
While the adoption of llms.txt is still in its infancy and met with skepticism from some corners of the tech industry, its inclusion in popular tools like Yoast SEO signals the industry’s inclination toward innovation. With AI increasingly shaping online experiences, tools like this may pave the way for more efficient content sharing between publishers and language models.Although its practical application remains to be seen, Yoast’s integration of llms.txt ensures users can stay ahead of the curve. For now, it’s an additional tool in the broader SEO toolkit aimed at keeping websites AI-ready.
