What Is llms.txt? How to Add llms.txt in WordPress

Last month, I noticed crawlers from companies like OpenAI and Google in my website analytics. My first reaction was concern: Was my content being scraped without my permission? I also worried that too many requests from AI or search crawlers might slow down my site for visitors.

But then I started thinking: What if I could actually turn this into an opportunity? What if I could guide AI tools—like ChatGPT—to the content I want them to see?

That’s when I discovered something called llms.txt. It’s a new file format designed to help large language models (LLMs) understand which pages on your site are most useful. This can improve how your content shows up in AI-generated answers and even help your site get mentioned as a source.

In this guide, I’ll show you how to create an llms.txt file using a plugin or a manual method. Whether you want more AI visibility or simply more control, this is a great way to start shaping how AI interacts with your content.

What Is an llms.txt File and Why Do You Need One?

An llms.txt file is a new proposed standard that gives AI tools like ChatGPT or Claude a structured list of the website content you want them to use when generating answers.

This file lets you point to your most helpful posts, tutorials, or landing pages—content that’s clear, trustworthy, and AI-friendly.

Think of it like a welcome mat for AI. You’re saying: “If you’re going to use my site in your answers, here’s what I recommend you look at first.”

The file itself lives at the root of your site (like example.com/llms.txt) and is written in plain Markdown. It can include links to your sitemap, cornerstone content, or anything else you’d want cited.

Including your sitemap ensures AI tools can find a complete index of your site—even if they don’t follow every link listed individually.

This is part of a broader approach called Generative Engine Optimization (GEO). You might also hear it called AI content optimization or AI search visibility. The idea is to help AI models give better answers—and increase the chances of your site being linked as a source.

Just keep in mind that llms.txt is still an emerging format. Not all AI companies support it yet, but it’s a smart step if you’re looking to shape your content’s role in AI search results.

llms.txt vs. robots.txt: What’s the Difference?

You might be wondering how llms.txt compares to robots.txt, since both files deal with bots and visibility.

The key difference is this:

• robots.txt tells crawlers what they’re allowed to index and cache.
• llms.txt gives AI models a curated list of the content you want them to reference when generating AI-powered answers.

Here’s a side-by-side look:

For a complete AI strategy, you can use both files at the same time. You can use llms.txt to welcome the AI bots you want, while using robots.txt to block the ones you don’t.

My guide will show you how to use both files to manage your AI content strategy. You can use the quick links below to jump to the method that best fits your strategy:

Method 1: Create an llms.txt File Using AIOSEO (Recommended)

The easiest way to create an llms.txt file in WordPress is by using the All in One SEO plugin (AIOSEO). I recommend this method because it does all of the work for you.

It automatically creates a helpful llms.txt file that guides AI crawlers to your content, and it keeps the file updated as you add new posts and pages.

Step 1: Install and Activate AIOSEO

First, you’ll need to install and activate the AIOSEO plugin.

For a full walkthrough, you can see our step-by-step guide on how to properly set up All in One SEO.

The great news is that the llms.txt feature is enabled by default in all versions of AIOSEO, including the free version.

However, since we’re talking about taking full control of your content and SEO, it’s worth mentioning a few powerful features you get if you upgrade to the AIOSEO Pro license.

While you don’t need these for llms.txt, they are incredibly helpful for growing your website traffic:

• Advanced Rich Snippets (Schema): The Pro version gives you more schema types, which helps you get those eye-catching rich results in Google (like reviews, recipes, or FAQs). Adding schema markup can also help your content appear in AI search.
• Redirection Manager: This tool makes it easy to redirect bots or users from certain pages, fix broken links, and track 404 errors. It gives you more control over how both visitors and crawlers navigate your site.

So, while the llms.txt feature is free, upgrading gives you a much more powerful toolkit for managing and growing your website’s presence.

Step 2: Verify Your llms.txt File

Because this feature is turned on by default, there’s nothing you need to do to set it up. AIOSEO is already helping guide AI bots for you.

You can see the settings by navigating to All in One SEO » General Settings and clicking the ‘Advanced’ tab.

Here, the ‘Generate an LLMs.txt file’ toggle is on by default.

When you click the ‘Open LLMs.txt’ button, you’ll see that the file is a list of links to your content.

This is exactly what you want for GEO. It’s a clear signal to AI bots that you are welcoming them and have provided a helpful guide for them to follow.

Just keep in mind that llms.txt is not an enforceable rule—AI tools may or may not choose to follow it.

Method 2: Create an llms.txt File Manually

If you prefer not to use a plugin, then you can still create a helpful llms.txt file manually. This approach involves creating a text file with a list of links to your most important content.

Step 1. Create a New Text File

First, you need to open a plain text editor on your computer (like Notepad on Windows or TextEdit on Mac).

Create a new file and save it with the exact name llms.txt.

Step 2. Add Your Content Links

Next, you need to add links to the content you want AI bots to see. The goal is to create a simple, clear map of your site using markdown headings and lists.

While you can just list your most important URLs, a best practice is to organize them into sections. You should always include a link to your XML sitemap, as it’s the most efficient way to show bots all of your public content.

Then you can create separate sections to highlight your most important posts and pages.

Here is a more structured template you can copy and paste into your llms.txt file. Just be sure to replace the example URLs with your own:

# My Awesome Website

## Sitemaps

- [XML Sitemap](https://example.com/sitemap.xml)

## Key Pages

- [About Us](https://example.com/about-us/)
- [Contact Us](https://example.com/contact/)

## Key Posts

- [Important Guide](https://example.com/important-guide/)
- [Key Article](https://example.com/key-article/)

Step 3. Upload the File to Your Website

Once you’ve saved your file, you need to upload it to your website’s root directory. This is usually named public_html or www.

You can do this using an FTP client or the File Manager in your WordPress hosting dashboard. Simply upload the llms.txt file from your computer into this folder.

Step 4. Verify Your File Is Live

Finally, you can verify that your file is working correctly by visiting yourdomain.com/llms.txt in your browser.

You should see the list of links you just created.

Bonus: How to Block AI Bots Using Your robots.txt File

While using llms.txt to guide AI bots is great for GEO, you may decide you want to block them instead. If your goal is to prevent AI companies from using your content for training, then the official method is to add rules to your robots.txt file.

The robots.txt file is a powerful tool that gives instructions to web crawlers. For a complete overview, I recommend our full guide on how to optimize your WordPress robots.txt file.

Method 1: Edit robots.txt Using the AIOSEO Plugin (Recommended)

If you already use All in One SEO, this is the safest and easiest way to block AI bots. The plugin has a built-in robots.txt editor that prevents you from making mistakes.

First, navigate to All in One SEO » Tools in your WordPress dashboard. From there, find and click on the ‘Robots.txt Editor’ tab.

First, you need to click the toggle switch to enable custom robots.txt.

Then you will see an editor where you can add your custom rules. To block a specific AI bot, you need to add a new rule by clicking the ‘Add Rule’ button. Then you can fill in the fields for the User-agent (the bot’s name) and a Disallow rule.

For example, to block OpenAI’s bot, you would add:

User-agent: GPTBot
Disallow: /

You can add rules for as many bots as you like. I’ve included a list of common AI crawlers at the end of this section.

Once you’re done, just click the ‘Save Changes’ button.

Method 2: Edit robots.txt Manually via FTP

If you don’t use a plugin, you can edit the file manually. This requires you to connect to your site’s root directory using an FTP client or the File Manager in your hosting account.

First, find your robots.txt file in your site’s root folder and download it. Do not delete it.

Next, open the file in a plain text editor. Add the blocking rules you want at the end of the file.

For example, to block Google’s AI crawler, you would add:

User-agent: Google-Extended
Disallow: /

After you save the file, upload it back to the same root directory, overwriting the old file.

Common AI Bots to Block

Here is a list of common AI user agents you might want to block:

• GPTBot (OpenAI)
• Google-Extended (Google AI)
• anthropic-ai (Anthropic / Claude)
• CCBot (Common Crawl)

You can add a separate block of rules for each one in your robots.txt file.

FAQs About llms.txt and robots.txt in WordPress

I often get questions about managing AI crawlers. Here are some of the most common ones.

1. Will adding an llms.txt file affect my website’s SEO?

No, creating an llms.txt file won’t affect your regular SEO rankings. Search engines like Google still rely on your robots.txt file and other SEO signals to decide what gets indexed and ranked.

llms.txt is different. It’s designed for AI tools, not search engines, and is used to support Generative Engine Optimization (GEO). While it may help AI models better understand and cite your content, it doesn’t influence how your site appears in traditional search results.

2. Will using an llms.txt file help me get more traffic from AI?

No, using an llms.txt file isn’t a guaranteed way to get more traffic from AI tools. It can help by pointing language models like ChatGPT to content you want them to see—but there’s no promise they’ll use it or link back to your site.

llms.txt is still new, and not all AI platforms support it. That said, it’s a smart step if you want more control over how your content might be used in AI-generated answers.

3. What is the difference between llms.txt and robots.txt?

An llms.txt file acts like a guide for AI models, pointing them to the content you want them to see—your most helpful posts, tutorials, or pages. It’s meant to improve your GEO strategy by highlighting what’s worth citing.

In contrast, a robots.txt file is used to block search crawlers and AI tools from accessing specific parts of your site. You use llms.txt to say “look here,” and robots.txt to say “don’t go there.”

Final Thoughts on Managing Your Content’s Future

The world of AI and Generative Engine Optimization is changing fast. So, I recommend checking in on your strategy every few months.

A bot you block today could be a major source of traffic tomorrow, so being ready to adapt is key. You can always switch from blocking to guiding (or vice-versa) as your business goals evolve.

I hope this guide has helped you make an informed decision about the future of your content in the world of AI. If you found it useful, you might also like our other guides on growing and protecting your site:

If you liked this article, then please subscribe to our YouTube Channel for WordPress video tutorials. You can also find us on Twitter and Facebook.