How to Create the Perfect llms.txt File (With Templates)

The web was built for humans and search engine crawlers. Neither of those audiences needed a plain-language overview of what a website is about — humans could see it, and crawlers could index it. AI language models need something different. They need a concise, structured summary they can process before deciding how to use your site’s content.

That is exactly what llms.txt provides. It is a plain text file (Markdown-formatted) served at your domain root that answers three questions for AI models: What is this site? What does it offer? Where are the key pages?

The concept was proposed by Jeremy Howard in late 2024 and has rapidly gained traction. Our protocol adoption research shows that llms.txt is the fastest-adopted AI protocol, with over 15% of top-10K websites now serving one. The full protocol landscape covers how llms.txt fits alongside NLWeb and MCP.

AgentReady™ checks for llms.txt as part of the AI Protocols factor. Sites with a well-structured llms.txt file consistently score higher on AI comprehension metrics.

Understanding the consumption flow helps you write a better file. When an AI model encounters your domain — either through a user query or during a research task — it follows a predictable sequence.

First, it checks robots.txt to confirm it has crawl permission. Then, if it is looking for site-level context (rather than a specific page), it checks for llms.txt at the domain root. If the file exists, the model parses it to understand your site’s purpose, structure, and key pages before deciding which pages to crawl in depth.

This is important: llms.txt does not replace deep crawling. It supplements it. Think of it as a table of contents that helps the model navigate your site intelligently rather than randomly crawling pages.

The llms.txt specification also supports an extended version at /llms-full.txt. The base llms.txt should be concise (under 500 words). The extended version can include full content from key pages, which is useful for AI models that prefer to consume comprehensive context in a single request.

The llms.txt format follows a simple Markdown structure. The spec is intentionally minimal to maximize adoption. Here are the key elements:

The file starts with an H1 heading (#) containing your site or company name. This is followed by a blockquote (>) with a one-line description of what your site does.

After the description, you include sections organized by H2 headings (##). The most common sections are:

## Docs — links to your main documentation or content pages. ## Optional — supplementary links that provide useful but non-essential context. ## Blog or ## Resources — links to frequently referenced content.

Each link is formatted as a standard Markdown link with a colon-separated description: - Page Title: Brief description of what this page covers.

Keep the core llms.txt under 500 words. Be specific in your descriptions — do not just list page titles. Tell the AI model what it will find on each page.

Different site types need different llms.txt approaches. Below are ready-to-use templates for the five most common site types. Copy the template that matches your site, fill in your specifics, and deploy. Each template is designed to provide maximum signal to AI models with minimum verbosity.

E-commerce and local business sites have distinct needs. An e-commerce llms.txt should help AI models understand your product catalog structure, while a local business llms.txt needs to communicate location, services, and service area clearly.

For e-commerce, prioritize your category pages over individual products. AI models querying about your store typically want to understand what you sell and how your catalog is organized, not details about a specific SKU. Include your returns policy and shipping information since those are among the most common AI-generated queries about e-commerce sites.

For local businesses, geographic context is critical. AI models frequently answer location-based queries (“best plumber near me”) and your llms.txt should explicitly state your service area, locations, and hours.

Your llms.txt file must be accessible at https://yourdomain.com/llms.txt. It should return a 200 status code with a text/plain or text/markdown content type. Do not put it in a subdirectory or behind authentication.

Here are platform-specific deployment instructions for the most common setups:

WordPress: Upload the file to your theme root or use a plugin like Yoast SEO (which now supports llms.txt) to manage it. Alternatively, add a rewrite rule in your .htaccess file to serve a file from wp-content/uploads/llms.txt.

Shopify: Add the file through Settings → Files, then create a URL redirect from /llms.txt to the file URL. Alternatively, use a Shopify app that supports llms.txt deployment.

Vercel / Next.js: Place the file in your public/ directory. It will be automatically served at the root path.

Cloudflare Pages: Place the file in your build output’s root directory, or create a Cloudflare Worker that serves the content at /llms.txt.

Static Sites (Netlify, GitHub Pages): Place llms.txt in the root of your build output directory alongside your index.html.

After deployment, verify by navigating to yourdomain.com/llms.txt in a browser. Confirm the content type header with curl -I yourdomain.com/llms.txt.

The llms.txt specification includes support for an extended file at /llms-full.txt. While your base llms.txt is a concise summary (under 500 words), the full version can include complete content from your key pages. Some AI models, particularly those operating in research mode, prefer to consume a single comprehensive document rather than crawling multiple pages.

The llms-full.txt file follows the same Markdown format but expands each section with actual content. For a documentation site, this might mean including the full text of your getting started guide, API reference, and key concept pages.

For dynamic sites with frequently changing content (e-commerce catalogs, news sites, SaaS changelogs), consider auto-generating your llms.txt. Build a script or serverless function that pulls your current key pages, product categories, or latest articles and outputs a fresh llms.txt on each request or at a scheduled interval.

Keep the dynamic version cached with a reasonable TTL (1–24 hours). AI crawlers do not need real-time updates, and regenerating on every request wastes server resources.

After reviewing thousands of llms.txt files through AgentReady scans, these are the patterns that consistently reduce effectiveness:

Mistake 1: Writing for humans instead of models. Your llms.txt is not a marketing brochure. Skip the superlatives and buzzwords. AI models need factual descriptions, not persuasive copy. Instead of “The world’s most innovative platform,” write “A project management platform for remote teams with time tracking, task boards, and reporting.”

Mistake 2: Linking to every page. More links does not mean more coverage. Curate. Link to the 10–15 most important pages that represent your site’s core content. AI models will discover deeper pages through crawling.

Mistake 3: Stale content. If your llms.txt references pages that return 404 errors or describes products you no longer offer, it damages your credibility signal. Set a reminder to review the file quarterly.

Mistake 4: Missing the description block. The blockquote description after your H1 heading is the single most important line in the file. It is what AI models use for site-level classification. Write it carefully.

Mistake 5: Wrong content type. If your server returns text/html instead of text/plain, some AI parsers will fail. Verify your content type headers after deployment.