HTML Discovery Tags: Help AI Systems Find Your Markdown Content

Why this matters

HTML discovery tags are small <link> and <meta> elements in your page's <head> that tell AI crawlers where to find the clean, Markdown version of your content. They bridge the gap between your regular website and the AI-readable endpoints Legible provides.

Legible can inject these tags automatically for Cloudflare proxy customers, or you can add them manually. This guide explains what each tag does, how to enable automatic injection, and how to verify the tags are working.

What are discovery tags?

Discovery tags are standard HTML elements placed in the <head> of your pages. They work the same way that <link rel="alternate" hreflang="en"> tells search engines about translated versions of a page — except here, they tell AI systems about the Markdown version.

Legible uses three discovery tags. Together they give AI crawlers a complete map of your AI-readable content without requiring them to guess URLs or parse your HTML.

<link rel="ai-index"> — Points to your llms.txt file, which is the top-level index of all AI-readable content on your site.
<meta name="legible:ai-sitemap"> — Points to your ai-sitemap.json, a structured JSON index that AI systems can parse programmatically.
<link rel="alternate" type="text/markdown"> — Appears on individual content pages and points directly to the Markdown version of that specific page.

Automatic injection with Cloudflare

If your site uses Cloudflare proxy mode, Legible can inject discovery tags automatically. The Cloudflare Worker that already serves your Markdown content uses HTMLRewriter — a streaming HTML transformer — to add the tags to every page as it passes through the edge.

This means zero changes to your CMS, templates, or code. The tags appear in the server-rendered HTML that crawlers see, which is important because most AI crawlers do not execute JavaScript.

Go to Site Settings → Discovery in your Legible dashboard.
Toggle "Enable automatic discovery tag injection" on.
Choose which tags to inject (all three are enabled by default).
Save. The Worker updates within seconds.

What gets injected

When automatic injection is enabled, every HTML response from your origin gets three global tags appended to <head>. On pages whose URL matches your content collection path (for example /blog/my-post), a fourth per-page tag is added.

<!-- Global tags (every page) -->
<link rel="ai-index" type="text/plain" href="https://read.yoursite.com/llms.txt" />
<meta name="legible:ai-content-base" content="https://read.yoursite.com/" />
<meta name="legible:ai-sitemap" content="https://read.yoursite.com/ai-sitemap.json" />

<!-- Per-page tag (content pages only) -->
<link rel="alternate" type="text/markdown"
      href="https://read.yoursite.com/blog/my-post.md" />

Choosing which tags to inject

The Discovery settings panel lets you toggle each tag independently. You might want to start with just the ai-index tag if you are testing, or disable the per-page alternate tag if your collection path setup is still in progress.

All three tags are safe to enable together and that is the recommended configuration. The GEO Readiness audit awards up to 5 points for having both the ai-index and alternate Markdown tags present.

Manual setup for proxy-free sites

If you are using proxy-free mode — where Legible hosts your Markdown on a subdomain like read.yoursite.com but does not sit in the request path of your main site — automatic injection is not available. You will need to add the tags to your site's <head> manually.

The exact method depends on your platform. In Webflow, you paste the tags into Project Settings → Custom Code → Head Code. In WordPress, the Legible plugin handles it automatically. For other platforms, add the HTML to your global head template.

<!-- Add to your site's global <head> -->
<link rel="ai-index" type="text/plain"
      href="https://read.yoursite.com/llms.txt" />
<meta name="legible:ai-content-base"
      content="https://read.yoursite.com/" />
<meta name="legible:ai-sitemap"
      content="https://read.yoursite.com/ai-sitemap.json" />

<!-- Add to CMS content page templates -->
<link rel="alternate" type="text/markdown"
      href="https://read.yoursite.com/blog/{{slug}}.md" />

Verifying discovery tags are working

The simplest check is to view the page source of any page on your site and search for ai-index or text/markdown in the raw HTML. The tags should appear inside <head>, not injected by JavaScript.

Legible's GEO Readiness audit also checks for these tags automatically. If you run or refresh the audit after enabling injection, the discovery signals score should increase.

Open any page on your site in a browser.
Right-click → View Page Source (not Inspect — you want the raw server-rendered HTML).
Search for "ai-index" — the <link> tag should be visible.
On a content page, also search for "text/markdown" — the alternate link should point to the correct Markdown URL.
Run the GEO Readiness audit in your Legible dashboard to confirm the score reflects the tags.

Why server-rendered tags matter

Most AI crawlers — including GPTBot, ClaudeBot, and CCBot — do not execute JavaScript. They parse the raw HTML response and move on. If your discovery tags are injected by a client-side script, these crawlers will never see them.

That is why Legible's automatic injection uses Cloudflare's HTMLRewriter, which modifies the HTML stream at the edge before the response reaches the client. The result is indistinguishable from tags that were in your original template.

How discovery tags affect your GEO score

Legible's GEO Readiness scoring awards up to 5 points for discovery signals. Having both the ai-index link and the alternate Markdown link on a page earns the full 5 points. Having one of the two earns 4 points.

These tags work alongside the other scored signals — llms.txt presence, ai-sitemap.json, clean HTML structure, robots.txt permissions, and response headers — to give a complete picture of your site's AI readiness.

Frequently asked questions

Will these tags affect my regular SEO? No. Search engines already understand <link rel="alternate"> tags. Adding a Markdown alternate is no different from adding an hreflang alternate. It will not confuse or penalize search rankings.
Can I use discovery tags and a JavaScript snippet together? Yes. The JS snippet (coming soon) will check for existing tags before injecting and will skip duplicates. Using both gives you server-rendered tags for AI crawlers and a future extension point for analytics and badges.
Do I need to redeploy my Worker after changing settings? No. The dashboard pushes the updated INJECT_DISCOVERY_TAGS setting directly to your Worker's environment variables. Changes take effect within seconds.
What if my collection path changes? Update the collection path in your Legible site settings. The Worker reads it from the environment, so the per-page alternate tags will automatically use the new path.