How to Detect and Attribute AI-Engine (LLM) Traffic Using Referrers and User-Agents

If you run growth analytics for any content-heavy site, you have a new attribution problem: AI engines are sending you traffic, and most of it is being misclassified.

People ask ChatGPT, Perplexity, Copilot, or Gemini a question. The answer cites your page. Some of those users click through. In most analytics setups, that visit lands in Direct or a generic Referral bucket — and your SEO team gets zero credit for content that an LLM decided was worth citing.

I've spent the last couple of years building attribution pipelines that treat AI engines as a first-class marketing channel. This post is the generic, tool-agnostic version of that playbook: how to detect this traffic, how to separate browsing users from crawling bots, and how to model it into your funnel.

The three signals you have to work with

There are only three places AI-engine traffic reveals itself:

1. The HTTP referrer header — when a human clicks a citation link in an AI chat interface
2. The user-agent string — when a bot crawls or fetches your page on a user's behalf
3. UTM parameters — when the AI engine is polite enough to append them (some are!)

You need all three, because each one catches traffic the others miss.

Signal 1: Referrer domains

When a user clicks a cited link, the browser usually sends a referrer. These are the domains to pattern-match against (current as of writing — this list changes, so put it in a seed table, not in code):

A basic classification expression looks like this (BigQuery SQL, easily portable):

CASE
  WHEN REGEXP_CONTAINS(referrer, r'(chatgpt\.com|chat\.openai\.com)') THEN 'chatgpt'
  WHEN REGEXP_CONTAINS(referrer, r'perplexity\.ai')                   THEN 'perplexity'
  WHEN REGEXP_CONTAINS(referrer, r'(copilot\.microsoft\.com|bing\.com/chat)') THEN 'copilot'
  WHEN REGEXP_CONTAINS(referrer, r'(gemini|bard)\.google\.com')       THEN 'gemini'
  WHEN REGEXP_CONTAINS(referrer, r'claude\.ai')                       THEN 'claude'
  ELSE NULL
END AS ai_engine

The catch: a large share of AI-engine clicks arrive with no referrer at all — apps, native clients, and privacy policies strip it. That traffic looks Direct. Which brings us to signal 3 in a moment. First, the bots.

Signal 2: User-agents (separating bots from humans)

AI companies hit your site with distinct crawlers, and you must not count these as sessions. But you absolutely should track them separately — crawl activity is a leading indicator of citation activity. The big ones:

Note the pattern: most vendors now split training crawlers from user-triggered fetchers (ChatGPT-User, Perplexity-User). A user-triggered fetch means a human is reading your content inside an AI chat right now — arguably an impression, even though they never hit your analytics. If you only look at pageviews, this consumption is invisible. Log it from your CDN or server logs and model it as its own metric.

CASE
  WHEN REGEXP_CONTAINS(user_agent, r'(ChatGPT-User|Perplexity-User|Claude-User)')
    THEN 'ai_fetch'      -- human reading via AI, not a session
  WHEN REGEXP_CONTAINS(user_agent, r'(GPTBot|OAI-SearchBot|PerplexityBot|ClaudeBot|Google-Extended|Bytespider)')
    THEN 'ai_crawler'    -- indexing/training, exclude from everything user-facing
  ELSE 'human'
END AS agent_class

Signal 3: UTM parameters

Some engines append them (?utm_source=chatgpt.com has been observed in the wild), and you can encourage more of it: anywhere you control content that AI engines ingest — docs, integrations, partner listings — use tagged URLs. UTMs survive referrer stripping, which makes them your best defense against the dark-traffic problem.

Normalize aggressively: lowercase, trim, map chatgpt.com / chatgpt / openai to one channel value. UTM chaos is self-inflicted attribution damage.

Putting it together: the channel model

The order of precedence matters. My rule of thumb:

1. agent_class != 'human' → not a session (route to the bot/fetch models)
2. UTM says AI → AI channel (highest trust, explicit)
3. Referrer matches AI domain → AI channel
4. Referrer matches search engine → Organic Search
5. Referrer present → Referral
6. Nothing → Direct (accept that some AI traffic hides here; you can estimate it by watching Direct's trend against your AI-citation counts)

From there, AI traffic flows into your funnel like any other channel: sessions → engagement → leads → conversions, at whatever grain your model supports (mine go down to URL × country × device). Two findings that seem to generalize across sites I've looked at:

• AI-referred visitors are high intent. They arrive pre-qualified — the AI already answered their question, and they clicked anyway because they wanted depth. Conversion rates typically beat organic search by a meaningful multiple.
• Volume is small but compounding. Single-digit percentages of traffic today, but growing quarter over quarter while classic organic CTRs erode.

Operational notes

• Keep pattern lists in seed data (a dbt seed, a lookup table), not hardcoded in SQL. New engines and user-agents appear monthly.
• Backfill-proof your logic: when you add a new pattern, you want to re-classify history, not just tag traffic going forward. Idempotent classification models make this a re-run, not a migration.
• Monitor the unclassified bucket. A spike in Direct or unknown-referrer traffic is often a new AI surface you haven't mapped yet.
• Track citations upstream if you can. Rank trackers for AI answers exist now; joining "were we cited for query X" with "did we get AI-referred sessions on page Y" is where this gets genuinely fun.

The bigger picture

SEO measured whether you ranked. AEO measures whether you're cited. The pipeline work is the same discipline it's always been — parse the headers, classify the traffic, model the funnel — but the teams that wire this up now will have a year of channel history when their executives start asking "what's our AI traffic strategy?"

If you want help building this for your growth team, get in touch — this is exactly the kind of pipeline I build.