Here Is Why Your Favorite Web Analytics Is Failing You

Imagine a world where only robots go out and do things for you. You send your robot to stores to buy clothes and food for you. You send your robot to perform your job for you - accounting, plumbing, whatever. You send your robot to pick up your medications for you. You send your robot to court to pay your fines, serve jury duty, etc.

You exist but you stay at home and enjoy your life. Maybe you go out walking through your neighborhood every day for exercise. Or maybe you vacation in a foreign country. But whenever you need to talk to someone you tell your robot what to say and send it off to have the conversation.

At first you are unique. You're the "wealthy" person, the early adopter. Your robot goes around the landscape doing its thing. Sometimes it succeeds. Sometimes it fails.

The robot comes back and says it executed your instructions. Maybe it tells you how things went. Maybe you get what you asked for. Maybe it's empty-handed.

Meanwhile, your neighbors, co-workers, local governments don't even know you made the effort. That's because they didn't know you sent the robot. They were looking for a human being to show up and make the purchase, do the job, fill out the paper work, etc.

You refine your robot's programming. You upgrade to a better, more flexible, more human-like model.

People begin to notice the robot. A few of them interact with it. The robot completes more tasks.

Soon other people begin using robots to do their tasks for them. Life is is good. Everyone is happier because they can do whatever they want while the robots take the burden off them.

But there's a problem. Everyone thinks they're the only ones using robots. They don't see the robots their friends, neighbors, co-workers, customers, and vendors are using to take care of business.

That's because everyone is "robot-blind". No one thought to treat the robots as if they are real human beings because they're NOT real human beings. They're just robots wandering around the landscape doing things no one really cares about.

Robots grow in number, do more things, and people just keep living their lives. But they start to wonder why no one visits them any more. No one is calling. They hang up on or ignore calls from robots. They block robots from their businesses. They don't count the robots when they count the number of people they see and interact with.

This is happening on the Web right now

If you run a Website you're probably using some kind of Javascript-based analytics to track visits to your site. The analytics script must be activated by the browser or robot to record its visit and activity.

A few older bots do this. They are mostly bad bots designed to inflate clicks on advertising and scrape Websites.

Many so-called "agentic" bots are NOT clicking on the Javascript. They don't need to do that. Even if they have been designed to fill out a form, they're following their instructions (probably generated by some Large Language Model) to the letter. And those instructions don't include anything about triggering analytics scripts.

Even server-side analytics reports barely record the traffic. They're showing you raw hits but they cannot track real sessions because those sessions don't exist.

Agentic traffic doesn't look like browser traffic. It doesn't look like human traffic. It may represent a proxy server somewhere used to protect a person's privacy. It may be sent by an LLM to find answers to questions. Or maybe it's collecting data to train new models.

Agentic traffic doesn't explain what it's doing. It doesn't engage directly with Website owners. It just goes about its business. It will leave a few traces in obscure server data.

How you might be able to see agentic traffic today

A few Web applications - content management systems - will track hits for every page they serve. If you run one of these applications like a Web forum you'll see some spectacular analytics data. A site that saw a few hundred visits per day suddenly receives thousands, tens of thousands, hundreds of thousands of visits every day.

Those are agentic visits coming in from thousands of random IP addresses. They're probably using SIM card farms. SIM cards are the little chips you insert into smartphones so they can connect to cellular networks. A SIM card farm might look like a box with a lot of these cards inserted into slots.

Someone sent all those visits from all those IP addresses. They don't explain what they're doing. They're just using up your server resources and your Web hosting bandwidth. If you're lucky you're on a truly unlimited plan that can absorb all the bogus traffic. If you're unlucky your account will be suspended before the end of the month and/or you'll be hit with a higher than normal hosting bill.

According to some estimates I've seen, agentic traffic now accounts for about 80-90% of all Web visits.

But you're not seeing them because you're using obsolete analytics services like Google Analytics or Matomo.

And there's currently nothing available - certainly not in the free analytics category - that will show you what all these agentic fetchers are doing.

Model context protocol won't help you

We're at the dawn of the MCP age. 1-2 years from now most Websites will probably be running MCP servers. They'll handle a lot of the agentic traffic. We're being told that's a good thing and I won't argue with the sales pitch.

But I've yet to see anyone who is talking up MCP services explain how detailed or reliable the analytics will be for all these agentic interactions.

A good MCP layer should track what it's doing and report back home. I hope that's built into the MCP generation platforms people are starting to promote. If not, well, you've now got your next mission.

But meanwhile - for the time being - robots are walking all over you and you can't even see them. You have no idea of how many there are or what they are doing.

If you've been feeling a bit weird about your Website lately, that may be why.