This week brings a critical wordpress core milestone, fresh data on AI bot traffic overtaking human visitors, and practical guidance for agencies juggling large site portfolios. Our team has been testing, reading, and implementing across all three fronts. Here is what matters and what we are doing about it for our clients right now.

Key Takeaways

• WordPress 7.1 Release Candidate 1 is now available for testing — production installs should wait, but staging environments need it today.
• AI crawlers accounted for over 53% of all web traffic in 2025, and the problem is accelerating into 2026.
• robots.txt alone does not reliably control AI bot access; layered defences including bot protection and llms.txt are now essential.
• Organising large WordPress portfolios inside hosting dashboards saves real hours for agencies managing dozens of client sites.
• Hosting stability remains a baseline requirement — even minor downtime compounds when AI bots hammer servers around the clock.

WordPress 7.1 RC1: What Our Team Is Testing Right Now

The WordPress core team has shipped WordPress 7.1 Release Candidate 1. This is not a production-ready release. We have already spun it up on internal staging environments to check theme and plugin compatibility across our client base. Any agency still running automated updates without a staging layer is asking for trouble.

Key areas we are watching:

• Block editor refinements and how they interact with custom theme templates.
• REST API changes that could affect headless or decoupled builds.
• Performance tweaks — every millisecond matters for Core Web Vitals.

We recommend every WordPress site owner sets up a staging clone and runs RC1 against their full plugin stack before the stable release drops.

AI Crawlers Now Generate More Traffic Than Humans

Bots crossed the 53% threshold of all web traffic in 2025. By mid-2026, that figure is climbing further, as detailed in Kinsta’s breakdown of why AI crawlers are breaking the established rules of web crawling. For our clients, this means server resources are being consumed at scale by machines that never convert, never buy, and never fill in a contact form.

We are actively auditing server logs to identify which AI bots hit our clients’ sites hardest. The immediate action is rate-limiting aggressive crawlers and blocking known bad actors at the edge, before they reach WordPress at all.

robots.txt vs. llms.txt vs. Bot Protection: Which Actually Works?

A robots.txt directive is a polite request. Many AI crawlers ignore it entirely. Our team has been reviewing the practical differences between these control mechanisms, guided by a detailed comparison of llms.txt, robots.txt, and active bot protection for WordPress. The short answer: you need all three working together.

• robots.txt — still necessary, still ignored by some bots.
• llms.txt — a newer standard that tells large language models what content is available for training. Worth implementing now.
• Server-level bot protection — the only layer that can actually enforce rules. Cloudflare, Kinsta, and similar platforms offer this at the edge.

We are rolling llms.txt files into our standard site deployment checklist alongside robots.txt and sitemap configuration.

Managing Dozens of WordPress Sites Without Losing Your Mind

Agencies running 20, 50, or 100+ WordPress sites need structure. Labelling, tagging, and grouping sites inside a hosting dashboard is not glamorous work, but it prevents costly mistakes. We have adopted several of the organisational strategies outlined in Kinsta’s guide to organising multiple WordPress sites in MyKinsta — consistent naming conventions, environment labels, and team-based access controls.

Paired with reliable hosting infrastructure — something real users continue to validate through positive hosting reviews — this operational discipline keeps client sites stable and our team efficient.

The rest of 2026 will demand tighter bot management, faster adoption of WordPress core updates, and sharper operational workflows. We are building all three into every client engagement starting now.

Frequently Asked Questions

What is llms.txt and how does it differ from robots.txt for WordPress sites?

llms.txt is a file that tells large language models which content on your site is available for AI training, while robots.txt instructs traditional search engine crawlers. For proper AI access control in 2026, you need both files deployed alongside server-level bot protection.

How do web designers prepare client sites for WordPress 7.1?

Set up a staging environment and install WordPress 7.1 RC1 to test every active theme and plugin for compatibility issues. Never run release candidates on live production sites — catch breaking changes before the stable version ships.

Why are AI crawlers using so much of my WordPress site’s server resources?

AI bots now account for over half of all web traffic and many ignore robots.txt directives, hitting your server repeatedly without generating any revenue. Rate-limiting these crawlers at the edge and using active bot protection is the most effective way to reclaim server capacity.

How do agencies manage a large number of WordPress sites efficiently?

Consistent naming conventions, environment labels, and role-based access controls inside your hosting dashboard prevent costly errors at scale. Pairing this structure with automated update and backup workflows saves significant time each month.