mrkaluzny homepage
Get in touch
mrkaluzny homepage
Tips & Tricks

How to Fix Ahrefs Site Audit "Broken Page" 404 Errors from Cloudflare

Jun 19, 2025

Running an Ahrefs Site Audit only to find it flooded with "Page has links to broken page" errors? If these 404 errors all point to URLs containing /cdn-cgi/l/email-protection, you've found the culprit: a common conflict with Cloudflare's Email Address Obfuscation feature.

This issue is specific to how the Ahrefs Site Audit crawler interprets the protected email links that Cloudflare generates. The crawler flags these dynamically generated links as internal broken pages, skewing your audit results and making it difficult to identify genuine issues on your site.

This guide will show you how to configure Cloudflare to specifically allow the Ahrefs Site Audit to run without error. We will start with the most secure method, which involves whitelisting Ahrefs' IP addresses.

Why Editing robots.txt Won't Fix Your Site Audit

You might be tempted to disallow the /cdn-cgi/ directory in your robots.txt file. However, this will not solve the problem in your Ahrefs Site Audit. The crawler will still discover the links on your pages and, because it cannot crawl them, will continue to report them as broken. The only effective solution is to create a specific rule within your Cloudflare settings.

Method 1: The Most Secure Fix (IP Whitelist)

This is the preferred and most secure method for resolving the Ahrefs 404 error. It works by verifying that the crawler is genuinely from Ahrefs' network before disabling email obfuscation.

Note: This method requires access to create and manage IP Lists within your Cloudflare account (Manage Account > Configurations > Lists). If you do not have this access due to your plan level or user permissions, please proceed to Method 2.

Step 1: Create an IP List in Cloudflare

First, you need to tell Cloudflare which IP addresses belong to Ahrefs.

  1. In your Cloudflare dashboard, go to Manage Account > Configurations.
  2. Select the Lists tab and click Create list.
  3. Name it something memorable like ahrefs_crawlers.
  4. Choose IP as the content type.
  5. Add the official Ahrefs IPs. The complete, up-to-date list is available on the Ahrefs IP ranges page.
  6. Save the list.

Handy list to add it in one go

202.8.40.0/2219
15.235.27.0/2419
15.235.96.0/24
15.235.98.0/24
51.89.129.0/24
51.161.37.0/24
51.161.65.0/24
51.195.183.0/24
51.195.215.0/24
51.195.244.0/24
51.222.95.0/24
51.222.168.0/24
54.36.148.0/24
54.36.149.0/24
54.38.147.0/24
54.39.0.0/24
54.39.6.0/24
54.39.89.0/24
54.39.136.0/24
54.39.210.0/24
142.44.220.0/24
142.44.225.0/24
142.44.228.0/24
142.44.233.0/24
148.113.128.0/24
148.113.130.0/24
167.114.139.0/24
168.100.149.0/24
198.244.168.0/24
198.244.183.0/24
198.244.226.0/24
198.244.240.0/24
198.244.242.0/24
51.222.253.0/26
5.39.1.224/27
5.39.109.160/27
37.59.204.128/27
51.68.247.192/27
51.75.236.128/27
54.37.118.64/27
92.222.104.192/27
92.222.108.96/27
94.23.188.192/27
176.31.139.0/27
198.244.186.193
198.244.186.194
198.244.186.195
198.244.186.196
198.244.186.197
198.244.186.198
198.244.186.199
198.244.186.200
198.244.186.201
198.244.186.202
202.94.84.110
202.94.84.111
202.94.84.112
202.94.84.113

Step 2: Create the Configuration Rule

Now, create the rule that uses this list.

  1. Navigate back to your domain, then go to Rules click Create rule and select Configuration Rule from the dropdown.
  2. Name the rule (e.g., Ahrefs IP Whitelist for Site Audit).
  3. For the condition "If incoming requests match...", select the IP Source Address field, use the is in list operator, and choose your Ahrefs IPs list from the value dropdown.
  4. Under "Then...", set Email Obfuscation to Off.
  5. Click Deploy.

Method 2: A Reliable Alternative (User-Agent Rule)

If you cannot create IP lists (lack of access to that part of the account), this method is a perfectly functional alternative. It uses the crawler's user-agent to identify it. While effective, it's considered less secure, which we will explain below.

Step-by-Step Guide:

  1. Log into your Cloudflare dashboard and select your domain.

  2. From the left-hand menu, navigate toRules click Create rule and select Configuration Rule from the dropdown.

  3. Name your rule something clear, like Ahrefs Site Audit - Fix Email 404s.

  4. In the "If incoming requests match..." section, select Edit expression.

  5. Paste the following expression. It specifically targets the known user agents for the Ahrefs Site Audit and other Ahrefs crawlers.

    (http.user_agent eq "Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +[http://ahrefs.com/robot/site-audit](http://ahrefs.com/robot/site-audit))") or (http.user_agent eq "Mozilla/5.0 (Linux; Android 13) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.5359.128 Mobile Safari/537.36 (compatible; AhrefsSiteAudit/6.1; +[http://ahrefs.com/robot/site-audit](http://ahrefs.com/robot/site-audit))") or (http.user_agent eq "Mozilla/5.0 (compatible; AhrefsBot/7.0; +[http://ahrefs.com/robot/](http://ahrefs.com/robot/))")

    (For the latest user agents, you can always check the official Ahrefs Robot page.)

  6. In the "Then..." section below, configure the following setting:

    • Setting: Choose Email Obfuscation.
    • Value: Set the toggle to Off.

  7. Click Deploy to save and activate the rule.

Why This is an Alternative Solution

The main reason the user-agent approach is not the top recommendation is security. A user-agent is a piece of text sent by a browser or bot that can be easily copied and faked (spoofed) by malicious actors. A bad bot could potentially mimic the Ahrefs user-agent to bypass your email protection and harvest email addresses. The IP Whitelist method is more secure because it verifies the request is coming from Ahrefs' actual servers, which is significantly harder to fake.

Get Accurate SEO Audits

By implementing one of these methods, you will resolve the "Page has links to broken page" 404 errors within your Ahrefs Site Audit. This ensures your audit data is accurate and actionable.

For the highest level of security, use the IP Whitelist (Method 1). If you lack the necessary permissions, the User-Agent Rule (Method 2) is a reliable alternative that will get the job done. In either case, your SEO audits will run smoothly, and your email addresses will remain protected from unwanted bots.