Screaming Frog

Just released @screamingfrog SEO Spider v.23: screamingfrog.co.uk/blog/seo-spide… Includes - 💡Lighthouse & PSI Updated to Insight Audits ⚙️Crawl Retention Config 🧑‍🤝‍🧑Semantic Similarity Embedding Rules 📊Display All Links in Visualisations 🍇Links in Content Cluster Diagram & more!

Optimizing your website for generative AI features on Google Search - developers.google.com/search/docs/fu… from @googlesearchc

For years, SEOs have found internal links using anchor text matching. However, you can use @screamingfrog to find links based on SEMANTIC SIMILARITY:

New post from yours truly. The Case for Cloaking for Large Language Models. ipullrank.com/cloaking-for-l… *Cue outrage.*

@zigojacko @Shopify @ahrefs @ShopifySupport Yeah, sorry don't know their product to know if you can create custom headers. 👍

@screamingfrog @Shopify @ahrefs @ShopifySupport @screamingfrog I have no idea how to create a signature for @ahrefs 🤷- I know this is possible with other crawlers such as yours though...

Since @Shopify changed the way they handle bots this past week, they are now blocking @ahrefs site audit mobile crawler by default and it is just returning a 403 response (desktop UA still can crawl though) - anyone else noticed this or know how to resolve? cc @ShopifySupport

AI Citation drift: How stable are sources in AI search results? - sistrix.com/blog/ai-citati… from @sistrix

There's a status code that doesn't exist in the official HTTP spec, your SEO tools don't surface it, and it may be quietly making your content invisible to ChatGPT and Perplexity. Not underperforming. Invisible. We've been seeing a clear pattern in our AI Search Pilot work and when we dug into the data with the team at Profound across 700K pages, the numbers were hard to ignore. The fix isn't complicated. But you have to know what you're looking for first. AI Search has a prerequisite layer that most SEOs have never thought about. Visibility isn't possible without eligibility. The full breakdown, what's happening, why it's happening, and what to do about it, is in my new post 👇🏾 ipullrank.com/page-speed-imp…

Google to no longer support FAQ rich results searchengineland.com/google-to-no-l…

Technical SEOs, your final boss fight is here. Introducing Keyword Hunt, a technical SEO puzzle by @snippet_digital 20 keywords are cloaked across the site using real technical SEO, AI SEO, and edge engineering. The first 10 to crack get permanent Elite status in the Hall of Fame. Happy hunting: keyword-hunt.com

@HenningSeo You can use it on a couple of devices without any issue. Pop through the details to us via support@screamingfrog.co.uk and our team can take a look! Cheers

Hi @screamingfrog I have a question about the licence. I can no longer use Screaming Frog on my laptop with my licence. I have a PC in the office and a laptop for when I’m travelling. It was still working until recently.

@DanielTexasLira Ah damn! We give away swag at events, plus we will do a giveaway soon - so keep an eye out!!

@screamingfrog any chance that SF is giving away free merch, like t-shirts? My old SF t-shirt shrank. 👉🏼👈🏼😅

@alexisamand @amenfrance Few things worth trialing - #why-do-i-get-a-429-too-many-requests-error-response" target="_blank" rel="nofollow noopener">screamingfrog.co.uk/seo-spider/faq…

Je ne comprends pas trop ce qu'il se passe. Il semble que mon hébergement mutualisé chez @amenfrance bloque tous les outis SEO (type @screamingfrog, Xenu ou #LibreCrawl). J'ai uniquement des erreurs 429.

@cooperpatton Sorry, Cooper! We did have a bug here, but believe this has been fixed in our next version.

@screamingfrog what good is an X button if it doesn't terminate the process? looks pretty though. there goes my morning.

@TaylorDanRW Well done for finding, and completing! We actually announced the winner earlier I am afraid. Amazingly someone did it at the conference, instead of going to the talks 😂 - linkedin.com/posts/screamin…

@screamingfrog just realised I can't DM you to send you the code... SF-BE....

💡 A 3 Layer Framework to Measure AI Presence, Readiness and Business Impact: Redefining Metrics for the AI Search Era - The guide expanding the presentation I gave at @brightonseo today , going through how to asssess AI search success via 3 assessment driven AI metric layers: 1. Presence: Are you actually appearing, and how? 2. Readiness: Are you structurally prepared to be surfaced? 3. Business Impact: Are visibility and readiness translating into value? Read it here: aleydasolis.com/en/ai-search/a…

The @screamingfrog #brightonSEO beer mats

Just released @screamingfrog Log File Analyser v.7 : screamingfrog.co.uk/blog/log-file-… This update includes - 🤖Bot Verification for AI Bots 🧑‍🤝‍🧑User Agent Grouping ⚙️Customisable Bot Verification 🕵️Simpler Discovery of Unknown User Agents & lots more!

The brightonSEO @screamingfrog Crawling Clinic April/May '26 - screamingfrog.co.uk/blog/the-brigh…

Technical SEO Checklist: The Complete Guide For 2026 - debugbear.com/blog/technical… from @mjcachon

One to keep an eye on - DailyMail migrates to DailyMail dot com - dmgmedia.co.uk/news/daily-mai…