Screaming Frog SEO Spider

In short: Screaming Frog SEO Spider is crawling, site auditing and page speed software, with freemium pricing and plans from £199/year.

Desktop website crawler for technical SEO audits on Windows, macOS and Linux

Screaming Frog SEO Spider is a desktop website crawler made by Screaming Frog, a UK search marketing agency founded in 2010 and based in Henley-on-Thames. It is used by SEO specialists, agencies and in-house teams to audit websites of any size from their own computer.

The crawler finds broken links, server errors, redirects, redirect chains and loops, and audits titles, meta descriptions, headings, canonicals, directives, hreflang, pagination and images. It detects exact, near and semantically similar duplicate content, generates and checks XML sitemaps and visualises site architecture. An integrated Chromium engine renders JavaScript sites built with frameworks such as React, Angular and Vue, and it can crawl as Googlebot, an AI crawler or a custom user agent. Custom extraction and custom AI prompts using OpenAI, Gemini, Anthropic or Ollama run during a crawl.

Integrations pull in Google Analytics, Google Search Console and PageSpeed Insights data, plus link metrics from Majestic, Ahrefs and Moz, and crawl reports can feed Looker Studio. Scheduled crawls, crawl comparison and command-line automation support regular monitoring. The free version crawls up to 500 URLs, and an annual licence removes the limit.