The free SEO crawler that runs on your machine — no limits, no signup, no cloud.
A fast, concurrent site crawler for technical SEO audits. Crawl unlimited URLs, detect every common on-page issue, and export bulk reports — without sending a single byte to a third party. Built by the team behind Melbourne SEO Consultant.
$ open-seo-crawler --url https://example.com --workers 5 ✓ Detected CMS: WordPress · Yoast SEO ✓ Applied recommended settings (exclude: /wp-admin/*, /?p=*) → Crawling at 5 workers, 0.4s per-host delay… PAGES 1,247 WARNINGS 38 DURATION 2m 14s ERRORS 12 → Bulk reports ready: • Duplicate Titles (7) • Duplicate Metas (14) • Redirect Chains (3) • Missing H1 (2) • Hreflang issues (1) $ # Open http://localhost:5002 for the full report
Every check you'd run manually, automated.
Crawl your site, surface the issues that matter, and export the reports your team actually needs.
Concurrent crawling
1–20 workers, per-host adaptive politeness, smart back-off on 429 / 5xx. Crawls a typical 1,500-page site in under 3 minutes.
CMS-aware presets
Auto-detects 10+ platforms (WordPress, Shopify, Webflow, Wix, Squarespace, Kajabi, Ghost, Drupal, HubSpot, Joomla) and applies sensible exclude rules.
Severity-grouped issues
Every issue tagged Error / Warning / Info, grouped by type with worst-affected pages first. Fix the highest-impact problems before the cosmetic ones.
Bulk reports
All Titles, All Metas, All H1s, All Canonicals — plus Duplicate Titles, Duplicate Metas, Duplicate H1s, Duplicate Bodies, Redirect Chains, Response Codes, Deep Pages and Hreflang. Every report exports to XLSX.
Sitemap analysis
Cross-checks crawl against sitemap.xml. Flags orphans, sitemap-only URLs, non-200 responses, and redirects living in your sitemap.
Near-duplicate content
Shingle-based Jaccard similarity flags pages 90%+ identical (tunable). Catches thin variants and cannibalisation at scale.
Redirect chains
Multi-hop redirects flagged with the full hop list. Trailing-slash / www / HTTPS normalisations classified separately so they don't pollute the real chain report.
Hreflang validation
Every hreflang annotation extracted, validated, and cross-page consistency checked. Missing return-tags surface in one click.
Auto-updates
One-click update from the topbar. Background daily auto-updater on Linux / macOS / Windows pulls the latest commits, restarts, and rolls back automatically if anything breaks.
Dark mode + saved crawls
Per-user dark/light toggle (persists). Every crawl auto-saved — reopen any historical run from the Load Saved modal. LAN-wide name resolution so the team knows who ran what.
Privacy by default
Runs entirely on your machine. No accounts, no cloud, no telemetry. The only outbound requests are to the site you're crawling.
JavaScript rendering
Optional headless rendering for SPA sites — adds support for React, Vue, Svelte, and other client-rendered frameworks. Install separately when you need it.
What you get vs. typical paid tools.
Most desktop SEO crawlers cap free use at 500 URLs and gate everything else behind annual licences. Most cloud audit tools meter every URL and lock the data in their dashboards. Here's how an open, self-hosted tool stacks up.
| Capability | Open SEO Crawler | Typical free desktop crawler | Typical paid desktop crawler | Typical cloud audit tool |
|---|---|---|---|---|
| URL limit | Unlimited | 500 | Unlimited | Metered per credit |
| Price | Free, MIT | Free | $200+ per year | $100+ per month |
| Self-hosted | Yes | Yes | Yes | No (cloud-only) |
| Data never leaves your machine | Yes | Yes | Yes | No |
| Source code available | Yes (MIT) | No | No | No |
| Auto-update (daily + on-demand) | Yes | Manual | Manual | Managed |
| CMS-aware crawl presets | 10+ platforms | No | Partial | Partial |
| JavaScript rendering | Optional | Paid only | Yes | Yes |
| Bulk XLSX exports | Every report | Limited | Yes | Yes |
| One-line install | Linux / macOS / Windows | Installer download | Installer download | n/a |
Comparison is descriptive only. We don't claim equivalence with any specific commercial product; capabilities of named categories vary by vendor and tier.
Who runs it, and why.
Audit every client without per-seat licensing
Install once on a shared box, point every account manager at the same URL. No per-user fees, no URL caps as your client roster grows. Pairs well with our link-building services for ongoing campaigns.
Pre-launch + post-migration verification
Crawl staging before the cutover, fix every Error, then re-crawl after launch and diff the redirect-chain and hreflang reports. Catch regressions before users do — or hand the audit to our audit team for a second opinion.
Faceted-navigation cleanup at scale
Glob exclude ?variant=* and */cart/*, run a real crawl of just the indexable surface, and export the duplicate-title report your dev team can actually action.
Bill-ready technical audits
Run a full crawl, export bulk reports to XLSX, hand the spreadsheets to the client's dev team. No "you'll need to upgrade your plan" upsell mid-engagement.
On-page audits before local SEO work
Get the technical baseline right before investing in citations, reviews, and local content. A clean crawl rules out the silent issues that cap local-pack performance.
Structured-data + content depth for AI search
Surface every page missing schema, thin content, or weak headings — the three things AI engines (ChatGPT, Perplexity, Google AI Overviews) need to cite you confidently.
One line to install, auto-starts on boot.
Each installer registers the crawler as a background service plus a daily auto-updater. Your browser opens to http://localhost:5002 when it's done.
curl -fsSL https://raw.githubusercontent.com/puneetindersingh/open-seo-crawler/master/install.sh \ -o install.sh && chmod +x install.sh && ./install.sh
curl -fsSL https://raw.githubusercontent.com/puneetindersingh/open-seo-crawler/master/install-macos.sh \ -o install-macos.sh && chmod +x install-macos.sh && ./install-macos.sh
iwr https://raw.githubusercontent.com/puneetindersingh/open-seo-crawler/master/install-windows.ps1 -OutFile install.ps1; powershell -ExecutionPolicy Bypass -File .\install.ps1
Prefer manual? git clone + venv + pip install works on any platform with Python 3.10+.
Need done-for-you SEO?
The crawler handles the audit. We handle everything else.
Audit your site the way it should be: free and on your terms.
MIT licensed. Built and maintained in the open. Every commit visible, every report yours to keep.
No signup. No credit card. No "starter tier".
