CrawlSEO
Self-hosted, MIT-licensed SEO monitoring that joins Google Search Console data, a site crawler and Core Web Vitals in one dashboard, with an MCP server so AI agents can query it
CrawlSEO is a self-hosted, MIT-licensed SEO dashboard that pulls your own Google Search Console data, crawls your site, and measures Core Web Vitals in one place, then exposes all of it to AI agents through a Model Context Protocol server. It is aimed at founders rather than SEO specialists: the free half is everything Google already reports about you, and the parts that need outside data — keyword volume and backlinks — are bring-your-own-key against a paid provider. It is an early project, so read it as a monitoring layer that tells you what changed, not as a tool that decides what to do about it.
What CrawlSEO is
CrawlSEO (github.com/crawlseo/crawlseo) is a self-hosted SEO monitoring dashboard built with Next.js, PostgreSQL and Prisma, published under the MIT licence by Brandson Digital. You connect it to your own Google Search Console property over OAuth, point its crawler at your site, and it holds the two views side by side: what Google reports about your pages, and what your pages actually contain.
- Search Console analytics keywords, pages, clicks, impressions and position, with a 28-day comparison and delta indicators.
- Site crawler up to 2,000 pages per crawl, with a health score, 16 issue types and per-page remediation notes.
- Core Web Vitals LCP, CLS, INP and TTFB pulled from PageSpeed Insights, mobile and desktop side by side. Our own definition of those metrics is at Core Web Vitals.
- Opportunity detection striking-distance keywords, low-CTR pages, content decay, and keyword cannibalization.
- Alerts traffic drops, position changes, new 404s and vitals degradation, delivered by email, Slack, Telegram or webhook.
- An MCP server ten tools that let Claude Code, Claude Desktop or Cursor query the same data over Model Context Protocol.
Deployment is Docker Compose against a prebuilt multi-architecture image, or a manual Node and PostgreSQL install. The project is young: it was created in April 2026, four people have contributed, and its only tagged release is v0.1.0. Read the star count as interest rather than maturity, and expect to be an early user rather than a customer.
What it reads for free, and what needs a paid key
The split matters more than the feature list, because it decides what the tool can tell you on day one with no budget.
| Data | Where it comes from | Cost |
|---|---|---|
| Clicks, impressions, position, queries | Your own Search Console property, over OAuth | Free |
| Crawl issues, titles, status codes | CrawlSEO's own crawler, on your server | Free |
| Core Web Vitals | Google PageSpeed Insights | Free |
| Keyword volume, difficulty, CPC | DataForSEO, bring your own key | Paid; free fallback is Google Autocomplete |
| Backlinks, referring domains, anchor text | DataForSEO, bring your own key | Paid |
That last row is not a corner the maintainer cut. We went looking for the same thing while building our own Search Console reporting and confirmed it: the Search Console API exposes no links endpoint at all, and the Links report exists only in the web UI. So any tool built on GSC data has to reach outside GSC for backlinks or leave the panel empty. Worth knowing before you judge a free SEO tool for gating the one number everybody wants.
Where it fits a founder's growth stack
Monitoring is the cheap half of SEO. A dashboard can tell you a page slipped from position 8 to 14, that a template change put a canonical tag on the wrong URL, or that a route started returning 404s after a deploy. It cannot write the page that ranks, earn the link, or decide which of forty flagged issues is worth an afternoon. That gap is the actual work, and it is where AgentCeres — the AI Growth Officer at agentceres.com — sits: a roster of specialists that drafts the content, the outreach and the ad copy, with a human approving anything that goes out.
The class of problem a monitor catches is real, and we hit one. Our own marketing site prerenders thousands of pages across eight locales, and a canonical problem surfaced in Search Console as pages Google crawled and then declined to index under our URL. No test caught it, because nothing was broken: the pages rendered fine. If you are shipping a large or programmatically generated site, something has to be watching Search Console on your behalf, and self-hosting that something against your own database is a reasonable answer. If the site is small, start with SEO for a brand-new website, and read crawl budget before assuming crawling is your bottleneck.
FAQ
- Is CrawlSEO a replacement for Ahrefs or Semrush?
- Not for the reason most people buy those. Ahrefs and Semrush sell you their crawl of the whole web: competitor keywords, a backlink index, rank data for domains you do not own. CrawlSEO reports on the site you control, using data Google already gives you, and rents third-party data only if you supply a key. If your question is "what is happening to my site", it is a fair swap. If your question is "what is my competitor ranking for", it is not.
- What does the MCP server actually let an agent do?
- It exposes ten tools to any Model Context Protocol client such as Claude Code, Claude Desktop or Cursor: listing sites, pulling keywords, pages and traffic, starting a crawl and reading its issues, and fetching vitals and opportunities. In practice you can ask an agent about your Search Console data in the same session where you are editing the page, instead of exporting a CSV first. The data stays on your own instance.
- Is it production-ready?
- Treat it as early. It is MIT-licensed, ships multi-architecture Docker images, runs migrations on container start, and has had real security work done on it — the crawler was hardened against SSRF and compose database credentials moved to environment variables in July 2026. But it is a young project with four contributors and a single v0.1.0 tag. Self-host it behind your own auth, pin an image tag instead of tracking latest, and keep an export of anything you would miss.
You built it. Now grow it.
AgentCeres is a managed AI marketing team — specialists draft the SEO, social, and outreach that fill your links, you approve what ships. 14-day free trial, from $39/month.