ICEcrawl

FastAPI, Redis, Docker, Claude, MCPrepo

Give it a URL for clean markdown of one page, or turn it loose to crawl an entire domain.
FastAPI in front of a Redis job queue and a headless browser, with SSRF guards and robots-aware rate limiting.
Claude does schema extraction when structured data is wanted instead of text.
Ships an MCP server, so an agent can call it as a tool directly.
115 tests, one Docker Compose file, no third-party API to pay for.