GcrawlAI

GcrawlAI is an open-source web scraping API, built by Gramosoft, that converts websites into structured, LLM-ready output for RAG pipelines and AI applications. It exposes scrape, crawl, map, screenshot, and search endpoints, with JavaScript rendering, anti-bot stealth mode, and rotating residential proxies included in every request rather than sold as add-ons. Because the full engine is published on GitHub under the MIT license, teams can self-host the entire scraping stack instead of depending on a hosted API, or use GcrawlAI's managed cloud version if they'd rather not run infrastructure. Pricing is per-request rather than credit-based: every account gets 50 free requests per month with no card required, and paid usage starts at roughly $19/month (₹1,587) for 3,000 requests. It also supports distributed crawling and real-time progress updates over WebSocket for long-running crawl jobs.