<p align="center"> <a href="" rel="noopener"> <img width=200px height=200px src="https://i.imgur.com/6wj0hh6.jpg" alt="Project logo"></a> </p>
The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s
Async and sync crawler for json object
Agnostic tree traversal library.
A specification compliant robots.txt parser with wildcard (*) matching support.
Official Firecrawl nodes for n8n - scrape, crawl, map, search, and extract data from websites. Supports AI Agent tool usage.
A community node for n8n to integrate Tavily API for web search and content extraction.
JavaScript SDK for Firecrawl API
Schema-driven relayfile adapter generator and runtime
Mdream Crawl generates comprehensive llms.txt artifacts from a single URL, using mdream to convert HTML to Markdown.
CLI tool for Genspark Tool API - search, crawl, analyze images, generate media
JavaScript SDK for Firecrawl API
A React component to crop images/videos with easy interactions
Simple JS stack with auto run for node and browsers
Stylish, intuitive and user-friendly prompt system. Fast and lightweight enough for small projects, powerful and extensible enough for the most advanced use cases.
Official JavaScript library for Tavily.
[](https://www.npmjs.com/package/recrawl) [](https://github.com/aleclarson/recrawl/actions/workflows/release.yml) [![codeco
plugin/hooks interface
FireCrawl nodes for n8n
Extra Cypress query commands for v12+
Fast, token-efficient web content extraction - fetch web pages and convert to clean Markdown
Cypress command for flexible test data setup
SEO analytics and indexing diagnostics module with Google Search Console integration and AI-ready reports
An iteration of the Node.js core streams with a series of improvements
Dead simple yet powerful Ruby crawler for easy parallel crawling with support for an anonymity.
SuperCrawler allows you to easily crawl full web sites or web pages (extracting internal links and assets) in few seconds.
Access book data on ARBookFinder
Simple API to transform from simple HTML to JSON to entire website to JSON.
Easy to use extension of Net::HTTP to let you obey robots.txt while crawling/scrapping/mining
Easy way to enable AdSense crawler to login and see private or custom pages in your rails application. Basically one custom login filter. Gem enables you to easily slightly increase revenues from Google AdSense/AdWords. It makes it easy to enable crawling on private pages and so get better targeted ads even in pages behind login screen.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.