Utilities to use for Spider Web Crawler.
Shared utility functions and types for the spider-lib ecosystem
hyper utilities
Proc-macros for the spider-lib web scraping framework.
Downloader component for the spider-lib web scraping framework
A Rust-based web scraping framework inspired by Scrapy (Python).
Core functionality for the spider-lib web scraping framework.
Pipeline implementations for the spider-lib web scraping framework.
Middleware implementations for the spider-lib web scraping framework.
HTML processing utilities for spider_agent — cleaning, content analysis, and diffing.
Transformation utils to use for spider
A web crawler and scraper, building blocks for data curation workloads.