A specification compliant robots.txt parser with wildcard (*) matching support.
Tame the robots crawling and indexing your Nuxt site with ease.
Gatsby plugin that automatically creates robots.txt for your site
Generate a robots.txt for Astro
Express middleware to serve and generate robots.txt
A simple, accessible, tagging component ready to drop into your React projects.
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
Validate urls against robots.txt rules.
Get the name of a dependency from a module file path.
Streaming parser for robots.txt files
parse arguments with recursive contexts
parse arguments with recursive contexts
A reliable robots.txt generator for Astro projects, offering zero-config setup and Verified Bots support.
Vite plugin for generating robots.txt for production and development builds
TypeScript definitions for react-tag-autocomplete
EasyZoom is an elegant, highly optimised jQuery image zoom and panning plugin based on the original work by Alen Grakalic.
Express middleware for blocking ai-robots
HTML assertions for Chai.
Static-SPA site framework for Plumile React documentation and marketing sites
Parse robot directives within HTML meta and/or HTTP headers.
Hapi plugin to serve up robots.txt
Hardened Fetch is a tiny wrapper for `global.fetch` adding request timeouts, throttling, rate limit awareness, retries, and descriptive errors. It makes working with APIs without SDKs and building robust web scrapers easier.
Automatic folder-based routing with colocation for React Router v7+
A simple rate limited queue for asynchronous operations. Restricts the number of operations executed per time interval.