An express.js middleware for handling noisy robots.txt
A specification compliant robots.txt parser with wildcard (*) matching support.
Tame the robots crawling and indexing your Nuxt site with ease.
Gatsby plugin that automatically creates robots.txt for your site
Generate a robots.txt for Astro
Express middleware to serve and generate robots.txt
A simple, accessible, tagging component ready to drop into your React projects.
Hardened Fetch is a tiny wrapper for `global.fetch` adding request timeouts, throttling, rate limit awareness, retries, and descriptive errors. It makes working with APIs without SDKs and building robust web scrapers easier.
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
A convenient wrapper for node-mocks-http which makes testing Express or Koa controllers and middleware easy.
Validate urls against robots.txt rules.
Get the name of a dependency from a module file path.
Streaming parser for robots.txt files
parse arguments with recursive contexts
A reliable robots.txt generator for Astro projects, offering zero-config setup and Verified Bots support.
parse arguments with recursive contexts
TypeScript definitions for react-tag-autocomplete
Retry a request.
Vite plugin for generating robots.txt for production and development builds
EasyZoom is an elegant, highly optimised jQuery image zoom and panning plugin based on the original work by Alen Grakalic.
Error class for Octokit request errors
Like request, but smaller.
Send parameterized requests to GitHub's APIs with sensible defaults in browsers and Node
Core Promise support implementation for the simplified HTTP request client 'request'.
No description provided.
No description provided.
No description provided.
No description provided.