The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s
Strip UTF-8 byte order mark (BOM) from a string
Check if a buffer contains valid UTF-8
A triple-linked lists based DOM implementation
This repository contains a list of of HTTP user-agents used by robots, crawlers, and spiders as in single JSON file.
Create and parse HTTP Content-Type header
A library to recursively retrieve and serialize Notion pages with customization for machine learning applications.
This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.
Strip UTF-8 byte order mark (BOM) from a Uint8Array
Strip UTF-8 byte order mark (BOM) from a stream
Analyzes license information for multiple node.js modules (package.json files) as part of your software project.
Parse HTTP Content-Type header according to RFC 7231
TypeScript definitions for utf-8-validate
zlib port to javascript - fast, modularized, with browser support
Text Decoder
UTF-8 only polyfill for the Encoding Living Standard's API.
Parses, serializes, and manipulates MIME types, according to the WHATWG MIME Sniffing Standard
Used to run a web crawler that checks for errors on specified pages.
Optimise inline SVG with PostCSS.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Converts text to and from UTF-7 (RFC 2152 and IMAP)
Inspecting Node.js's Network with Chrome DevTools
Text recoding in JavaScript for fun and profit!
Device detection module for Nuxt