BETAmodules.com is in beta — open to partnerships & joint ventures.Build with us

cross-ecosystem search · live

Results for crawling

Found in 6 of 7 ecosystemsnpm 1–24 of 1,329 · 529 matches across other registries

npm1329 PyPI1 crates.io385 RubyGems12 Maven11 NuGet120

How we search: free-text on npm, crates.io, RubyGems, NuGet and Maven. PyPI and Go do exact-name lookup only. Tip: click an ecosystem chip below to filter; click Show all ecosystems to come back.

Sort

Auto-load on scroll

npm matches

Showing 24 of 1,329 · JavaScript

See all npm →

metro-file-mapv0.84.4

npm

[Experimental] - 🚇 File crawling, watching and mapping for Metro

Maintained. Maintained, actively maintained.

firewormv0.7.2

npm

A crawling file watcher.

Abandoned. Last published 4 years ago.

crawlingv1.0.1

npm

A simple crawler made in JavaScript for Node.

Abandoned. Last published over a year ago.

storycrawlerv5.0.1

npm

Utilities to build Storybook crawling tools with Puppeteer

Abandoned. Last published over a year ago.

@nuxtjs/robotsv6.0.9

npm

Tame the robots crawling and indexing your Nuxt site with ease.

Maintained. Maintained, actively maintained.

limiterv3.0.0

npm

A generic rate limiter for the web and node.js. Useful for API clients, web crawling, or other tasks that need to be throttled

Aging — last published over a year ago — check before adopting.

@prantlf/storycrawlerv8.1.1

npm

Utilities to build Storybook crawling tools with Puppeteer

Aging — last published 6 months ago — check before adopting.

thredds-catalog-crawlerv0.0.7

npm

A module for crawling thredds catalogs

Abandoned. Last published 2 years ago.

find-nearest-package-jsonv2.0.1

npm

Find the nearest package.json by recursively crawling parent directories

Abandoned. Last published 8 years ago.

fetch-filecache-for-crawlingv5.1.1

npm

Implementation of a `fetch` that extends the implementation from `node-fetch` to add an HTTP cache using a local cache folder for crawling purpose.

Abandoned. Last published 2 years ago.

crawlbasev1.0.2

npm

Dependency free module for scraping and crawling websites using [Crawlbase](https://crawlbase.com) API

Abandoned. Last published over a year ago.

@crawlee/linkedomv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

crawleev3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/browserv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/httpv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/corev3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/basicv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/cheeriov3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/cliv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/playwrightv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/puppeteerv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

@crawlee/jsdomv3.16.0

npm

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Maintained. Maintained, actively maintained.

limiter-es6-compatv2.1.2

npm

A generic rate limiter for the web and node.js. Useful for API clients, web crawling, or other tasks that need to be throttled