ScrapeGraph AI CLI tool
Scrape documentation frameworks to Mintlify docs
JavaScript SDK for Firecrawl API
Search from DuckDuckGo and use it's spice APIs.
JavaScript SDK for Firecrawl API
Promise queue with concurrency control
A Node.js scraper for humans.
Official Firecrawl nodes for n8n - scrape, crawl, map, search, and extract data from websites. Supports AI Agent tool usage.
The core scraping functionality of scrape-it.
Command-line interface for Firecrawl. Scrape, crawl, and extract data from any website directly from your terminal.
Scrape Window Metadata
Scrape From primbon.com
A lightning fast package to scrape YouTube search results. This was made for Discord Bots.
Scrape instagram posts from Username, Hashtag or Location pages. Download media and save them to a ZIP archive. Create JSON/CSV files with a post information. No login required
A slim module for scraping Facebook event data in milliseconds.
The library scraper for WhatsApp bot or Restfull API's
Fetch and normalize job descriptions across every major ATS (Greenhouse, Lever, Ashby, Workday, and more) — for your AI assistant, no copy-paste.
Scraper app with Fastify server and React UI
Compatible version of p-queue
A library to easily scrape metadata from an article on the web using Open Graph, JSON+LD, regular HTML metadata, and series of fallbacks.
extend an object
A type-safe marriage of `EventTarget` and `EventEmitter`.
Return an object representing the diffs between two objects. Supports jsonPatch protocol
The string_decoder module from Node core
The Firecrawl gem implements a lightweight interface to the Firecrawl.dev API. Firecrawl can take a URL, scrape the page contents and return the whole page or principal content as html, markdown, or structured data. In addition, Firecrawl can crawl an entire site returning the pages it encounters or just the map of the pages, which can be used for subsequent scraping.
This is just a small project that scrapes the classes from D&D from the site dndbeyond. It then lets you choose a class and gives a short scenario(again sraped from dndbeyond.com). You can confirm or choose again. After it gives you a random monster to fight scraped from http://www.chaoticshiny.com/monstergen.php win or lose you can fight another random monster.
Welcome to World's top movies. This gem scrapes IMDB and gives you a list of only the highest rated movies of all time. You can search by genre or just by rating. Also, you can save your favourite ones and leaves notes.
(***moved to the gem 'vaccine-spotter'***) This gem will notify you when COVID-19 vaccine appointments are available matching certain criteria (a list of zip codes, type of vaccine, etc). It currently pretty much just wraps the very beta API from the absolutely wonderful vaccinespotter.org, though I hope to add my own website scraping soon too so as to improve response times.
Easy to use DSL that helps scraping data from websites. Thanks to it, writing web crawlers would be very fast and intuitive. Traversing through html nodes and fetching all of the HTML attributes, would be possible. Just like in jQuery - you will find methods like parent, children, first, find, siblings etc. Furthermore, you are able to download images, web pages, and store all content in the database. Please visit my Github account for more details.
pikuri-core is the lean, audit-friendly foundation of the pikuri family: Pikuri::Agent (a thin wrapper around ruby_llm's chat loop) with its Configurator + Extension protocol, the strict Pikuri::Tool framework, a listener surface for rendering / budgets / sub-agents, and four bundled stateless tools (calculator, web search, web scrape, fetch). Extensions (skills, MCP, workspace, coding stack, named-agent personas) live in sibling gems so a privacy-conscious user can install just this core and audit a minimal dependency tree. For the convenience bundle that pulls in everything, see the +pikuri+ metagem.