A library to test if a url(request) is crawled, usually used in a web crawler. Compatible with `request` and `node-crawler`
A redis repo adapter for seenreq
A mongodb repo adapter for seenreq
seenreq wrapper for middleware of floodesh
Floodesh is a distributed web spider/crawler written with Nodejs.
No description provided.