robotstxt
v0.5.4RubyGems· RubyRobotstxt Parser allows you to the check the accessibility of URLs and get other data. Full support for the robots.txt RFC, wildcards and Sitemap: rules.
The verdict
Abandoned. Last published 16 years ago. No recent activity — look for a maintained alternative.
No recent activity — look for a maintained alternative.
Live from the RubyGems registry · derived rules, not AI
How it scores
MaintenanceAbandoned
PopularityNiche
SecurityClean
LicenseUnknown
DepsZero deps
Maintenance
Last published 16 years ago.
Popularity
22 downloads / week
Security
No known advisories for this version (OSV).
License
No license declared.
Dependencies
No runtime dependencies
Recent releases
- 0.5.416 years ago
- 0.5.316 years ago
- 0.5.216 years ago
- 0.5.116 years ago
- 0.5.016 years ago