perl-WWW-RobotRules

Database of robots.txt-derived permissions
Current version: 6.02

History

Effective Date Table ID Version Download URL Project URL Version URL
2024-08-26 12 6.02 download project version