robots-parser

v3.0.1

A specification compliant robots.txt parser with wildcard (*) matching support.

npm install robots-parser
Bundle size
Dependencies
Dependencies(0)
    Peer Dependencies(0)
      Dev Dependencies(3)