Crawlers fetch HTML and never run JavaScript, so a browser script can’t see
them. @pug-sh/crawlers runs in your site’s own request handling
instead and reports each crawler request to Pug.
New
- The Overview has an AI and search crawlers section: requests by category, the crawlers, companies and categories behind them, whether each was verified, and the pages they read, 404s included.
- Events names the crawler on its rows, and a crawler’s id opens a page of its own.
- Dashboards offer two crawler tile templates.
- Insights and the tile editor have a bots chip. A query that reads crawler data includes bot traffic on its own, and the chip shows it locked.