May not be of much use if they're ignoring robots.txt, but I just tripped across the following fascinating resource that's attempting to provide "A List of Known AI Agents on the Internet" https://darkvisitors.com/
Paul R. Pival
Research Librarian – Qualitative Data Analysis
Economics Liaison Librarian
University of Calgary
TFDL 460C
[log in to unmask]
Pronouns: he, him, his
|