I don’t know how well this applies to your specific use of screen-scraping, but for libraries’ broader use of crawlers to build archives, the Section 108 Study Group Recommendations are a good source of guidance (though not law). They propose specific copyright exceptions for libraries in regard to collecting and archiving “publicly accessible online content”. Their recommendations are clear & sensible… they run from page 80-87 of the report. http://www.section108.gov/docs/Sec108StudyGroupReport.pdf Tracy Seneca California Digital Library ________________________________________ From: Code for Libraries [[log in to unmask]] on behalf of Nate Hill [[log in to unmask]] Sent: Sunday, October 02, 2011 7:23 PM To: [log in to unmask] Subject: [CODE4LIB] screen scraping A question: what are the 'rules' around screen scraping? If one site doesn't offer an RSS feed and you want to grab (for example) their weekly top ten list with a script and then redisplay it on another site, is that bad form? Or even illegal? Thanks- Nate -- Nate Hill [log in to unmask] http://www.natehill.net