Thanks everyone! This was very interesting (and at various points quite amusing). You know how sometimes you don't know you need something until you learn about the possibilities? Well, I still don't need web scraping lol but I'll definitely be saving this thread for the future because thanks to you all I now know better what's possible and in what situations it might be the best tool. I hope it was helpful to others as well. Brad (hope others continue to share their stories too..) On Wed, Nov 29, 2017 at 11:27 AM, Charles Ed Hill <[log in to unmask]> wrote: > I scrape data from a large book retailer we sometimes order from to make > our acquisitions workflow a bit easier. In particular, I ask people to make > wishlists and scrape those (I can do individual items too, but the large > retailer doesn't like that, even though we're trying to purchase from > them), check those titles against our holdings, then line them up with some > info in a spreadsheet for people. It is NOT the way I would suggest going > about things if you can help it, as pages change frequently, large > retailers block IP addresses, large retailers time out a lot, and so on, > but when up against a rock and a hard place. > > Not for libraries, but I also recently had to put together a scraper when > looking for daycares in Massachusetts as the page listing daycares was > helpful but really, really clunky. Saved me quite a bit of sanity. > > On Wed, Nov 29, 2017 at 8:50 AM, Ross Singer <[log in to unmask]> > wrote: > > > Due the absence of APIs, we have to scrape III WebBridge and EBSCO > > LinkSource link resolver results to determine electronic holdings for > > things. > > > > Neither of them make it particularly easy, since they don't provide many > > semantic clues in the markup as to what you're looking at and there are > all > > kinds of other conditions you have to account for (e.g. direct linking to > > certain sources, etc.). > > > > It's generally one of those things I avoid at all costs since the pages > you > > want/need to scrape are the most likely to be the most frustrating to > work > > with. > > > > -Ross. > > > > On Tue, Nov 28, 2017 at 1:26 PM Brad Coffield < > [log in to unmask] > > > > > wrote: > > > > > I think there's likely a lot of possibilities out there and was hoping > to > > > hear examples of web scraping for libraries. Your example might just > > > inspire me or another reader to do something similar. At the very > least, > > > the ideas will be interesting! > > > > > > Brad > > > > > > > > > -- > > > Brad Coffield, MLIS > > > Assistant Information and Web Services Librarian > > > Saint Francis University > > > 814-472-3315 <(814)%20472-3315> > > > [log in to unmask] > > > > > > -- Brad Coffield, MLIS Assistant Information and Web Services Librarian Saint Francis University 814-472-3315 [log in to unmask]