I'm going to use the "FIFIO API" term from now on...

For a while I scraped Ex Libris's status webpage (http://status.exlibrisgroup.com/) to monitor the status of our catalogue/discovery layer instances and, in the event of an issue, automatically put a notice on the discovery layer to tell users what was wrong, that it was being worked on, and what other options they had in the meantime.

Eventually Ex Libris created a RESTful API with the same content so I've gratefully upgraded my code to use that instead.

Deborah

-----Original Message-----
From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of Jason Bengtson
Sent: Wednesday, 29 November 2017 8:00 a.m.
To: [log in to unmask]
Subject: Re: [CODE4LIB] Anyone web scraping to benefit their library?

I use web scraping sometimes to extract data from systems that lack APIs.
I'm doing this to get current library job openings from our University jobs application, for instance. I use the structure of their website in a way similar to an API query, scrape the results, and extract only what I need.
I jokingly call it a FIFIO API (Fine, I'll Figure It Out). Obviously, such a tool is inherently unstable, and has to be closely managed. When used with things like the jobs application, which maintain a relatively stable uri structure over time, however, it can be a pretty good tool when you have nothing else. I also used screen scraping as part of a tool I built years ago to allow authorized staff to create announcements within a special libguide that they then pushed to the EZ Proxy login page. I wrote
a book chapter on that one:   "Leveraging LibGuides as an EZProxy
Notifications Interface." Innovative Libguides Applications: Real World Examples. New York: Rowman & Littlefield, 2016

Best regards,

*Jason Bengtson*


*http://www.jasonbengtson.com/ <http://www.jasonbengtson.com/>*

On Tue, Nov 28, 2017 at 12:24 PM, Brad Coffield <[log in to unmask]
> wrote:

> I think there's likely a lot of possibilities out there and was hoping 
> to hear examples of web scraping for libraries. Your example might 
> just inspire me or another reader to do something similar. At the very 
> least, the ideas will be interesting!
>
> Brad
>
>
> --
> Brad Coffield, MLIS
> Assistant Information and Web Services Librarian Saint Francis 
> University
> 814-472-3315
> [log in to unmask]
>

________________________________
P Please consider the environment before you print this email.
"The contents of this e-mail (including any attachments) may be confidential and/or subject to copyright. Any unauthorised use, distribution, or copying of the contents is expressly prohibited. If you have received this e-mail in error, please advise the sender by return e-mail or telephone and then delete this e-mail together with all attachments from your system."