I think this episode also illustrates, once again, that preservation is not about technology at all, it's about *institutional commitment*. The kind of institutional commitment that would have implemented and maintained the kinds of procedures that Jonathan described. Without institutional commitment, no technology on earth can save you. Roy On 9/1/09 9/1/09 € 9:00 AM, "Jonathan Rochkind" <[log in to unmask]> wrote: > I'd add that not only does it sound like GPO maintained no failover > backup, it sounds, based on Jonathan Lebreton's report, like they > didn't even maintain an offline backup, since they're needing to > regenerate the purl database from raw data, rather than simply restoring > from a backup, which would generally be much quicker then the process > that Jonathan Lebreton seems to be describing. > > From what info we have, it sounds like GPO simply, well, was very very > far from 'best practices' for a service meant to be robustly reliable. > On the other hand, we're just going from sort of third hand hearsay, > maybe they were doing things more right than it sounds, but some kind of > catastrophic unexpected 'perfect storm' still happened to bring > everything down. Maybe 48 hours of outage in 10 years (how long has GPO > purl been running? Have there been outages like this before?) is > appropriate reliability for the level of importance of this service. I > dunno. > > Jonathan > > Jonathan Lebreton wrote: >> This is indeed an interesting problem - we are all dependent on a >> centralized service node. >> >> Just got off the phone with GPO 9 am 9/1/09. >> I was told they are now up to 50% or PURLs restored but the script is >> running very slowly line-by-line since the server (they're updating the >> production server while it is up) is experiencing unusually heavy load >> from the user community and bots scheduled to troll at beginning of the >> month. >> >> Jonathan LeBreton >> Sr. Associate University Librarian >> Temple University Libraries >> voice: 215-204-8231 >> fax: 215-204-5201 >> email: [log in to unmask] >> email: [log in to unmask] >> >> >> >> >> >> >> >>> -----Original Message----- >>> From: Code for Libraries [mailto:[log in to unmask]] On Behalf >>> >> Of >> >>> James Jacobs >>> Sent: Monday, August 31, 2009 6:06 PM >>> To: [log in to unmask] >>> Subject: Re: [CODE4LIB] GPO PURLs >>> >>> Hi all, (cross-posted to purl-dev) >>> >>> I'm a documents librarian (and member of the Depository Library >>> Council) >>> and usually just a lurker over here. Thanks Keith and Patricia for the >>> easy workaround. I shared this with govdoc-l and on my blog: >>> >>> http://freegovinfo.info/node/2704 >>> >>> See especially the comment that as of today, only 3,677 PURLs out of >>> 116,237 have been restored (3.1%). I would love to hear your >>> thoughts/ideas for how this kind of critical system failure can be >>> averted in the future from a technological standpoint. Is it possible >>> to >>> mirror a purl server? Will the same issue occur when GPO moves to >>> handles in FDsys (http://www.handle.net/)? Will a distributed >>> infrastructure as I've briefly mapped out be able to handle these >>> >> types >> >>> of critical system crashes better? >>> >>> Please let me know and I'd be happy to share your ideas with GPO and >>> the >>> documents community. >>> >>> Best, >>> >>> James Jacobs >>> >>> >>> >>> >>> Keith Jenkins wrote: >>> >>>> Thanks to everyone who helped me confirm that the GPO PURL server is >>>> down. An official announcement on the GPO Listserv said: >>>> "The PURL Server is currently inaccessible. GPO is working with >>>> >> IT >> >>>> staff to restore service as soon as possible. We regret any >>>> inconvenience caused by the server problems. An updated listserv >>>> >> will >> >>>> be sent once service is restored." >>>> >>>> While the server is down, here is one workaround (thanks to Patricia >>>> >>> Duplantis): >>> >>>> 1. Go to http://catalog.gpo.gov/ >>>> 2. Click "Advanced Search" >>>> 3. Search for word in "URL/PURL", enter the PURL >>>> 4. Click "Go" >>>> 5. The original URL at the time of cataloging should appear in a >>>> >>> 53x note. >>> >>>> This incident, however, illuminates a weakness in PURL systems: >>>> >>> access >>> >>>> is broken when the PURL server breaks, even though the documents are >>>> still online at their original URLs. >>>> >>>> Maybe someone more familiar with PURL systems can tell me... is >>>> >> there >> >>>> any way to harvest data from a PURL server, so that a backup/mirror >>>> can be available? >>>> >>>> Keith >>>> >>> -- >>> James R. Jacobs >>> International Documents Librarian >>> Green Library, Stanford University >>> P: (650) 725-1030 E: [log in to unmask] >>> AIM: LibrarianJames T: @freegovinfo >>> >>> The more beautiful questions demand the more beautiful answers, >>> and if we can learn to ask them, we stand a chance of steering >>> clear of shipwreck on our jury-rigged and not so distant star. >>> --Lewis Lapham, Lapham's Quarterly I(3), Summer, 2008, p.17. >>> >>> ----------------------------------------------- >>> This message may have been intercepted and read by U.S. government >>> agencies including the FBI, CIA, and NSA without notice or warrant or >>> knowledge of sender or recipient. >>> >>> (\ >>> {|||8- >>> (/ >>> >> >> >