 |
 |
I've used the Perl Digest::SHA module for setting fixity for the Census 2000 and 2010 data we received, and I'm currently using the same in a project to add fixity setting for the records we receive from the various congressional committees (our ERA CRI instance). It requires a Perl environment on your system, of course; for the projects I'm doing in a Windows environment, I'm using cygwin as the base environment. One of the issues I have run across with this particular solution in a Windows environment is if you are processing a large number of files in a single script, and those files reside on a network drive, the Digest::SHA module will occasionally kick out an error attempting to calculate the SHA-256 hash code for a file. For example, in our Census 2000 project, I found we were encountering approximate 3 errors for every 100,000 files processed. I had to build in a retry mechanism to work around this problem which worked since the errors were never on contiguous files in the list. Regards, Kevin McCarthy Computer Scientist National Archives and Records Administration Electronic Records Archives (ERA) 8601 Adelphi Rd. Room B530 College Park, MD 20740-6001 (301) 837-1657 [log in to unmask] On Wed, Apr 2, 2014 at 1:16 PM, Shawn Rounds <[log in to unmask]> wrote: > Hello, > > Reading over the minutes from the last call and draft of the fixity paper > prompts me to pose a question to all of you. Do you use or know of any > stand-alone tools/programs for automated, server-side fixity checks of > large files batches (thousands of files)? Not necessarily looking for open > source. Any pointers to online resource lists on this topic? > > Thanks very much, > Shawn > > Shawn Rounds > State Archivist > State Archives Department > Division of Library and Collections > Minnesota Historical Society > 345 Kellogg Boulevard West > Saint Paul, Minnesota 55102 > > [log in to unmask] > 651.259.3265 > MNHS online: www.mnhs.org > Facebook: www.facebook.com/minnesotastatearchives > > > > > > On Wed, Mar 26, 2014 at 12:51 PM, Howard, Barrie <[log in to unmask]> wrote: > >> Good afternoon NDSA Infrastructure Working Group Members, >> >> >> >> I tidied up the notes from yesterday's call and have posted them to the >> wiki, >> http://www.loc.gov/extranet/wiki/osi/ndiip/ndsa/index.php?title=Tuesday,_Mar_25,_2014 >> >> >> >> >> Kind regards, >> >> >> >> Barrie Howard >> >> NDSA Secretariat >> >> +1-202-707-1830 >> >> >> >> >> > >