 |
 |
Also just stumbled across this on stackoverflow. http://stackoverflow.com/questions/12115824/compute-the-hash-of-only-the-core-image-data-of-a-tiff On Wed, Jan 28, 2015 at 10:32 AM, Ronald Houk < [log in to unmask]> wrote: > Hello, > > I like Danielle's idea. I wonder if it wouldn't be a good idea to > decouple the metadata from the data permanently. Exiftool allows you to > export the metadata in lots of different formats like JSON. You could > export the metadata into JSON, run the checksums and then store the photo > and the JSON file in a single tar-ball. From there you could use a JSON > editor to modify/add metadata. > > It would be simple to reintroduce the metadata into the file when needed. > > On Mon, Jan 26, 2015 at 10:27 AM, danielle plumer <[log in to unmask]> > wrote: > >> Kyle, >> >> It's a bit of a hack, but you could write a script to delete all the >> metadata from images with ExifTool and then run checksums on the resulting >> image (see >> http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=4902.0). >> exiv2 might also work. I don't think you'd want to do that every time you >> audited the files, though; generating new checksums is a faster approach. >> >> I haven't tried this, but I know that there's a program called ssdeep >> developed for the digital forensics community that can do piecewise >> hashing >> -- it hashes chunks of content and then compares the hashes for the >> different chunks to find matches, in theory. It might be able to match >> files with embedded metadata vs. files without; the use cases described on >> the forensics wiki is finding altered (truncated) files, or reuse of >> source >> code. http://www.forensicswiki.org/wiki/Ssdeep >> >> Danielle Cunniff Plumer >> >> On Sun, Jan 25, 2015 at 9:44 AM, Kyle Banerjee <[log in to unmask]> >> wrote: >> >> > On Sat, Jan 24, 2015 at 11:07 AM, Rosalyn Metz <[log in to unmask]> >> > wrote: >> > >> > > >> > > - How is your content packaged? >> > > - Are you talking about the SIPs or the AIPs or both? >> > > - Is your content in an instance of Fedora, a unix file structure, >> or >> > > something else? >> > > - Are you generating checksums on the whole package, parts of it, >> > both? >> > > >> > >> > The quick answer to this is that this is a low tech operation. We're >> > currently on regular filesystems where we are limited to feeding md5 >> > checksums into a list. I'm looking for a low tech way that makes it >> easier >> > to keep track of resources across a variety of platforms in a >> decentralized >> > environment and which will easily adopt to future technology >> transitions. >> > For example, we have a bunch of stuff in Bepress and Omeka. Neither of >> > those is good for preservation, so authoritative files live elsewhere >> as do >> > a huge number of resources that aren't in these platforms. Filenames are >> > terrible identifiers and things get moved around even if people don't >> mess >> > with the files. >> > >> > We also are trying to come up with something that deals with different >> > kinds of datasets (we're focusing on bioimaging at the moment) and fits >> in >> > the workflow of campus units, each of which needs to manage tens of >> > thousands of files with very little metadata on regular filesystems. >> Some >> > of the resources are enormous in terms of size or number of members. >> > >> > Simply embedding an identifier in the file is a really easy way to tell >> > which files have metadata and which metadata is there. In the case at >> hand, >> > I could just do that and generate new checksums. But I think the generic >> > problem of making better use of embedded metadata is an interesting one >> as >> > it can make objects more usable and understandable once they're removed. >> > For example, just this past Friday I received a request to use an image >> > someone downloaded for a book. Unfortunately, he just emailed me a copy >> of >> > the image, described what he wanted to do, and asked for permission but >> he >> > couldn't replicate how he found it. An identifier would have been handy >> as >> > would have been embedded rights info as this is not the same for all of >> our >> > images. The reason we're using DOI's is that they work well for anything >> > and can easily be recognized by syntax wherever they may appear. >> > >> > On Sat, Jan 24, 2015 at 7:06 PM, Joe Hourcle < >> > [log in to unmask]> >> > wrote: >> > >> > > >> > > The problems with 'metadata' in a lot of file formats is that they're >> > > just arbitrary segments -- you'd have to have a program that knew >> > > which segments were considered 'headers' vs. not. It might be easier >> > > to have it be able to compute a separate checksum for each segment, >> > > so that should the modifications change their order, they'd still >> > > be considered valid. >> > > >> > >> > This is what I seemed to be bumping up against so I was hoping there >> was an >> > easy workaround. But this is helpful information. Thanks, >> > >> > kyle >> > >> > > > > -- > Ronald Houk > Assistant Director > Ottumwa Public Library > 102 W. Fourth Street > Ottumwa, IA 52501 > (641)682-7563x203 > [log in to unmask] > -- Ronald Houk Assistant Director Ottumwa Public Library 102 W. Fourth Street Ottumwa, IA 52501 (641)682-7563x203 [log in to unmask]