Robert,
Your work also sounds very interesting and definitely overlaps with some of what we want to do. It seems like a lot of people are trying to get useful "format" information out of MARC records and it's unfortunate that it is so complicated. I would be very interested to see your logic for determining format and dealing with self-contradictory records. Runtime from the 008 is, as you say, pretty straightforward, but not always filled out and useless if the resource is longer than 999 minutes.
It's interesting that you mention identifying directors. We have also been working on a similar, although more generalized, process. We're trying to identify all of the personal and organizational names mentioned in video records and, where possible, their roles. Our existing process is pretty accurate for personal names and for roles in English. It tends to struggle with credits involving multiple corporate bodies and we're working on building a lexicon of non-English terms for common roles. We're also trying to get people to hand-annotate credits to build a corpus to help us improve our process. (Help us out at http://olac-annotator.org/. And if you're willing to be on call to help with translating non-English credits, email me with the language(s) you'd be able to help out with. We also just started a mailing list at https://lists.uoregon.edu/mailman/listinfo/olac-credits)
Matching MARC records for moving images with external data sources is also on our radar. Most feature film type material can probably be identified by the attributes you mention: title, original date and director (probably 2 out of 3 would work in most cases). We are also hoping to use these attributes (and possibly others) to cluster records for the same FRBR work.
It would be great to talk with you more about this off-list.
Kelley
[log in to unmask]
________________________________________
From: Robert Haschart [[log in to unmask]]
Sent: Monday, December 02, 2013 10:49 AM
To: Code for Libraries
Cc: Kelley McGrath
Subject: Re: [CODE4LIB] Looking for two coders to help with discoverability of videos
Kelley,
The work you are proposing is interesting and overlaps somewhat both
with work I have already done and with a new project I'm looking into
here at UVa.
I have been the primary contributor to the Marc4j java project for the
past several years and am the creator of the project SolrMarc which
extracts data from Marc records based on a customizable specification,
to build Solr index records to facilitate rich discovery.
Much of my work on creating and improving these projects has been in
service of my actual job of creating and maintaining the Solr Index
behind our Blacklight-based discovery interface. As a part of that
work I have created custom SolrMarc routines that extract the format of
items similar to what is described in Example 3, including looking in
the leader, 006, 007 and 008 to determine the format "as-coded" but
further looking in the 245 h, 300 and 538 fields to heuristically
determine when the format "as-coded" is incorrect and ought to be
overridden. Most of the heuristic determination is targeted towards
Video material, and was initiated when I found an item that due to a
coding error was listed as a "Video in Braille format".
Further I have developed a set of custom routines that look more closely
at Video items, one of which already extracts the runtime from the
008[18-20] field,
To modify it from its current form that currently returns the runtime in
minutes, to instead return it as HH:MM as specified in your xls file,
and to further handle the edge case of 008[18-20] = "000" to return
"over 16:39" would literally take about 15 minutes.
Another of these custom routines that is more fully-formed, is code for
extracting the Director of a video from the Marc record. It examines
the contents of the fields 245c, 508a, 500a, 505a, 505t, employing
heuristics and targeted natural language processing techniques, to
attempt to correctly extract the "Director". At this point I believe
it achieves better results than a careful cataloger would achieve, even
one who specializes in film and video.
The other project I have just started investigating is an effort to
create and/or flesh out Marc records for video items based on heuristic
matching of title and director and date with data returned from
publicly-accessible movie information sites.
This more recent work may not be relevant to your needs but the custom
extraction routines seem directly applicable to your goals, and may also
provide a template that may make your other goals more easily achievable.
-Robert Haschart
|
