Andrea,
Andy seemed very interested in input on metrics we'd like to see, so doing some thinking on that ourselves and reaching out to him would be great.
I bet one of the most useful things we could do would be trying to specify exactly what we meant by a particular metric, laying the groundwork for a real apples-to-apples comparison. As you say, this is where it gets complicated.
When I was poking around on the cost issue, I found a pretty good description from Hathitrust on what they include in the fees they charge (or will charge others), which was interesting to see. Since they are charging others for storage in a way that NARA is not, they had a motivation to figure out what to charge. That could be a little different what it costs them, of course. http://www.hathitrust.org/cost
Meg
>>> "Goethals, Andrea" <[log in to unmask]> 6/7/2011 10:05 AM >>>
Hi Meg,
Thanks for sharing the discussion that came out of the Estonian
conference. Metrics like those are definitely of interest to me - they
seem fundamental to managing preservation repositories. I think that
there would need to be a lot of up-front work in defining terms and
metrics to make them really usable and comparable. Even some of the ones
that seem simpler at first read (cost per GB per year) can vary greatly
for the same institution depending on what's included in the cost.
Recently at my institution we were trying to put a total cost on the
purchase of our current storage system and there were some real gray
areas (consulting fees, incremental storage, etc.) that had a large
effect on the total number.
If the group (and/or the Infra. Group) is interested in this topic,
would it make sense for us to contribute to Andy's project - maybe by
thinking of relevant metrics?
Andrea
-----Original Message-----
From: The NDSA Standards working group list
[mailto:[log in to unmask]] On Behalf Of Meg
Phillips
Sent: Monday, June 06, 2011 2:33 PM
To: [log in to unmask]
Subject: [NDSA-STANDARDS] interesting topics from the ANADP conference
in Estonia
All:
On today's call I mentioned being intrigued by some of the discussions
that came up at the recent conference on Aligning National Approaches to
Digital Preservation in Estonia (sponsored by LC and Educopia, among
others - thanks, guys!) Jimi wants to see if we can stir up some
discussion on the listserv, so I'll jump in.
Andy Rauber of the Technical University of Vienna made a plea for better
testing, metrics, and benchmarking of digital preservation systems and
better sharing of that information so we could start identifying
technical set-ups that work best for particular purposes. Right now,
very few people have hard data, and even fewer are sharing what they
have. In the context of the discussion in Estonia, the idea was largely
to generate competition among vendors selling digital preservation
solutions, but it would be very useful to institutions no matter what
kind of solution we're using.
Andy was going to start an initiative to gather ideas for metrics people
would like to see, then see where we could take it.
Some of the things that might be useful to test and compare are below.
Does anyone know what already does exist in these areas? Are there
other areas that would be useful to compare?
- ingest throughput rates (maybe even step by step: what particular
processes are the limiting factors on ingest rates)
- accuracy rates for format identification, characterization, or
migrations tools
- bit error rates (especially at scale)
- cost per GB per year for storage/preservation (especially at scale)
- robustness of security stance: has anyone tried to hack their own
systems and change data? How hard was it to do and how hard to detect?
("Assume an attitude of distrust in order to become trustworthy.")
I don't know if topics like these are of interests to the Standards and
Practices group, or whether they're better suited to the infrastructure
group. Regardless, the idea of measuring and then identifying some
benchmarks that would help us all know whether we're doing as well as we
could be for our environment could be really useful.
Meg
############################
To unsubscribe from the NDSA-STANDARDS list:
write to:
mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-S
TANDARDS&A=1
############################
To unsubscribe from the NDSA-STANDARDS list:
write to: mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-STANDARDS&A=1
############################
To unsubscribe from the NDSA-STANDARDS list:
write to: mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-STANDARDS&A=1
