Hi Meg, Thanks for sharing the discussion that came out of the Estonian conference. Metrics like those are definitely of interest to me - they seem fundamental to managing preservation repositories. I think that there would need to be a lot of up-front work in defining terms and metrics to make them really usable and comparable. Even some of the ones that seem simpler at first read (cost per GB per year) can vary greatly for the same institution depending on what's included in the cost. Recently at my institution we were trying to put a total cost on the purchase of our current storage system and there were some real gray areas (consulting fees, incremental storage, etc.) that had a large effect on the total number. If the group (and/or the Infra. Group) is interested in this topic, would it make sense for us to contribute to Andy's project - maybe by thinking of relevant metrics? Andrea -----Original Message----- From: The NDSA Standards working group list [mailto:[log in to unmask]] On Behalf Of Meg Phillips Sent: Monday, June 06, 2011 2:33 PM To: [log in to unmask] Subject: [NDSA-STANDARDS] interesting topics from the ANADP conference in Estonia All: On today's call I mentioned being intrigued by some of the discussions that came up at the recent conference on Aligning National Approaches to Digital Preservation in Estonia (sponsored by LC and Educopia, among others - thanks, guys!) Jimi wants to see if we can stir up some discussion on the listserv, so I'll jump in. Andy Rauber of the Technical University of Vienna made a plea for better testing, metrics, and benchmarking of digital preservation systems and better sharing of that information so we could start identifying technical set-ups that work best for particular purposes. Right now, very few people have hard data, and even fewer are sharing what they have. In the context of the discussion in Estonia, the idea was largely to generate competition among vendors selling digital preservation solutions, but it would be very useful to institutions no matter what kind of solution we're using. Andy was going to start an initiative to gather ideas for metrics people would like to see, then see where we could take it. Some of the things that might be useful to test and compare are below. Does anyone know what already does exist in these areas? Are there other areas that would be useful to compare? - ingest throughput rates (maybe even step by step: what particular processes are the limiting factors on ingest rates) - accuracy rates for format identification, characterization, or migrations tools - bit error rates (especially at scale) - cost per GB per year for storage/preservation (especially at scale) - robustness of security stance: has anyone tried to hack their own systems and change data? How hard was it to do and how hard to detect? ("Assume an attitude of distrust in order to become trustworthy.") I don't know if topics like these are of interests to the Standards and Practices group, or whether they're better suited to the infrastructure group. Regardless, the idea of measuring and then identifying some benchmarks that would help us all know whether we're doing as well as we could be for our environment could be really useful. Meg ############################ To unsubscribe from the NDSA-STANDARDS list: write to: mailto:[log in to unmask] or click the following link: http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-S TANDARDS&A=1 ############################ To unsubscribe from the NDSA-STANDARDS list: write to: mailto:[log in to unmask] or click the following link: http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?SUBED1=NDSA-STANDARDS&A=1