Hi Amy and group,

The type of documentation you describe in your first case for mistakes is exactly the type of documentation I've been trying to figure out how and where to store. It seems important to preserve but doesn't fit well with the PREMIS data model. I almost think there needs to be another entity in the data model for the repository itself so you can associate events, business processes, etc. with it and not have to have always tie events to objects.

Thank you Kate M. for reminding me to start using the new email list instead of the old one! Apologies to anyone who isn't interested in this thread... 

Andrea

> -----Original Message-----
> From: Amy Kirchhoff [mailto:[log in to unmask]]
> Sent: Wednesday, March 19, 2014 12:49 PM
> To: Goethals, Andrea; [log in to unmask]
> Subject: RE: documenting events not associated with content
> 
> Hi all ~
> 
> We have a couple of ways to do "extra systems" processing.
> 
> 1) We can delete things from the archive when we make a mistake.  It is
> very painful, mostly because of the bureaucracy and paperwork
> requirements we put around it.  In such cases, everything done is
> documented and that document is stored in SVN (at the moment ...
> eventually, we want to keep this type of documentation in the Archive
> proper, as well).  These type of deletions require my approval and they
> are lot of work for everyone involved, so the team works very hard to
> put things in the Archive properly and avoid this. :-)  In these
> instances, there are no stubs or metadata remaining in the archive.
> 
> 2) We have designed our system to allow us to create off-system event
> records.  Sometimes, the type of manipulation we need to make to the
> content cannot be easily implemented within our content processing
> system and we have to do them off-system.  In such cases, the off-
> system processing creates an event record that is slurped up in the
> preservation metadata file when the content makes it back into the
> content processing system and a record of the off-system processing is
> stored as an event in the final archival unit.
> 
> ~ Amy
> 
> -----Original Message-----
> From: The NDSA Standards working group list [mailto:NDSA-
> [log in to unmask]] On Behalf Of Goethals, Andrea
> Sent: Wednesday, March 19, 2014 12:12 PM
> To: [log in to unmask]
> Subject: Re: [NDSA-STANDARDS] documenting events not associated with
> content
> 
> Thanks Amy and Stephen for your answers! We have a couple use cases
> where I think what you suggest will work - keep the metadata and record
> the expungement event in it, even though we are actually deleting the
> content (not inactivating it). The 2 cases are when we legally have to
> get rid of the content (e.g. if we were to inadvertently collect child
> porn. during a web crawl), or when we want to clean out large amounts
> of test data. We also support "basic" deletions where we do something
> similar to you Amy - we flag the content as deleted so that it can't be
> accessed by end users but allow it to be restored if necessary.
> 
> We have a third odd case where we're actually doing a metadata-only
> expungement. Odd case I know but we're in the middle of a really big
> metadata migration between our old and new repository and need to be
> able to recover from errors. The idea is that if we find a mistake in
> the metadata migration for any set of content we'll be able to get rid
> of the metadata for that content in the new repository and rerun the
> migration on it. In this case there's nowhere to record the event but
> we may be able to log it.
> 
> This has been helpful.
> 
> thanks,
> Andrea
> 
> > -----Original Message-----
> > From: Amy Kirchhoff [mailto:[log in to unmask]]
> > Sent: Tuesday, March 18, 2014 12:26 PM
> > To: Goethals, Andrea; [log in to unmask]
> > Subject: RE: documenting events not associated with content
> >
> > Hi Andrea ~
> >
> > At Portico, each archival unit may have multiple content units (and
> > each content unit may or may not be ACTIVE).
> >
> > An e-journal article would be an archival unit.  We may have several
> > versions of the e-journal article (this is different from different
> > versions of specific files within the article).  We would address
> this
> > problem by making the original content unit inactive, so that it
> could
> > not be accessed by end users.  We would create a new content unit
> that
> > contained the original metadata, plus an explanation of the
> retraction.
> > Most of our publishers, send us retractions in this way -- it comes
> as
> > an update with modified metadata to explain the retraction and
> usually
> > the original PDF, just stamped with the word "retraction" on every
> > page.
> >
> > If we were under a court order to actually delete the content (as
> > opposed to just inactivate it), we would still leave an archival unit
> > behind with the original metadata and an explanation, if at all
> > legally possible.
> >
> > ~ Amy
> >
> > -----Original Message-----
> > From: The NDSA Standards working group list [mailto:NDSA-
> > [log in to unmask]] On Behalf Of Goethals, Andrea
> > Sent: Tuesday, March 18, 2014 11:21 AM
> > To: [log in to unmask]
> > Subject: [NDSA-STANDARDS] documenting events not associated with
> > content
> >
> > Hi all,
> >
> > Since we have 'Practices' in our name I thought I'd ask you all how
> > you handle something that we need to address in our repository. For
> > various reasons we have the need to support 'expungements' where we
> > delete a particular set of content and its metadata. We document
> other
> > significant events in PREMIS metadata embedded in object descriptors,
> > but after this action there won't be any metadata for this content to
> > record the event.
> >
> > I'd like to keep some kind of record that the expungement happened
> > though but am not aware of any 'standard' or common ways of recording
> > high-level repository events outside of PREMIS events associated with
> > content. Do any of you have a method for doing this or know of any
> > used in other preservation repositories?
> >
> > thanks,
> > Andrea
> >
> > _________________________________________
> > Andrea Goethals
> > Digital Preservation and Repository Services Manager Harvard Library
> > [log in to unmask]
> > (617) 495-3724
> >
> >
> > ############################
> >
> > To unsubscribe from the NDSA-STANDARDS list:
> > write to: mailto:NDSA-STANDARDS-SIGNOFF-
> > [log in to unmask]
> > or click the following link:
> > http://list.digitalpreservation.gov/scripts/wa-
> DIGITAL.exe?SUBED1=NDSA
> > -
> > STANDARDS&A=1
> 
> ############################
> 
> To unsubscribe from the NDSA-STANDARDS list:
> write to: mailto:NDSA-STANDARDS-SIGNOFF-
> [log in to unmask]
> or click the following link:
> http://list.digitalpreservation.gov/scripts/wa-DIGITAL.exe?SUBED1=NDSA-
> STANDARDS&A=1