LISTSERV mailing list manager LISTSERV 16.5

Help for NDSA-ALL Archives


NDSA-ALL Archives

NDSA-ALL Archives


NDSA-ALL@LISTS.CLIR.ORG


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

NDSA-ALL Home

NDSA-ALL Home

NDSA-ALL  June 2013, Week 4

NDSA-ALL June 2013, Week 4

Subject:

Re: provenance projects?

From:

Sayeed Choudhury <[log in to unmask]>

Reply-To:

The NDSA organization working group list <[log in to unmask]>

Date:

Wed, 26 Jun 2013 01:33:02 +0000

Content-Type:

multipart/mixed

Parts/Attachments:

Parts/Attachments

text/plain (139 lines) , Poster192.pdf (139 lines)

In response to Michelle's email below, I'm sharing the abstract and
conclusion that comes from a paper my co-authors and I are submitting
related to the Data Conservancy (https://dataconservancy.org/) provenance
and lineage service. The "stack model" mentioned below is described in the
attached poster from the most recent IDCC conference. This stack model is
also described in the YouTube clip available at
https://www.youtube.com/watch?v=F6iYXNvCRO4

Sayeed


Data Conservancy Provenance, Context, and Lineage Services: Key Components
for Data Preservation and Curation

Matthew S. Mayernik, G. Sayeed Choudhury, Tim DiLauro, Ruth Duerr, Elliot
Metsger, Anne Thessen

For submission to the Data Science Journal: http://www.codata.org/dsj/

Abstract
Among the key services that an institutional data management
infrastructure must provide is provenance and lineage tracking and the
ability to associate data with the contextual information needed to
understand and use the data. These functionalities are critical for
addressing a number of key issues faced by data creators/collectors and
users such as trust in data, results traceability, transparency of data
modification or analysis processes, data citation support, etc. In this
paper, we describe the support for these services within the Data
Conservancy Service (DCS) software. The DCS provenance, context and
lineage services provide functionalities that cross the four layers in the
DCS data curation stack model: storage, archiving, preservation, and
curation.


Conclusion
In order to ensure that data remain understandable and usable, data
infrastructure must manage relationships and processes in addition to the
resources themselves. Provenance, context and lineage services provide
scalable ways to trace and manage relationships between numerous
resources, and to track the processes used to manipulate, transform, and
preserve resources. Within the Data Conservancy Service, the provenance
functionality documents internal processes, the lineage service
establishes linkages between successive versions of submitted resources,
and the ability to record relationships between resources allow users to
associate provenance and contextual information with a resource. These
services reflect the characteristics of provenance ready computational
systems noted by Groth, et al. (2012). The results of completed processes
are compiled into the provenance and lineage streams as system events by
the DCS, reflecting how the processes were executed, and their success or
failure. DC provenance and lineage streams are autonomously created by the
DCS as system events occur, and are immutable once created.

From a service stack perspective, the DCS provenance and lineage services
provide functionalities that cross the four layers in the Data Conservancy
Service stack model: storage, archiving, preservation, and curation.

● Storage - The DCS begins storing provenance information upon ingest
along with the resources themselves. The stream of DCS-generated
provenance information can then be leveraged to assess the success and
failure of individual processes, and of sets of processes, such as the
ingest process.

● Archiving - The DCS creates identifiers for individual entities as they
are ingested, and creates a lineage identifier that identifies a set of
related resources. The lineage identifier also provides the
mechanism by which lineage information can be queried and retrieved.

● Preservation - Capturing system events as provenance and lineage
information is one component of a preservation-ready system, in that these
streams allow people and machines other than the original data producer to
use and interpret the data. Providing users the capability of associating
a resource in defined ways with other information irrespective of whether
that information is held within the DCS or not; maximizes the chances that
all of the information needed by future users will be available.

● Curation - The DCS allows additional features to be built on the lineage
service and the ability to define relationships between resources both
within and outside of the DCS. The lineage service will
underpin user interfaces that allow users to navigate through changes in
versions, understand an objects history and context. This flexibility and
customizability is a design feature of the software. In the alpha release
of the Data Conservancy software, the reference user interface hides the
details of the lineage service from the user.

Provenance, context and lineage functionalities are essential components
of infrastructure for digital research data. The Data Conservancy stack
model helps to identify where provenance and lineage functionalities fit
within the storage, archiving, preservation, and curation service layers.

Future extensions of the work described in this paper will investigate the
consequences of allowing users to assert pre-existing provenance events
about objects during accession by the DCS archive, in order to, for
example, represent format conversions that took place prior to objects
being uploaded to the DCS. Other extensions will explore the potential for
a unified event model between the business layer and the archive, and
develop an understanding of how preservation actions may affect the
lineage of archival object graphs.


On 6/18/13 8:17 AM, "Gallinger, Michelle" <[log in to unmask]> wrote:

>We are interested in learning more about what people are doing in regards
>to provenance.
>
>Are you working on a project that has a provenance component? do you know
>of interesting provenance work being done by others? Please share!
>
>Thanks so much.
>
>All best,
>
>Michelle
>
>Michelle Gallinger
>Digital Programs Coordinator
>Library of Congress
>101 Independence Ave. SE
>Washington, DC 20007
>202-707-1619
>[log in to unmask]
>
>
>############################
>
>To unsubscribe from the NDSA-ALL list:
>write to: mailto:[log in to unmask]
>or click the following link:
>http://list.digitalpreservation.gov/scripts/wa-DIGITAL.exe?SUBED1=NDSA-ALL
>&A=1


############################

To unsubscribe from the NDSA-ALL list:
write to: mailto:[log in to unmask]
or click the following link:
http://list.digitalpreservation.gov/scripts/wa-DIGITAL.exe?SUBED1=NDSA-ALL&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014, Week 3
March 2014, Week 2
March 2014, Week 1
March 2014
February 2014, Week 4
February 2014, Week 3
February 2014, Week 2
February 2014, Week 1
January 2014, Week 5
January 2014, Week 4
January 2014, Week 3
January 2014, Week 2
January 2014, Week 1
December 2013, Week 3
December 2013, Week 2
December 2013, Week 1
November 2013, Week 4
November 2013, Week 3
November 2013, Week 2
November 2013, Week 1
October 2013, Week 5
October 2013, Week 4
October 2013, Week 3
September 2013, Week 4
September 2013, Week 3
September 2013, Week 2
September 2013, Week 1
August 2013, Week 5
August 2013, Week 4
August 2013, Week 3
August 2013, Week 2
August 2013, Week 1
July 2013, Week 5
July 2013, Week 4
July 2013, Week 3
July 2013, Week 2
June 2013, Week 4
June 2013, Week 3
June 2013, Week 2
June 2013, Week 1
May 2013, Week 5
May 2013, Week 4
May 2013, Week 3
May 2013, Week 2
May 2013, Week 1
April 2013, Week 5
April 2013, Week 4
April 2013, Week 3
April 2013, Week 2
March 2013, Week 4
March 2013, Week 3
March 2013, Week 2
March 2013, Week 1
February 2013, Week 4
February 2013, Week 3
February 2013, Week 2
February 2013, Week 1
January 2013, Week 5
January 2013, Week 4
January 2013, Week 3
December 2012, Week 4
December 2012, Week 3
December 2012, Week 2
November 2012, Week 5
November 2012, Week 3
November 2012, Week 2
November 2012, Week 1
October 2012, Week 5
October 2012, Week 4
October 2012, Week 3
October 2012, Week 2
October 2012, Week 1
September 2012, Week 4
September 2012, Week 3
September 2012, Week 2
September 2012, Week 1
August 2012, Week 5
August 2012, Week 4
August 2012, Week 3
August 2012, Week 1
July 2012, Week 5
July 2012, Week 3
July 2012, Week 2
July 2012, Week 1
June 2012, Week 4
June 2012, Week 3
June 2012, Week 2
June 2012, Week 1
May 2012, Week 5
May 2012, Week 4
May 2012, Week 3
May 2012, Week 2
May 2012, Week 1
April 2012, Week 5
April 2012, Week 4
April 2012, Week 3
April 2012, Week 2
April 2012, Week 1
March 2012, Week 5
March 2012, Week 4
March 2012, Week 2
February 2012, Week 5
February 2012, Week 4
February 2012, Week 3
February 2012, Week 2
February 2012, Week 1
January 2012, Week 4
January 2012, Week 3
January 2012, Week 2
January 2012, Week 1
December 2011, Week 5
December 2011, Week 3
December 2011, Week 2
December 2011, Week 1
November 2011, Week 4
November 2011, Week 3
November 2011, Week 2
November 2011, Week 1
October 2011, Week 4
October 2011, Week 3
October 2011, Week 2
October 2011, Week 1
September 2011, Week 4
September 2011, Week 3
September 2011, Week 2
September 2011, Week 1
August 2011, Week 5
August 2011, Week 2
August 2011, Week 1
July 2011, Week 4
July 2011, Week 2
June 2011, Week 2
May 2011, Week 1
April 2011, Week 5
April 2011, Week 4
April 2011, Week 3
April 2011, Week 2
April 2011, Week 1
March 2011, Week 4
March 2011, Week 3
March 2011, Week 2
March 2011, Week 1
February 2011, Week 4
February 2011, Week 3
February 2011, Week 2
January 2011, Week 5
January 2011, Week 4
December 2010, Week 3
November 2010, Week 4
October 2010, Week 3
September 2010, Week 2

ATOM RSS1 RSS2



LISTS.CLIR.ORG

CataList Email List Search Powered by the LISTSERV Email List Manager