On Jul 23, 2014, at 5:29 PM, Kyle Banerjee wrote:

> We've been facing increasing requests to help researchers publish datasets.
> There are many dimensions to this problem, but one of them is applying
> appropriate metadata and mounting them so they can be explored with a
> regular web browser or downloaded by expert users using specialized tools.
> 
> Datasets often are large. One that we used for a pilot project contained
> well over 10,000 objects with a total size of about 1 TB. We've been asked
> to help with much larger and more complex datasets.
> 
> The pilot was successful but our current process is neither scalable nor
> sustainable. We have some ideas on how to proceed, but we're mostly making
> things up. Are there methods/tools/etc you've found helpful? Also, where
> should we look for ideas? Thanks,


The tools I use are too customized for our field to be of much use to anyone else, so can't help on that part of the question.


I'd really recommend trying to reach out to someone working in data informatics in the field that the data is from, as they would have recommendations on specific metadata that should be captured.


For the general 'data publication' community, it's coalescing, but still a bit all over the place.  Here are some of the ones that I know about:

	JISC has a 'Data Publication' mailing list:

		https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DATA-PUBLICATION
	
	ASIS&T runs a 'Research Data Access & Preservation' conference and mailing list:

		http://www.asis.org/rdap/
		http://mail.asis.org/mailman/listinfo/rdap

	... and they put most of the presentations up on slideshare:

		http://www.slideshare.net/asist_org/

	The Research Data Alliance has two working groups on the topic, Publishing Services and Publishing Data Workflows:

		https://rd-alliance.org/group/rdawds-publishing-services-wg.html
		https://rd-alliance.org/group/rdawds-publishing-data-workflows-wg.html


I'm also one of the moderators of the Open Data site on Stack Exchange, which has some questions that might be relevant:

	Let's suppose I have potentially interesting data. How to distribute?
		http://opendata.stackexchange.com/q/768/263
	
	Benefits of using CC0 over CC-BY for data
		http://opendata.stackexchange.com/q/26/263

	... or just ask a new question.


I'd also recommend that when you catalog your data, that you also consider adding DataCite metadata, so that we can try to make it easier for others to cite your data.   (specific implementation recommendations for data citation are still evolving, but general principles have been released; if you have questions, feel free to ask me, as I think we need to add some clarification to what we mean on some of the items).

	http://www.datacite.org/
	https://www.force11.org/datacitation


As I see it, you're dealing with data that's in the problem range -- if it were larger, the department collecting the data would have a system in place already; if it were smaller, it's easier to manage as a single item for deposit.


-Joe