The hope is to take these bibliographies put it into more of a web
searchable/sortable format for researchers to make use out of them.  My
colleague was taking some inspiration from the Marlowe Bibliography (
https://marlowebibliography.org/), though we are hoping to possibly get a
bit more robust with the bibliography we are working on.  The important
first step it to be able to parse the existing OCRed bibliography scans we
have into a database, possibly a custom XML format but a database will
probably be easier to append and expand down the road.

On Thu, Jun 18, 2015 at 1:11 PM, Kyle Banerjee <[log in to unmask]>
wrote:

> How you want to preprocess and structure the data depends on what you hope
> to achieve. Can you say more about what you want the end product to look
> like?
>
> kyle
>
> On Thu, Jun 18, 2015 at 10:08 AM, Matt Sherman <[log in to unmask]>
> wrote:
>
> > That is a pretty good summation of it yes.  I appreciate the suggestions,
> > this is a bit of a new realm for me and while I know what I want it to do
> > and the structure I want to put it in, the conversion process has been
> > eluding me so thanks for giving me some tools to look into.
> >
> > On Thu, Jun 18, 2015 at 1:04 PM, Eric Lease Morgan <[log in to unmask]>
> wrote:
> >
> > > On Jun 18, 2015, at 12:02 PM, Matt Sherman <[log in to unmask]>
> > > wrote:
> > >
> > > > I am working with colleague on a side project which involves some
> > scanned
> > > > bibliographies and making them more web
> > searchable/sortable/browse-able.
> > > > While I am quite familiar with the metadata and organization aspects
> we
> > > > need, but I am at a bit of a loss on how to automate the process of
> > > putting
> > > > the bibliography in a more structured format so that we can avoid
> going
> > > > through hundreds of pages by hand.  I am pretty sure regular
> > expressions
> > > > are needed, but I have not had an instance where I need to automate
> > > > extracting data from one file type (PDF OCR or text extracted to Word
> > > doc)
> > > > and place it into another (either a database or an XML file) with
> some
> > > > enrichment.  I would appreciate any suggestions for approaches or
> tools
> > > to
> > > > look into.  Thanks for any help/thoughts people can give.
> > >
> > >
> > > If I understand your question correctly, then you have two problems to
> > > address: 1) converting PDF, Word, etc. files into plain text, and 2)
> > > marking up the result (which is a bibliography) into structure data.
> > > Correct?
> > >
> > > If so, then if your PDF documents have already been OCRed, or if you
> have
> > > other files, then you can probably feed them to TIKA to quickly and
> > easily
> > > extract the underlying plain text. [1] I wrote a brain-dead shell
> script
> > to
> > > run TIKA in server mode and then convert Word (.docx) files. [2]
> > >
> > > When it comes to marking up the result into structured data, well, good
> > > luck. I think such an application is something Library Land sought for
> a
> > > long time. “Can you say Holy Grail?"
> > >
> > > [1] Tika - https://tika.apache.org
> > > [2] brain-dead script -
> > > https://gist.github.com/ericleasemorgan/c4e34ffad96c0221f1ff
> > >
> > > —
> > > Eric
> > >
> >
>