Thanks, that is interesting since we can export from the PDFs, and while
the OCR text is a little messy it is in decent shape.  I'll certainly look
into that.

On Thu, Jun 18, 2015 at 3:13 PM, Gordon, Bonnie <[log in to unmask]>
wrote:

> We¹re actually also working on getting a bibliography from a Word Doc to a
> more structured format. We¹re using regular expressions in LibreOffice
> Writer to mark up the citations, then insert tabs between the elements,
> and then copy into a spreadsheet (similar to what¹s described in
> http://programminghistorian.org/lessons/understanding-regular-expressions
> ).
>  However, our bibliography was originally a Word Doc, not OCRed text. This
> method is pretty reliant on consistent formatting, though, so messy OCR
> could complicate things. Another thing to note is that it¹s easiest when
> you know what format the citation is for (e.g., a book or article), since
> that impacts how the citation is structured.  I¹d be happy to provide a
> sample citation in each step of the process.
>
> All the best,
> Bonnie
>
>
>
> On 6/18/15, 1:52 PM, "Matt Sherman" <[log in to unmask]> wrote:
>
> >The hope is to take these bibliographies put it into more of a web
> >searchable/sortable format for researchers to make use out of them.  My
> >colleague was taking some inspiration from the Marlowe Bibliography (
> >https://marlowebibliography.org/), though we are hoping to possibly get a
> >bit more robust with the bibliography we are working on.  The important
> >first step it to be able to parse the existing OCRed bibliography scans we
> >have into a database, possibly a custom XML format but a database will
> >probably be easier to append and expand down the road.
> >
> >On Thu, Jun 18, 2015 at 1:11 PM, Kyle Banerjee <[log in to unmask]>
> >wrote:
> >
> >> How you want to preprocess and structure the data depends on what you
> >>hope
> >> to achieve. Can you say more about what you want the end product to look
> >> like?
> >>
> >> kyle
> >>
> >> On Thu, Jun 18, 2015 at 10:08 AM, Matt Sherman
> >><[log in to unmask]>
> >> wrote:
> >>
> >> > That is a pretty good summation of it yes.  I appreciate the
> >>suggestions,
> >> > this is a bit of a new realm for me and while I know what I want it
> >>to do
> >> > and the structure I want to put it in, the conversion process has been
> >> > eluding me so thanks for giving me some tools to look into.
> >> >
> >> > On Thu, Jun 18, 2015 at 1:04 PM, Eric Lease Morgan <[log in to unmask]>
> >> wrote:
> >> >
> >> > > On Jun 18, 2015, at 12:02 PM, Matt Sherman
> >><[log in to unmask]>
> >> > > wrote:
> >> > >
> >> > > > I am working with colleague on a side project which involves some
> >> > scanned
> >> > > > bibliographies and making them more web
> >> > searchable/sortable/browse-able.
> >> > > > While I am quite familiar with the metadata and organization
> >>aspects
> >> we
> >> > > > need, but I am at a bit of a loss on how to automate the process
> >>of
> >> > > putting
> >> > > > the bibliography in a more structured format so that we can avoid
> >> going
> >> > > > through hundreds of pages by hand.  I am pretty sure regular
> >> > expressions
> >> > > > are needed, but I have not had an instance where I need to
> >>automate
> >> > > > extracting data from one file type (PDF OCR or text extracted to
> >>Word
> >> > > doc)
> >> > > > and place it into another (either a database or an XML file) with
> >> some
> >> > > > enrichment.  I would appreciate any suggestions for approaches or
> >> tools
> >> > > to
> >> > > > look into.  Thanks for any help/thoughts people can give.
> >> > >
> >> > >
> >> > > If I understand your question correctly, then you have two problems
> >>to
> >> > > address: 1) converting PDF, Word, etc. files into plain text, and 2)
> >> > > marking up the result (which is a bibliography) into structure data.
> >> > > Correct?
> >> > >
> >> > > If so, then if your PDF documents have already been OCRed, or if you
> >> have
> >> > > other files, then you can probably feed them to TIKA to quickly and
> >> > easily
> >> > > extract the underlying plain text. [1] I wrote a brain-dead shell
> >> script
> >> > to
> >> > > run TIKA in server mode and then convert Word (.docx) files. [2]
> >> > >
> >> > > When it comes to marking up the result into structured data, well,
> >>good
> >> > > luck. I think such an application is something Library Land sought
> >>for
> >> a
> >> > > long time. ³Can you say Holy Grail?"
> >> > >
> >> > > [1] Tika - https://tika.apache.org
> >> > > [2] brain-dead script -
> >> > > https://gist.github.com/ericleasemorgan/c4e34ffad96c0221f1ff
> >> > >
> >> > > ‹
> >> > > Eric
> >> > >
> >> >
> >>
>