I suspect the tool wil only be able to handle select languages, and very
doubtful you could develop a tool to handle non-LCG text.
For a fully internationalised tool, you would have fo ignore all text
layers in a PDF and run all PDFs through OCR to generate text.
Then you'd need to apply very sophisticated word boundary identification
On 12/10/2013 9:40 AM, "Mark Pernotto" <[log in to unmask]> wrote:
> Very cool tool, thank you!
> Putting my devil's advocate hat on, it doesn't parse foreign documents well
> (I got it to break!). I also got inconsistent results feeding it PDF files
> with tables embedded (but haven't been able to figure out what it is about
> them it doesn't like).
> Just from a curiosity standpoint, what encoding is being utilized? I know
> nothing about Perl. It seemed to have no problem parsing a dash (-) if it
> was up against another character (2007-2012), but barfs when it's by itself
> (2007 � 2012). I'm only referring to 'extracted text' mode.
> If it helps, I can send along *most* of my test PDF files used.
> Thank you!
> On Fri, Oct 11, 2013 at 10:58 AM, Eric Lease Morgan <[log in to unmask]>
> > On Oct 11, 2013, at 1:49 PM, Matthew Sherman <[log in to unmask]>
> > wrote:
> > >> For a limited period of time I am making publicly available a
> > >> program called PDF2TXT -- http://bit.ly/1bJRyh8
> > >
> > > Very slick, good work. I can see where this tool can be very helpful.
> > It
> > > does have some issues with some characters, but this is rather common
> > with
> > > most systems.
> > Again, thank you for the support. Yes, there are some escaping issues to
> > be resolved. "Release early. Release often." I need help with the graphic
> > design in general.
> > Here's an enhancement I thought of:
> > 1. allow readers to authenticate
> > 2. allow readers to upload documents
> > 3. documents get saved in readers' cache
> > 4. allow interface to list documents in the cache
> > 5. provide text mining services against reader-selected documents
> > 6. go to Step #1
> > It would also be cool if I could figure out how to finish the
> > of Tesseract to enable OCRing. 
> >  OCRing -
> > http://serials.infomotions.com/code4lib/archive/2013/201303/1554.html
> > --
> > Eric Morgan