Matt,
There are probably a dozen ways to do this, but it would be really helpful to know what operating system you are on? For example, if you are using Linux, you can run it through sed using
cat <OCR_FILE> | sed 's/\n//' >> <STRIPPED_OCR_FILE>
see http://stackoverflow.com/a/800644/2896744 for more info
________________________________________
From: Code for Libraries [[log in to unmask]] On Behalf Of Matt Sherman [[log in to unmask]]
Sent: Monday, August 03, 2015 10:29 PM
To: [log in to unmask]
Subject: [CODE4LIB] Looking for Ideas on Line Breaks in OCR Text
Hi Code4Lib folks,
I was wondering if anyone had some experience cleaning up OCR text.
Particularly I am trying to figure out how I can deal with the random
line breaks that come from OCR. I am trying to parse out a
bibliography with regex. I think I've figured out which queries I
need to run to break it up so I can make it into a tab delimited text
file but I noticed that the text does the classic thing of OCR
inserting line breaks where they physically are on the page. This
will obviously be a bit of an issue since it would break the
annotation into a bunch of lines rather than leaving it one block so I
can manipulate it into a database. So I am wondering if anyone who
has worked with OCR text before has a suggested way to clean up those
line breaks without doing 300 + pages by hand? Any thoughts would be
welcome.
Matt Sherman
|