I have ~800,000 MARC records from an indexing service (http://natlib.govt.nz/about-us/open-data/innz-metadata CC-BY). I am trying to generate:

(a) a list of person authorities (and sundry metadata), sorted by how many times they're referenced, in wikimedia syntax

(b) a view of a person authority, with all the records by which they're referenced, processed into a wikipedia stub biography

I have established that this is too much data to process in XSLT or multi-line regexps in vi. What other MARC engines are there out there?

The two options I'm aware of are learning multi-line processing in sed or learning enough koha to write reports in whatever their reporting engine is.

Any advice?

cheers
stuart
--
I have a new phone number: 04 463 5692