> From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of
> Bill Dueber
> Sent: Friday, March 05, 2010 03:45 PM
> To: [log in to unmask]
> Subject: Re: [CODE4LIB] Q: XML2JSON converter
> 
> I guess my concern here is that the specification, as you're describing
> it, is closing off potential uses.  It seems fine if, for example, your
> primary concern is javascript-in-the-browser, and browser-request,
> pagination-enabled systems might be all you're worried about right now.
> 
> That's not the whole universe of uses, though. People are going to want
> to dump these things into a file to read later -- no possibility for
> pagination in that situation.

I disagree that you couldn't dump a paginated result set into a file for reading later.  I do this all the time not only in Javascript, but may other programming languages.

> Others may, in fact, want to stream a few thousand
> records down the pipe at once, but without a streaming parser that
> can't happen if it's all one big array.

Well, if your service isn't allowing them to be streamed a few thousand records at a time, then that isn't a issue :)

Maybe I have been mislead or misunderstood JSON streaming.  My understanding was that you can generate an arbitrary large outgoing stream on the server side and can read an arbitrary large incoming stream on the client side.  So it shouldn't matter if the result set was delivered as one big JSON array.  The SAX like interface that JSON streaming uses provides the necessary events to allow you to pull the individual records from that arbitrary large array.

> I worry that as specified, the *only* use will be, "Pull these down a
> thin pipe, and if you want to keep them for later, or want a bunch of
> them, you have to deal with marc-xml."

Don't quite follow this.  MARC-XML is an XML format, MARC-JSON is our JSON format for expressing the various MARC-21 format, e.g., authority, bibliographic, classification, community information and holdings in JSON.  The JSON is based on the structure of MARC-XML which was based on the structure of ISO 2709.  Don't see how MARC-XML comes into play when you are dealing with JSON.  If you want to save our MARC-JSON you don't have to convert it to MARC-XML on the client side.  Just save it as a text file.

> Part of my incentive is to *not* have to use marc-xml, but in this 
> case I'd just be trading one technology I don't like (marc-xml) 
> for two technologies, one of which I don't like (that'd be marc-xml 
> again).

Again not sure how to address this concern.  If you are dealing with library data, then its current communication formats are either MARC binary (ISO 2709) or MARC-XML, ignoring IFLA's MARC-XML-ish format for the moment.  You might not like it, but that is life in library land.  You can go develop your own formats based on the various MARC-21 format specifications, but are unlikely to achieve any sort of interoperability with the existing library systems and services.

We choose our MARC-JSON to maintain the structural components of MARC-XML and hence MARC binary (ISO 2709).  In MARC, control fields have different semantics from data fields and you cannot merge them into one thing called field.  If you look closely at the MARC-XML schema, you might notice that the controlfield and datafield elements can have non-numeric tags.  If you merge everything into something called field, then you cannot distinguish between a non-numeric tag for a controlfield vs. a datafield element.  There are valid reasons why we decided to maintain the existing structure of MARC.


Andy.