Hi Kyle,

You might want to have a look at our Diva viewer (http://ddmal.music.mcgill.ca/diva/).

We've tested it on books that are over 100GB total, and images that are around 200MB each. For example:

http://coltrane.music.mcgill.ca/salzinnes/experiments/diva-cci-tif/

Each page is about 180MB each (uncompressed TIFF).

Here's some features:

-- Supports JPEG2000 and Pyramid TIFF via the IIP Image Server.
-- Almost immediate viewing. You're only downloading the parts of the page that you're seeing, so even if a book is huge, you won't download it until you need it. So you don't need to try to do tons of compression or conversion to greyscale to try and get file size down.
-- Multiple zoom levels per page so you can get a very detailed look, or zoom out to quickly navigate the pages.
-- Grid layout for even faster navigation.
-- Easily create links to very specific parts of a page (e.g., http://coltrane.music.mcgill.ca/salzinnes/experiments/diva-cci-tif/#f=true&z=5&n=5&i=salz-1-002-recto.tif&y=5276&x=-1075)
-- We have a nifty HTML5 canvas view that lets you do some basic image manipulation in the browser (rotate, brightness, contrast, colour channel manipulation). Above each page there's a little gear icon; clicking this will take you to the image manipulator. Your manipulations are also stored locally, so you can return to the page with your modifications intact. We did this because our scholars wanted to view things like marginalia, or increase the contrast for faded inks.
-- You can integrate it into an existing page or digital collection.
-- Lots of public hooks for tying it in to other scripts, and a simple plugin API for extending it.

And, it's all open source. https://github.com/DDMAL/diva.js

Hope that helps.
-Andrew

On 2013-02-23, at 7:51 AM, Kyle Banerjee <[log in to unmask]> wrote:

>> I did PDF.  There are about no studies on PDF size and usability.  What I
>> did is go to gray scale for text pages to knock down file size, played with
>> optimizing, and broke super long (think 3K page book) files in smaller
>> chunks....
>> 
>> When I looked at other big long books online, I found they tended to use
>> 300 dpi gray scale or 600 dpi black and white.  I just looked at government
>> documents, because that's what I worked with.
>> 
> 
> Hadn't thought about doing government reports. That's a good use case for
> what we have -- one of ours got downloaded about 7,000 times last month
> alone. I could probably make those really manageable in size as most pages
> would be just fine bitonal.
> 
> Can you say a bit about what you discovered while playing with
> optimization? Since archival books are a prime target, fidelity is
> important -- there is significant artwork on many  pages. I'll have to
> scale down and would really like to avoid grayscaling if necessary.
> 
> kyle