Quality of code in general:   How well-designed is the code 
architecture, for maintenance and debugging?   [This not only matters if 
you plan to do in-house development with it, but matters for predicting 
how likely the product is to stay 'alive' and continue to evolve with 
the times, instead of you just being stuck with exactly the version you 
first installed forever.]

Developer Community: Is there a developer community around this 
software, with multiple people from multiple institutions contributing? 
Or is it just one founder maintaining it?  [One founder maintaining it 
_can_ work fine, as long as that founder keeps maintaining it. MarcEdit 
is a great example.  But the more of a community there is, again, the 
higher the reliability that the software will continue to evolve in the 
future, even if the founder bows out for some reason. ]

A related topic:  Do individual institutions do extensive local 
customization to core code, which does not end up merged back into the 
'main' distribution?  Again, this effects long-term sustainability of 
the software. 

I wrote a bit on judging one aspect of open source in a Library Journal 
article here: http://www.libraryjournal.com/article/CA6611591.html
I also compiled some opinions from me, Bill Dueber, and others, in what 
'good code' looks like in open source here:
http://wiki.code4lib.org/index.php/Category:Patterns

I could also pick nits with some of your criteria, but, hey, if they're 
important to someone they're important to someone. Some of htem are less 
important to me (For instance: "Is it deployed on LAMP" I'd generalize 
to "what are it's requirements and level of difficulty for deployment?"  
We are quite capable of deploying non-PHP solutions, but that doesn't 
mean that all non-PHP solutions are equal for ease of deployment 
either!.  )

Eric Lease Morgan wrote:
> What qualities and characteristics make for a "good" piece of open source software? And once that question is answered, then what pieces of library-related open source software can be considered "best"?
>
> I do not believe there is any single, most important characteristic of open source software that qualifies it to be denoted as "best". Instead, a number of characteristics need to be considered. For example, a program might do one thing and do it well, but if it is bear to install then that counts against it. Similarly, some software might work wonders but it is built on a proprietary infrastructure such as a closed source compiler. Can that software really be considered "open"?
>
> For my own education and cogitation, I have begun to list questions to help me address what I think is the "best" library-related open source software. [1] Your comments would be greatly appreciated. I have listed the questions here in (more or less) personal priority order:
>
>   * Does the software work as advertised?
>   * To what degree is the software supported?
>   * Is the documentation thorough?
>   * What are the licence terms? 
>   * To what degree is the software easy to install?
>   * To what degree is the software implemented
>     using the "standard" LAMP stack?
>   * Is the distribution in question an
>     application/system or a library/module?
>   * To what degree does the software satisfy some
>     sort of real library need?
>   
> What sorts of things have I left out? Is there anything here that can be measurable or is everything left to subjective judgement? Just as importantly, can we as a community answer these questions in light of distributions to come up with the "best" of class?
>
> 'More questions than answers.
>
> [1] There are elaborations on the questions in a blog posting. See: http://tinyurl.com/ybk2bef
>
>