While I think the author draws to strong of a line between Open Source 
and Closed Source, there is a good book about evaluating Open source 
software by Bernard Golden called "Succeeding with open source" [1].

Edward

[1] http://www.worldcat.org/oclc/55124574


Brett Bonfield wrote:
> I really like this topic, and I like how you're thinking about it. I
> tried to ask similar questions in an article I published in July:
> http://inthelibrarywiththeleadpipe.org/2009/w-e-b-s-i-t-e-find-out-what-it-means-to-me/
>
> I think Jonathan and Nicole nailed it with community health, though
> this leads to an additional consideration that I think is more nuanced
> than the application/system vs library/module distinction. Scriblio
> and SOPAC are built on top of very healthy (from a developer community
> perspective) software that has been created with moderately technical
> end-users in mind.
>
> This also gets back to Jonathan's very good generalization of your
> point about LAMP: "What are its requirements and level of difficulty
> for deployment?" When the first few steps are as comparatively easy
> for non-developers as a "Five Minute WordPress Install," I think that
> has to count for something.
>
> Brett Bonfield
>
> On Mon, Dec 28, 2009 at 2:21 PM, Nicole Engard <[log in to unmask]> wrote:
>   
>> I'm with Jonathan on the community health, one of the things I stress
>> when teaching my open source classes is that the developer and user
>> community is essential to the success and life of the product.
>>
>> Nicole C. Engard
>>
>> On Mon, Dec 28, 2009 at 12:51 PM, Jonathan Rochkind <[log in to unmask]> wrote:
>>     
>>> Quality of code in general:   How well-designed is the code architecture,
>>> for maintenance and debugging?   [This not only matters if you plan to do
>>> in-house development with it, but matters for predicting how likely the
>>> product is to stay 'alive' and continue to evolve with the times, instead of
>>> you just being stuck with exactly the version you first installed forever.]
>>>
>>> Developer Community: Is there a developer community around this software,
>>> with multiple people from multiple institutions contributing? Or is it just
>>> one founder maintaining it?  [One founder maintaining it _can_ work fine, as
>>> long as that founder keeps maintaining it. MarcEdit is a great example.  But
>>> the more of a community there is, again, the higher the reliability that the
>>> software will continue to evolve in the future, even if the founder bows out
>>> for some reason. ]
>>>
>>> A related topic:  Do individual institutions do extensive local
>>> customization to core code, which does not end up merged back into the
>>> 'main' distribution?  Again, this effects long-term sustainability of the
>>> software.
>>> I wrote a bit on judging one aspect of open source in a Library Journal
>>> article here: http://www.libraryjournal.com/article/CA6611591.html
>>> I also compiled some opinions from me, Bill Dueber, and others, in what
>>> 'good code' looks like in open source here:
>>> http://wiki.code4lib.org/index.php/Category:Patterns
>>>
>>> I could also pick nits with some of your criteria, but, hey, if they're
>>> important to someone they're important to someone. Some of htem are less
>>> important to me (For instance: "Is it deployed on LAMP" I'd generalize to
>>> "what are it's requirements and level of difficulty for deployment?"  We are
>>> quite capable of deploying non-PHP solutions, but that doesn't mean that all
>>> non-PHP solutions are equal for ease of deployment either!.  )
>>>
>>> Eric Lease Morgan wrote:
>>>       
>>>> What qualities and characteristics make for a "good" piece of open source
>>>> software? And once that question is answered, then what pieces of
>>>> library-related open source software can be considered "best"?
>>>>
>>>> I do not believe there is any single, most important characteristic of
>>>> open source software that qualifies it to be denoted as "best". Instead, a
>>>> number of characteristics need to be considered. For example, a program
>>>> might do one thing and do it well, but if it is bear to install then that
>>>> counts against it. Similarly, some software might work wonders but it is
>>>> built on a proprietary infrastructure such as a closed source compiler. Can
>>>> that software really be considered "open"?
>>>>
>>>> For my own education and cogitation, I have begun to list questions to
>>>> help me address what I think is the "best" library-related open source
>>>> software. [1] Your comments would be greatly appreciated. I have listed the
>>>> questions here in (more or less) personal priority order:
>>>>
>>>>  * Does the software work as advertised?
>>>>  * To what degree is the software supported?
>>>>  * Is the documentation thorough?
>>>>  * What are the licence terms?  * To what degree is the software easy to
>>>> install?
>>>>  * To what degree is the software implemented
>>>>    using the "standard" LAMP stack?
>>>>  * Is the distribution in question an
>>>>    application/system or a library/module?
>>>>  * To what degree does the software satisfy some
>>>>    sort of real library need?
>>>>  What sorts of things have I left out? Is there anything here that can be
>>>> measurable or is everything left to subjective judgement? Just as
>>>> importantly, can we as a community answer these questions in light of
>>>> distributions to come up with the "best" of class?
>>>>
>>>> 'More questions than answers.
>>>>
>>>> [1] There are elaborations on the questions in a blog posting. See:
>>>> http://tinyurl.com/ybk2bef
>>>>
>>>>
>>>>