41,000 sites and 21 million pages ( is a lot of

An admittedly quick check of the site didn't explain the vetting process to me, but did
profess a "...background in search technology..."  

Authoratative vetting of a large volume of resources is a hard problem.  I haven't seen
any good solutions, but am leaning toward crowd-sourcing with an authoratative crowd. :-)

Do you have any additional information on how AbleGrape vets these?

Tim Cornwell
National Science Digital Library (

> -----Original Message-----
> From: Code for Libraries [mailto:[log in to unmask]] On 
> Behalf Of Keith Jenkins
> Sent: Tuesday, September 29, 2009 5:35 PM
> To: [log in to unmask]
> Subject: Re: [CODE4LIB] Bookmarking web links - 
> authoritativeness or focused searching
> is a good example of a focused search engine that aims
> to index only "authoritative" sources within a particular disciple --
> in this case it's wine, enology, and viticulture.  It currently crawls
> about 40,000 vetted websites.
> It's a great search engine for the subject area it serves, and it
> probably helped that the creator was a VP at Inktomi.
> Keith
> On Tue, Sep 29, 2009 at 10:53 AM, Cindy Harper 
> <[log in to unmask]> wrote:
> > So that led me to speculate about a search engine that 
> ranked just by links
> > from .edu's, libraries sites, and a librarian-vetted list of .orgs,
> > scholarly publishers, etc.  I think you can limit by .edu 
> in the linked-from
> > in Google - I haven't tried that much. if anyone here has 
> experience at
> > using tha technique, I'd like to hear about it.  But I'm 
> thinking now about
> > the possibility of a search engine limited to sites 
> cooperatively vetted by
> > librarians, that would incorporate ranking by # links.  
> Something more
> > responsive than cataloging websites in our catalogs.
> >
> > Is anyone else thinking about these ideas?  or do you know 
> of projects that
> > approach this goal of leveraging librarian's vetting of 
> authoritative
> > sources?