This is really interesting. Do you have evidence (anecdotally or otherwise) that the people coming to you via search engines found what they were looking for? Sorry, I don't know exactly how to phrase this. To put it another way - are your patrons finding you this way? wayne -----Original Message----- From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of Sean Hannan Sent: Thursday, February 23, 2012 12:37 PM To: [log in to unmask] Subject: Re: [CODE4LIB] Local catalog records and Google, Bing, Yahoo! Our Blacklight-powered catalog (https://catalyst.library.jhu.edu/) comes up a lot in google search results (try gil scott heron circle of stone). Some numbers: 59% of our total catalog traffic comes from google searches 0.04% of our total catalog traffic comes from yahoo searches 0.03% of our total catalog traffic comes from bing searches For context, 32.96% of our total catalog traffic is direct traffic and referrals from all of the library websites combined. Anecdotally, it would appear that bing (and bing-using yahoo) seem to drastically play down catalog records in their results. We're not doing anything to favor a particular search engine; we have a completely open robots.txt file. Google regularly indexes our catalog. Every couple days or so. I haven't checked in awhile. We're not doing any fancy SEO here (though, I'd like to implement some of the microdata stuff). It's just a function of how the site works. We link a lot of our catalog results to further searches (clicking on an author name takes you to an author search with that name, etc). Google *loves* that type of intertextual website linking (see also: Wikipedia). We also have stable URLs. Search URLs will always return searches with those parameters, item URLs are based on an ID that does not change. All of that good stuff doesn't help us with bing, though. ...But I'm not really concerned with remedying that, right this moment. -Sean On 2/23/12 12:37 PM, "[log in to unmask]" <[log in to unmask]> wrote: > First of all, I'm going to say I know little in this area. I've done > some preliminary research about search indexing (Google's) and > investigated a few OPAC robot.txt files. Now to my questions: > > - Can someone explain to me or point me to research as to why local > library catalog records do not show up in Google, Bing, or Yahoo! search > results? > - Is there a general prohibition by libraries for search engines to > crawl their public records? > - Do the search engines not index these records actively? > - Is it a matter of SEO/promoted results? > - Is it because some systems don't mint URLs for each record? > > I haven't seen a lot of discussion about this recently and I know > Jason Ranallo has done a lot of work in this area and gave a great > talk at code4lib Seattle on microdata/Schema.org, so I figured this > could be part of that continuing conversation. > > I look forward to being educated by you all, > > Tod