 |
 |
Great input....thanks for the discussion. A few notes on what we're seeing on our end: a) We've just finished testing a content-based relevance ranking method with users; it works well for academic users, and will probably work better with some tweaking of field weights. I'll have more specific results in a month or two. b) We experimented with "boosting" some documents using holdings data. OCLC gave us both WorldCat-wide and UC-wide numbers to work with. David's observation is right on...the mix in WorldCat skews things for academic libraries, so we rejected that and used UC system-wide numbers in our experiments. And I do agree with Jonathan's excellent summary...summing the scores is not the right approach. And yet...I wish I could explain why it seems as though the clustering can tell us something. --Colleen David Walker wrote: >The only tricky thing about this with WorldCat, though, is that you have >such a large mix of libraries. > >In my own searching on WorldCat, I've noticed that a fair amount of >fiction and non-scholarly works appear near the top of results because >the public libraries are skewing the holdings of those titles. > >Not a bad thing in itself, if that's what I'm looking for, but our >students are looking for scholarly works (and still learning to >distinguish scholarly from not), so would be nice in our particular >context to limit only to academic libraries that own the title. > >--Dave > >========================= >David Walker >Web Development Librarian >Library, Cal State San Marcos >760-750-4379 >http://public.csusm.edu/dwalker >========================= > > > > > >-----Original Message----- >From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of >Hickey,Thom >Sent: Monday, April 10, 2006 12:52 PM >To: [log in to unmask] >Subject: Re: [CODE4LIB] Question re: ranking and FRBR > >I'd agree with this. > >Actually, though, 'relevancy' ranking based on where terms occur in the >record and how many times they occur is of minor help compared to some >sort of popularity score. WorldCat holdings work fairly well for that, >as should circulation data. The primary example of this sort of ranking >is the web search engines where ranking is based primarily on word >proximity and links. > >--Th > > >-----Original Message----- >From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of >Jonathan Rochkind >Sent: Monday, April 10, 2006 3:16 PM >To: [log in to unmask] >Subject: Re: [CODE4LIB] Question re: ranking and FRBR > >When you are ranking on number of holdings like OCLC is, a straight >sum makes sense to me---the sum of all libraries holding copies of >any manifestation of the FRBR work is indeed the sum of the holdings >for all the records in the FRBR work set. Of course. > >If you're doing relavancy rankings instead though, a straight sum >makes less sense. A relevancy ranking isn't really amenable to being >summed. The sum of the relevancy rankings for various >manifestations/expressions is not probably not a valid indicator of >how relevant the work is to the user, right? And if you did it this >way, it would tend to make the most _voluminous_ work always come out >first as the most 'relevant', which isn't quite right.---This isn't >quite the same problem as OCLC's having the bible come out on >top---since OCLC is ranking by holdings, it's exactly right to have >the bible come out on top, the Bible is indeed surely one of the >(#1?) most held works, so it's quite right for it to be on top. But >the bible isn't always going to be the most relevant result for a >user, just because it's the most voluminous! Summing is going to >mess up your relevancy rankings. > >Just using the maximum relevancy ranking from the work set seems >acceptable to me--the work's relevancy to the user is indicated by >the most relevant manifestation in the set. There might be a better >way to do it (Is a work with four manifestations with a relevancy >ranking .7 more relevant than a work with just one manifestation with >a ranking of .9? I don't think it probably is, actually; I think >just taking the maximum should work fine. But it depends on the >relevancy algorithm maybe.). I don't think I'm enough of a >mathematician to understand the point of the log of the sum, though, >hmm. > >--Jonathan > >At 2:38 PM -0400 4/10/06, Hickey,Thom wrote: > > >>We're doing straight sums of the holdings of all the manifestations in >>the work. It's hard for me to see the need to discount holdings in >>multiple manifestations. It does mean that 'bible' tends to come to >> >> >the > > >>top for many searches, but that's about the only work-set I see coming >>up unexpectedly to the top. >> >>If we had circulation data we'd certainly factor that in (or maybe just >>use it if it was comprehensive enough). >> >>--Th >> >> >>-----Original Message----- >>From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of >>Colleen Whitney >>Sent: Monday, April 10, 2006 2:04 PM >>To: [log in to unmask] >>Subject: Re: [CODE4LIB] Question re: ranking and FRBR >> >>Thanks...is it just a straight sum, Thom? >> >>--C >> >>Hickey,Thom wrote: >> >> >> >>>Here at OCLC we're ranking based on the holdings of all the records in >>>the retrieved work set. Seems to work pretty well. >>> >>>--Th >>> >>>-----Original Message----- >>>From: Code for Libraries [mailto:[log in to unmask]] On Behalf >>> >>> >Of > > >>>Colleen Whitney >>>Sent: Monday, April 10, 2006 1:06 PM >>>To: [log in to unmask] >>>Subject: [CODE4LIB] Question re: ranking and FRBR >>> >>>Hello all, >>> >>>Here's a question for anyone who has been thinking about or working >>> >>> >>with >> >> >>>FRBR for creating record groupings for display. (Perhaps others have >>>already discussed or addressed this...in which case I'd be happy to >>> >>> >>have >> >> >>>a pointer to resources that are already out there.) >>> >>>In a retrieval environment that presents ranked results (ranked by >>>record content, optionally boosted by circulation and/or holdings), >>> >>> >how > > >>>could/should FRBR-like record groupings be factored into ranking? >>>Several approaches have been discussed here: >>> - Rank the results using the score from the highest-scoring record >>> >>> >in > > >>a >> >> >>>group >>> - Use the sum of scores of documents in a group (this seems to me to >>>place too much weight on the group) >>> - Use the log of the sum of the scores of documents in a group >>> >>>I'd be very interested in knowing whether others have already been >>>thinking about this.... >>> >>>Regards, >>> >>>--Colleen Whitney >>> >>> >>> >>>