 |
 |
+1 for OpenRefine. Exactly what it is made to do. Chad On Fri, Mar 21, 2014 at 2:29 PM, Roy Tennant <[log in to unmask]> wrote: > Have you considered dumping it into Open Refine? [1] I haven't used it a > lot, but it is likely a good tool to find similar data and allow you to > globally replace with a canonical entry. > Roy > > [1] http://openrefine.org/ > > > On Fri, Mar 21, 2014 at 11:24 AM, Ken Irwin <[log in to unmask]> wrote: > > > Hi folks, > > > > I'm looking for a tool that can look at a list of all of subject terms in > > a poorly-controlled index as possible candidates for term consolidation. > > Our student newspaper index has about 16,000 subject terms and they > include > > a lot of meaningless typographical and nomenclatural difference, e.g.: > > > > Irwin, Ken > > Irwin, Kenneth > > Irwin, Mr. Kenneth > > Irwin, Kenneth R. > > > > Basketball - Women > > Basketball - Women's > > Basketball-Women > > Basketball-Women's > > > > I would love to have some sort of pattern-matching tool that's smart > about > > this sort of thing that could go through the list of terms (as a text > list, > > database, xml file, or whatever structure it wants to ingest) and spit > out > > some clusters of possible matches. > > > > Does anyone know of a tool that's good for that sort of thing? > > > > The index is just a bunch of MySQL tables - there is no real > > controlled-vocab system, though I've recently built some systems to > suggest > > known SH's to reduce this sort of redundancy. > > > > Any ideas? > > > > Thanks! > > Ken > > >