 |
 |
Hi Athina, The extractors are very different in terms of what they're optimized to work with and what they're designed to extract -- you need one designed for your purposes, and you may need more than one. A few years back, I experimented with a number of extractors before settling on Alchemy as best suited for the purpose at hand -- namely analyzing transcriptions of interviews to identify topical, place, and person access points. A few observations: 1. There was enormous variation in output from various extractors, and effectiveness varied dramatically depending on what we tested them against. Some are good at what they're designed to do, and none were any good at what they weren't designed to do. For example, automatic analysis of EAD finding aids was uniformly abysmal. 2. What you think is important and what an extractor thinks is important are different. Highly important terms often appear infrequently while irrelevant terms often appear repeatedly -- so term recurrence is a poor indicator of importance. Unless you let the extractor decide what's important, you'll get so much output that you'll need a postprocessing routine (e.g. comparison of terms against a list or at least pattern matching). 3. All extractors missed important terms and returned many irrelevant ones. However, enough important terms were returned for it to be worthwhile. Bottom line is that we found it a viable "quick 'n dirty" approach to generate lists of terms a human could quickly scan, but not good for anything else we were interested in. Good luck on your project, and I hope you'll share what you discover. kyle On Mon, Sep 16, 2019 at 9:20 AM Athina Livanos-Propst < [log in to unmask]> wrote: > Hi everyone, > > I'm starting to think around a project that would involve key terms from > other types of text (transcripts, captions, documents). I'm basically > trying to build a tool that I can use to extra key terms from larger > strings of text, i.e. pull out the important words from a larger sentence. > > Does anyone have recommendations for tools that already exist out there? > > Thanks, > Athina > > PS- I'm thinking specifically about state K-12 educational standards, if > anyone wants to talk more specifically with me. > > > Athina Livanos-Propst > Digital Librarian & Editorial Services, Manager | PBS Education > O: 703-739-5485 | [log in to unmask]<mailto:[log in to unmask]> >