 |
 |
I often recommend against stop word removal altogether. Is there any reason you need to remove them? The primary reason stop words get removed is to increase performance of queries with very common terms. If you are encountering that, using Solr's CommonGramsFilter(Factory) is a good solution to keep your stop words and alleviate the performance degradation potential. The HathiTrust folks have had success with the common grams capability. Erik On Nov 11, 2009, at 3:41 PM, Eric James wrote: > Has anyone already given some thought into refining the solr > stopwords.txt for library collections, particularly finding aids? > The words included in the out of the box stopwords.txt are of very > questionable unimportance: > > <an and are as at be but by for if in into is it not of on or s such > t that the their then there these they this to was will with> > > > > We were indexing a field id with "no." as one of its tokens (for > number), but wanted a query with "no" (where the person did not add > the period) to find the doc, but in actuality the "no" would get > stripped by the StopFilterFactory. And thus we stumbled upon this > list, and was a bit suprised by some of the inclusions (ex:"will"), > and exclusions( ex:"a"). > > > > Thanks, > > Eric James > > Yale University Libraries >