 |
 |
@Jason and @Michele: I'd rather stay away from a Google solution. The reason being that they don't index everything. Our sitemap is submitted nightly and out of about 6000 URLs only 1500 are indexed. I can't make sure Google indexes the PDFs or be sure that they always will. (If I'm misunderstanding this, please let me know.) @Péter: The VuFind solution I mentioned is very similar to what you use here. It uses Aperture (although soon to use Tika instead) to grab the full-text and shoves everything inside a solr index. The import is managed through a PHP script the crawls every URL on the sitemap. The only part I don't have is removing deleted, adding new, and updating changed webpages/files. I'm not sure how to rework the script to use a list of new files rather than the sitemap, but everything is on the same server so that should work. On Wed, Feb 20, 2013 at 12:53 PM, Nathan Tallman <[log in to unmask]> wrote: > My institution is looking for ways to provide search across PDFs through > our website. Specifically, PDFs linked from finding aids. Ideally searching > within a collection's PDFs or possibly across all PDFs linked from all > finding aids. > > We do not have a CMS or a digital repository. A digital repository is on > the horizon, but it's a ways out and we need to offer the search sooner. > I've looked into Swish-e but haven't had much luck getting anything off the > ground. > > One way we know we can do this through our discovery layer VuFind, using > it's ability to full-text index a website based on a sitemap (which would > includes PDFs linked from finding aids). Facets could be created for > collections, and we may be able to create a search box on the finding aid > nav that searches specifically that collection. > > But, I'm not sure how scalable that solution is. The indexing agent cannot > discern when a page was updated, so it has to re-scrape, > everything, every-night. The impetus collection is going to have about over > 1000 PDFs. And that's to start. Creating the index will start to take a > long, long time. > > Does anyone have any ideas or know of any useful tools for this project? > Doesn't have to be perfect, quick and dirty may work. (The OCR's dirty > anyway :-) > > Thanks, > Nathan > > > >