Thanks for all the ideas!

I'm a exploring a few of them this morning.

Patrick Galligan
Rockefeller Archive Center
Assistant Digital Archivist
914-366-6386

-----Original Message-----
From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of Ted Lawless
Sent: Monday, September 29, 2014 9:43 AM
To: [log in to unmask]
Subject: Re: [CODE4LIB] Reconciling corporate names?

OCLC's FAST contains corporate names derived from LCSH.

http://www.oclc.org/research/activities/fast.html

I wrote a simple proxy to the FAST API that can be used as reconciliation endpoint in OpenRefine.

https://github.com/lawlesst/fast-reconcile

Ted

On Mon, Sep 29, 2014 at 9:37 AM, Simon Brown <[log in to unmask]> wrote:
> You could always web scrape, or download and then search the LCNAF 
> with some script that looks like:
>
> #Build query for webscraping
> query = paste("http://id.loc.gov/search/?q=", URLencode("corporate 
> name here "), "&q=cs%3Ahttp%3A%2F%2Fid.loc.gov%2Fauthorities%2Fnames")
>
> #Make the call
> result = readLines(query)
>
> #Find the lines containing "Corporate Name"
> lines = grep("Corporate Name, result)
>
> #Alternatively use approximate string matching on the downloaded LCNAF 
> data query <- agrep("corporate name here",LCNAF_data_here)
>
> #Parse for whatever info you want
> ...
>
> My native programming language is R so I hope the functions like 
> paste, readLines, grep, and URLencode are generic enough for other 
> languages to have some kind of similar thing.  This can just be 
> wrapped up into a for
> loop:
> for(i in 1:40000){...}
>
> Web scraping the results of one name at a time would be SLOW and 
> obviously using an API is the way to go but it didn't look like the 
> OCLC LCNAF API handled Corporate Name.  However, it sounds like in the 
> previous message someone found a work around.  Best of luck! -Simon
>
>
>
>
>
>
> On Mon, Sep 29, 2014 at 8:45 AM, Matt Carruthers <[log in to unmask]> wrote:
>
>> Hi Patrick,
>>
>> Over the last few weeks I've been doing something very similar.  I 
>> was able to figure out a process that works using OpenRefine.  It 
>> works by searching the VIAF API first, limiting results to anything 
>> that is a corporate name and has an LC source authority.  OpenRefine 
>> then extracts the LCCN and puts that through the LCNAF API that OCLC 
>> has to get the name.  I had to use VIAF for the initial name search 
>> because for some reason the LCNAF API doesn't really handle corporate 
>> names as search terms very well, but works with the LCCN just fine 
>> (there is the possibility that I'm just doing something wrong, and if 
>> that's the case, anyone on the list can feel free to correct me).  In 
>> the end, you get the LC name authority that corresponds to your 
>> search term and a link to the authority on the LC Authorities website.
>>
>> Anyway,  The process is fairly simple to run (just prepare an Excel 
>> spreadsheet and paste JSON commands into OpenRefine).  The only 
>> reservation is that I don't think it will run all 40,000 of your 
>> names at once.  I've been using it to run 300-400 names at a time.  
>> That said, I'd be happy to share what I did with you if you'd like to 
>> try it out.  I have some instructions written up in a Word doc, and 
>> the JSON script is in a text file, so just email me off list and I can send them to you.
>>
>> Matt
>>
>> Matt Carruthers
>> Metadata Projects Librarian
>> University of Michigan
>> 734-615-5047
>> [log in to unmask]
>>
>> On Fri, Sep 26, 2014 at 7:03 PM, Karen Hanson 
>> <[log in to unmask]>
>> wrote:
>>
>> > I found the WorldCat Identities API useful for an institution name 
>> > disambiguation project that I worked on a few years ago, though my 
>> > goal wasn't to confirm whether names mapped to LCNAF.  The API 
>> > response
>> includes
>> > a LCCN, and you can set it to fuzzy or exact matching, but you 
>> > would need to write a script to pass each term in and process the results:
>> >
>> >
>> http://oclc.org/developer/develop/web-services/worldcat-identities.en
>> .html
>> >
>> > I also can't speak to whether all LC Name Authorities are 
>> > represented, so there may be a chance of some false negatives.
>> >
>> > OCLC has another API, but not sure if it covers corporate names:
>> > https://platform.worldcat.org/api-explorer/LCNAF
>> >
>> > I suspect there are others on the list that know more about the 
>> > inner workings of these APIs if this might be an option for you... 
>> > :)
>> >
>> > Karen
>> >
>> > -----Original Message-----
>> > From: Code for Libraries [mailto:[log in to unmask]] On 
>> > Behalf Of Ethan Gruber
>> > Sent: Friday, September 26, 2014 3:54 PM
>> > To: [log in to unmask]
>> > Subject: Re: [CODE4LIB] Reconciling corporate names?
>> >
>> > I would check with the developers of SNAC ( 
>> > http://socialarchive.iath.virginia.edu/), as they've spent a lot of 
>> > time developing named entity recognition scripts for personal and 
>> > corporate names. They might have something you can reuse.
>> >
>> > Ethan
>> >
>> > On Fri, Sep 26, 2014 at 3:47 PM, Galligan, Patrick <
>> [log in to unmask]
>> > >
>> > wrote:
>> >
>> > > I'm looking to reconcile about 40,000 corporate names against 
>> > > LCNAF to see whether they are authorized strings or not, but I'm 
>> > > drawing a blank about how to get it done.
>> > >
>> > > I've used http://freeyourmetadata.org/ for reconciling subject 
>> > > headings before, but I can't get it to work for LCNAF. Has anyone 
>> > > had any experience in a project like this? I'd love to hear some 
>> > > ideas for automatically dealing with a large data set like this 
>> > > that we did not create and do not know how the names were created.
>> > >
>> > > Thanks!
>> > >
>> > > -Patrick Galligan
>> > >
>> >
>>
>
>
>
> --
> Simon Brown
> [log in to unmask]
> simoncharlesbrown (Skype)
> 831.440.7466 (Phone)
>
> *Following our will and wind we may just go where no one's been -- 
> MJK*