On 1/14/14 10:45 PM, Edward Summers wrote:
> Just out of curiosity, does it work for a little bit then stop working? I know arXiv throttle crawlers, and am not sure if they throttle oai-pmh clients. Simeon Warner who helps run arXiv has been know to post code4lib, so maybe this will cross his radar.

The arXiv OAI endpoint at uses 503 
responses [1] control request frequency. I think most harvester 
libraries support this OK.

I'm sad to say that our server is a bit overloaded at 
the moment (mainly arXiv API and RSS load) and sometimes this affects 
the OAI-PMH performance. We are working on improving performance to 
handle the ever increasing load...



> In the meantime, could you share your harvesting script on or somewhere similar for us to take a look?
> //Ed
> On Jan 14, 2014, at 4:46 PM, Eka Grguric <[log in to unmask]> wrote:
>> Thanks for responding!
>> I initialized it as follows (following the code from the synopsis on the site).
>> my $harvester = Net::OAI::Harvester->new(
>> 		baseURL => ''
>> 	 );