On Fri, Jul 29, 2016 at 3:33 PM, Trail, Nate <[log in to unmask]> wrote:

> Ha.
>
> I don't have a good answer for that. I think It must vary from day to day,
> based on what our security provider software deems "insecure" at the
> moment, and they wouldn't tell us if we asked.
>

I've dealt with such things with my previous employer's enterprise's own
'security provider software'.

It's actually totally an impossible situation when it comes to automation
like the schemas are meant for. To have no specs for what requests are
allowed and what are not, and for it to change from day to day, meaning
your automation processes could break from day to day. The 'security
provider software' is probably focused on ordinary browser access, meaning
the requirements it imposes, changing from day to day, will not take
account of automated non-browser user-agents, which differ from
human-triggered browser user-agents in several ways, the user-agent string
being the most notable and obvious but not the only one.

I understand this is something out of the control of the unit that is
actually responsible for the LC-hosted standards documents that are
intended for automated access.

But it's probably time people start considering mirroring them at a more
reliable location.

Jonathan



>
> Nate
>
> -----------------------------------------
> Nate Trail
> Network Development & MARC Standards Office
> LS/ABA/NDMSO
> LA308, Mail Stop 4402
> Library of Congress
> Washington DC 20540
>
>
>
> -----Original Message-----
> From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of
> Gorman, Jon
> Sent: Friday, July 29, 2016 3:30 PM
> To: [log in to unmask]
> Subject: Re: [Code4Lib] Schema Validations at Library of Congress
>
> > For Security Reasons, the Library of Congress has begun filtering
> > (blocking) HTTP requests that do not express a userAgent in the header.
>
> Curiosity compels me to ask, is there a whitelist of user agents allowed?
> Or is it just the presence of any user agent, even something like
> "RadHackerzTotalAnnoyanceDDOSmytotalllyrandomstringperrequest", allowed?
>
>
> Jon Gorman
> Library IT
> University of Illinois
> 217 244-4688
>