This conversation is heading into the "draining the swamp" category.
Bill Denton started this thread with the suggestion that we use HTTPS everywhere. He did not make a specific case for it. I am just guessing that an argument for going that route would include security.
Regardless of whether this is a good idea, or whether there is a compelling reason for doing it, it seems to me that the possibility of its making it difficult for older scraping tools to scrape the site does not seem like a compelling reason not to do it.
The cost issue, on the other hand, would be a more compelling consideration.
On Nov 6, 2013, at 6:17 PM, Ross Singer <[log in to unmask]> wrote:
> How is security getting thrown under the bus?
> On Wednesday, November 6, 2013, Cary Gordon wrote:
>> It sounds like we are willing to throw security under the bus for an edge
>> case, although I am sure that I am missing some subtlety
>>>> (Question: Why does HTTPS complicate screen-scraping? Every decent
>>>> and library supports HTTPS, doesn't it?)
>>> Birkin asked me this same question, and I realized I should clarify what
>>> meant. I was mostly referring to existing screen scrapers/existing web
>>> sites. If you redirect every request from http to https, this will
>>> probably break things. I think the Open Library example that Karen
>>> mentioned is a good case study.
>>> And it's pretty different for a library or tool to support HTTPS and a
>>> specific app to be expecting it. If you follow the thread around that OL
>>> change, it appears there are issues with Java (as one example)
>>> consuming HTTPS (from what I understand, you need to have the cert
>>> locally?), but I don't know enough about it to say for certain. I think
>>> there would also probably be potential issues around mashups (AJAX, for
>>> example), but seeing as code4lib.org doesn't support CORS, not really a
>>> current issue. Does apply more generally to your question about library
>>> websites at large, though.
>>> Anyway, I agree with you that the option for both should be there. I'm
>>> just not convinced that HTTPS-all-the-time is necessary for all web use