On Mon, Oct 3, 2011 at 8:55 AM, Ed Summers <[log in to unmask]> wrote:
> Respecting robots.txt is key. It's normally easy to eyeball, but when
> in doubt something like robotparser  can be helpful. A bit of
> reflection on the Golden Rule probably is probably more important than
> pondering the legality of what you are doing.