 |
 |
On Mon, Oct 3, 2011 at 8:55 AM, Ed Summers <[log in to unmask]> wrote: > Respecting robots.txt is key. It's normally easy to eyeball, but when > in doubt something like robotparser [1] can be helpful. A bit of > reflection on the Golden Rule probably is probably more important than > pondering the legality of what you are doing. s/probably is/is/ //Ed