First off I’d like to give creds to Francis Brown and Rob Ragan who presented their talk Tenacious Diggity at Defcon20 where I found out about the apparent steam-rolling of Dropbox’s robots.txt file. For as far back as I can remember, the robots.txt file has been a ban-list of places that… →
