the.com/scrape

Taking a website's data without asking, then calling it research.

means To extract or harvest data (often from a website) automatically, typically without explicit permission from whoever put it there.

from The word 'scrape' goes back to Old English 'scrapian,' meaning to scratch or grate with something sharp — the same root gave us the literal act of scraping a pan or scraping your knee. The digital sense arrives much later, extending the physical image of dragging something hard across a surface to collect whatever comes loose: a 'scraper' tool drags across a webpage's code and pulls off the data clinging to it. The metaphor works because it's a little violent and a little crude — scraping isn't asking nicely, it's taking by friction.

legal status
public data scraping is mostly legal in the US
arms race
sites build walls, scrapers build ladders, forever
irony
Google scrapes everyone, then bans you for scraping Google
tell
your rate limit exists because someone already did this
the.com · every word