

25·
20 days agoThe Wikimedia Foundation does just that, and still, their infrastructure is under stress because of AI scrapers.
Dumps or no dumps, these AI companies don’t care. They feel like they’re entitled to taking or stealing what they want.
The Wikimedia Foundation does just that, and still, their infrastructure is under stress because of AI scrapers.
Dumps or no dumps, these AI companies don’t care. They feel like they’re entitled to taking or stealing what they want.
Apps such as https://kiwix.org/ uses the data dumps regularly made available by the Wikimedia Foundation at https://dumps.wikimedia.org/.
The entire Wikipedia might be large, especially with images, but e.g English Wikipedia without images is a couple of 10s of GB.