An Indian research university has assembled 73 million journal articles (without permission) and is offering the archive for unfettered scientific text-mining

The JNU Data Depot is a joint project between rogue archivist Carl Malamud (previously), bioinformatician Andrew Lynn, and a research team from New Delhi's Jawaharlal Nehru University: together, they have assembled 73 million journal articles from 1847 to the present day and put them into an airgapped respository that they're offering to noncommercial third parties who want to perform textual analysis on them to "pull out insights without actually reading the text."

This text-mining process is already well-developed and has produced startling scientific insights, including "databases of genes and chemicals, map[s of] associations between proteins and diseases, and [automatically] generate[d] useful scientific hypotheses." But the hard limit of this kind of text mining is the paywalls that academic and scholarly publishers put around their archives, which both limit who can access the collections and what kinds of queries they can run against them.

By putting 73 million articles in a repository without having to bargain with the highly concentrated and notoriously rent-seeking scholarly publishing industry, the JNU Data Depot team are able to dispense with the arbitrary restrictions put on data-mining. They believe that they are on the right side of Indian copyright law as well, as they are a scholarly institution that is making a single digital copy for local use, and not circulating the articles on the internet; they believe that these precautions might shield them from a lawsuit.

They're relying on precedent set in a 2016 Delhi High Court Ruling that turned on the legality of a copy shop that sold photocopied selections from expensive textbooks, where the court held that section 52 of the 1957 Copyright Act allows reproduction of copyrighted works for education and research. Read the rest