The OECD has just come out with their annual rankings of the academic knowledge and skills of the world’s 15-year-olds, and Shanghai aced the test. The city ranked number-one in mathematics, reading, and science, utterly kicking-ass compared to its Asian neighbors, let alone those backwards Europeans and Americans. Slight caveat here: the survey makes no sense and is borderline useless.
The most glaring stat-wrenching is that the OECD survey measures the academic performance of entire countries, not of cities, except for in China. Shanghai, Hong Kong, and Macao are the only non-countries on the list (with the arguable inclusion of “Chinese Taipei”). Although “Shanghai-China” ranks several positions higher than Japan, that doesn’t really mean anything at all, because Japan’s sample included representatives of the entire country, while China’s only included residents from its wealthiest, most affluent city.
Shanghai v. Tokyo could be an interesting comparison, as could China v. Japan, but this present system gives Chinese cities a huge advantage. The city/country disconnect is not addressed on the OECD website, or at least not as prominently as one would think, considering how hugely this skews the data.
If one loads up the OECD’s spreadsheets for “Results for regions within countries,” no Asian countries make the list. To find any kind of recognition that “oh hey, it might be weird and wildly unfair to compare cities with countries,” one has to go to page 51 of the full “Key Findings” report, in a subsection of the Math results, where we find:
For entities other than those for which full samples were drawn, namely Chinese Taipei, Hong Kong-China, Macao-China and Shanghai-China, it is not possible to calculate a rank order; but the mean score provides the possibility of comparing subnational entities against the performance of countries and economies.
Essentially, then, there is a recognition that the scope and scale of the results coming from “entities other than those for which full samples were drawn” (i.e. not countries) is completely different from the scale of real countries, but fuck it, we’ll still put the stats into the mix.
This isn’t to say that Shanghai shouldn’t be slapping itself on the back for having some of the world’s highest education statistics, it’s simply saying that we have no idea who has the world’s highest education statistics, because fundamental parts of this survey make no sense. “Shanghai-China” outpaces Korea and Japan, but maybe not Seoul or Tokyo.
Also, if the OECD insists on measuring China’s academic performance by cities and not by the nation as a whole, we recommend pulling in a few yellow-earth 4th-tier cities to counterbalance the metropolises.