[Off Topic] The amazing truth about PISA scores: USA beats Western Europe, ties with Asia.

What an interesting thesis indeed.  I need to mull it over a bit.

Make sure you read the entire post over at his Super-Economy blog, for more of interest, including his policy implications.

The main theme in my blog is that we shouldn’t confuse policy with culture, and with demographic factors.

For instance, education scholars have known for decades that the home environment of the kids and the education levels of the parents are very important for student outcomes. We also know that immigrant kids have a more difficult time at school, in part because they don’t know the language.

Take me as an example. The school me and my brother attended was in a basement in Tehran, had no modern resources, and largely focused on religious indoctrination. But we had a good home background. Our father attended a college in the west a few years (our mother didn’t, despite stratospheric scores test scores, because at the time you didn’t send a good Kurdish girl to another city to study). So we did well in school. Conversely, the first few years in Sweden I had bad grades, in part because I didn’t master the language.

The point I am trying to make is that the school in Sweden was objectively superior to the school in Iran. But I scored lower in Sweden, because of factors outside the control of the education system. If you want to compare the effect of the school, you have to isolate those external factors and make an apples-to-apples comparison.

However, this is not at all how the media is presenting the recent PISA scores. For example there is a lot of attention of the score of the kids in Shanghai, the according to the NYT is supposed to “stun” us or something.

It’s dumb to compare one of the most elite cities in a country with entire nations, and to draw policy-inference from such a comparison. Shanghai has 3 times the average income of China! It is also naive to trust the Chinese government when they tell us the data is representative of the entire nation. Either you compare Shanghai to New York City, or you compare the entire country of China, including the rural part, with other large nations. Most of the news and policy conclusions we read about PISA-scores in the New York Times is thus pure nonsense.

1. Correcting for the demography:

In almost all European countries, immigrants from third world countries score lower than native born kids.

Why? No one don’t know exactly why. Language, culture, home environment, income of parents, the education level of the parents and social problems in the neighborhood and peer groups norms are among likely explanations. But it is generally not true that the schools themselves are worse for immigrants than natives. In welfare states, immigrants often (thought not always) go to the same or similar schools and have as much or likely more resources per student.

So the fact that immigrant students in mixed schools do worse than Swedish kids used to a few decades ago in homogeneous schools does not it out of itself prove that Swedish public schools have become worse.

Of course, the biggest myth that the media reporting of PISA scores propagates is that the American public school system is horrible.

The liberal left in U.S and in Europe loves this myth, because they get to demand more government spending, and at the same time get to gloat about how much smarter Europeans are than Americans. The right also kind of likes the myth, because they get to blame social problems on the government, and scare the public about Chinese competitiveness.

We all know that Asian students beat Americans students, which “proves” that they must have a better education system. This inference is considered common sense among public intellectuals. Well, expect for the fact that Asian kids in the American school system actually score slightly better than Asian kids in North-East-Asia!

So maybe it’s not that there is something magical about Asian schools, and has more to do with the extraordinary focus on education in Asian culture, with their self-discipline and with their favorable home environment.

There are 3 parts to the PISA test, Reading, Math, and Science. I will just make it simple and use the average score of the 3 tests. This is not strictly correct, but in practice it doesn’t influence the results, while making it much easier for the reader. (the reason it doesn’t influence the results is that countries that are good at one part tend to be good at other parts of the test.)

The simplest thing to do in order to get an apples-to-apples comparison is to at least correct for demography and cultural background. For instance, Finland scores the best of any European country. However first and second generation immigrant students in Finland do not outperform native Swedish, and score 50 points below native Finns (more on this later).

On PISA, 50 points is a lot. To give you a comparison, 50 points is larger than the difference between Sweden and Turkey. A crude rule of thumb here is that 50 points is 0.5 standard deviations.

The problem is that different countries have different share of immigrants. Sweden in 2009 PISA data had 17%, and Finland 4%. It’s just not fair to the Swedish public school system to demand that they must produce the same outcome, when Sweden has many more disadvantaged students. Similarly schools with African-American students who are plagued by racism, discrimination, crime, broken homes, poverty and other social problems are not necessarily worse just because their students don’t achieve the same results as affluent suburbs of Chicago. In fact, the most reliable data I have seen suggests that American minority schools on average have slightly more money than white schools. It’s just that the social problems they face are too much to overcome for the schools. It is illogical to blame the public school system for things out of its hands.