Hang onto your hats! PISA is just around the corner!

You know, I have to admit, all has been pretty quiet on the education front recently. Although there has been a bit of noise made about report cards here in Nova Scotia, and a bit of concern about the removal of some academic accolades for students in schools out West, the latter part of 2013 has been relatively quiet. Where 2011 and 2012 saw such head line grabbers as the “No zero” policy, budget cuts  and what seemed like a constant barrage of attacks against the teaching profession in general, 2013 has been, to some extent, blissful.

Well, folks, mark December 3rd on your calendar. Because that’s when the “educational expert” bombast and table thumping will begin anew.

Because on December 3rd, PISA is scheduled to release its 2012 test results.

For those of you who may not know, PISA stands for the Programme for International Student Assessment. Essentially, it is an examination that is written every three years by somewhere around a half  million 15-year-old students in over 60 countries around the world. Developed by the Organization for Economic Co-operation and Development it purports to test ability in math, reading and science. The OECD takes the results from the PISA and ranks countries against each other. It then publishes these rankings.

And that’s where the fun begins.

You see, if a country scores high in the PISA, they become the model after which everyone else wants to build their system. In 2010, Finland became the darlings of the education world, as their PISA scores from 2009 were shown to be impressively high, a trend that had been occurring over a number of PISA cycles. Pasi Sahlberg’s book, Finnish Lessons, became a must read for policy makers, and educational commentators of every stripe (including this one) were prying suggestions out of their system when looking for ways to improve education. Contrarily,  to score low on the PISA is somewhat of a black eye for a country’s education system. This was the case for the US  when it placed well out of the top ten in the 2009 results. (Last place that year belonged to Kyrgyzstan.)

PISA rankings, however, can be broken down even further, and districts within countries can be compared. We saw this here in  Canada in 2010 when the PISA results became a “mine is better than yours” crowing match amongst the Canadian provinces, with two of Canada’s richest Provinces, Alberta and Ontario, coming out on top. And, as always, when ever there are numbers attached to education, there are those who spin them to further an agenda of self-serving changes to the system. Certainly, here at home, local think tanks and their supporters were quick to cite PISA scores as evidence that schools needed “fixing”, despite Nova Scotia’s strong showing.

Well, before Tuesday rolls around, and our schools are again paraded or berated based on their PISA results, a word of caution. Just like last time, PISA will provide us with some interesting and informative numbers about the education system here in Nova Scotia. And, just like last time, there will be those who interpret these data to push agendas and sway public opinion. But at the end of the day, the PISA is a small piece in a much larger puzzle.

Large scale standardized tests may be popular providers of media sound bites, and make easy ammunition for those who like to spin data, but they can also narrow educational focus, and take precious resources away from the classroom. My gravest concern with them, however, is that they cause reactionary decisions to be made about education. Somehow, a score on a test becomes not just a measure of how kids did in that subject on that day, but rather how the education system is doing overall. Scores in this PISA, which was focused on mathematics, must be taken in context.

Tests can only measure what they measure, and there is much more to public education than I have ever seen encapsulated in any test, no matter how revered.

High results on PISA do not mean we can stop looking for ways to get better at meeting the needs of our students any more than low results mean we should shred our entire system. The results are, at the end of the day, simply another lens through which to view education.

I hope on Tuesday, everyone remembers that.



Filed under Uncategorized

2 responses to “Hang onto your hats! PISA is just around the corner!

  1. Finnish miracle: fata morgana?
    Finnish students’ achievement (15 y) declined significantly: study of University Helsinki
    University of Helsinki – Faculty of Behavioral Sciences, Department of Teacher of Education Research Report No 347Authors: Jarkko Hautamäki e.a. Learning to learn at the end of basic education: Results in 2012 and changes from 2001
    S.: The change between the year 2001 and year 2012 is significant. The level of students’ attainment has declined considerably: under the mean of the scale used in the questions. The difference can be compared to a decline of Finnish students’ attainment in PISA reading literacy from the 539 points of PISA 2009 to 490 points, to below the OECD average. The mean level of students’ learning-supporting attitudes still falls above the mean of the scale used in the questions but also that mean has declined from 2001.
    Since 1996, educational effectiveness has been understood in Finland to include not only subject specific knowledge and skills but also the more general competences which are not the exclusive domain of any single subject but develop through good teaching along a student’s educational career. Many of these, including the object of the present assessment, learning to learn, have been named in the education policy documents of the European Union as key competences which each member state should provide their citizens as part of general education (EU 2006).
    In spring 2012, the Helsinki University Centre for Educational Assessment implemented a nationally representative assessment of ninth grade students’ learning to learn competence. The assessment was inspired by signs of declining results in the past few years’ assessments. This decline had been observed both in the subject specific assessments of the Finnish National Board of Education, in the OECD PISA 2009 study, and in the learning to learn assessment implemented by the Centre for Educational Assessment in all comprehensive schools in Vantaa in 2010.
    The results of the Vantaa study could be compared against the results of a similar assessment implemented in 2004. As the decline in students’ cognitive competence and in their learning related attitudes was especially strong in the two Vantaa studies, with only 6 years apart, a decision was made to direct the national assessment of spring 2012 to the same schools which had participated in a respective study in 2001.
    The goal of the assessment was to find out whether the decline in results, observed in the Helsinki region, were the same for the whole country. The assessment also offered a possibility to look at the readiness of schools to implement a computer-based assessment, and how this has changed during the 11 years between the two assessments. After all, the 2001 assessment was the first in Finland where large scale student assessment data was collected in schools using the Internet.
    The main focus of the assessment was on students’ competence and their learning-related attitudes at the end of the comprehensive school education, but the assessment also relates to educational equity: to regional, between-school, and between- class differences and to the relation of students’ gender and home background to their competence and attitudes.
    The assessment reached about 7 800 ninth grade students in 82 schools in 65 municipalities. Of the students, 49% were girls and 51% boys. The share of students in Swedish speaking schools was 3.4%. As in 2001, the assessment was implemented in about half of the schools using a printed test booklet and in the other half via the Internet. The results of the 2001 and 2012 assessments were uniformed through IRT modelling to secure the comparability of the results. Hence, the results can be interpreted to represent the full Finnish ninth grade population.
    Girls performed better than boys in all three fields of competence measured in the assessment: reasoning, mathematical thinking, and reading comprehension. The difference was especially noticeable in reading comprehension even if in this task girls’ attainment had declined more than boys’ attainment. Differences between the AVI-districts were small. The impact of students’ home-background was, instead, obvious: the higher the education of the parents, the better the student performed in the assessment tasks. There was no difference in the impact of mother’s education on boys’ and girls’ attainment. The between-school-differences were very small (explaining under 2% of the variance) while the between-class differences were relatively large (9 % – 20 %).
    The change between the year 2001 and year 2012 is significant. The level of students’ attainment has declined considerably. The difference can be compared to a decline of Finnish students’ attainment in PISA reading literacy from the 539 points of PISA 2009 to 490 points, to below the OECD average. The mean level of students’ learning-supporting attitudes still falls above the mean of the scale used in the questions but also that mean has declined from 2001.
    The mean level of attitudes detrimental to learning has risen but the rise is more modest. Girls’ attainment has declined more than boys’ in three of the five tasks. There was no gender difference in the change of students’ attitudes, however. Between-school differences were un-changed but differences between classes and between individual students had grown. The change in attitudes—unlike the change in attainment—was related to students’ home background: The decline in learning-supporting attitudes and the growth in attitudes detrimental to school work were weaker the better educated the mother. Home background was not related to the change in students’ attainment, however. A decline could be discerned both among the best and the weakest students.
    The results of the assessment point to a deeper, on-going cultural change which seems to affect the young generation especially hard. Formal education seems to be losing its former power and the accepting of the societal expectations which the school represents seems to be related more strongly than before to students’ home background. The school has to compete with students’ self-elected pastime activities, the social media, and the boundless world of information and entertainment open to all through the Internet. The school is to a growing number of youngpeople just one, often critically reviewed, developmental environment among many.
    The change is not a surprise, however. A similar decline in student attainment has been registered in the other Nordic countries already earlier. It is time to concede that the signals of change have been discernible already for a while and to open up a national discussion regarding the state and future of the Finnish comprehensive school that rose to international acclaim due to our students’success in the PISA studies.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s