The science of teaching science: An exploration of science teaching practices in PISA 2015

OECD Education Working Papers No. 188

With Alfonso Echazarra and Hélène Guillou

This paper explores the relationship between various science teaching strategies and students’ science-related outcomes. The focus is on enquiry-based science teaching, teacher-directed instruction, adaptive teaching and teacher feedback. The outcomes of interest include students’ science performance, and students’ dispositions and attitudes towards science. The findings show that the negative association between enquiry-based science teaching and science performance is greatly attenuated when lessons are delivered in disciplined science classes. This approach could help close the gender gap between girls and boys when it comes to attitudes towards science and to the decision to pursue a career in STEM-related fields. The results also show that teacher-directed instruction is a reliable strategy that is positively associated with students’ science outcomes regardless of school climate and resources. Adaptive teaching is positively correlated with science performance in the majority of countries, particularly in countries known for the use of personalised learning approaches, while teacher feedback is weakly but positively associated with science performance once students’ achievement in mathematics and reading is accounted for. In general, all teaching strategies have the potential to foster enjoyment of and interest in science, and students’ epistemic beliefs, self-efficacy in science and expectations of a career in science.

Link to paper in PDF

http://dx.doi.org/10.1787/f5bd9e57-en

 

The Science of Teaching Science: Evidence from PISA 2015

The Science of Teaching Science: Evidence from PISA 2015

Education experts have spent the last 50 years debating over a seemingly simple question: what’s the best way to teach science? On one side of the divide are those who support self-guided, enquiry-based approaches, under which students direct their own learning. On the other are proponents of teacher-directed instruction, who say this approach makes it easier for teachers to manage classrooms and cover a wider range of content. Complicating the debate even further is the increasing diversity of student populations, which has raised demands for science curricula to adapt to student needs through adaptive teaching approaches.

We take a closer look at each of these strategies in the latest issue of PISA in Focus. Using new evidence from PISA 2015, we found that each approach has advantages and drawbacks for learning – and that identifying the most effective strategy isn’t as clear cut a proposition as it may seem.

In almost all of the 68 countries and economies that participated in PISA, students in the least disciplined science classes perform worse when exposed to enquiry-based science teaching. But in 33 countries and economies, this negative association disappears when students are learning in a disciplined environment.

In Thailand, exposure to enquiry-based teaching accounted for a four-point increase in performance among students in the most disciplined science classes. But students exposed to enquiry-based teaching in the least disciplined classrooms, scored about 13 points lower than those in more disciplined environments. The benefits gained from attending disciplined science classes with enquiry-based teaching are largest in Georgia (+20 points), Kosovo (+15 points), Lebanon (+13 points), Malta (+14 points), and Slovenia (+13 points).

Our findings shed light on the real-world complexity of teaching.

In OECD countries, enquiry-based teaching seems like the most promising way to nurture positive attitudes toward science – including interest and enjoyment in science-related topics, and participation in science-related activities. We also found that all three teaching practices – enquiry-based, teacher-directed and adaptive teaching are associated with higher expectations among students to pursue a career in science. This association is particularly strong among girls who are exposed to enquiry-based teaching.

Teacher-directed science instruction, on the other hand, is associated with better science performance in almost all countries. This positive association is equally strong across all science sub-domains and proficiency levels, and does not vary with student and school characteristics (e.g. disciplinary climate, student composition, resources, etc.). Based on these findings, we can conclude that teacher-directed practices are likely to deliver good results regardless of environment.

Our findings also show that adapting science lessons to students’ needs is correlated with stronger science performance in the majority of countries, even after accounting for student and school characteristics. This relationship is particularly strong in the Nordic countries, which are known for their comprehensive education systems and their reliance on differentiated learning approaches.

So which strategy would be most effective for science teachers to deploy? Our findings suggest a combination of all three. For example, teachers with strong classroom-management skills and professional knowledge could guide student learning with explicit instruction of basic ideas, then ask them to carry out enquiry-based activities to consolidate their knowledge. At the same time, teachers could also adapt their science lessons to account for differences among students, and help those who have difficulty understand a particular topic.

This conclusion may not be satisfying for those who firmly support one approach over the other. But our findings shed light on the real-world complexity of teaching in various classroom environments – where teachers often have to find the right mix of different practices to achieve the best results for their students.

Science teachers’ satisfaction: Evidence from the PISA 2015 teacher survey

OECD working paper EDU/WKP(2018)4

With Judit Pál

In 2015, for the first time in its history, PISA (the Programme for International Student Assessment) asked teachers to describe the various aspects of their working environment and teaching practices. This paper examines how teacher, student and school characteristics are related to science teachers’ satisfaction in 19 PISA-participating countries and economies. The findings show that the most satisfied science teachers tend to be those who are initially motivated to become teachers. The results also highlight the positive relationship between science teachers’ satisfaction and teacher collaboration, good disciplinary climate in science classes, availability of school resources, and the opportunity to participate in professional-development activities.

Download paper in PDF

http://dx.doi.org/10.1787/1ecdb4e3-en

How Consistent is Respondent Behaviour to Allow Linkage to Health Administrative Data over Time?

With Richard Wiggins

CLS Working paper 2015/3.

This study constitutes the first longitudinal exploration of consent to link survey and administrative data. It relies on a theoretical framework distinguishing between passive, active, consistent and inconsistent consent behaviour. The findings show that, in general, consent behaviours are both passive and consistent. First, consent rates indicate that most respondents behave consistently over time. Secondly, the regression analyses show that for the majority of respondents, consent is not driven by personal convictions but rather depends on the circumstances of the respondent at the time of the interview and on the impact of the interviewers. The findings also show that in longitudinal surveys cross-sectional analyses of consent can be misleading. The changes in the magnitude and in the significance of the results when the temporal dimension of consent is taken into account is a clear indication that consent should be treated as a dynamic phenomenon.

Download paper in PDF

Variation within Households in Consent to Link Survey Data to Administrative Records Evidence from the UK Millennium Cohort Study

CLS Working paper 2014/8.

This study expands our knowledge of consent in linking survey and administrative data by studying respondents’ behaviour when consenting to link their own records and when consenting to link those of their children. It develops and tests a number of hypothesised mechanisms of consent, some of which were not explored in the past. The hypotheses cover: parental pride, privacy concerns, loyalty to the survey, pre-existing relations with the agency holding the data, and interviewer effects. The study uses data from the longitudinal Millennium Cohort Study to analyse the correlates of consent in multiple domains (i.e. linkage of education, health and economic records). It relies on a multivariate probit approach to model the different consent outcomes, and uses fixed and random effects specifications to estimate the effects of interviewers.The findings show that respondent’s behaviour vary depending on the consent domain (i.e. education, health, and economic records) and on the person for whom consent is sought (i.e. main respondent vs. cohort member). In particular, the cohort member’s cognitive skills and the main respondent’s privacy concerns have differential effects on consent. On the other hand, loyalty to the survey proxied by the longitudinal response history has a significant and strong impact on consent irrespective of the outcome. The findings also show that interviewers account for a large proportion of variations in consent even after controlling for the characteristics of the interviewer’s assignment area. In total, it is possible to conclude that the significant impact of some of the correlates will lead to sample bias which needs to be accounted for when working with linked survey and administrative data.

Download paper in PDF

Handling attrition and non-response in the 1970 British Cohort Study

With Richard Wiggins

CLS Working paper 2014/2.

The 1970 British Cohort Study (BCS70) is a continuing multi-purpose, multidisciplinary longitudinal study based on a sample of over 17,000 babies born in England, Wales and Scotland in 1970. The study has collected detailed information from the cohort members on various aspects of their lives, including their family circumstances at birth, education, employment, housing and partnership histories. There have been nine sweeps of data collection so far: at birth and at ages 5, 10, 16, 26, 30, 34, 38 and most recently age 42 (2012). This paper studies the extent of attrition in BCS70 and how it affects sample composition over time. We examine the determinants of response then construct inverse probability weights. In the last section, we use a simulation study to illustrate the effectiveness of weights and imputations in dealing with unit non-response and item missingness respectively. Our findings show that when the predictive power of the response models is weak, the efficacy of non-response weights is undermined. Further, multiple imputations are effective in reducing the bias resulting from item missingness when the magnitude of the bias is high and the imputation models are well specified.

Download paper in PDF

Learning and Wellbeing Trajectories Among Older Adults in England

With Andrew Jenkins

DBIS Research Paper N° 92.

Published by the Department for Business, Innovation and Skills – British Government.

In an ageing society such as the UK, there is much interest in factors which can contribute to the wellbeing of older adults. It is not implausible to suppose that participation in learning could have beneficial effects, yet most research on the wider benefits of learning has tended to focus on young people or those in mid-life and there is currently rather little evidence on the impact of learning on the wellbeing of older adults. Insofar as evidence does exist, most of it is qualitative, and while of much value and interest, it is based on very small, and possibly not very representative, samples of the older population. This research aimed to provide new, quantitative evidence drawing on a large, nationally representative sample, on the effects of participation in learning on the wellbeing of older adults.

Download paper in PDF

Trends in job quality in Europe

With Francis Green

Eurofound report published by the publications office of the European Union, Luxembourg.

This study measures job quality in the 27 countries of the European Union, as well as the seven additional countries in Europe that participated in the European Working Conditions Survey (EWCS). Four indices were constructed for the study: earnings, prospects, intrinsic job quality and working time quality. The four indices cannot be reduced into a single index of job quality because associations between them are weak, and none can increase over time nor move in similar directions. They are, however, theoretically and conceptually coherent.

The intention was to fi nd an objective means of assessing the principle established in a number of EU directives that work should adapt to the workers. The indices constructed for this study do not rely on subjective measurement such as preferences and attitudes, but are built on the self-reported features of jobs that are associated with workers’ well-being.

Download report in PDF

Measuring the Impact of Universal Pre-School Education and Care on Literacy Performance Scores

With Andy Green

LLAKES Research Paper N° 36.

The objective of this paper is to simulate the effects of universal pre-school education and care (PSEC) on reading performance scores and educational inequalities in the UK and Sweden. We utilize the PISA 2009 data and start by estimating a fixed effects multilevel model for each country in order to determine the returns to PSEC attendance. Then we simulate the effects of universal PSEC provision using counterfactual data. More precisely, after estimating the multilevel model, we progressively universalize PSEC participation starting with the lowest economic, cultural and social status (ESCS) decile and moving up to reach the top decile. At each stage of the universalisation process we compute the average predicted performance scores for each ESCS decile and for each country as well as their dispersions. This allows us to measure the change in average predicted literacy scores and the change in the level of inequality.

Our findings show that all social groups benefit from universalizing PSEC with the lowest groups getting the highest additional benefits from universalisation. Further, the international rankings of both Sweden and the UK improve after the universalisation of PSEC. The UK moves 12 positions up the OECD league table and Sweden moves up seven positions. We also find that inequalities in test scores drop until reaching a minimum when the lower seven ESCS deciles are attending PSEC and then starts to increase again. In conclusion, our findings clearly show that universalising PSEC would be an effective policy instrument that boosts educational performances while reducing inequalities in their distribution.

Download paper in PDF

Pre-School Education and Care – a ‘Win-Win’ Policy?

With Andy Green

LLAKES Research Paper N° 32.

Pre-school education and care (PSEC) is often claimed as a ‘win-win’ policy which simultaneously enhances both economic competitiveness and social cohesion. High levels of PSEC are said to raise living standards by increasing female employment rates and improving young people’s skills and to mitigate inequalities by reducing social gaps in learning outcomes. Much of the evidence for this rests on analysis of data for a small number of countries. In this paper we test the claims using cross-national time series data for a large number of OECD countries. The analysis of determinants of employment rates, using a variety of controls, does confirm the association between PSEC participation levels and female employment rates. However, the cross-national analysis does not support the argument that raising aggregate levels of PSEC participation necessarily reduces social gaps in attainment at 15 years of age. Participation in PSEC increases educational performance at 15 by similar amounts for children of all social groups in most countries. Social gaps in performance at 15 may only be mitigated by high levels of PSEC provision where children from less advantaged families get more – or better quality – provision. The recently announced Department for Education plan to extend free provision of PSEC for fifteen hours a week to two-year-old children from disadvantaged families (i.e. in care or qualifying for free school meals) therefore points in the right direction. However, it remains to be seen whether this will bias participation towards this group sufficiently to reduce inequalities in learning outcomes.

Download paper in PDF