How to Analyze Data from International Large-Scale Assessments (ILSAs) Using Statistical Software
This section describes available resources that support more experienced users in understanding the specifics of ILSAs, effectively merging and analyzing datasets, and conducting advanced statistical analyses using different statistical software packages. We also demonstrate how to perform analyses using either open-source software (such as R) or commercial statistical software.
First, it is advisable to familiarize yourself with the available documentation, including technical reports from the studies and user manuals provided with the datasets. This will help you understand the study design, the sampling scheme, and how the sampling is reflected in the analytical weights. Pay particular attention to the use of variables relating to student achievements, as well as the rules for merging different datasets (e.g., datasets containing students’ data and teachers’ data).
Tools supporting statistical data analysis
Tools for analysing International Large-Scale Assessments (ILSAs)
International Large-Scale Assessments require advanced statistical tools, which take into account complex sampling designs, replicate weights and plausible values.
The most convenient tool for working with these data is the IEA IDB Analyzer Software. Users of mainstream statistical software such as R, SPSS, Stata, or SAS can also analyze these data using additional packages or macros designed for International Large-Scale Assessments.
The table below presents a review of statistical software best suited for the analysis of data from such studies.
We encourage you to use the tutorials that show how to analyze data from International Large-Scale Assessments with selected statistical software packages.
IDB Analyzer
The IEA IDB Analyzer is a free tool developed by IEA for analyzing data from International Large-Scale Assessments (ILSAs), such as those conducted by IEA, OECD, and other organizations. This software allows users to take into account the specific design of the study and perform analyses in different statistical packages, including SPSS, SAS, and R. The program requires a Windows operating system and the installation of the relevant statistical software.
The IEA IDB Analyzer features a graphical user interface that reads the data files from the user’s directory and generates syntax for merging and analyzing data in SPSS, SAS, or R. It consists of three main modules: the Merge Module, the Analysis Module, and a new module introduced in version 5.0, which enables the conversion of SPSS system files to R data files. The Merge Module generates syntax to merge files from different countries and levels (e.g., students, teachers, schools), while the Analysis Module generates syntax for analyzing these data. The generated code enables users to conduct statistical analyses, including calculating descriptive statistics, hypothesis testing, and running basic linear and logistic regression models.
More information about the IEA IDB Analyzer is available here.
Tutorial: “Analysis of ILSA data using the IEA IDB Analyzer” is available here.
R
R is an open-source statistical software environment offering advanced analytical tools, available through additional packages. Once loaded to the R environment, these packages extend its functionalities with new features. There are packages that, among other features, allow researchers to incorporate sampling design, analytic and replicate weights, as well as plausible values in analyses, as well as to conduct cross-country comparisons. The packages vary in functionality and complexity, allowing users to tailor the tools to their individual needs.
Recommended packages:
IEA IDB Analyzer
Creates and exports scripts for data merging and basic data analysis from studies.
BIFIEsurvey
A versatile package developed to analyze data from various studies with complex sampling designs, replicate weights, and plausible values.
SPSS
A popular commercial statistical software widely used in social science research. It does not directly support the analysis of data containing plausible values and replicate weights. To incorporate these, additional macros or the IEA IDB Analyzer are required.
More information about the IEA IDB Analyzer is available here.
Tutorial: “Analysis of ILSA data using the IEA IDB Analyzer” is available here.
Stata
Stata is an advanced, commercial statistical software package valued for its intuitiveness and versatility in social science research analyses.
The svyset function allows users to specify sampling designs, weights, and replication methods. Stata does not directly support plausible values, which are essential for ILSA data analysis. However, they can be handled indirectly by working with imputed datasets or by installing additional packages.
Recommended packages can be installed by typing the following command in the console: ssc install package_name:
repest
- Predefined study designs
- Analysis of data with replicate weight
- High flexibility in multilevel analyses
pv
- A dedicated module for working with plausible values
- Supports multiple variable sets in complex studies
pisatools
- A dedicated package for analysis of PISA data
- Built-in support for plausible values and replicate weights (BRR)
- Enables calculation of descriptive statistics, regression and decomposition analyses
Tutorial: Analysis of ILSA data using the Stata repest package is available here.
SAS
An advanced, commercial statistical software package for analyzing data in social and educational research.
The SURVEY procedures allow users to handle data from complex sampling designs. Plausible values can be incorporated using procedures for imputed data, and macros can be applied to obtain averaged results.
Mplus
A commercial statistical software package that includes psychometric functionalities and multilevel modeling features useful for ILSA data analyses. Plausible values can be incorporated using procedures for imputed data.