The Effectiveness of Restorative Justice Practices: A Meta-Analysis

2. Meta-Analysis

Similar to traditional quantitative research methods, the meta-analytic process involves three basic steps:

  1. literature review – identifying and gathering relevant research studies
  2. data collection – extracting data through pre-determined coding procedures
  3. data analysis – analysing the aggregated data using statistical techniques

A meta-analysis can be understood as a statistical analysis of a collection of studies that aggregates the magnitude of a relationship between two or more variables (Glass, McGaw & Smith, 1981). These studies may differ on several important characteristics, such as operationalization of independent and dependent variables, sample size, sample selection techniques and design quality. Meta-analytic statistics can describe the typical strength of the effect under investigation, the degree of statistical significance, the variability, as well as provide researchers the opportunity to explore and identify potential moderating variables. The outcome of a meta-analysis is an effect size, which can be interpreted as the estimated effect of the independent variable on the dependent variable. For example, an average effect size estimate of +.10 translates into the independent variable accounting for a 10 percent change in the dependent variable (Rosenthal, 1991).

Meta-analytic reviews are generally regarded as a superior method of research synthesis compared to traditional narrative reviews as the former are “more systematic, more explicit, more exhaustive, and more quantitative” (Rosenthal, 1991, p.17). Meta-analytic techniques have been used across such diverse fields as education and medicine, and have more recently been adopted within the social sciences (Lipsey & Wilson, 1993). In the area of criminal justice research in particular, meta-analytic studies have investigated the prediction (Bonta, Law, & Hanson, 1998; Dowden & Brown, in press; Gendreau, Little, & Goggin, 1996; Hanson & Bussière, 1998) and treatment (Andrews et al., 1990; Dowden & Andrews, 1999, 2000; Latimer, 2001; Lipsey, 1995; Losel, 1995; Whitehead & Lab, 1989) of criminal behaviour.

Critics argue that one of the major limitations of meta-analytic techniques is that the sampling procedures are biased in favour of including predominantly published studies. It is surmised that the probability of publishing a study is increased by the statistical significance of the results so that published studies are not actually representative of the entire body of research that has been conducted in that area. Consequently, a calculated effect size, based exclusively on published studies, may be overestimating the relationship. Coined the “file drawer problem” (Rosenthal, 1991, p. 103), this suggests that if unpublished studies were included in the meta-analysis, the effect size estimate would be smaller.

Bonta, Wallace-Capretta, and Rooney (1998) conducted a preliminary meta-analysis of programs that contained elements of restorative justice and exclusively focussed on their role in reducing offender recidivism. The results revealed that these programs yielded mild reductions in re-offending (+.08). However, the authors used a very broad operational definition of restorative justice as they included both court-ordered restitution and community service programs. This definition is somewhat problematic as it fails to fully incorporate some of the fundamental principles of restorative justice – namely, the voluntary nature of both offender and victim participation and the face-to-face encounter. A need therefore existed to quantitatively aggregate the findings of the literature using a more precise definition of restorative justice.

Date modified: