The Effectiveness of Restorative Justice Practices: A Meta-Analysis

4. Results

4. Results

Twenty-two unique studies that examined the effectiveness of 35 individual restorative justice programs generated 66 effect sizes. A summary of specific study characteristics is presented in Table 2. The frequencies presented in Table 2 are based upon the 35 programs with the exception of the type of outcome measure and study source, which are based upon 66 effect sizes and 22 unique studies respectively.

Table 2. Descriptive Program/Study Characteristics
VARIABLE FREQUENCY (%)
Restorative Justice Model  
Conferencing 8 (22.9)
Victim−offender mediation 27 (77.1)
Entry Point  
Pre-charge 7 (20.0)
Post-charge 6 (17.1)
Pre-sentence 1 (2.9)
Post-sentence 1 (2.9)
Mixed 20 (57.1)
Outcome Measure  
Victim satisfaction 13 (19.7)
Offender satisfaction 13 (19.7)
Restitution compliance 8 (12.1)
Recidivism 32 (48.5)
Gender  
Predominantly male (>70%) 33 (94.3)
Mixed 2 (5.7)
Ethnicity  
Predominantly Caucasian (>70%) 14 (40.0)
Other 2 (5.7)
Mixed/unspecified 19 (54.3)
Age Group  
Adult 9 (25.7)
Youth 26 (74.3)
Study Source  
Published 10 (45.5)
Unpublished 12 (54.5)

The vast majority of the effect sizes were derived from programs that targeted predominantly male (94%), young (74%) offenders. Interestingly, a large proportion of the effect sizes were drawn from studies that were not published in peer-reviewed academic journals (55%), which, as discussed previously, is typically not the case in meta-analytic work.

As shown in Table 2, studies commonly included one or more of the following outcome measures: victim satisfaction, offender satisfaction, restitution compliance and recidivism reduction. Each of these issues will be discussed accordingly in the following subsections.

4.1 Victim Satisfaction

The overall mean effect size for the 13 tests of treatment that explored the impact of restorative justice programming on victim satisfaction was +0.19 (SD=.18) with a 95 percent confidence of +0.30 to +0.08 (see Figure 1). Although the effect sizes ranged from +0.44 to -0.19, the latter was the only negative value found in the distribution. In other words, participation in a restorative justice program resulted in higher victim satisfaction ratings when compared to a comparison group in all but one of the 13 programs examined.

Figure 1. Distribution of Effect Size Estimates (VICTIM SATISFACTION)

Figure 1. Distribution of Effect Size Estimates (VICTIM SATISFACTION)
[Description of Figure 1]

It should be noted that the one negative result was found in the only program that operated at the post-sentence (or corrections) entry point. Compared to victims who participated in the traditional justice system, victims who participated in restorative processes were significantly more satisfied (t (12) = 3.89, p < 0.01).

Given the relatively wide range of effect sizes, additional analyses were conducted to explore whether characteristics of the study sample or methodological considerations could explain this variability. Initially, we had hoped to explore a relatively large number of potential moderators, such as gender, ethnicity, criminal history, offence type, etc. The relative homogeneity of the offenders used in the studies, however, as well as the large amount of missing data, rendered many of these analyses untenable. On the other hand, this homogeneity increases our confidence in the generalizability of the findings to this population. Therefore, the analyses reported below focus on six factors: random assignment, offender age, publication source, restorative justice model, entry point and control/comparison group type.

As indicated previously, we used two coding methods for capturing information on the control/comparison group used in the studies. First, we combined multiple control/comparison groups from the same study to calculate a single effect size; second, we calculated individual effect sizes for each control/comparison group. For the comparison of control groups in Table 3 (and subsequent Tables), we used the latter coding technique. This allowed us to compare restorative justice programs with individuals who were referred to a restorative justice program but refused participation versus all other control group types (i.e. probation, court, prison).

The mean effect sizes for each value of the moderator variable, along with their corresponding significance tests, are presented in Table 3. Although these variables did not yield significant between-group differences, studies using non-randomized comparisons and studies published in academic journals displayed a higher mean effect size than their counterparts. In addition, VOM models tended to display higher victim satisfaction rates than conferencing models when compared to the non-restorative approaches. The lack of significance between moderating variables might be due to the low number of effect sizes.

Table 3. Moderator Analyses for Victim Satisfaction
VARIABLE N EFFECT SIZE Unweighted T value (p)
AGE Youth 8 .20 -.09
Adult 5 .19 (ns)
RANDOM ASSIGNMENT Yes 3 .14 .94
No 10 .21 (ns)
STUDY SOURCE Published 3 .30 -1.42
Unpublished 10 .16 (ns)
ENTRY POINT (earliest) Pre-charge Other entry points 6 6 .16 .18 -.31 (ns)
ENTRY POINT (latest) Pre-charge Other entry points 3 9 .15 .18 -.32 (ns)
MODEL CONTROL GROUP Conferencing 4 .14 .94
V−O mediation 9 .21 (ns)
Non-participation Other control 9 6 .22 .18 .62 (ns)

v–o = victim–offender
ns = not significant

4.2 Offender Satisfaction

The overall mean effect size for the 13 tests of the impact of restorative justice programming on offender satisfaction was +0.10 (SD=.28), while the effect sizes ranged from +0.31 to -0.71 (see Figure 2). While offenders who participated in restorative justice programs displayed higher satisfaction with the process than their comparisons, the one-sample t-test indicated that this difference was not statistically significant. Since the 95 percent confidence interval included zero, this further decreased our confidence that these programs have had any discernible impact on offender satisfaction.

This conclusion is mitigated, however, by the finding that although there were two negative effect sizes contributing to this result, the -0.71 was a clear outlier. Moreover, given that the sample size used in this outlier study was extremely small (n=7), we removed the study from the analysis. This increased the mean effect size to +0.17 and substantially reduced the standard deviation (SD=.13). Furthermore, and more importantly, removal of this study resulted in the confidence interval not including zero, thus suggesting that these programs have a moderate to weak positive impact on offender satisfaction. The difference in offender satisfaction between restorative and non-restorative participation also becomes significant (t (11) = 4.52, p < 0.01). Interestingly, the -.71 effect size was drawn from the same post-sentence entry point program as the only negative victim satisfaction effect size.

Figure 2. Distribution of Effect Size Estimates (OFFENDER SATISFACTION*)

Figure 2. Distribution of Effect Size Estimates (OFFENDER SATISFACTION*)
[Description of Figure 2]

*Does not include -.71 outlier

To account for this substantial discrepancy, we presented the results both with the outlier (Table 4.1) and without the outlier (Table 4.2). Given this extreme outlier, interpreting these results was inappropriate as the conclusions would be drastically different in each case based upon the inclusion or exclusion of one value.

Table 4.1. Moderator Analyses for Offender Satisfaction (With Outlier)
VARIABLE N EFFECT SIZE Unweighted T Value (p)
AGE Youth 8 .15 -.47
Adult 5 .05 (ns)
RANDOM ASSIGNMENT Yes 4 .09 .17
No 9 .12 (ns)
STUDY SOURCE Published 1 .08 .00
Unpublished 12 .11 (ns)
ENTRY POINT (earliest) Pre-charge Other entry points 8 5 .15 .03 .17 (ns)
ENTRY POINT (latest) Pre-charge Other entry points 5 8 .09 .11 -.13 (ns)
MODEL CONTROL GROUP Conferencing 6 .11 .12
V−O mediation 7 .09 (ns)
Non-participation Other control 8 7 .10 .10 .02 (ns)

v–o = victim–offender
ns = not significant

Table 4.2. Moderator Analyses for Offender Satisfaction (Without Outlier)
VARIABLE N EFFECT SIZE Unweighted T Value (p)
AGE Youth 8 .15 1.71
Adult 4 .22 (ns)
RANDOM ASSIGNMENT Yes 4 .09 1.33
No 9 .21 (ns)
STUDY SOURCE Published 1 .08 .00
Unpublished 11 .18 (ns)
ENTRY POINT (earliest) Pre-charge Other entry points 8 4 .15 .22 -1.2 (ns)
ENTRY POINT (latest) Pre-charge Other entry points 5 7 .09 .23 -1.79 (ns)
MODEL CONTROL GROUP Conferencing 6 .11 .12
V−O mediation 6 .23 (ns)
Non-participation Other control 7 7 .21 .10 1.93 (ns)

v–o = victim–offender
ns = not significant

4.3 Restitution Compliance

One of the potential advantages of a restorative justice approach is that it could be more effective in ensuring offender compliance with restitution agreements. This would be a significant contribution as the victims would have a greater likelihood of receiving compensation for the harm caused by the criminal activity and the offenders would be actively accepting responsibility. The results of the studies that included a measure of restitution compliance are reported below.

Only eight studies examined the impact of restorative justice programming on restitution compliance. Although this number may seem small, it may have been, in part, due to the inclusion criteria for this meta-analysis (i.e. the study used a comparison group). Overall, the mean effect size of +0.33 (SD=.24) was quite high, indicating that offenders who participated in restorative justice programs tended to have substantially higher compliance rates than offenders exposed to other arrangements. Furthermore, there was a great deal of variability in the effect sizes found in these studies, with values ranging from +0.63 to -0.02 (see Figure 3). Compared to the comparison/control groups not participating in a restorative justice program, offenders in the treatment groups were significantly more likely to complete restitution agreements (t (7) = 3.87, p < 0.01).

The small number of effect sizes (k=8) made conducting the moderator analyses inappropriate.

Figure 3. Distribution of Effect Size Estimates (RESTITUTION COMPLIANCE)

Figure 3. Distribution of Effect Size Estimates (RESTITUTION COMPLIANCE)
[Description of Figure 3]

4.4 Recidivism

Arguably, one of the most important outcome variables for any form of criminal justice intervention is recidivism. Considerable public and institutional support for correctional programming rests on its ability to reduce future criminal activity. Therefore, the ability of restorative justice programs to reduce recidivism was felt to be particularly important for the present meta-analysis.

The overall mean effect size for the 32 tests that examined the effectiveness of restorative justice programming in reducing offender recidivism was +0.07 (SD=.13) with a 95 percent confidence interval of +0.12 to +0.02. Although the effect sizes ranged from +0.38 to -0.23, more than two thirds of the effect sizes were positive (72%). In other words, restorative justice programs, on average, yielded reductions in recidivism compared to non-restorative approaches to criminal behaviour. In fact, compared to the comparison/control groups that did not participate in a restorative justice program, offenders in the treatment groups were significantly more successful during the follow-up periods (t (31) = 2.88, p < 0.01).

One of the major areas of debate in the correctional treatment literature is the impact of different methodological and demographic characteristics on program effectiveness. Subsequently, we conducted moderator analyses to explore the impacts of several

variables on recidivism reduction. The results of these analyses are presented in Table 5 and are discussed below.

Figure 4. Distribution of Effect Size Estimates (RECIDIVISM)

Figure 4. Distribution of Effect Size Estimates (RECIDIVISM)
[Description of Figure 4]

Table 5: Moderator Analyses for Recidivism
VARIABLE N EFFECT SIZE Unweighted T value (p)
AGE Youth 24 .06 .60
Adult 8 .10 (ns)
RANDOM ASSIGNMENT Yes 8 .06 .33
No 24 .07 (ns)
STUDY SOURCE Published 12 .12 -1.73
Unpublished 20 .04 (ns)
ENTRY POINT (earliest) ENTRY POINT (latest) Pre-charge Other entry points 16 16 .07 .06 .17 (ns)
Pre-charge Other entry points 8 24 .06 .07 -.16 (ns)
MODEL Conferencing 8 .06 .22
V−O mediation 24 .07 (ns)
CONTROL TYPE Non-participation Other control 9 31 .02 .12 -1.73 (ns)

v–o= victim–offender
ns = not significant

As stated previously, one of the primary criticisms lodged against meta-analysis is its predominant reliance on published studies and the subsequent problem of potential publication bias. This issue has been addressed in the present meta-analysis by conducting searches of governmental and non-governmental reports, graduate theses and dissertations and by directly contacting researchers active in the field for unpublished research. Nevertheless, we directly tested the impact of publication source on effect size. Inspection of Table 5 reveals that the mean effect size for studies from published sources was somewhat higher than the mean effect size found in unpublished sources. This, in combination with the reported difference above in victim satisfaction rates, lends support to the “file-drawer” problem in meta-analytic work.

Date modified: