Stronger Families in Australia study: the impact of Communities for Children

4. Validity of contrast sites as the counterfactual

The validity of the evaluation requires that the outcomes for children and families in CfC sites would have been the same as those that occurred in the contrast sites had there been no CfC intervention (that is, that they provide a valid counterfactual).

The validity of the contrast sites as a control group can be assessed by testing statistically whether there are systematic differences in the outcome variables and various demographic and socioeconomic characteristics that are known to impact on child development in the CfC and non-CfC sites prior to the intervention (that is, at Wave 1). Small differences can be taken into account through the estimation techniques used (see Section 3.2).

Of the 14 outcome variables assessed at Wave 1, only two were statistically significantly different between CfC and non-CfC sites at the 95 per cent confidence level (see Appendix D). Children’s physical health was better in contrast sites compared to CfC sites at Wave 1, and the reverse was true for maternal mental health at Wave 1. However, subsequent estimates of differences between key outcomes for Wave 1 suggest that there were no statistically significant differences between the 14 outcome measures; that is, after controlling for observed individual and community characteristics at Wave 1.²¹

The demographic characteristics of families residing in the CfC sites and contrast sites at Wave 1 were also similar (Table 2). No significant differences were found in relation to child gender and age, family type, size of the sibling group and parent work status. However, there was a greater percentage of Aboriginal or Torres Strait Islander children in CfC sites than the contrast sites. In contrast sites, a higher proportion of mothers spoke a language other than English at home. A higher proportion of mothers and fathers had also completed Year 12 in contrast sites. While the percentage of families where parents’ combined income was $1,500 per week or above was similar, a higher proportion of families in contrast sites earned less than $800 per week.

The overall conclusion of this analysis is that the contrast sites provided a valid comparison group. There were no statistically significant differences on the outcome variables at Wave 1 or Wave 2 after adjusting for differences in demographic and socioeconomic characteristics. While there were some differences in demographic and socioeconomic characteristics between families in CfC and contrast sites, these were restricted to income, education and cultural background and could be accommodated by using statistical techniques.

Table 2: Demographic characteristics of families in CfC and contrast sites at Wave 1
Wave 1 characteristics	Contrast %	CfC %
Study child’s gender
Male	51.1	50.1
Female	48.9	49.9
Age of study child (months)
27–30	24.2	23.0
31–33	28.2	27.9
34–36	27.6	28.8
37–40	20.0	20.3
Family type
Couple family	76.2	73.5
Both biological parents	75.2	71.8
Other (for example, step/blended)	1.0	1.7
Single-parent family	23.8	26.5
Siblings
Only child	21.4	18.6
One sibling	43.7	42.9
Two or more siblings	34.9	38.5
Cultural background
Child is an Aboriginal or Torres Strait Islander	4.5	10.1*
Mother speaks a language other than English at home	35.4	18.1*
Work status
Both parents work or lone parent works	35.6	38.1
One parent works (couple family)	38.7	37.6
No parent works	25.8	24.3
Educational status
Mother completed Year 12	55.3	50.1*
Father completed Year 12	58.8	43.6*
Parents’ combined income ($ per week)
Less than 800	46.4	36.7*
800–1,499	38.7	47.4*
1,500 or more	14.9	15.9

Note: * Indicates that there was a statistically significant difference between the CfC and contrast sites. The percentages included all families who participated at Wave 1.

Source: SFIA.

[ top ]