Stronger Families in Australia study: the impact of Communities for Children

Executive summary

This report presents the results of the evaluation of the short-run impacts of the Communities for Children (CfC) initiative on child, family and community outcomes. CfC was one of three models of service delivery funded under the Australian Government’s Stronger Families and Communities Strategy (SFCS) 2004–2009. The CfC initiative aimed to:

improve coordination of services for children 0 to 5 years and their families
identify and provide services to address unmet needs
build community capacity to engage in service delivery
improve the community context in which children grow up.

Under the CfC initiative, the Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) funded non-government organisations as ‘Facilitating Partners’ in 45 disadvantaged geographic areas around Australia to develop and implement a whole-of-community approach to enhancing early childhood development.

The Stronger Families in Australia (SFIA) study is an evaluation of the impact of CfC. It was a central component of the SFCS evaluation 2004–2008 that was undertaken by the Social Policy Research Centre (SPRC) at the University of New South Wales and the Australian Institute of Family Studies (AIFS).

The overarching aim of the SFIA evaluation study was to measure changes in child, family and community outcomes in CfC communities over the funding period, and potentially beyond. This aspect of the evaluation was designed to:

identify whether the CfC initiative had an impact on child, family and community-level outcomes
ascertain whether there were any differences in these outcomes for different groups of children.

The SFIA evaluation study was based on a three-wave longitudinal study of 2,202 families living in 10 sites that had a CfC program and five sites that did not have a CfC program but were in other ways comparable to the CfC sites (contrast sites).

The first wave of fieldwork was conducted during the consultation and partnership-building phase of the CfC initiative, and this provided a baseline of data with which impacts of the CfC initiative were identified. Implementation of CfC services and programs varied across the sites, commencing around the time of the Wave 2 data collection. Wave 3 was conducted in 2008, approximately one year after CfC program activities were under way. At that stage, any impact of CfC was expected to be small, given that the evaluation was of short-run effects and the intervention was intended to have an impact on all families living in CfC sites and not just those who directly accessed services.

The effects of the CfC initiative were estimated using statistical techniques that allowed child, family and community outcomes in the CfC sites to be compared to what they would have been in the absence of the CfC intervention (using outcomes in the contrast sites). The outcomes measures related to four priority areas:

healthy young families—child injuries requiring medical attention, child and parent physical health, children’s experiences of emotional and behavioural problems, children’s prosocial behaviour, children being overweight, and parents’ mental health
supporting families and parents—harsh parenting, parenting self-efficacy or self-confidence, parent relationship conflict, and living in a jobless household
early learning and care—children’s receptive vocabulary achievement and verbal ability, and the quality of the home learning environment
child-friendly communities—parents’ involvement in community service activities, the level of support parents receive from others to raise children, the quality of the neighbourhood as a place to raise children, parents’ sense of community social cohesion, their perception of the quality of facilities in the community, and the level of unmet service needs.

The strengths of the study include having:

a large sample, representing 42 per cent of the population of 2 year-old children in 10 CfC sites and five contrast sites at Wave 1
low attrition (that was not systematic) at longitudinal follow-up
a sophisticated approach to design and analysis and robust measurement of outcomes
an appropriately matched comparison group (families and children in the five contrast sites were a valid counterfactual).

The overall conclusion is that, on balance, there is evidence that CfC had positive impacts. The positive impacts were that:

fewer children were living in a jobless household
parents reported less hostile or harsh parenting practices
parents felt more effective in their roles as parents.

It also appears that the CfC intervention was associated with parents reporting lower levels of child physical functioning. There is some evidence of a negative impact on the mental health of mothers with low education (defined as Year 10 education or less), and poorer general health of mothers in relatively lower income households. However, it is unclear whether this reflects an actual deterioration in child and parent health in CfC sites compared to children and parents in non-CfC sites. It is possible, for example, that exposure to CfC programs and activities brought parents and their children to the attention of professionals and others who may have recognised undiagnosed health conditions, or that CfC programs and services in some other way increased parents’ understanding of their own and their children’s actual health needs.

For many of the outcome measures, the estimated impact of CfC was not statistically significant, which is not surprising given that any effects were likely to be small in the short-run. Depending on the statistical model estimated, between two-thirds and three-quarters of the outcome variables indicated a positive, although not necessarily statistically significant, effect. Although non-significance means that it is not possible to say with a high level of confidence that the individual effect was not different from zero, the skewed pattern of results towards positive effects provides support for the conclusion that CfC has had some positive impacts in the short-run.

The SFIA evaluation study also estimated whether the CfC intervention had different impacts for three groups that are at particular risk for poor child outcomes and who have been shown in some studies to be less likely to benefit from area-based interventions. The groups are:

hard-to-reach households with at least one of the following characteristics: no father present, mother not employed and father not working/not present, low household income, maternal education Year 10 or less, a parent born overseas, and child is of Aboriginal or Torres Strait Islander origin
households with low incomes ($485 a week or less)
households with mothers with low education (Year 10 or less).

These groups were defined according to their characteristics pre-intervention (that is, at the time of the Wave 1 interview). The evaluation provided evidence that the CfC intervention had a positive effect for at least some of the measures of wellbeing among low-income, low maternal education and generally hard-to-reach households.

Positive and statistically significant findings in relation to these hard-to-reach groups included:

higher levels of receptive vocabulary and verbal ability among children of mothers with Year 10 education or less
less hostile/harsh parenting among hard-to-reach parents
higher involvement in community service activities among parents in households with relatively lower income
higher involvement in community service activities in households comprising mothers with Year 10 education or less
fewer children in jobless households across all three subgroups
increased parental perception of community social cohesion reported in relatively lower income households at the p<0.07 level of statistical significance.

Consistent with the estimates for the population as whole, there were also some negative findings from the CfC intervention on health outcomes for the hard-to-reach, low-education and low-income groups. They were:

decreased reported mental health of mothers with Year 10 education or less
decreased reported general health of mothers in relatively lower income households
decreased reported child physical functioning among children in all three subgroups.

Similar to the overall pattern for the population as a whole, the overall pattern of significant and non-significant estimates was consistent with the interpretation that there was some evidence of CfC having a positive impact. Specifically:

69 to 78 per cent of the outcomes at Wave 3 for low-income households reflected a positive impact of the CfC initiative
50 to 64 per cent of the outcomes at Wave 3 for households where the mother had low levels of education suggested positive impacts of the CfC initiative
60 to 65 per cent of outcomes at Wave 3 for hard-to-reach households suggested positive impacts of the CfC initiative.

Overall, these findings demonstrate that the CfC initiative had some success in improving outcomes among the most vulnerable children and families in relation to children’s early receptive vocabulary and verbal ability, joblessness rates and mothers’ involvement in community activities.

The effect sizes of the CfC impacts on all outcomes were small, but can be considered positive relative to what was observed in the early phase of Sure Start (a large-scale area-based initiative in the United Kingdom). The current results were also comparable in size to those found in the later impacts evaluation of Sure Start, where 3 year-old children were exposed to more developed programs from birth.

An important question, however, is the extent to which these effects compare with alternative early childhood interventions that target specific client groups and seek to enhance child outcomes through other processes, such as centre-based programs, home visiting programs, case management interventions and parenting programs. A 2005 review of the effectiveness of early childhood interventions found that most studies reported effect sizes on parenting and child outcomes that were negligible to small (Wise et al. 2005). It should also be noted that most of these evaluations measured outcomes for children who were directly enrolled in the program, whereas CfC is aimed at improving outcomes for children in the whole community.

The fact that the effect sizes of CfC were comparable to, if not greater than, many alternative early childhood interventions, and that these effects were evident irrespective of whether parents and children in the CfC communities had actually received services, seems to point towards an additional effect over and above the provision of new, stand-alone services, possibly as the result of a better coordinated local system of early childhood services and/or other enhancements to the community context in which children develop.

[ top ]