Stronger Families in Australia study: the impact of Communities for Children

Endnotes

1. The study was identified as the Stronger Families in Australia (SFIA) study for general communication purposes such as recruitment and participant feedback, and described as the Families Study in the framework for the national evaluation (Social Policy Research Centre [SPRC] & Australian Institute of Family Studies [AIFS] 2005).

2. The questionnaires developed for the Wave 1, 2 and 3 fieldwork are provided in Web Appendixes 1–3, available at <www.aifs.gov.au/cafca/pubs.html#evaluation>.

3. Conceptual frameworks that are currently influential in developmental psychopathology for explaining the determinants of child developmental outcomes are the risk and resiliency framework (Garbarino 1995; Rutter 1999), and the bioecological developmental systems perspective (Bronfenbrenner & Morris 2006).

4. Similarly, the Sure Start impact evaluation also did not employ randomisation (NESS Team 2005).

5. There were also no statistically significant differences at the 95 per cent confidence level at Wave 1 for the low-income, low maternal education and hard-to-reach subgroups.

6. As the difference-in-difference models made use of the longitudinal nature of the data, it is also necessary to control for clustering within individuals. In practice though, the design effect was far larger for community clustering than clustering within individuals. Consequently, adjusting standard errors for community clustering was preferred, as it was a more conservative approach.

7. Clustering of children within families did not need be taken into account as data on only one child from each family was collected.

8. An alternative approach would be to use multilevel modelling, also known as hierarchical linear modeling (Rabe-Hesketh & Skrondal 2005) to account for the clustering of children within communities. These types of models can also be used to estimate the effect of neighbourhood characteristics and family/child characteristics on children's developmental outcomes. Given that the purpose of the evaluation is to estimate the impact of the CfC intervention and we are not interested in neighbourhood and family effects as such, we have chosen to not directly model clustering at the community level. As a sensitivity check we have estimated multilevel models for the Wave 3 cross-section. The results are essentially unchanged.

9. The validity of this method requires that there be no unobserved differences that would result in different levels of wellbeing between CfC and contrast sites in the absence of the intervention.

10. SEIFA consists of four indexes developed by the Australian Bureau of Statistics. Each index summarises a different aspect of the socioeconomic conditions of the Australian population, using a combination of variables from the Census of Population and Housing.

11. Another check of the robustness of findings was also performed. Outcomes that were statistically significant using the Wave 3 regression were robust to a site fixed effect (a dummy variable for every site).

12. Although the $485 cut-off for being a low-income household is based on the distributional pattern of income in the sample, it is correlated with the financial wellbeing of the household. For example, 42.0 per cent of families who were defined as low-income households at Wave 1 could not raise $2,000 for an emergency, while a further 19.3 per cent would have to do something drastic and another 20.6 per cent could raise $2,000, but it would involve some sacrifices (for example, reduced spending, selling a possession).

13. These percentages are for the total sample. For the balanced sample, 16.2 per cent of households had a mother who had Year 10 education or less and 14.4 per cent were low-income households at Wave 1.

14. The actual bottom quintile of income cut-off differs slightly from that used in the analysis of the full sample because of the need to exclude several 'outliers' from the analysis. Approximately 20 persons were identified as recording high-income data, which was not correct, and were excluded from the analysis. This resulted in a slight increase in the quintile cut-off from $485 to $500, and meant seven additional people were coded as hard-to-reach. It made little discernable difference to the analysis for the hard-to-reach families, and results are comparable to those produced for the main SFIA analysis, which was based on the slightly lower quintile cut-off.

15. Teenage parents were excluded from the hard-to-reach sample if they did not meet the other criteria; fewer than 15 teenage parents in the whole balanced panel sample were thus excluded.

16. The Australian Early Development Index (AEDI) is a community measure of young children's development, based on the scores from a teacher-completed checklist consisting of over 100 questions. The AEDI checklist measures five areas of child development: physical health and wellbeing; social competence; emotional maturity; language and cognitive skills; and communication skills and general knowledge. Teachers of children in their first year of formal schooling complete the checklist on each child after they have had a chance to observe their development. Between 2004 and 2006, the AEDI was trialled in 54 Australian communities over six states and territories on more than 30,000 children.

17. Random sampling of dwellings in the study areas was considered, but was found to be prohibitively expensive. An administrative database was therefore needed. The best available source of administrative data was the FTB database. The major advantages of the FTB database are that the age of the children can be directly identified, it has excellent coverage of lone-mother families (approaching 100 per cent), it has excellent coverage of couple families with lower to upper middle income, it has geographic information, and it has been successfully used as a sampling frame for other surveys.

18. This includes one person who requested that their contact details be removed from the database.

19. Each time a household visit occurred to attempt contact, interviewers left behind an I-view calling card. Where families were difficult to locate, interviewers were required to locate the family by asking neighbours, the local shop or a community group for the best time to visit or call on the family.

20. The percentage of Wave 1 respondents participating at Wave 3.

21. There was also no evidence of statistically significant differences at the 0.05 level at Wave 1 for the groups tested in the differential effects.

22. Effect sizes were calculated as the difference in average scores between treatment and control groups, divided by the pooled standard deviation of the two groups. In the evaluation of the CfC intervention, measures of effect size were estimated by controlling for key control variables and for outcomes for the Wave 3 regression and difference-in-difference models. In the instance of difference-in-difference estimation, we would expect the estimates of the effect size to be conservative estimates, while in the case of the estimates from the regression analysis, they would be upper-bound estimates of the effect size.

23. Another possible explanation for a pattern such as this is that the Wave 1 scores were skewed in one direction and therefore any change was likely to be towards the mean (regression to the mean). This explanation could be ruled out in this case because the distribution of the outcome measures at Wave 1 were similar for the CfC and contrast sites.

24. Full details of the statistical models used to produce Tables 5 to 7 appear in Web Appendix 4, available at <www.aifs.gov.au/cafca/pubs.html#evaluation>.

25. Subsequent analyses for children with receptive vocabulary and verbal ability one standard deviation below the population mean suggested that children with mothers with low education who resided in CfC sites were 2.56 times more likely (p=0.05) not to have below-average verbal skills than children with mothers with below-average education from contrast sites. This method was used by Love et al. (2002).

26. A danger when attempting to estimate the impact of an intervention on a number of outcomes is that a statistically significant effect can be due to sampling error. In this case, the number of significant results was larger than what one would expect from chance. In addition, the pattern of results held both for the overall results and the subgroups, suggesting that the significant effects were not due to chance.

27. Hard-to-reach families had at least one of the following characteristics at Wave 1: no father present in the household; mother of household aged 20 or less; mother unemployed/or not in the labour force and father not working/not present; parental income $500 or less; maternal education Year 10 or less; and having a parent born overseas.

28. For not-low-income households, 65 per cent of regression and 57 per cent of difference-in-difference estimates were in the positive direction.

29. For households with mothers with Year 11 education or more, 65 per cent of regression and 75 per cent of difference-in-difference estimates suggested a positive impact of CfC.

30. For not-hard-to-reach households, 88 per cent of regression and 54 per cent of difference-in-difference estimates were positive.

31. Parents of 3 year-old children had lower levels of hostile parenting and provided a better home learning environment. Children in SSLP areas had better social competence and self-regulation that appeared to be a consequence of the benefits of parenting. Rates of immunisation were higher and children in SSLP areas had fewer accidental injuries than in similar areas that did not have a SSLP. Families also used more child and family-related services than in other areas.

32. The following definition of negative mood was provided to the interviewer: 'A negative mood includes fussing, pouting, whining, crying and vocal or physical expression of anger'.

33. The following definition of fearfulness was provided to the interviewer: 'The extent to which the child shows fear toward you when you make a social or physical approach toward the child or when you present material to the child'.

34. PPVT-III-LSAC Australian Short-Form-developed by Dr Sheldon Rothman, Australian Council for Educational Research, Melbourne.

35. The Rasch modelling was conducted by Dr Sheldon Rothman from the Australian Council of Educational Research. Raw scores were de-identified to preserve the anonymity of the children. See Appendix C.

36. See Web Appendix 4, available at <www.aifs.gov.au/cafca/pubs.html#evaluation>

[ top ]