• Table of Contents
• Executive Summary
• Introduction
• Background
• Methodology
• Communities for Children service outcomes
• Communities for Children sector outcomes: service coordination and collaboration
• Communities for Children outcomes for families, children and communities
• Communities for Children model: strengths and weaknesses
• Local Answers
• Invest to Grow
• What works in early intervention? Cross-strategy findings from Communities for Children, Invest to Grow and Local Answers
• Appendix A: Methodological limitations
• List of shortened forms
• Endnotes
• References
• List of Tables and Boxes

National evaluation (2004–2008) of the Stronger Families and Communities Strategy 2004–2009

• Previous: Invest to Grow
• Next: Appendix A: Methodological limitations

10. What works in early intervention? Cross-strategy findings from Communities for Children, Invest to Grow and Local Answers

10.1 Broad conclusions
10.2 Program/project implementation
10.3 Data collection and evaluation
10.4 Funding and timelines
10.5 Sustainability

[ top ]

10.1 Broad conclusions

• The SFCS 2004–2009 successfully raised the profile of early intervention and the need for a coordinated approach to the early years across Australia.
• Overall, CfC was successfully implemented and achieved its objectives. Data limitations have prevented the evaluation from making strong conclusions regarding the outcomes of LA or ItG.
• SFCS 2004–2009 successfully engaged families traditionally thought of as hard-to-reach.
• CfC has shown that the community-based, coordinated, capacity building approach has substantial potential.
• The three strands, however, were not well integrated.

[ top ]

10.2 Program/project implementation

Strategic issues

• Most projects were effectively designed to meet individual needs, with tailored engagement strategies and activities for specific groups or individuals.

Key elements

• Programs/projects/activities were community-focused: they were community-based and allowed community members to contribute to the design, implementation and management of the project.
• Non-government organisations (NGOs) which delivered programs and activities that were well established, respected and locally known were effective in implementing projects and recruiting families.
• Programs/projects/activities were outcome-focused, which was positive for families and communities, as NGOs and others regularly rethought objectives.
• Agencies benefited from implementing changes as projects progressed and lessons were learnt through early feedback, evaluation and experience.

Challenges

• Remote areas with limited infrastructure, high costs, workforce issues and sometimes seasonal weather were very challenging for SFCS 2004–2009. Implementation took much longer and was less successful in remote areas than in regional or metropolitan areas.
• Overall, it appears that CfC did not meaningfully engage with the private sector. A more strategic approach was needed for this to occur.
• The technical and operational support available was in many ways inadequate for such a major, complex program. In particular, many projects required support in areas such as financial planning, community asset mapping, commissioning evaluations, data collection, and implementing evidence-based interventions.

Workforce

Key elements

• Staff development opportunities and training were beneficial.
• Quality staff were essential for all aspects of the initiative. Project managers in particular needed to be:
  • knowledgeable about the local community
  • effective leaders
  • competent managers of people and finances
  • able to network with other organisations/initiatives
  • effective facilitators for service integration.
• Cultural competence among staff was important and community liaison officers were beneficial for organisations in targeting, engaging and effectively supporting families and children from specific cultural backgrounds.

Challenges

• Recruitment and retention of adequately qualified staff was challenging for many projects, especially in regional and remote areas. Recruitment problems led to implementation delays and other problems in many cases. Staff turnover was generally very disruptive for projects and initiatives.
• Recruitment of Indigenous staff was particularly challenging.

Collaboration and community engagement

• Interagency and inter-sectoral working was an important component of all three strands. All were relatively successful in engaging with a range of other stakeholders.

Key elements

• Partnership arrangements needed to be flexible and to adapt to events as they unfolded.
• Understanding of the different roles and responsibilities of agencies and practitioners was important in effective working relationships.
• Coordination at strategic and operational levels was important.
• Funding for service coordination activities helped greatly in getting agencies to work actively together.
• Having a key leader with the ability to facilitate the coordination of services and networks was important for successful collaboration (especially where they were able to facilitate transparent and effective consultation and communication processes, and resolve conflicts).
• Previous history of collaboration was helpful in developing partnerships.
• Collaboration between NGOs and local councils was effective in implementing place-based early intervention activities and strategies. These relationships worked well.
• Mentoring and support from large NGOs was helpful for smaller organisations.
• The use of soft entry points—taking services to families, rather than expecting families to come to services—was very effective across all three strategies. This helped to build relationships and trust between workers and families, and engaged families who would otherwise have been unlikely to attend particular services.

Challenges

• Collaborations between state and territory government departments (such as health and education) and NGOs are important for early intervention programs. Many NGOs struggled to get these relationships to work effectively.
• There were some challenges for the collaborators, especially those who did not have a financial stake in SFCS 2004–2009 and who tended to lose interest after a while.
• Engagement of community members in strategic roles when they were not employees of community organisations was extremely challenging for projects and initiatives.
• All strands of the SFCS 2004–2009 found engagement with Indigenous communities challenging; it required in-depth consultation, longer periods of time, skilled, well-trained staff, and workers or volunteers with close links to the community.

[ top ]

10.3 Data collection and evaluation

Data collection

Key elements

• SFCS 2004–2009 had a very ambitious plan for data collection. The focus on outcomes as well as outputs was innovative.
• Combining data collection for management and accountability purposes with evaluation was also innovative.

Challenges

• However, data collection was hampered by a number of technical and other factors:
  • The electronic data collection tool was developed quickly and needed to be regularly upgraded for its 18 months of operation.
  • Data definitions were complex and difficult to standardise across the initiative.
  • Projects found it difficult to report on outcomes. This was done mainly in qualitative format, which was difficult and expensive to analyse.
  • The quantity of data collection was very challenging for many service providers, especially small NGOs who were not used to this level of reporting.

Evaluation

• Local evaluations had some benefit for CfC and were integral to ItG.

Challenges

• There were a number of issues for local evaluations:
  • difficulties in commissioning evaluations by projects and initiatives due to lack of understanding by project managers of the practicalities of program evaluation
  • too little funding for evaluations to address outcomes
  • lack of expertise in this sort of evaluation within academia and consultancy
  • lack of control groups in most local evaluations
  • future evaluations of area-based interventions that differ by community needs would benefit from greater funding of local evaluations and better integration of evaluation results.
• The national evaluation also had to address some significant challenges with implications for future evaluations of this sort. These include:
  • lack of secondary data at the local level suitable for the outcome indicator framework, which highlighted the need for a national social indicators project designed to set up a national indicators framework
  • limitations of the progress reporting template in providing reliable data that could be analysed quantitatively
  • low response rates to questionnaires from service providers requiring multiple reminders and eventually a change of method to computer assisted telephone interviewing (CATI)
  • a lack of capacity within many projects to respond adequately to requests for promising practice profiles (PPP) submissions, and a consequent need to support them
  • evaluation timescales that limited the time for analysing many of the ItG evaluation final reports, resulting in only preliminary findings for some ItG projects
  • confusion about the relationship between the national and local evaluations, and the consequent lack of pre and post-outcome data on outcomes for children and families who participated in SFCS programs and activities.
• Further waves of the SFIA study are required to assess whether the short-term effects continue into medium and long-term outcomes and to confirm whether the trend findings become significant. This is important to further inform future policy.

[ top ]

10.4 Funding and timelines

• SFCS funding increased service provision and capacity in disadvantaged areas.

Key elements

• SFCS funding provided opportunities to:
  • assess assets
  • address service gaps
  • establish preventative services
  • trial innovative programs.

Challenges

• The short-term nature of the funding created difficulties for all three strands of SFCS 2004–2009. It affected the recruitment and retention of staff, the capacity to provide flexible and effective services, and the commitment of other agencies to the initiative.
• In some cases the funding rules created difficulties:
  • Projects were required to be implemented quickly. For some sites, the timeframes were not reasonable, especially in remote areas.
  • The perception that funding had to be wholly committed in the initial stages limited the flexibility of some projects and sites to respond to changing circumstances and opportunities.
  • The relationship between Australian, state and territory government responsibilities constrained development in some cases. For example, the requirement that SFCS funding should not be provided for services which are a state responsibility.

[ top ]

10.5 Sustainability

Key elements

• Positive changes in outcomes for children, families and communities were evident, however it is too early to assess their sustainability.
• Some networks and service coordination may continue.

Challenges

• Sustainability of CfC within the short timeframe is unrealistic.
• Some services may be sustainable if provided by an agency with ongoing capacity and access to other funding sources.
• A paid facilitator may be important in ensuring the sustainability of some service networks and service coordination.

• Previous: Invest to Grow
• Next: Appendix A: Methodological limitations

[ top ]