Performance Framework and Data Collection
- What changes are there in the FaRS, FLS and Specialist Services Annual Service Report in 2011-12?
- What changes are there in the CfC and Community Playgroups Performance Reports in 2011-12?
- What are the data collections methods for Client feedback/follow-up?
- What will the data be used for?
- What is the FSP Performance Framework?
- How was the FSP Performance Framework developed?
What changes are there in the FaRS, FLS and Specialist Services Annual Service Report in 2011-12?
The first Annual Service Report due in 2012 under the new Funding Agreement (2011-14) will be significantly changed from previous years. Consultation with the sector was undertaken to develop a template that reflects the new reporting requirements to demonstrate the effectiveness of the FSP.
The new Annual Service Report will include sections for service providers to report their immediate and intermediate client outcome data for each service they are funded to deliver.
From 2012, the Annual Service Report is due on 31 July each year in order to meet timelines for reporting FSP outcomes in the FaHCSIA Annual Report. This new 31 July due date will mean consistent outcomes reporting due dates for all FSP services.
What changes are there in the CfC and Community Playgroups Performance Reports in 2011-12?
CfC and Community Playgroup service providers report to FaHCSIA using a six monthly Performance Report. In previous years, Performance Reports were consistent for both halves of each financial year. From 2011-12 onwards, the July to December period Performance Report (due on 28 February each year) will be a shorter version focusing on key basic information. The report for the January to June period (due on 31 July each year) will include additional sections to report annual data on items such as performance indicators, complaints, client access plans and approval requirements where applicable.
What are the data collection methods for Client feedback/follow-up?
The 2011-14 Funding Agreements require service providers to collect data under the FSP Performance Framework.
All services are encouraged to collect feedback from clients throughout the year to assist with internal evaluation and continuous improvement of services. Only 20 consecutive working days (4 weeks) of client feedback, collected at the time of service, is required to be reported to FaHCSIA. During the service provider nominated 20 day reporting period, all clients attending services should be asked to complete an anonymous survey. It is anticipated that a minimum of 50% of clients will respond. This relates to the 50% specified as the sample size in your Funding Agreement.
All services will also be required to seek intermediate client outcome data by following up clients who accessed their service in the previous 3 to 6 months, or in the case of continuing contact, 6 months after commencing the service. The Department will identify a small sample number for each service, based on the July December client numbers.
Service providers will need to seek consent from clients to be recontacted 3 to 6 months after receiving services. Parental permission should routinely be sought for a person under 18 years of age to provide feedback.
The Department has produced a list of suggested questions for both immediate (at the time of service) and intermediate (follow-up) client feedback data collection. A pilot was undertaken with some CfC providers to test the questions and collection tools. Consultation with other service providers was also undertaken to finalise the list. This list will give providers possible questions to ask clients for each Performance Indicator.
Exemptions from providing intermediate outcome data may be provided to services in exceptional circumstances, for example, if receiving minimal funding from FSP or for a service that engages a transient client group where follow-up would not be achievable.
Service providers will also nominate a small number of external partner agencies with whom they collaborate. These agencies will be contacted by an agency contracted by FaHCSIA and invited to participate in a survey to assess the contribution of the service providers to integrated and coordinated service delivery.
Family and Relationships, Specialist, and Family Law Services provide the immediate and intermediate client data as aggregated client responses in the Annual Service Report due on 31 July each year from 2012. Examples of action taken or improvements made based on open-ended immediate feedback may also be included. Service providers will be required to collect client feedback against all four immediate outcomes and the two intermediate outcomes most relevant to each service.
The FRSP Online Client Feedback form is no longer used for the collection of immediate outcomes. Other data collection for FRSP online will remain the same. As defined in your Funding Agreement, client data must be entered into FRSP Online within 28 days of the service being provided.
Communities for Children will provide the immediate and intermediate client data as aggregated client responses in the Performance Report due on 31 July each year. Examples of action taken or improvements made based on open-ended immediate feedback may also be included.
CfC providers need to collect data on the two immediate client outcome performance indicators specified in their Funding Agreement. CfC providers must collect and report data on the two intermediate indicators most relevant to their service.
Service providers will need to seek consent from clients to be recontacted 3-6 months later. Parental permission should routinely be sought for a person under 18 years of age to provide feedback.
What will the data be used for?
FaHCSIA needs data and information from service providers funded under the FSP to:
- Assess service performance under the funding agreement between FaHCSIA and the service provider including benchmarking
- Report on the overall performance of the FSP and its components; and
- Contribute to evaluations to inform future program and policy development and improve program effectiveness and efficiency for clients.
The box below shows in more detail how different types of data will be used.
| Data to be collected by FSP service provider | Will the data be used by FaHCSIA to monitor service performance? | Will the data be collated by FaHCSIA for reporting at the national (program) level? |
|---|---|---|
|
Funding agreement performance indicators |
Yes |
Some |
|
Activity information |
Yes |
Yes |
|
Client information |
Yes |
Yes |
|
Deliverables (CfC) |
Yes |
Yes |
|
Story/narrative |
Can contribute |
No but can be used as supporting examples. |
|
Client feedback/ Immediate client outcomes |
Yes |
Yes |
|
Intermediate outcomes |
No |
Yes |
|
Evaluation data |
No |
Yes |
|
Annual service report (FaRS, FLS and Specialist services) |
Yes |
Some |
Benchmarking
As specified in 2011-12 – 2013-14 funding agreements, service providers need to meet activity specific benchmarks. The data for all providers offering a similar service will be analysed twice a year, commencing in February 2012, to determine a benchmark. Where individual providers are significantly above or below the benchmark, Funding Agreement Managers will review their performance with them in more detail to determine whether there are valid reasons for variations. The benchmarking methodology and review process has not yet been finalised and further information will be available in the future.
What is the FSP Performance Framework?
The FSP Performance Framework is designed to provide a set of parameters against which to measure program outcomes. Drawing on data aggregated across the program, the Performance Framework provides a consistent, logical and streamlined basis for reporting on the achievement of FSP objectives at the program level. The Performance Framework also helps to ensure that services are clearly focused on achieving positive outcomes for clients.
The Performance Framework covers all activities delivered under the FSP. Each service only contributes to those aspects of the framework that are relevant to their specific activities and their clients. The framework also links with funding agreement performance indicators and requirements.
Specifically the FSP Performance Framework aims to:
- Ensure that services are clearly focused on achieving positive outcomes for clients;
- Provide a logical and consistent approach for measuring outcomes across the program. It does so by defining a common set of high-level outcomes and performance indicators which are applicable to all FSP services, and can be adapted or modified as required to measure specific activities; and
- Link the performance reporting of specific activities and the overall program performance including the achievement of FSP outcomes.
How was the FSP Performance Framework developed?
FaHCSIA has developed the FSP Performance Framework in partnership with some services within the Family Support Program to improve data collection. Consultation with Children and Parenting services began in 2009, following the announcement of the FSP. The new framework was built on the FRS Performance Framework, and has been adapted to be appropriate across the whole FSP.
All Family Relationship Services were invited to participate in pilot testing of data collection methods in 2010. Separate pilot testing involving selected Children and Parenting services was undertaken in 2011. Family Relationships providers have been supported by consultants to collect data on longer-term client outcomes in 2010-11. These results have informed the Performance Framework and the further development of program performance indicators.
The Performance Framework needs to be able to encompass all activities delivered under the FSP. Most services types have been given the opportunity to be involved in the development of the Performance Framework.
Each service only contributes to those aspects of the Performance Framework that are relevant to their specific activities and their clients. The Performance Framework also links with funding agreement performance indicators and requirements. Edition 1 of the FSP Performance Framework became effective on 1 July 2011.