Family Relationship Services Guidelines

4 Performance Management and Evaluation

4.1 Performance Measurement

A funding recipient's performance is measured against the Funding Agreement requirements, which include an annual Service Assessment (see 4.6 Service Assessments), an annual Service Report and input into FRSP Online (see 4.4 FRSP Online).

From July 2010 and the commencement of the implementation of the Performance Framework (see 4.2 FRS Performance Framework); reviews of a funding recipient's performance will include the outcomes it achieved for clients.

4.2 FRS Performance Framework

The FRS Performance Framework (Framework) has been developed in partnership with the Joint Sector-Government Working Group on Quality and Performance Measurement and the FRS sector. The Framework (see Table 4) asks four key questions about service delivery:

how much did we do (outputs)
how well did we do it (processes)
did we make a difference in the short term (immediate outcomes), and
did we make a difference in the longer term (intermediate outcomes)?

Much of the data required under the Framework is collected through FRSP Online, particularly in relation to the first three questions. Collection of new data including for National Find and Connect Service where required will be phased in and some of the data collection may be done by occasional surveys or service specific data management systems.

Data collected under the Framework is required for reporting and FRS program management purposes. This data complements research and evaluation activities, and sits along side other quality assurance mechanisms such as:

competitive selection processes
FRS Administrative Approval Requirements compliance processes, and
service assessments (see section 4.5 Quality Assurance and Risk Management) against client targets and compliance with the Funding Agreement which will increasingly focus on achievement of longer term outcomes following full implementation of the Framework.

As the Framework is implemented it is intended that data will be available for use at the national and Funding Recipient level.

The Framework's program logic shows the relationship and linkages between the FRS outcomes, processes and outputs. The best practical performance indicators, data sources and counting rules point towards achievement of these. Performance indicators have been mapped as either ‘Key' or ‘Other' to specific FRS services.

For example, the FRS program's achievements in helping parents improve the way they work together to parent their children (improved parenting arrangements) is indicated by the percentage of clients making parenting arrangements and the percentage of clients reporting that the parenting arrangements are workable. Importantly, Funding Recipients will not need to report against all performance indicators for individual clients of specific FRS services.

The Commonwealth will continue to work with the sector including National Find and Connect Services to develop the best approach to collect and report longer-term outcome data. The pilot testing of the forms developed for the collection of client outcome information and data on co-ordinated service delivery will provide information that will be used to inform the data collection methodology.

While the staged implementation of the Framework will commence in July 2010, some aspects of the Framework will need to be phased in over time. The Department is preparing the FRS Performance Framework Implementation Guide to help providers through the process.

For the implementation of the Family Support Program (FSP) from July 2011, we will need to have in place the performance framework that will capture all FSP components, meaning we need to include performance indicators for other programs under the FSP. The FRS indicators will remain but will become a part of the whole of FSP Framework.

Table 4: Family Relationship Services Performance Framework
Program objective: Improved family functioning
Program logic	Performance indicator	Data source	Counting rule	FREST	MFR	SFVS	AMFT	FRSFSC	FRSHE	FRSC	KIDS IN FOCUS	MENSLINE	Couns	FRC	FDR	RFDR	POP	CCS	PSCP	SCASP	FRAL	TDRS	RRFSS	NFC
Intermediate outcomes: did we make a difference in the longer term?
Improved family functioning	% clients with improved family functioning including child wellbeing	Service provider survey	# clients better able to cope or deal with the issues they received support with as a % of followed up clients who answered the question	K¹	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	-
Immediate outcomes: did we make a difference in the short term?
Reduced harmful behaviour	% clients with reduced harmful behaviour	FRSP Online Client Feedback Form	# clients reporting there is less conflict in the family and/or members of the family are safer as a % of all clients providing feedback to the questions	K	K	K	K	K	K	K	K	-	K	-	-	-	-	-	-	-	-	-	-	-
Improved family relationships	% clients with improved relationships (parents/partners/children/communities)	FRSP Online Client Feedback Form, Mensline and FRAL statistical reports	# clients reporting improved relationship with partner/ex partner and/or with children as a % of all clients providing feedback to the questions	K	K	K	K	K	K	K	K	O	K	O	O	O	K	O	K	K	O	O	K	-
Improved knowledge and/or skills	% clients reporting improved knowledge and/or skills related to their presenting need	FRSP Online Client Feedback Form, Mensline and FRAL statistical reports	# clients reporting that they can apply a new skill and/or knowledge to their relationship and/or situation as a % of all clients providing feedback to the question	K	K	K	K	K	K	K	K	K	K	K	O	O	K	O	O	O	K	O	K	-
Improved parenting arrangements	% clients making parenting arrangements	FRSP Online Client Activity Outcome Form	# clients who made full or partial agreement as a % registered clients with presenting issues 'parenting' or 'post separation parenting' or 'step parenting'	O	_	_	K	_	_	_	_	_	O	K	K	K	K	_	O	_	_	K	_	-
	% clients reporting parenting arrangements are workable	FRSP Online Client Feedback Form	# clients reporting parenting arrangements workable as a % of all clients providing feedback to the questions	O	_	_	K	_	_	_	_	_	O	K	K	K	K	_	O	_	_	K	_	-
Satisfaction with assistance received	% clients satisfied that the support service they received was relevant to their circumstances and needs	FRSP Online Client Feedback Form, Mensline and FRAL statistical reports, NFC statistical reports	# clients reporting they would recommend a service to others as a % of all registered clients providing feedback	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
Improved access to the wider service system	% clients satisfied that they received adequate information and/or referrals	FRSP Online Client Feedback Form, Mensline and FRAL statistical reports, NFC statistical reports	# clients satisfied that they received adequate information and/or referral as a % of registered clients providing feedback to the question	K	K	K	K	K	K	K	K	K	K	K	O	O	O	O	O	O	K	O	O	K
More effective resolution of complex family separation issues	Extent of linkages between services and (1) the courts, (2) legal assistance providers and (3) other family law system services to the benefit of clients	Not currently collected - suggest reports from services (and their partner agencies) in relation to the working with (1) the courts, (2) legal assistance providers and (3) other family law system services	Suggest proportion of positive comments within and across reports	-	-	-	-	-	-	-	-	-	-	K	-	-	K	-	-	-	O	-	-	-
Processes: how well did we do it?
Inclusive practices	% clients from priority groups	FRSP Online, Mensline and FRAL statistical report, NFC statistical reports	# Indigenous, CALD, vulnerable, child, male and low income clients as a % of all clients (for services reporting in FRSP Online, refers to registered clients) # Elderly, terminally ill as a percentage of all clients for Find and Connect Service	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
Inclusive practices	% clients from priority groups satisfied with the appropriateness of the service for their background/needs	FRSP Online Client Feedback Form, Mensline and FRAL statistical reports, NFC statistical reports.	# clients reporting that they were treated with respect as a % of clients from priority groups providing feedback	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
Integrated service delivery/coordination	% referrals taken up	FRSP Online Referral Out Details and Notification Form, Mensline, FRAL statistical reports	Suggest # clients reporting referrals to other FSP service taken up during course of intervention as a % of all registered relevant clients	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	-
Integrated service delivery/coordination	% partner agencies reporting satisfaction with the contribution of the funding recipient to integrated service delivery/coordination	Departmental survey	# organisations whose partner agencies reporting satisfaction with the contribution of the funding recipient to integrated service as a % of total partner agencies providing feedback to the question	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
Service quality	% funding recipients that meet accreditation/approval requirements including safety	FRS annual status report and internal and external assessments	# funding recipients meeting accreditation/ approval requirements at point in time as a % of all funding recipients	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
Outputs: how much did we do?
Intake and assessment, information and referral, Education and skills training Counselling, Family/Dispute resolution, Change-over and supervised contact, Support Community development/ Access strategies, Outreach, Coordination and familiy histories.	# clients	FRSP Online Client Details Form, Mensline, FRAL and Find and Connect statistical reports	Sum of # clients assisted	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
	# services	FRSP Online Session Details Form, Mensline, FRAL and Find and Connect statistical reports	Sum of # services provided	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K
	# referrals	FRSP Online Referral Details Form, Mensline, FRAL and Find and Connect statistical reports	Sum of # referrals made	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K	K

1 The capital letter 'K' stands for 'key performance indicator' and the capital letter 'O' for 'other performance indicator'. This designation reflects the policy intent of individual service types and does not diminish the importance of individual performance indicators.

The Framework also:

responds to the increased Commonwealth focus on measuring the longer-term benefits of service delivery and reducing red tape, and
complements ongoing research and evaluation activities to provide funding recipients and the Commonwealth with data on the benefits of FRS services for Australian families.

4.3 Performance Recognition and Improvement

FaHCSIA will negotiate Performance Improvement Plans with those funding recipients whose Service Assessments identify underperformance issues. As there may be a range of reasons for underperformance, Performance Improvement Plans will be agreed between the funding recipient and the Network. Monitoring will occur in the context of ongoing contract management activities.

Performance Improvement Plans will:

identify performance indicators that are not being met by the funding recipient;
require the funding recipient to develop strategies in consultation with the department to improve its performance (such strategies could include additional training for staff or seeking the assistance of other funding recipients); and
identify a date (no more than six months from the development of the plan) when the funding recipient's performance will be reviewed.

On review, if there has been no or limited improvement in the funding recipient's performance, FaHCSIA will, at its discretion, consider extending or refining the Performance Improvement Plan or terminating the Funding Agreement.

4.4 FRSP Online

FRSP Online is the FaHCSIA developed system for the FRS Program. FRSP Online consists of a web-based portal for data entry and a secure database for storage of the data.

Most performance information (e.g. client characteristics and service usage) is collected by funding recipients and entered directly into the web-based portal for FRSP Online. Alternatively information may be transferred through another approved software application (third party system) directly into the FRSP Online Database. If a third party system is used it is the funding recipient's responsibility to ensure the system can be adapted to FRSP Online development requirements within a reasonable timeframe agreed to by the Department.

Data is to be collected and reported in compliance with the FRSP Online Protocols (Appendix B).

The FRSP Online reporting portal enables:

the Department to monitor funding recipient performance, and
funding recipients to interrogate their own data and compare it against national benchmarks.

FRSP Online training is available from the FRSP Online Support Centre (see 7.7 Information Technology Support).

Funding recipients may also be required to collect additional data items through FRSP Online.

4.5 Quality Assurance and Risk Management

Applicants for funding under FRS must comply with the Approval Requirements (Appendix A). These are a set of 15 standards including governance, data provision, training, confidentiality and safety. Compliance against these standards is initially externally assessed and reviewed every three (3) years by an independent FaHCSIA approved body. Ongoing self-assessment against the Approval Requirements is recorded in an annual service report which is confirmed by the funding recipient's Board. This process helps to provide families and staff with assurance that quality services are being delivered by a proficient organisation.

4.6 Service Assessments

Under the terms of the Funding Agreement, FRS funding recipients are required to participate in, and achieve, satisfactory Service Assessments. Service Assessments are conducted by the FaHCSIA Network. These assessments are generally conducted between 15 February and 15 May of each year. Service Assessments incorporate an annual desk assessment of funding recipient's compliance against their obligations under the Funding Agreement and a review of previous risk assessments, performance issues and unresolved complaints.

4.7 Site Visits

Site visits are conducted in addition to desk assessments:

for all funding recipients delivering a new service (one site visit at the main outlet for each service in their first year of operation);
at a minimum of at least once during the term of the Funding Agreement;
if issues raised in the Service Assessment indicate the need for additional site visits; and
as required to address performance and/or compliance issues at the outlet level.

Site visits provide an opportunity for funding recipients and Network staff (where appropriate National Office staff) to discuss any issues related to compliance with these Guidelines and/or other funding requirements.

Network staff will meet with the funding recipient's senior management and/or outlet staff during this process. Visits may be conducted at a funding recipient's head office or at an outlet.

4.8 Financial Accountability

FaHCSIA is obliged to comply with the Financial Management and Accountability Act 1997⁸ to ensure the efficient, effective and ethical use of Commonwealth resources. FaHCSIA requires funding recipients to provide annual audited financial statements in accordance with the Funding Agreement and ensure funding is only used for the purposes for which it was provided. As FRS is a Discretionary Grant Program, the above obligations sit alongside the Commonwealth Grant Guidelines⁹.

The Australian National Audit Office (ANAO) undertakes independent financial statement audits of all Commonwealth entities in accordance with recognised professional standards. Performance Assessments of selected Commonwealth public sector activities include financial reporting, administration, control and accountability.

In consideration of the Commonwealth's substantial financial investment, ANAO has an interest in the management and administration of the FRS.

4.9 Evaluation and Research

FaHCSIA periodically undertakes or commissions program evaluations and reviews to ensure FRS are high quality, effective, efficient and appropriately targeted to the needs of families. With the development of the new Performance Framework, there may also be a need to conduct or commission occasional surveys with providers and/ or clients to allow data collection against performance indicators.

FaHCSIA will notify funding recipients of:

the intention to conduct an evaluation/ data collection;
the focus of the evaluation/ data collection;
who is conducting the evaluation/ data collection;
the time period in which it is to be conducted;
where appropriate, the involvement sought from the funding recipient ; and
any special strategies, or other information relevant to the collection of data

FRS funding recipients are required to cooperate with external evaluators and researchers commissioned by the Commonwealth to undertake research and evaluation of family law reforms and associated services. Funding recipients may also undertake additional research activities. These optional activities may include local research projects, with or without the assistance of research partners.

http://www.finance.gov.au/financial-framework/fma-legislation/index.html
Effective of 1 July 2009 available at http://www.finance.gov.au/publications/index.html

[ top ]