Executive summary

SGS Economics and Planning Pty Ltd (SGS) and Tallegalla Consultants Pty Ltd (Dan Gillespie) were commissioned in January 2005 to complete an evaluation of the Fixing Houses for Better Health (FHBH) Projects 2, 3 and 4 for the then Australian Government Department of Family and Community Services (FaCS). In January 2006 the Office of Indigenous Policy Coordination (OIPC) and the Australian Government of Family and Community Services merged to form the Australian Government Department of Families, Community Services and Indigenous Affairs.

The purpose and scope of the evaluation

The purpose of the evaluation was to evaluate the performance of the FHBH Projects in achieving their objectives and to make recommendations for future FHBH Project rounds.

The scope of the evaluation was to:

understand the social, economic and political context in which the FHBH Projects operate
evaluate the effectiveness of the development and design of the FHBH Projects
understand the interrelationship between FHBH Projects and state and territory housing policy
assess the implementation of the FHBH Projects with an emphasis on community involvement and the development of on-community housing maintenance capacity
assess FHBH Projects against their objectives
assess the cost-effectiveness of FHBH Projects.

Approach and method

The evaluation was guided by an evaluation framework, which was developed in conjunction with the steering group for the evaluation at the outset of the commission. The evaluation framework contained the following elements:

a distillation of key program objectives for the FHBH Projects
a list of key evaluation research questions
nomination of potential data sources
nomination of suggested performance measures.

As part of developing the evaluation framework, Stage 1 consultation was undertaken with key stakeholders such as government officers and other individuals directly involved in the design, development and implementation of FHBH Projects and 'housing for health' policy in general. Stage 1 consultations helped to clarify perspectives on the purpose and objectives of the FHBH Projects, and provided information for further analysis in the development of findings.

Having settled the evaluation framework, Stage 2 consultations involved visits to five case study communities, with cases selected to achieve variability in physical and social context. The consultants participated in the Survey Fix 2 stage of a FHBH Project in one case. Discussions were held with community-level representatives, and observations were made regarding FHBH Project outcomes.

Further research involved the collection and analysis of parts of the FHBH Projects database, to explore whole-of-program and case-study-level data. This necessitated cooperation with Healthabitat Pty Ltd, the developer of the database, to understand how to use the database.

On the basis of the above research, findings were developed against each of the key evaluation research questions. Based on the findings, a set of recommendations was produced. Some of the recommendations refer to program change and improvements, while others reflect on the potential broader learning from the FHBH Projects.

Limitations

The evaluation has openly acknowledged certain limitations, which included:

The evaluation was not expected to explore quantitative connections between FHBH Projects and improved health outcomes for Indigenous people as a direct result of FHBH implementation.
Although an assessment of the design and operation of the 'housing for health' methodology was an important part of the evaluation, there was no intention to evaluate the core philosophy and principles behind the FHBH Projects.

Other limitations as they apply to specific aspects of analysis undertaken during the evaluation are explained within the report.

The contents of the report

Apart from the first two sections, which introduce the evaluation and the method employed, the report contains:

a section describing the context for and design of the FHBH Projects
a section describing the evaluation's detailed findings
a section setting out overall conclusions and recommendations for program change.

Three appendixes are also provided: Appendix A, which sets out in some detail observations made for each of the case study communities; Appendix B, which provides some examples of key documents used during the FHBH Projects and for the evaluation; and Appendix C, the evaluation framework.

Findings

The key findings of the evaluation are set out in summary form below. They are grouped according to the evaluation framework's Key Program Objectives (KPOs) and the Key Evaluation Research Questions relevant to each KPO.

KPO 1 To improve the safety and functioning of housing within the Indigenous communities where FHBH has been implemented, and in a cost-effective way

KPO 1 Summary of findings

There were very significant problems with Indigenous housing conditions in all FHBH communities prior to the commencement of FHBH Projects.
Given the spread/range/number of communities receiving a FHBH Project, and that no community 'failed' the feasibility assessments, it is likely that the state of Indigenous housing conditions in FHBH Project communities is reasonably representative of other rural, remote and very remote Indigenous communities.
Key problems with Indigenous housing condition prior to FHBH Projects were found in relation to most critical Healthy Living Practice (HLP) areas, and particularly Fire, Structure and access, and Drains.
FHBH Projects fixed a significant number of problems over the average six-month period between Survey Fix 1 and Survey Fix 2. In fact, in most cases, the least functional housing at Survey Fix 2 was more functional than the average level of functionality prior to Survey Fix 1.
The Survey Fix method moved a considerable number of communities towards having a large proportion of their housing 100 per cent OK, particularly with regard to Power, water and waste, Flush toilet working, Shower working, Electricity, and Laundry. In addition, improvements in average scores were also recorded for Electricity, Gas, Structure and access, and Drains.
Despite these significant improvements, there are still a significant number of problems remaining with Indigenous housing conditions after FHBH Projects. After Survey Fix 2, there was still no community that had between 80 to 100 per cent of its housing 100 per cent OK against all critical HLPs. Problems remain with structural elements of houses (for example, egress associated with escaping from fires), bathing children, and storing and preparing food.
Limited financial information was available, but it would seem that the majority of FHBH Project budgets have been spent on capital upgrades/fix and repair work-over 60 per cent on average and up to 80 per cent in some communities. Expenditure shares align reasonably well with critical HLPs requiring the most attention. On average, about 6 to 7 per cent of FHBH Project budgets were spent on project establishment and design specification and tender. This does not seem excessive and indeed seems necessary to ensure tasks required can be managed and completed.
Given the improvements, and the analysis of the types of jobs completed, it would seem that, in general, the most critical problems are being fixed. Although the number of low-priority and high-priority tasks finalised are almost equal among the case study communities, tradespeople are being used appropriately and efficiently to fix critical (high-priority) problems in the majority of cases, whereas local Survey Fix Teams are fixing many of the low-priority jobs.
Based on the fact that no community achieved between 80 and 100 per cent of their housing 100 per cent OK against all critical HLPs at Survey Fix 2, it would seem that the average of $5,000 per house was not sufficient to achieve the FHBH Project standard in any community. This was confirmed by a limited financial analysis for Case Study A, which indicated that, all other things being equal, to complete the remaining jobs required to bring all housing to 100 per cent OK against the FHBH Project standard after Survey Fix 2, an indicative total budget of around $12,000 per house would be required.

KPO 2 To transfer housing maintenance systems, skills and employment to the Indigenous communities (and Indigenous Community Housing Organisations) in which FHBH has operated

KPO 2 Summary of findings

There were generally good levels of community involvement/Indigenous Community Housing Organisations (ICHO) involvement during FHBH Projects.
Those community members who did participate were formally employed during the FHBH Project survey assessments, receiving a market wage for their time.
Methods for training community members were 'excellent'. Training was provided in simple fix techniques, data entry and the survey check process.
However, only a very basic level of training and skill was provided. The vast majority of the necessary higher-level skills (such as housing repair work) are 'imported' with FHBH Projects.
Sustained skills transfer was very limited, but there are some examples of communities using skills learned during FHBH projects and continuing with rigorous housing maintenance systems after a FHBH Project had been implemented. It was hard to attribute any ongoing new employment to FHBH Projects alone but anecdotal accounts suggested this has happened on occasion.
Direct evidence of the consistent reuse of skills learned during a FHBH Project across the whole program was hard to find. However, the general view suggested that, in communities where there is a certain level of pre-existing housing maintenance and general governance capacity, there has been a lot of success in the ongoing take-up of some aspects of the FHBH Project method. Many stakeholders agree there is a great need for systemised approaches to service delivery-similar to the FHBH Project-in other areas of community management.
The 'silo' model for service delivery is a threat to the true potential of a FHBH Project to achieve community-wide, ongoing benefits.
Expectations were that further capacity development and training would be needed to achieve ongoing application of FHBH Project systems and skills in most cases.
Resource limitations are often the main determinant of the housing maintenance method employed or preferred at the community level.

KPO 3 To encourage states and territories to adopt housing assessment and maintenance programs in their asset management systems

KPO 3 Summary of findings

Other states and territories, and regions and communities, are aware of and in some cases are using or have used the FHBH Project method independently.
There is a degree of mild resistance to adopting the FHBH Project method independently among some states and territories, perhaps due to perceptions about the sufficiency of pre-existing historical responsibilities/approaches and differing perspectives on the healthy housing debate.
Licensing arrangements for the FHBH Project were said to be an impediment to wider adoption of the method at the state and territory level.
States and territories recognise that the FHBH Projects should rightly be recognised as one approach to understanding and developing Indigenous housing maintenance requirements, but that there are merits in other approaches/philosophies.
There is a growing push among states and territories for better-resourced and documented maintenance programs, but it was not clear the extent to which FHBH Projects had been responsible for raising the standard.
Some states and territories expressed a view that whole-of-government coordination and cooperation around Indigenous housing, including the adoption of specific systems of maintenance such as FHBH, should not be imposed but negotiated.
Many examples were found of where FHBH Project funds were used in conjunction with other resources and funding sources to leverage better housing maintenance outcomes, including in most of the case study communities.
It was often found that FHBH Project funds could 'take care of the basic essentials' in maintenance needs, thereby freeing up other funding sources to focus on 'big ticket' improvements such as minor and major upgrades. This aspect was widely recognised as a very successful aspect of the FHBH Projects and demonstrated the power of coordinated efforts between different levels of government.

KPO 4 To provide a point-in-time analysis of the quality of housing stock in Indigenous communities (to determine progress toward Building a Better Future outcomes)

KPO 4 Summary of findings

The FHBH Projects database is an excellent framework for understanding housing conditions in Indigenous communities. It provides a very 'necessary', 'detailed', 'contextualised', 'comparable' and 'objective' baseline statement of Indigenous housing conditions.
It was also recognised that there are very significant (and misleading) limitations in other data sets that are often used to understand and predict housing need in Indigenous communities, increasing the importance of the FHBH Projects database.
The FHBH database is not a census of the entire Indigenous population. The FHBH Projects are focused upon rural and remote Indigenous communities. Thus, while the outcomes of this study are very important indicators of the condition of Indigenous housing in many areas, generalisations of the data across all Indigenous communities should not be made.
Nonetheless, the database is a sound and strengthening indication of the likely condition of Indigenous housing in rural, remote and very remote areas of Australia. The database does provide an effective point-in-time analysis of housing condition in FHBH Project recipient communities.
The FHBH Projects' data collection and handling frameworks have consistently evolved based on field experiences and learnings as the various generations of projects have occurred.
The ongoing usefulness of the FHBH Project database as a measure of housing condition in Indigenous communities was very widely embraced.
The FHBH Projects database has taken great strides in developing a much deeper understanding of housing asset maintenance needs among Indigenous communities in a very broad range of contexts with differing capacities. For this outcome, it is a model to be roundly applauded.
No matter how successful or effective a program might be-and FHBH Projects have been-it will still be necessary to find ways and means of better coordination between the various efforts of different agencies if program outcomes are to be maximised and sustained. That is, a good understanding of the problem as developed via a high-quality database is a necessary but not sufficient tool in its own right.

Conclusions and recommendations

In making conclusions and recommendations, the evaluation recalled the scope of and terms of reference for the study which required investigation of the following aspects of the FHBH Program:

program context and development
program design
program implementation
program outcomes
program costs
program cost-effectiveness
program change.

With these areas of interest in mind, and reflecting on the evaluation's findings, the conclusions and recommendations were categorised and discussed under the following headings:

A total of 16 recommendations have been made. A brief discussion providing a rationale for each set of recommendations under each of the above headings was provided. The recommendations are set out below.

Achievement of program objectives

Recommendation 1

That the success of the FHBH Projects in achieving the primary objectives of fixing the most critical health hardware deficiencies of dwellings located in participating communities and compiling a comprehensive database which records the 'point-in-time' condition of Indigenous housing be acknowledged and the FHBH Projects' primary objectives be strongly endorsed as a means of improving Indigenous housing outcomes.

Recommendation 2

That the FHBH Project delivery method be acknowledged and endorsed as a successful means of program delivery, particularly with regard to good resource planning and achieving practical outcomes in relation to 'on-the-spot' fixing of health hardware deficiencies. It is a conceptually straightforward methodology which accords with best practice asset management principles, and which can be successfully applied by FHBH Project managers and participating communities. It has been shown to be appropriate and adaptable to its circumstances and to provide an objective 'evidence-based' means of assessing the status of Indigenous housing.

Recommendation 3

That the demonstration role of the FHBH Projects in capacity and partnership building be built upon, but with explicit regard for the limits to what this role can achieve, and with a recognition of the pressures inter-program coordination can place upon local project managers. High-level whole-of-government policy and program coordination (such as the Building a Better Future framework) should continue to be promoted as the primary means of improving the context in which the FHBH Projects operate, and should seek to leverage the demonstrated benefits that the projects provide.

Potential for improvements

Recommendation 4

That the FHBH Projects be acknowledged for widely applauded success in providing critically required practical improvements for housing, collecting useful information about housing conditions, actively engaging communities in project delivery, and winning the support and enthusiasm of community members in particular.

Recommendation 5

That, once there is sufficient information available, a program-wide evaluation of financial data be undertaken to investigate the relationship between 'average' critical health hardware function at Survey Fix 1 and the resources required to achieve 100 per cent OK for health hardware, as a means of establishing an effective average budget per house for the FHBH Projects.

Recommendation 6

That the principle of introducing flexibility in budget setting post-Survey Fix 1 be adopted, and that research be undertaken into developing a budget-setting formula based on scores achieved in the initial survey. This could produce two stages for setting budgets for FHBH Projects:

Stage 1-a standard minimum average allocation per house to allow for preparation and implementation of Survey Fix 1
Stage 2-a budget allocation based on the results of Survey Fix 1 for further fix work/capital upgrades and Survey Fix 2.

Recommendation 7

That the funding for FHBH Projects in each state and territory be based on a multi-year budgetary cycle of three to five years.

Recommendation 8

That the HLP ratings of health hardware function are validated by an independent verification of the assessment method and the relationship between HLP assessments and health risks. This verification would, as a minimum, have regard for mainstream benchmarks for housing standards that demonstrate a connection to health outcomes.

Recommendation 9

That the housing condition assessments undertaken by FHBH Projects should continue to collect information about elements of critical health hardware that would require major structural changes to dwellings to achieve better outcomes (so as to inform other responses such as improvements in housing design). However, the success of a FHBH Project in improving HLPs in this category should be assessed with resource limitations understood.

Recommendation 10

That changes to the FHBH information system be implemented so as to enable project-by-project financial information to be incorporated, and that all available financial information previously gathered be integrated into this system.

Recommendation 11

That, for the benefit of advancing the national understanding of the condition of Indigenous housing, nationally aggregated FHBH Project data be held by and accessible via a suitable public or non-profit body, which would regulate the use of the data under a suitable public licence and monitor access.

Sustainability of outcomes and relationship to other housing and environmental health programs

Recommendation 12

That regionally-based delivery of FHBH Projects and subsequent routine maintenance programs be investigated as an option for servicing smaller remote communities with limited capacities; and that the feasibility of using Shared Responsibility Agreements as a means of supporting the sustainability of FHBH Project outcomes be investigated further. When investigating these options, regard should be given to the risks associated with the potential collapse of regional delivery systems and agreement-based approaches, which could leave individual communities stranded without the skills and support necessary to manage housing.

Recommendation 13

All housing-related programs should be preceded by a standardised and comprehensive 'planning assessment' of community conditions. This planning assessment would identify and assess opportunities for the implementation of housing programs and threats to the sustainability of housing program outcomes. The planning assessment would assess areas such as governance, human resources, asset management capability and the influence of remoteness. The planning assessment would also identify or prescribe the need for other non-housing programs, such as community capacity-building programs, to operate ahead of or alongside housing programs.

The planning assessment would inform all subsequent strategic planning for a coordinated program response at the community level.

Recommendation 14

To maximise the FHBH Project's value as a resource planning and outcomes evaluation tool:

That consideration is given to adopting Survey Fix 1 as a standard, comprehensive baseline assessment of individual dwelling condition in all communities. This baseline assessment of dwelling condition would then inform the allocation of resources from all housing and infrastructure programs towards the repair and provision of housing and housing related infrastructure

and

That Survey Fix 2 is conducted on a periodic basis as a tool for evaluating progress and the sustainability of outcomes for all housing and infrastructure programs.

Recommendation 15

That the data collected via standardised Survey Fix 1 and Survey Fix 2 assessments be used to maintain the national FHBH Project database as the definitive measure of Indigenous housing condition, so as to facilitate nationally consistent longitudinal monitoring and assessment of housing standards, and to coordinate program responses over the long term.

Recommendation 16

That consideration be given to, where required, supporting FHBH Projects with a complementary household environmental health and capacity-building program which could be mobilised during or subsequent to a FHBH project, with the aim of contributing to and sustaining better healthy housing outcomes.

Return to top