The research reported in this document was conceived to develop a typology to assist policy makers and others with an interest in the provision of family services in making some distinctions among the wide range of service activities collectively known as 'relationship education'. The research provides some insights into the many providers and the diverse range of service activities that are available for individuals, couples and families across selected groups of providers across Australia.
The process of attempting to capture the details of the many different forms of service activities presented a considerable challenge for both providers of programs and the researchers. The process of describing and classifying their services using tightly defined criteria was not in keeping with some providers' views of services as organic entities that shift in focus and outcomes each time they are delivered. This inherent tension itself is a significant point of interest. Tightly defined and detailed descriptions of service activities are a necessary first step if those concerned with providing funds for these services are to be persuaded of the efficacy and capacity of services to deliver outcomes to address key issues relating to the health and well being of individuals, couples and families.
This chapter commences with a synthesis of the key outcomes from the study. It first analyses the nature of relationship education service activities. Key characteristics of these activities are then analysed as a means of further enunciating the basis for the development of a typology of service activities. The final section develops, from the findings of the research, a number of significant implications and presents some recommendations for future research and development activities.
8.1 Conceptualising relationship education
It is immediately apparent from the data collected that the provision of relationship education services is far broader and more complex than it might appear. These services are being provided in a number of different ways by a wide range of service providers within the health, human services and education sectors. The popular notion of relationship education as dominated by church organisations conducting stand-alone programs for a narrow range of client groups is strongly challenged by the diversity apparent in the data presented in Chapter 4 of this report. Based on that information, relationship education can be conceived variously as:
- a form of public education;
- a form of primary health care;
- a specific component of the school curriculum;
- a form of adult, liberal education;
- a form of education directed at social and moral development specifically within faith traditions or specific socio-cultural groups; and
- one component of an integrated and holistic continuum of social services that extends from prevention and early intervention through to crisis intervention.
The wider field of relationship education can be conceptualised as a series of freestanding or embedded activities. These can be offered as part of a range of social and welfare services offered by community-based health care facilities, hospitals, schools, adult and community education organisations, juvenile justice and corrections facilities, hospitals, youth services, religious and specific cultural groups. They are also available to the defence forces, in Employee Assistance Programs, maternal and child health centres, and neighbourhood and community centres. As the construction of the typology of programs demonstrates, within that broad framework there are groups of programs with particular characteristics that are conducted in more or less specific domains.
[
top ]
The phenomena of embedded programs highlights the potential for existing relationship education providers to expand their work by forming collaborative partnerships with organisations and groups working in related areas such as health and aged care. They are also perhaps indicative of the market for the adoption of service design structures based around the development of a series of modules that can be combined flexibly with a range of other content areas such as dealing with illness or disability or caring for an ageing partner (see, for example, the range of products built around the research of Howard Markman and his colleagues in both the USA and Australia).
The research also provided some insights into the various groups of relationship education service providers. Three primary groups of providers were identified:
- church-based;
- church-affiliated; and
- secular.
The distinction between church-based and church-affiliated providers is significant as it highlighted the diversity in approaches to service provision that exists in services connected with particular faith traditions. Church-affiliated and secular programs share many similarities in relation to the structure, content and organisation of relationship education service activities, but the analysis contained in Chapter 4 illustrates the manner in which these two groups of providers have developed 'niche' markets to service particular target groups and geographical locales. Additionally, the data appear to be suggestive of some gaps in the provision of service activities specifically for committed, pre-marriage and married individuals and couples by secular service providers.
8.2 Development of the typology
The process of developing the typology revealed that service activities could be distinguished on the basis of a number of key characteristics including the
- types of participants addressed by the service activity (target groups);
- cost of provision;
- protective factors addressed in the service activity;
- type of agency or provider (church-based, church-affiliated or secular).
Conversely there appears to be considerable uniformity amongst service providers in relation to the qualifications and skills they required their educators to hold, approaches to on-going professional development for educators, the learning domains that are addressed within programs, the approaches used in assessing participants' learning and evaluation practices. The data in Chapter 5, therefore, provide a basis for identifying the key characteristics or dimensions of relationship education service activities which are likely to be of most value to funding providers in seeking to be able to distinguish the nature of relationship education service activities offered by various providers. The profiles of the four groupings that comprise the typology outlined in Chapter 5 provide a framework that can be used to group programs for evaluation purposes and other research purposes.
[
top ]
8.3 Inventories as a form of relationship education service activity
The response to the survey of inventory users was low. However, it is possible to make some cautious comments about the practices involved in using the inventories based on the data received. Judging by this small sample the use of inventories differs from structured programs largely in scale. Aside from the administration of and feedback from the inventory itself, the content of additional sessions is similar to programs: family of origin, money and budgeting, and communication and conflict resolution are common. At least two thirds of inventory users indicate that skills training is incorporated into the sessions with couples, however given that on average there are only four sessions of approximately three hours each it would appear that the amount of skills training is rather limited in the majority of cases. The maximum length of a series of sessions was reported as between 12 (ENRICH users) and 16 (FOCCUS users) hours. Given that the first two sessions would be dominated by the inventory itself it seems that few couples are receiving comprehensive training in important relationship skills. From an educative point of view the one-on-one attention that an individual couple would receive is more desirable than learning such skills in a group setting. However, in both cases it is likely that the critical time required to practice these skills is not being provided in either program group- or inventory-based programs. Inventory-based programs are in a position to make lasting and measurable improvements to couples' relationships because of the close attention a couple receives, however it is suggested that the limited number of such sessions may nullify these benefits to some extent.
An additional concern regarding inventories relates to the follow up of couples. The nature of follow up varied markedly across respondents, ranging from offers or invitations to attend further sessions or programs or to make contact for counselling if needed, to personal contact or outreach via pastoral care, liturgy or newsletters. Subsequent contact is often left to the couple to initiate if they feel it necessary although in some cases appointments for later review sessions to address any additional needs are made at the time of completing the inventory. Some practitioners indicated that they were in the process of developing a formal follow up process. This is to be welcomed.
8.4 Issues in undertaking research on relationship education service activities
Developing and documenting relationship education service activities
The research processes used for the development of the typology proved effective in terms of providing the sorts of data required for the development of the typology, albeit that the data was limited in terms of the overall number of programs for which complete sets of data could be obtained. The data collection methods utilised in the study required respondents to provide significant amounts of information relating to their services, some of which was not easily accessible or available in a format that was amenable to public scrutiny. While accessibility to program information may in some cases be subject to the need to ensure the confidentiality of commercial products and services, the nature and quality of the available data, combined with the absence in some cases of documentation of service activities, raises some issues relating to the processes of program development and documentation used by service providers.
The issue of the availability of documentation relating to service activities is a complex one. On the one hand, some providers might argue that the documentation of service activities (in the form of a curriculum document, for example), represents a rationalist approach to service provision which minimises the potential for services to be responsive to the emerging needs of clients and downplays the value of educators' tacit knowledge and ability to respond 'in action' to clients' needs. There is always a considerable gap between the curriculum as planned and the curriculum as delivered. Planned service activities often need to be adapted for a range of reasons including the characteristics of the clients and the available resources. Additionally, a compelling argument may also be made in relation to the resource and time intensive nature of development and documentation processes that ultimately reduce resources for service delivery.
On the other hand, the development of documentation that outlines in detail the intended outcomes, content, teaching strategies, intended assessment and evaluation processes can be viewed as an attempt to capture the underpinning rationale behind the intended learning experiences for clients, which can then be subject to feedback from interested stakeholders (including clients). The documentation of the intended curricula for service activities provides opportunities for the internal logic of the service activity to be tested and revised in a way that provides the best potential for the intended outcomes to be achieved by the clients. Processes such as peer review and facilitating input from clients during program development become a possibility. Documenting intended service activities also provides a benchmark against which revisions and adaptations of service delivery can be mapped. This has the potential to add to the body of knowledge about effective service delivery and the ways in which the needs of various target groups might be best met. In the absence of documentation of the development processes and current status of service activities, evidence for subsequent program improvement and development can be left to rely on subjective information collected from evaluation processes, anecdotal evidence based on gut feelings and limited systemic observation and application of these data to educators' perceptions of the nature, scope and outcomes of service delivery activities. Processes measuring service quality which encourage service providers to develop systematic and transparent processes for the design of relationship service activities including exploring strategies such as peer review of proposed service activities and the inclusion of clients in service development activities potentially have a lot to offer in the quest to increase the overall quality of service provision.
[
top ]
Evaluating relationship education service activities
One of the other key issues that this research sought to address was the identification of 'best practice' models using pre-existing evaluation data from service providers. In attempting to meet this objective, several issues were raised in relation to the quality of existing evaluation data held by service providers including a lack of shared understanding of the potential focus and scope of evaluation activities, the types of data collected in evaluation processes and its generalisability in terms of making assertions about 'best practice' models of service delivery.
In broad terms evaluation of relationship service activities is concerned with a number of purposes
inter alia accountability to funding bodies, measuring outcomes and providing data from which improvements to service activities can be ascertained. Evaluation is concerned with the nature of the service activity as delivered and experienced by clients and also the impact of the service delivery, including some indication of the learning that has taken place for clients. This latter focus is usually referred in the Australian education literature as assessment and is distinguished as a separate but interrelated component of the overall evaluation process (Figure 8.1).
Figure 8.1: The relationship between assessment of learning and the evaluation of service activities
Assessment focuses on the process of
learning and is concerned with the collection of data in order that some judgement is made as to the progress that clients (learners) have made towards the goals that are the focus of the relationship education service activity. Assessment is concerned with changes in the clients in terms of their knowledge, skills, attributes and abilities. Assessment processes usually rely upon the articulation of the learning outcomes or objectives that then form the focus of the service activity. These outcomes can be pre-determined (as in the case of a service activity that has a pre-determined curriculum with stated objectives or learning outcomes) or they can be negotiated with or determined by the clients as an integral part of the service activity. The process of assessment necessarily requires that data be collected over time so that service providers (and usually clients) are able to provide input into decisions about progress in relation to learning against the stated outcomes. Hence, assessment should ideally have both formative and summative dimensions. As a formative process, assessment can be viewed as an integral part of the learning process. This requires service providers to think through how assessment activities might be integrated into the learning processes within programs as one means of tracking the progress of clients (see Figure 8.2). As a summative activity, assessment requires that judgements be made in relation to the extent of progress of clients towards and achievement of the stated learning outcomes / objectives of the service activity, thus contributing important data to the overall process of evaluating the service activity.
Evaluation, in contrast to assessment, usually focuses on an
evaluand, such as a course or product. It is concerned with collecting a range of data that will enable further development and improvement as well as providing data to address the accountability requirements established by funding providers and / or sponsoring institutions. Evaluation usually includes an interest in outcomes (in the case of relationship education, an assessment of the learning that has taken place) along with issues relating to client satisfaction on a range of program characteristics. In some instances evaluation will also be concerned with collecting data relating to the transfer of learning from the program context into the 'lived worlds' of the participants and the impact of this learning within that world.
Most existing approaches to assessment and evaluation reported by service providers (see Chapter 4) relied almost exclusively on general observation and post-service evaluation data collected from clients or educators. Very few instances of using data from referring agencies or agents or collecting data in follow up surveys or interviews were reported. The subjective nature and limited scope of these data did not allow any significant meta-analyses in relation to establishing best practice benchmarks to be performed. The quality of these data raises questions about ways in which a comprehensive evaluation framework for relationship education service activities might be developed, particularly in the light of the limitations to conducting evaluations using empirical, experimental designs noted in Chapter 2.
Tomison (2000) has canvassed these issues in relation to the evaluation of child abuse prevention programs and emphasises the importance of developing a variety of evaluation processes that includes, but is not limited to, empirical experimental studies. Owen and Rogers (1999) emphasise the importance of developing models of evaluation that are comprehensive and systematic and which equally address issues relating to input, process and outcome evaluation. Put simply, frameworks for evaluation need to be developed that contain a number of different levels or purposes that collectively provide detailed descriptive data on service delivery and measures for determining success. These frameworks provide a comprehensive approach to evaluation and emphasise the systematic collection of data related to developing an in-depth understanding of the service activity and some measure of the 'success' of the service (Tomison 2000, p. 11).
[
top ]
A number of evaluation frameworks are available that may provide useful starting points for service providers. One of the most well known and extensively used in the field of human resource development (arguably a field of work where the press to measurement of outcomes equals that felt by relationship education providers) is Kirkpatrick's levels of evaluation (cited in Tomison 2000, p. 11). This model consists of four levels of evaluation that could be undertaken successively and where each level of evaluation require more complex approaches to collecting data in order to provide valid measures of the behavioural change that has taken place:
- Ascertaining reactions (collecting data about levels of satisfaction with learning experiences);
- Ascertaining learning (changes to knowledge, skills and attitudes that have occurred as a result of the learning experiences);
- Ascertaining transfer (how has the learning been transferred to the 'everyday context'?); and
- Ascertaining impact (how has this learning impacted in that context?).
Similarly, Owen and Rogers (1999) have developed an evaluation framework consisting of five forms of evaluation. Each of these five forms can be used on its own or in combination with other forms, depending on the reasons for the evaluation process:
- Proactive evaluation (used to determine if a need exists for a program and how this need might best be met);
- Clarificative evaluation (used to clarify the design of a service and analyse the logic of a service in terms of the links between various service elements and intended outcomes);
- Interactive evaluation (used to during the implementation of a service to identify points for continuous improvement);
- Monitoring evaluation (used to examine service processes and outcomes as a basis for continuous improvement and provide accountability for use of resources); and
- Impact evaluation (used to establish worth of a service and provide accountability to stakeholders including clients).
Tomison (2000) and Owen and Rogers (1999) have noted that as expectations grow in relation to the degrees of accountability that should be provided by agencies and organisations in receipt of funding for service delivery, so too does the realisation that attempts to bring methodological rigour to program evaluations is fraught with challenges. The issue then is not the wholesale rejection of empirical control group studies with their associated attempts to identify causal relationships between outcomes and specific program elements. The issue is one of attempting to encourage 'greater pluralism' (Tomison 2000, p. 15) in evaluation practices and the importance of encouraging and supporting service providers to adopt comprehensive and systematic approaches to evaluation that can complement those empirical evaluations undertaken by researchers working collaboratively with service providers (see for example Halford 2000). The typology developed in this research study will provide valuable information in relation to the knowledge of the relationship education services available and provide some standardised way of providing detailed descriptions of the activities (including the target groups and information relating to a range of characteristics of activities) that can be used as a basis for future efforts in these areas.
[
top ]
8.5 Recommendations
The experience and outcomes of the research process to develop the typology of relationship education service activities point to some avenues worthy of further attention in order to progress the provision of quality relationship education services.
Recommendation 1
In the light of data examining the nature and availability of relationship education service activities to specific target groups, it is recommended that steps be taken to examine ways in which activities might be expanded to meet the identified gaps particularly in relation to services for
- Adolescents both outside and inside the school system who do not have the opportunity to benefit from existing opportunities to experience those aspects of the school curriculum addressing relationship issues;
- Older Australians;
- Persons from culturally and linguistically diverse groups;
- Individuals and couples from specific community groups;
- Individuals and couples with mental illnesses and intellectual disabilities; and
- Indigenous Australians.
[
top ]
Recommendation 2
In the light of evidence in relation to importance of systematic development and accurate and detailed documentation of services for accountability and research activities, it is recommended that ways in which 'best practice' in relation to service development and documentation be promoted.
Options for this could include
- the development and wide dissemination of discussion papers that critically reflect and discuss issues impacting on the processes used in the development and documentation of relationship education service activities, and
- examination of ways in which support for demonstration projects, action research or 'good practice' projects in the implementation of comprehensive service delivery development and documentation processes (including the incorporation of client perspectives) might be provided.
Recommendation 3
In the light of the issues canvassed in this report in relation to the evaluation of relationship education service activities, it is recommended that in addition to examining the conduct of empirical, control group studies to measure the outcomes of relationship education service activities, active steps are taken to engage service providers in discussion and debate with key academics and stakeholders in relation to ways in which systematic, developmental evaluation frameworks (covering input, process and outcome evaluations) might be implemented in cost effective and realistic ways within current service activity contexts.
Recommendation 4
In the light of data relating to current practices in relation to assessment of learning and evaluation processes adopted by service providers, it is recommended that active steps be taken to build the capacity of service providers in these aspects of service provision using a range of strategies that could include
- assessment of educators' current levels of competence in relation to assessment and evaluation practices to identify and prioritise gaps;
- the development of support materials to promote up-skilling in these areas of educators' work; and
- the provision of support for demonstration projects, action research or 'good practice' projects in the assessment and evaluation of relationship education service activities.
[
top ]
Recommendation 5
In the light of the growing use of inventories as a mechanism for providing relationship education and the importance of ensuring that the services provided using these tools are comprehensive as possible, it is recommended that active steps be taken to examine ways in which service providers who use inventories as part of their relationship education service activities can be encouraged to evaluate the extent to which their services embrace and use current understandings relating to systematic relationship skills development as part of the inventory process. Additionally, service providers should also be encouraged to incorporate systematic evaluation and follow-up procedures as part of their service provision. Strategies such as action research projects, demonstration or best practice studies, developed in partnership with organisations responsible for administering these inventories, may provide some useful ways of advancing these issues amongst educators and service providers.
REFERENCES
Andrews, M, 2000,
Threshold, no. 66, p. 30.
Andrews, M 1994, 'Taking family seriously: a national strategy to enhance marriage and family',
Threshold, no.44, pp. 14 - 20.
Cranton, P 1989,
Planning instruction for adult learners, Wall and Emerson, Toronto
Geasler, M.J. & Blaisure, K.R. (1998), A review of divorce education program materials.
Family Relations, 47(2), 167-175.
Gottman, J. & Silver, N. 1999,
The seven principles to making marriage work, Crown Publishers, New York.
Halford, W. K. 1999,
Australian Couples in Millennium Three. A research and development agenda for marriage and relationship education, Australian Academic Press, Brisbane.
Halford, W. K., Kelly, A. & Markman, H. J. 1997, The concept of a healthy marriage, in W.K. Halford & H. J. Markman (eds)
Clinical handbook of marriage and couple interventions, Wiley. Chichester, England, pp. 3 - 12.
Harris, R., Simons, M., Willis, P. & Barrie, A. 1992,
Love, sex and waterskiing. The experience of pre-marriage education in Australia, Centre for Human Resource Studies, University of South Australia, Adelaide.
House of Representatives Standing Committee on Legal and Constitutional Affairs, 1998,
To Have and To Hold. A report of enquiry into aspects of family services, The Parliament of the Commonwealth of Australia, Canberra.
Markman, H. J. 1981, 'The prediction of marital distress: a five year follow-up',
Journal of Consulting and Clinical Psychology, no. 49, pp. 760-762.
Markman, H. J., Floyd, F. J., Stanley, S. M. & Storaasli, R. D. 1988, 'Prevention of marital distress: a longitudinal investigation',
Journal of Consulting and Clinical Psychology, vol 56, no. 2, pp. 210 - 217.
Markman, H. J., Renick, M. J., Floyd, F., Stanley, S., & Clements, M. 1993, 'Preventing marital distress through communication and conflict management training: a four and five year follow-up',
Journal of Consulting and Clinical Psychology, no. 61, pp. 70-77.
Markman, H. J., Stanley, S. M. & Blumberg, S.L. 1994,
Fighting for your marriage: positive steps for preventing divorce and preserving a lasting love, Jossey Bass, San Francisco.
Owen, J & Rogers, J 1999, P
rogram Evaluation. Forms and Approaches, Allen and Unwin, Sydney.
Parker, R. (1999),
A framework for future research in pre marriage education. Australian Family Briefing No. 8. Australian Institute of Family Studies.
Simons, M., Harris, R. & Willis, P. 1994,
Pathways to marriage: learning for married life in Australia, Centre for Research in Education and Work, University of South Australia, Adelaide.
Tomison, A. 2000,
Issues in Child Abuse Prevention. Evaluating child abuse prevention programs, Paper No. 12, National Child Protection Clearinghouse, Australian Institute of Family Studies, Melbourne.