The previous chapter provided a rationale for the development of a framework or typology that could be used to assist policy makers, researchers and service providers develop a deeper understanding of relationship education services and how these might be organised for evaluative purposes. The study that is the focus of this report was designed to assist in the development of such a typology that would provide a mechanism by which programs could be grouped according to key characteristics and examined and classified according to their stated outcomes. This chapter provides a detailed description of the research process developed to achieve this outcome.
3.1 The key tasks of the study
Given the overarching aim of the study, the population for the study comprised all relationship education service activities in Australia. The range of programs, their distribution across the country and the relatively unknown nature of a large number of these services made the procedures for finding and recruiting providers a relatively complex and time consuming process. In developing the research process a balance was sought between maintaining a high level of methodological rigour and maximising the opportunities to capture a sample of providers that was representative of the relationship education field.
The brief from the former Department of Family and Community Services stated that the study needed to provide data that would assist in identifying the range and volume of relationship education activities across the nation. At the first Steering Committee meeting for the study this overarching goal was divided into two parts (see Figure 3.1). The first part of the study was to seek detailed information in relation to as many relationship education programs as could be reasonably captured given the financial and time limitations attached to the study.
Figure 3.1: Key tasks in the development of a Relationship Education (RE) Typology
The Steering Committee also requested that in part two of the study the research team undertake two parallel tasks. Firstly, they requested that the research team make particular efforts to develop a 'snapshot' of the nature and extent of programs offered by a number of key organisations which they considered were worthy of detailed investigation. Organisations of particular interest included:
- community neighbourhood houses;
- community health services,
- schools (public, private and independent);
- hospitals;
- correctional services;
- the defence forces;
- those which target specific cultural groupings;
- the major faith communities; and
- those providing Employee Assistance Programs (EAPs).
In capturing this 'snapshot' the Committee hoped that information relating to availability of programs from these organisations would help to identify the potential scope of the relationship education field and where expansion of services might productively take place.
Secondly, the Steering Committee were also interested in the practices of providers who used inventories such as PREPARE /ENRICH or FOCCUS. In order to provide this information, the research team agreed to undertake a small, indicative study of the activities of these providers as an adjunct to the main activity of the development of the typology of services.
3.2 The parameters of the research
The first task of the research team was to define the term 'relationship education activities' so that decisions relating to the inclusion or exclusion of programs in the project could be made appropriately for each of the three tasks that were to be undertaken.
Arriving at a definition of relationship education activities that allows programs to be properly characterised was difficult. Relationship education can potentially encompass a wide range of life stages and life events including
- pre-marriage;
- post wedding (including early, middle and later married life);
- separation and divorce;
- the transition into parenting;
- re-partnering; and
- step parenting.
Furthermore, relationship education can be aimed at particular sub groups within the broader population such as adolescents or parents-to-be, and/or to particular issues or sets of topics or skills. Additionally, relationship education can be the focus of an entire program or it can exist as a component of a program that focuses on a related issue (for example parenting). The content of programs, while largely constructed around a core of communication and conflict resolution skills, varies in accordance with the professional, pastoral and personal characteristics and leanings of the providers, and the client group for which the program is intended.
Hence, any workable definition needed to be broad enough to allow the inclusion of a reasonably representative range of programs while keeping the project within the bounds of manageability. In discussions with the Steering Committee, the research team made the decision that a program would be able to be included in the study if it met certain criteria. In order to be included in the study a program needed to:
- target adults or adolescents aged 16 years or more;
- have an early intervention/prevention focus;
- have as its key focus the development of knowledge, skills, attributes and values that contribute to and promote healthy stable relationships between adults with the aim of preventing relationship instability and family breakdown;
- be offered to couples or individuals at different points in the life cycle;
- be of minimum two hours duration;
- have an intentional focus on fostering /promoting learning for couples or individuals;
- have been conducted in past 12 months;
- have a curriculum that was either documented or could be documented as part of the research process.
These criteria were considered to be broad enough to capture the anticipated diversity of contexts and providers. In addition, these criteria were also designed to facilitate the inclusion of programs that address couple relationship issues in the context of a larger program, such as parenting programs, programs that target specific ethnic groups, or those focussing on groups such as same-sex couples or individuals and couples where one or both partners has some illness or disability. Programs focussing on anger management, communication, assertiveness or decision-making that may actually target individuals, were also able to be included if they addressed the application of those skills to particular relationships.
3.3. Sampling Method
There is no known, complete database of Australian relationship education service providers. Thus it was necessary to assemble a customised database from which potential project participants could be sought. This was achieved using three sources of information about providers of relationship education services.
The first source was the database of providers who participated in the Competency Standards for Relationship Educators project conducted in 2001. Permission to access the database was obtained from the Assistant Secretary of the Family Relationships Branch in July 2001. As providers were recruited into the project they were also asked to suggest other providers of similar or the same services whom the research team could contact. Through this process, information on one hundred and forty two programs was received.
A sampling procedure that enabled the project to cover the range of programs available across Australia outside of the known providers identified from the competency standards project was designed. Local Government Areas (LGAs) in each state and territory formed the sampling frames. A random sample of forty per cent of both rural and metropolitan LGAs was selected (142 of 355 rural, 74 of 186 metropolitan). Only half of these were to be contacted in the initial stage of the research process, with the additional LGAs being selected and used as a 'pool' from which replacement LGAs could be drawn when or if a LGA originally included could not be contacted or service providers could not be recruited into the sample.
Since a different system of local government exists in the Australian Capital and Northern Territories the selection of agencies/providers in those areas was managed separately from those in the states. In the ACT contacts were made through phone books and the networks of major agencies with branches in Canberra. Local government in the NT is in the form of three separate groups: Municipal Councils, Community Government Councils, and Independent Associations. Only the sample of Municipal Councils was used to select potential contacts.
The Council offices of each selected LGA were contacted for information regarding the existence and nature of relationship education services in their area. In most cases the Community Service Directory for the region was sent to the research team (either electronically or in hard copy), from which contact information for service providers was obtained. In some cases (for example where the community was particularly small) no Community Service Directory existed or was available. In these instances the Council Officer was asked for any known contacts or for other basic information such as details of local hospitals and welfare or community agencies that could be used as a starting point for identifying and contacting local providers (as well as themselves being assessed as suitable for inclusion in the project). If there appeared to be little or no information to be found and/or no service providers in the area, the LGA was replaced with the next LGA selected into the sample (as described above). The sample of LGAs drew forty-four programs into the study. (The selected LGAs are listed in Appendix A).
The third source was the contacts and networks established through use of the database from the competency standards project and the LGAs. Identifying and contacting service providers was expected to be difficult in some instances, particularly in regions where directories or lists of service providers were unavailable or did not exist. In these cases researchers supplemented the sample via the networks of established contacts. As part of the initial recruitment process, providers were asked to suggest other providers of the same or similar services in the general area who could be contacted with a view to participating in the project. In addition, if in the process of consultations with service providers, the inclusion of particular unselected councils appeared warranted then additions to the sample could be made. Using these connections a further thirty-four programs were included in the project. Contacts were also made with a range of organisations suggested by the Steering Committee (see 3.1).
3.4 Questionnaire Design
The questionnaire used to develop a detailed picture of existing programs was based on the 'Divorce Education Program Review Form' developed by Geasler and Blaisure (1998). This form was designed for use in assessing 37 divorce education programs offered in the state of Michigan, USA and is a relatively brief measure covering the major components of program development and operation: hours and sessions, content, conceptual bases, teaching strategies and evaluation. Sections from this measure were adapted and expanded to apply to relationship education programs.
Whereas Geasler and Blaisure designed the form to be completed by researchers, the relationship education service activities questionnaire was adapted for completion by service providers. Given that the nature and focus of some of the information requested in the questionnaire would not be familiar to some providers, a guide explaining some of the terms was created. (A copy of the Guide and Questionnaire can be found in Appendix B). Development of the questionnaire proceeded through several iterations and was informed by piloting and feedback from four experienced relationship education service providers.
3.5 Research method
Using one of the three sources of contact information mentioned above, phone contact was made with provider organisations and individuals and the project explained. In the first instance it was necessary to determine whether the agency or individual offered programs that may fit the project criteria. If such programs were conducted, the request for participation in the project was made. Where a provider conducted more than one program of interest, they were asked to complete a questionnaire for each. If the service provider agreed to participate, details of the preferred method of delivering the project materials was established (mail or e-mail). Reply-paid envelopes were provided for those receiving questionnaires by mail. The Guide and a letter explaining the rationale and parameters of the project, and the contact details of the project team accompanied questionnaires.
3.6 Response rates
A total of 1377 agencies or providers were contacted. Of the 228 agencies/organisations identified as conducting relationship education programs (as defined for this project), 221 (97%) agreed to participate (the remainder were unable to participate or declined participation).
Table 3.1 Contacted – participating organisations
| State |
Contacted |
No. participating |
No. progs* |
| Vic |
350 |
46 |
103 |
| NSW |
365 |
45 |
113 |
| Qld |
216 |
39 |
108 |
| ACT |
32 |
8 |
18 |
| WA |
107 |
27 |
74 |
| SA |
181 |
38 |
113 |
| NT |
51 |
5 |
9 |
| Tas |
75 |
13 |
23 |
| Total |
1377 |
221 |
561 |
Note: * Approximated as many program managers/educators indicated that they might have overestimated or overlooked programs
Of the 561 questionnaires expected, 250 were received. Thirty of these questionnaires were returned blank, perhaps indicating the amount of effort required to complete each questionnaire, especially where several programs were to be described. The final number of programs for which information was obtained was 220, an overall response rate of 39 per cent (see Table 3.2).
Table 3.2: Questionnaires returned
| STATE |
# USABLE |
STATE |
# USABLE |
| VIC |
41 |
WA |
41 |
| NSW |
43 |
SA |
52 |
| QLD |
26 |
NT |
4 |
| ACT |
8 |
TAS |
5 |
3.7 Coding frames and analysis
Both qualitative and quantitative information was requested of program providers. A great deal of the information was coded to indicate the presence or absence of a particular characteristic. To ensure coding was consistent, a process of coding and crosschecking was followed until a high level of agreement across coders was achieved. This process was particularly important for data relating to program content, learning outcomes, teaching strategies and skills taught in programs. In many instances individual providers had developed their own unique ways of describing these aspects of their programs and it was necessary to develop some common descriptors that would provide meaningful parameters around which the typology of programs could be developed.
The research team's knowledge of the field and a sample of twenty program questionnaires formed the basis of the initial set of likely responses to items requesting details of
program content. These categories were then tested on a random sample of fifty questionnaires, coded independently by each research assistant. The codes were compared and the categories reviewed and refined. A further fifty randomly selected questionnaires were again coded independently. The agreement between coders was 75 per cent. Further discussion of the instances where the coders disagreed took place to clarify their understanding and coding proceeded. Time constraints and the small sample of questionnaires remaining to be coded precluded conducting another iteration in the process of independent coding and checking of a further sample of questionnaires, so the coders worked simultaneously through the remainder of the questionnaires, discussing and reconciling the allocation of codes for those categories which had proved problematic throughout the process.
The same process was followed for the coding of the data for
learning outcomes. This process was facilitated by the use of three taxonomies drawn from the general adult education literature. These taxonomies describe the domains of learning (cognitive - knowledge; psychomotor – skills; affective - attitudes and values) and also provide an indication of the level of learning within these domains. The taxonomies are summarised in Figure 3.2. and further discussed in Chapter 5.
Whilst it is acknowledged that the division of a complex process such as learning into domains is problematic, the use of such tools provides a relatively simple method for ascertaining the exact nature of the learning that is the focus of a program and hence providing a benchmark outcome by which the participants' learning (and hence the overall outcomes of the program) might be assessed.
In the coding exercise for the learning outcomes, there was disagreement between the raters on only eight of ninety-six domains coded, and when both the level and domain of learning outcomes were considered the coders agreed in 75 per cent of cases.
In coding
teaching approaches, a list of twenty-five strategies was compiled. The presence or absence of each of these was recorded and each strategy was then coded using the groupings developed by Geasler and Blaisure (1998) according to the level of involvement it required of participants passive involvement, limited involvement or active involvement. The grouping of teaching strategies under these three headings is shown in Table 3. 3. and is discussed further in Chapter 5.
Table 3.3 Grouping of teaching strategies
| Passive |
Limited |
Active |
| Guest speakers |
Brainstorming |
Coaching |
| Handouts |
Case studies |
Games |
| Lecture / talk |
Couple time / pair discussion |
Artwork |
| Videos |
Discussion groups |
Mediation / relaxation |
|
Scenarios |
Psychodrama |
|
Homework |
Role play |
|
Journal writing |
Small group activities |
|
Large group work |
Genograms |
|
Question box |
|
|
Reflective activities |
|
|
Individual work / reflection |
|
* Couple time and pair discussion were combined into a single category.
Skills, and the time spent engaged in
skills training, were recorded as reported. For many participating providers, allocating time to each step of the skills training process was not possible due to the nature of the session. Many left this part of the questionnaire blank, therefore the average amount of time devoted to explaining, demonstrating, practising and giving feedback on each skill is based on limited data. Names attached to skills varied quite markedly and for ease of understanding groups of like skills were developed and these groups then used as a basis for classifying the information provided by respondents. The following ten skills were derived from the questionnaire information:
| Affective Issues |
Assertiveness |
| Anger |
Communication |
| Conflict Resolution |
Goal Setting |
| Parenting |
Relationship Management |
| Self Knowledge |
Stress Management |
Procedures for data analysis were undertaken using SPSS. These are further elaborated in chapters four, five and seven.
3.8 Limitations of the study
Although the research design and methods were selected to ensure high levels of validity and reliability in relation to the data collected, some limitations in relation to the conduct of a study of this nature need to be recognised.
The first limitation arises from the nature of the data collected in the study. The data collected from the service providers represents information relating to programs
as they were planned to be delivered – that is they represent the intended curricula and may, in some instances, not be representative of the programs
as they were delivered to clients. The researchers were also aware that data required to be provided by service providers was highly detailed and, in some instances, placed significant demands on service providers in terms of the time required to complete questionnaires and the degree to which they had ready access to the types of information required by the researchers. While every endeavour was made to ensure the quality of the data through mechanisms such as cross checking with providers and providing assistance in the completion of forms the quality of the data was limited, in some instances, by a lack of documentation of programs.
The second limitation arises from the sampling methods used in the study. As stated previously, no consolidated list of relationship education providers exists and a customised database had to be developed for the purposes of this study. Hence statements about the generalisability of the findings of this study across all relationship education programs need to be made with some caution since there are no established parameters defining relationship education activities that could form the basis from which these assertions might be made.
Miles and Huberman (1994, pp. 278 - 280) have provided a framework of questions, clustered into five main issues that provide a useful basis on which judgements about the quality of this study may be asserted:
- Objectivity / confirmability
- Reliability / dependability / auditability
- Internal validity / credibility / authenticity
- External validity / transferability / fittingness
- Utilisation/ application / action orientation
Issue 1: This issue relates to the acknowledgment of researcher biases and the explicit statement of the research process in order that other researchers might replicate it. This report provides an explicit public statement of the research process. Objectivity and confirmability are asserted in the approaches used to draw conclusions from the data collected during the study.
Issue 2: This issue effectively addresses the 'quality control' measures that have been put in place over the period of the research process. A statement of research questions provided the focus of the inquiry and a rationale has been enunciated for the selection of research methods based on the nature of these questions. The respondents for the study were selected on the basis that they were best placed to address directly the core issues set out in the research questions.
Issue 3: The credibility and authenticity of the study can be asserted by the use of approaches to data collection that were most able to provide the types of data required to address the core issues of the study. The study was based on the use of data collection tools specifically developed for the study. Different data collection processes were clearly ascribed to specific research issues. This clarity was carried through into the data analysis process where a number of clearly specified strategies were used to analyse the data collected from the service providers.
Issue 4: This issue addresses the generalisability or 'trustworthiness' of the findings generated by the study. On the basis of the restriction noted above it is not possible to assert the generalisability of the findings
per se. However, the issue of generalisability can be gauged with respect to a number of key characteristics of the study including:
- the use of a sample of providers that is theoretically diverse enough to allow assertions of broader applicability;
- provision of a detailed description of the programs included in the study which allowed comparisons to be made between this and other studies;
- acknowledgment of the boundaries of the study;
- provision of a level of detail in the presentation of the findings from the study that facilitates the reader reaching conclusions in relation to the generalisability of the findings based on their experience and knowledge; and
- the development of linkages between the findings from this study and other prior studies in terms of their congruence, connection and confirmation of prior theory.
Issue 5: The value of this research can also be determined in terms of its potential to open up new ways of viewing issues that affect the provision of relationship education services people and to illuminate new ways of thinking about evaluating the impact of these services.
3.9 Conclusion
This chapter has focused on a detailed explanation of the research methods used in the study and the design of the processes used to collect data. The aim of this chapter was to provide a justification of the research approach, including consideration of issues relating to judgements about the quality of the study and the value and potential contribution that the study can make to the provision of relationship education services. The following chapter builds on the contextual foundations that have been established in this and the previous chapter and commences the process of describing and analysing the data gathered during the research process.