Within the first 18 months of the Leaders North of the Bridge, the Program rolled out in 4 extra sites around NSW - South Eastern Sydney, Southern NSW, the Central Coast, and the Inner West of Sydney. We though of this as one large pilot, rather than 5 separate Projects.
In reflecting the local context of the Projects, several had more specific aims:
- The Southern NSW program specifically aimed to integrate Aboriginal services with mainstream services via mentoring relationships;
- The Central Coast program was looking to enhance relationships between government and non-government services, on the back of some wider co-design work;
- The Inner West was looking to better join up youth services with the wider family services system.
Except for Southern NSW, all Projects continued their full course. The Southern NSW Project broke down completely due to a number of critical factors, including the withdrawal of lead organisation support, and the local Project Coordinator having to leave their role.
A large number of evaluation questions were asked of the Program as a whole. Rather than seeing any of the questions as a success/fail dichotomy, we were interested as to whether mentoring would affect a wide range of variables. As such, some of our Program evaluation questions were:
- Would mentoring relationships be a good way for junior workers to learn?
- Would mentoring increase the leadership skills and abilities of mentors?
- Would the support of a mentor improve how a junior worker felt about the sector (this has links to staff retention and reducing turnover)?
- Would the program format support relationships and collaboration more widely than the mentor-mentee relationship?
- Would there be any flow-on impacts to end-clients of the involved services?
As is to be expected, some of the Projects progressed more smoothly than others. Some had higher retention rates, while other experienced higher rates of relationship breakdown between mentor and mentee.
Across all of the Program, we started with 144 participants, and 35 submitted evaluations at the 18 month checkpoint, giving a retention rate of 24%. While this seems low, there were a number of participants who completed the Program, but didn't complete final evaluations, and I have explored some of the other factors at work in the 'learning' area below.
Let's look at some of the evaluation questions:
1. Would mentoring relationships be a good way for junior workers to learn?
Strong "Yes". Final evaluation results showed that 100% (55% strongly agree, 45% agree) of mentees agreed that they were learning new things. 100% (82%SA, 18%A) also felt that they were improving in their leadership capacity. 100% (31%SA, 69%A) of mentors also felt that they had seen growth and development in their mentee.
2. Would mentoring increase the leadership skills and abilities of mentors?
Strong "Yes". 92% (15%SA, 77%A) of mentors felt that they were learning new things as a mentor. 46% also felt that being a mentor had changed the way they approach their role in their organisation in some way.
3. Would the support of a mentor improve how a junior worker felt about the sector ?
Strong "Yes". 81% (36%SA, 45%A) of mentees felt better about working in the human services serctor as a result of the mentoring they were receiving.
4. Would the program format support relationships and collaboration more widely than the mentor-mentee relationship?
Moderate "Yes". For those outside the human services sector, collaboration between organisations and workers across organisational boundaries is a really big deal for improving client experience and outcomes. 63% (36%SA, 27%A) of mentees felt that the Program had increased their links to local networks, either through the group sessions that were run, or through their mentor's contacts. 62% (8%SA, 54%A) of mentors felt that thier relationship had produced closer ties between their organisation and the organisation of their mentee, and 46% of mentors felt that there were improved ties between organisations involved in the Program.
4. Would there be any flow-on impacts to end-clients of the involved services?
Strong "Yes". 90% (45%SA, 45%A) of mentees said that because of their mentoring, they had improved client outcomes in some way. 61% (15%SA, 46%A) of mentors also said that they had observed improved outcomes for either their mentee's clients or organisation.
In summary, for those who completed the Program, the mentoring structure was an effective way to facilitate learning of more junior workers. It increased the leadership abilities of mentors, marginally affected collaboration locally, and have good flow-through to practice with mentee's clients.
Given the low cost of the Program to implement locally, and its flexibility to be adapted to local contexts, I'm hoping that we can take this Program to greater scale in future.
Learning and future improvements
While the evaluations are overly positive, there are some issues with the Program design that will need to be addressed in the next round of local Programs.
- The local partnership model is necessary to success: Each of the Projects had a slightly different aims which were particular to the needs of the partner. These reflected the needs of the local service system, and seemed to improve uptake by both mentors and mentees locally. The contacts of the local Project manager were also critical to the success of the Project, and also increased the efficiency of the model overall. Of note, when the local partner in Southern NSW had to withdraw from the coordination role, it was almost impossible to coordinate the program remotely.
- How to deal with the retention rate: While the model works for those who complete the 18 months, and evaluations at 3 and 12 month evaluation checkpoints were almost equally good, the significant non-completion rate is still a major concern. While some relationships broke down because of differences between mentor and mentee or a lack of commitment of one party, many fell away due to fairly natural change - changing roles, organisations, areas etc. Decreasing the length of the Program to 12 months may help to buffer against some of the drop-off. I may also explore how a commitment to contact can be maintained across distance, or how a model of reduced contact can be created for those who can no longer commit to the full Program structure.
- Group reflections were a valuable component of the Program design: The 'bringing together' of the groups to reflect on progress, challenges, and show them the Project's progress and evaluation added structure (or phases) to individual mentoring relationships. This seemed particularly valued by mentors, who often felt a little out of their depth or unsure of their mentoring ability. These groups also enhanced some of the wider collaborative outcomes.
- The design needs to be more culturally appropriate for Aboriginal workers: When designing, I checked the content and structure with an Aboriginal consultant, but the complete failure of the Southern NSW Project was foreshadowed by a high number of relationship issues. Some of the thoughts or feedback I have had since:
- The relationship between Aboriginal mentees and non-Aboriginal mentors could be problematic. The junior and senior members of Aboriginal communities have relationships which are different in nature to those between professional colleagues. For young Indigenous workers, navigating the differing expectations of these roles could be difficult, and warrants greater consideration and nuance in the Program design;
- Much of the initial mentor training centred around leadership constructs, such as goal-setting, interpersonal skills and encouraging follow-through/accountability. While many of the Aboriginal mentors enjoyed and engaged with the sessions, some thought should be given to tailoring the content and the delivery style to accommodate the different ways that Aboriginal workers may see these topics.