Data-Driven Program Assessment and Evaluation Participant data drove the mentorship pro- gram’s education component. A needs assess- ment was developed and disseminated to better understand mentee knowledge, con- fidence, skills, prior training, and interests related to the EH field. The resulting informa- tion enabled program sta to design the men- torship experience to meet mentee needs and interests. The needs assessment helped iden- tify training topics, preferred communication methods, learning styles, and informed the curriculum developed by NEHA. Mentors then created presentations and provided resources based on the curriculum. Evaluations were implemented during and after the mentorship program to ensure a high-quality experience. Two quality improve- ment check-ins provided real-time feedback. Mentees and mentors completed logs to track progress and challenges. Program adjustments were made based on collected information, including a Spanish language post-webinar discussion for mentees who spoke Spanish. Program improvements were implemented as needed. The goal was to ensure a high- quality mentorship experience. As such, evaluations were conducted during and after the program to monitor progress and inform adjustments. Weekly mentee logs were shared with mentors, and mentor logs were shared with program sta. Program changes were implemented based on collected data. Weekly logs and real-time feedback provided opportunities for improvement. Quality improvement check-ins allowed for real-time adjustments to enhance the program. Fur- ther, a post-program survey was administered to both mentees and mentors, and interviews with select mentees were conducted. Overall, data and feedback were used to optimize the mentorship experience for participants. Results Based on mentor logs submitted twice a month, mentors spent an average of 7 hr/week prepar- ing activities, assignments, and webinars for the program. Mentors spent approximately 4 hr/week communicating with their mentees via various channels. Most mentors did not meet with each mentee weekly but indicated that weekly time with mentees was ideal. A total of 13 mentees submitted at least one weekly log during the mentorship, with
5 mentees completing one log for each week. On average, mentees spent 3.5 hr/week on activities, webinars, and reading resources and reported spending slightly less than 1 hr/ week in communication with their mentor. Overall, mentors were satisfied with the program and rated their experience as a men- tor as very good (75%) or good (25%). Men- tors felt the program length was appropriate, although one mentor expressed that the pro- gram could have been longer to dive deeper into the topics and to create stronger relation- ships with mentees. Likewise, mentors perceived the program to have a good balance between instruction and one-on-one opportunities; one mentor expressed that there could have been more structured instruction. Figure 1 shows men- tor agreement with the various goals and components of the virtual mentorship. A total of 10 mentees completed the post- program survey. In all, 80% of respondents rated the mentorship program as excellent, and 20% rated it as very good. Of the men- tees, 90% found the mentorship program to be an extremely valuable experience. Overall, mentees were satisfied with their mentor and the learning opportunities oered in this program. They reported that they found their mentor knowledgeable and prepared, and the webinars increased their knowledge. Mentees felt supported in their learning and had opportunities to practice new skills. They also found the topics rele- vant and felt more prepared for their job. Most mentees (80%) felt that the time spent with their mentor was the appropriate amount of time and that the 12-week program was the right duration for a virtual mentorship; how- ever, 20% reported that they felt the program was not long enough. Furthermore, 90% of mentees felt the mentorship program bal- anced instruction and one-on-one opportu- nities well. Mentees highly rated webinar topics, one-on-one opportunities with their men- tor, and assigned activities. They were also satisfied with participation expectations but rated the message board and weekly activ- ity logs lower. To assess the knowledge and skills acquired from the mentorship pro- gram, mentees were asked to indicate their confidence in key EH areas before and after the program (Figure 2). Mentee confidence in key areas increased by an average of 40%
after the mentorship program. The most sig- nificant improvements were in interpersonal skills, professional written communication skills, and human relations. Discussion The successes of the mentorship program far outweigh the challenges. Similar themes were captured from the surveys and interviews from mentor and mentee perspectives. This program demonstrated that NEHA program sta could pivot from an in-person to a vir- tual experience and still provide a high-qual- ity, interactive, and personalized experience. The mentors and mentees were highly satis- fied with the knowledge and skills acquired, and mentees expressed that the program was relevant to their daily work. Language barriers, however, were a com- mon theme throughout the feedback process. We recommend and encourage the inclusion of multilingual program facilitators for vir- tual and in-person mentorship programs to address any language and trust barriers. It is important to emphasize that these programs are not successful without several supportive factors. Based on NEHA’s exten- sive experience implementing these types of programs, necessary factors include: • buy-in from interested parties such as part- ners, funders, and employers; •clear communication about the impor- tance of the program; • relevance of program materials; and •a trusting relationship among mentees, mentors, and program facilitators. Mentorship Program Strengths First, the relationship between mentors and mentees was the main success and is essential to establish. As one mentee stated, “You need to understand the person to make an impact. It is important to establish communication and respectful interactions.” Mentors were knowledgeable, listened, and provided expert guidance and exper- tise. One mentee explained that the best part of the program was “getting to meet the mentors and working one-on-one with my mentor. It was an eye-opening experience to see the scope of the EH field and learn from professionals with vast experience on the topics discussed.” Second, mentors and mentees noted that the EH topics and content areas covered were
11
January/February 2024 • Journal of Environmental Health
Powered by FlippingBook