Background: The role of infection control professionals has evolved with the increasing complexity of healthcare and the diversity of professional backgrounds, including nursing, epidemiology, and public health. This diversity presents challenges for orientation and professional development. Recognizing the importance of competency-based frameworks, the Alberta Health Services Provincial Infection Prevention and Control (IPAC) program implemented a strategic initiative to define and operationalize competencies across IPAC roles. This study evaluated the initiative using the RE-AIM framework.
Kathryn Bush, MSc1*, Christian Tsang, MSc2, Blanda Chow, RN, MPH2, Obeleye Tamuno, MPH2, Heather Gagnon, MPH2
1 Infection Prevention and Control, Interior Health, Kelowna, British Columbia, Canada
2 Infection Prevention and Control, Alberta Health Services, Calgary, Alberta, Canada
*Corresponding author:
Kathryn Bush
Interior Health, Kelowna
505 Doyle Ave.
Kelowna, B.C. V1Y 0C5
Email: kathryn.bush@interiorhealth.ca
ABSTRACT
Background: The role of infection control professionals has evolved with the increasing complexity of healthcare and the diversity of professional backgrounds, including nursing, epidemiology, and public health. This diversity presents challenges for orientation and professional development. Recognizing the importance of competency-based frameworks, the Alberta Health Services Provincial Infection Prevention and Control (IPAC) program implemented a strategic initiative to define and operationalize competencies across IPAC roles. This study evaluated the initiative using the RE-AIM framework.
Methods: The IPAC competency framework was launched in March 2022. A multi-method evaluation was conducted between January and March 2024, consisting of an anonymous survey distributed to 217 IPAC staff and follow-up interviews with senior leadership. The RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance) guided the evaluation.
Results: Sixty percent of staff (130/217) completed the survey. Under the Reach domain, most staff reported using and understanding the competencies, though some found prioritization challenging. Effectiveness was reflected in 79% of staff setting learning goals and 88% developing actionable plans. Adoption showed high satisfaction, with 88% of respondents using competency tools, though some reported perceived redundancy.
Conclusions: Factors influencing the use of the IPAC competency tools included alignment with program goals, leadership engagement, delegation of champions, and barriers to staff engagement. Ongoing evaluation was recommended to monitor progress and enhance sustainability. The RE-AIM framework proved useful in identifying both successes and potential risks to the long-term success of the competency framework. Key barriers included limited team size and competing priorities; however, continued training and perceived value were identified as critical to sustained engagement.
KEYWORDS
Professional development, infection control professional, infection preventionist, self-assessment, development conversations, coaching conversations
INTRODUCTION
IPAC Canada defines competencies as “the expectations of one who is able to perform effectively in the roles and functions required by his or her position and within the team and organization” (IPAC Canada, 2022). There is increasing emphasis on the professional development of infection control professionals, as a stable workforce ensures the continuity of Infection Prevention and Control (IPAC) program activities that translate “data to action” in support of improved patient safety (Bernard, 2018; Gilmartin, 2021). The retention and development of IPAC staff have been shown to increase employee tenure and engagement while reducing turnover (Gilmartin, 2021).
At Alberta Health Services, development conversations are used as a form of ongoing coaching, focused on goal setting and self-directed professional growth aligned with both team and organizational objectives (Alberta Health Services, 2024). These conversations are distinct from traditional performance evaluations, as they emphasize coaching questions that encourage employees to reflect on their development goals and create actionable plans for achieving them. In March 2022, the IPAC program introduced a competency framework that outlined both specialized and shared competencies for all IPAC roles. A strategic one-year goal was set for all IPAC staff to establish learning goals and create development plans aligned with this framework (Bush, 2022).
The RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance) is an evidence-based model within the field of implementation science. It has been applied in Infection Prevention and Control (IPAC) contexts, including the evaluation of an IPAC link nurse program and a review of an antimicrobial stewardship program in an intensive care unit (Dekker, 2023; Holtrop, 2021; Nkosi, 2021; Trivedi, 2023). The purpose of the current project was to apply the RE-AIM framework to evaluate the use of competencies during development conversations with Alberta Health Services’ provincial IPAC staff, two years after the competency framework was implemented.
METHODS
Setting
At the time of the evaluation, Alberta Health Services (AHS) was the sole provincial health authority in Alberta, Canada, serving a population of 4.5 million residents. AHS operated 106 acute care hospitals, comprising 8,605 acute care beds, 29,124 continuing care beds, and 3,176 addiction and mental health beds (Alberta Health Services, 2024). The provincial Infection Prevention and Control (IPAC) program supported the continuum of care with 217 staff members across ten distinct roles, excluding the IPAC senior medical director and IPAC physicians. IPAC teams were aligned with the geographic clinical operations zones and supported by a provincial team responsible for coordinating program activities, including surveillance and resource development. Two of the IPAC teams were located in Alberta’s major urban centres (Calgary and Edmonton), while the remaining three zones (North, Central, and South) primarily served rural populations. Table 1 presents the demographics of the survey participants.
Intervention
The IPAC competency framework introduced in March 2022, consists of 1) tools – a booklet with role competencies, defined proficiency levels, and a competency checklist for self-assessment; and, 2) process – a reflective competency self-assessment discussed in the context of an annual development conversation with the staff’s leader (Alberta Health Services, 2024; Bush, 2022). IPAC leadership implemented the competency framework and worked with IPAC staff to identify one or more learning goals and developed a plan to achieve the goal.
Study design
A multi-methods study, utilizing both quantitative and qualitative survey tools and qualitative interviews, was conducted between January and March 2024. An ARECCI (A Project Ethics Community Consensus Initiative) assessment (https://arecci.albertainnovates.ca/) identified this project as a quality improvement initiative; therefore, Institutional Review Board approval was not required. The five domains of the RE-AIM framework – Reach, Effectiveness, Adoption, Implementation, and Maintenance – were used to guide the evaluation. 1) Reach – proportion of those receiving the intervention; 2) Effectiveness – identifying whether the primary outcome was affected by the intervention; 3) Adoption – describing relevant characteristics of the delivery setting and those involved in the intervention; 4) Implementation – evaluating the extent that the intervention is delivered as intended; and, 5) Maintenance – describing whether the setting continues to deliver the intervention and the long-term effects on staff (King, 2020). Reach, Effectiveness, and Implementation focus on the staff/individual level, and Adoption and Maintenance focus on the staff/individual and the program/setting levels (Glasgow, 2019).
The RE-AIM framework informed the design of the survey questions regarding staff experiences with the IPAC competency framework. RedCap (v14.0) was used to create the online, anonymous survey, which was distributed to IPAC staff via email in February 2024 and remained open for four weeks. During this period, email and verbal reminders were sent to encourage participation. IPAC senior medical directors and IPAC physicians were excluded from the survey as their roles did not have defined IPAC staff competencies. At the end of the survey, virtual semi-structured interviews were conducted with IPAC leadership using MS Teams. These interviews were led by members of the evaluation team, and although full transcription was not completed, each interview included a designated note-taker.
Data processing and analysis
Closed-ended survey questions were analyzed using frequencies and proportions, incorporating all complete and incomplete responses to minimize response bias. Survey results were grouped, and those with fewer than five responses were not reported. Free-text fields from both the survey and interviews were analyzed in three stages: coding, generation of specific beliefs, and identification of prominent domains through thematic analysis (Atkins, 2017). Two members of the evaluation team (CT and OT) independently coded the free-text survey responses, while another two team members (KB and BC) coded the interview data, all using the RE-AIM dimensions. Each pair included at least one individual experienced in qualitative interviews and data analysis. Afterward, the two members of each pair met to establish a coding scheme through consensus. The final data coding was compiled into a matrix to aid in data interpretation. Counts of domains and themes were used to assess the findings.
RESULTS
Sixty percent of IPAC staff (130/217) responded to at least one question. Respondents represented various IPAC roles, with the majority being infection control professionals (70.8%), and 42.3% holding a graduate degree (see Table 1).Respondents reported feeling well-integrated and valued within the IPAC program, with over 90% indicating an understanding of their roles and how they fit into the broader team (Table 2). A strong sense of contribution to patient safety and quality care was expressed by 88% of respondents. Seventy-nine percent of staff identified a learning goal, 88% developed a learning plan to achieve their goal, and 79% of staff completed items on their plan. A higher proportion of responses from urban zone staff indicated disagreement or strong disagreement with understanding how their role fit into the program, being valued for their expertise, and feeling they contributed strongly to patient safety (17/171, 10%) compared to staff in the other three zones (3/49, 6%), though the difference was not statistically significant. Eighty-five percent (17/20) of staff who indicated being very dissatisfied or dissatisfied with the IPAC competency framework were from an urban zone.
Seven interviews were conducted with the senior leadership team, representing the provincial IPAC program teams. Table 3 presents themes derived from both the free-text responses of the staff survey and the leadership interview comments, analyzed using the RE-AIM framework.
Reach
Themes of understanding, perspective, priority, and representation emerged (Table 3). Both staff and leaders indicated that the competencies were understood, with one staff member noting, “…I think they do an excellent job of breaking down the role into chunks that can be understood, communicated, and planned improvements made.” Staff also felt that the competencies were prioritized, with one leader commenting, “What surprised me was that it was so well received – ‘do we have to do this?’ or people just not doing it – that did not happen.” The representation of competencies across diverse roles was a point of contention. While some staff respondents felt the competencies adequately represented the IPAC roles, others believed that the competencies did not fully reflect the responsibilities and day-to-day tasks of infection control professionals, and that not all competencies outlined in the framework were relevant to the roles they held.
Effectiveness
Themes of perception, professional development, and self-assessment emerged. One staff member shared, “It helped to identify secondary areas of focus… where I was looking for opportunities to explore through my work, but I hadn’t identified before then as an area of professional development.” Self-assessment was key, with a leader noting: “…you’ll have some people who are too humble and will under-rate themselves and not see their strengths. It’s a great opportunity to have that discussion with them to help them see the things that they’ve done. And then those who thought they were quite advanced in everything, that’s a really interesting conversation to have… my view and their view could be quite different.”
Adoption
The RE-AIM framework includes two categories for adoption: the program/setting level and the staff/individual level. At the program/setting level, system drivers were key: “What helps is that we talk about it, it becomes common language; we try to embed it, even with strategic planning.” Themes for staff/individual adoption included acceptance and value. Some staff mentioned opportunities for further training and education: “I found the checklist kept me on track and provided useful links to tools to help familiarize myself with different areas on the Alberta Health Services website.”
Implementation
Themes emerged regarding both the tools and the process. All leaders reported using the tools in their conversations with staff. Staff reported varied experiences, highlighting both challenges and effective implementation practices. Staff used the competency tools as a preparatory exercise for development conversations and felt that the tools were helpful in demonstrating their value in their roles. One staff member emphasized the importance of ongoing discussions: “Unless they are regularly addressed (more often than a once-a-year conversation and progress is noted and reviewed), they simply aren’t usable enough from my perspective.”
Maintenance
As with adoption, maintenance was categorized at both the program/setting and staff/individual levels. At the program/setting level, staff suggested that reminders to complete the competency checklist be sent out regularly and that educational opportunities addressing specific competencies be made available. At the staff/individual level, maintenance of IPAC competencies was supported through a “baseline” assessment followed by regular assessments to track progress and identify areas for continuous professional development. The continuous process of re-evaluating and improving the proficiency of IPAC competencies was seen as rewarding: “I use the competency profile as a ‘journey’ to improve my professional and personal development.”
DISCUSSION
RE-AIM is a comprehensive evaluation tool for interventions such as the introduction of the IPAC competency framework. Reach and adoption of the intervention was high as the competency tools were used by all leadership and with most staff. In terms of effectiveness, many staff identified a learning goal, developed a learning plan to achieve their goal, and completed items on their plan. Implementation was facilitated by both the tools and the process, which were positively received by leaders and staff. IPAC leadership adoption was considered at a program/setting level and the leaders played a critical role by using the tools to guide annual development conversations. Leadership adoption was influenced by program drivers including the IPAC strategic goal for staff professional development, and through support and role-modelling from leadership champions who created the tools. Leadership training supported a common understanding on the competency and proficiency definitions and how to use the tools in their staff development conversations.
Staff adoption was contingent on perceiving value in the use of the tools. Survey results indicated high staff satisfaction, with staff comments highlighting the supportive nature of the tools in considering professional development goals. Leaders were surprised by the adoption, as there had been initial concerns that the tools would be viewed as redundant with existing organizational forms. Although some staff felt the tools did not fully represent all aspects of their role, the tools were developed from published competency frameworks, including those from IPAC Canada, the Association for Professionals in Infection Control and Epidemiology (APIC), and the National Center for Healthcare Leadership (Bush, 2022).
For most individuals, annual performance appraisals result in evaluation biases from both the leader and the staff member being assessed (Kromrei, 2015). The process of observation, evaluation, and score assignment by the leader is typically done to (rather than with) the staff member. Incorporating a self-assessment tool can enhance effectiveness by increasing engagement and reflection from the staff members (Kromrei, 2015). However, self-assessment carries inherent biases, and weak correlations have been observed between self-reported ability estimates and actual performance, including leadership and interpersonal skills (Karpen, 2018). Notably, the requirements for the Certification in Infection Control (CIC) exam, administered by the Certification Board of Infection Control and Epidemiology (CBIC), serve as an external competency proficiency assessment within the APIC competency model, providing an objectively validated method for competency assessment (Billings, 2019).
The Dunning-Kruger effect (Kruger, 1999) describes a bias in individuals with both low and high competency proficiency. Those with low competency tend to overestimate their abilities due to a lack of metacognitive awareness, meaning “they don’t know what they don’t know.” Conversely, high performers often underestimate their abilities due to the false-consensus effect, believing they are comparable to their peers, who are, in fact, less proficient (Karpen, 2018). One strategy to mitigate these biases is to provide externally generated criteria, such as the competency tools in this context. Staff and leader discussions also help reduce self-assessment bias and identify continuous improvement opportunities beyond formal learning courses (Karpen, 2018). As one infection control professional noted: “I keep this posted at my desk as a reminder to be aware of opportunities that will increase my competencies in my goal areas.”
One of the urban zones reported that 85% of respondents were dissatisfied or very dissatisfied, which may have been attributed to the high number of IPAC staff in the zone, identified as an implementation barrier in the leadership interviews. Since short, frequent conversations are essential to maintaining motivation and momentum for professional development, leaders with large teams may consider integrating brief discussions into other meetings or encouraging staff to reference their professional development goals when identifying participation in new projects (Fraser-Thill, 2023).
Maintaining an intervention once initiated is always challenging, as sustainability is difficult to achieve. Interventions built on a strong theoretical foundation tend to be more effective, and the quality of the intervention design further supports long-term success (Jalali, 2019). In this case, the use of published role competency models to create the IPAC competency framework provided staff with an opportunity to engage in reflection during the performance appraisal process (Bush, 2022; Kromrie, 2015). Intervention erosion may occur if adoption factors are removed, such as changes in the IPAC program’s strategic goals or the loss of key champions (Jalali, 2019). To support long-term sustainability, it is important to maintain adoption drivers, including ongoing leadership training and minimizing barriers by incorporating more frequent and shorter updates on professional development progress.
Other program/setting maintenance issues included the ongoing review and improvement of the tools to ensure their relevance and usability. At the staff/individual level, establishing a baseline to compare ongoing professional development improvements was seen as an exciting prospect and supports sustainability. As noted by one leader: “Great follow-up for the following conversation – it’s great to compare the baseline to their progress over time.”
There are limitations to this work. Response bias could influence the results if individuals who chose not to participate differed from those who responded. For example, staff with more positive or negative experiences with the IPAC competency framework might have been more motivated to complete the survey, potentially skewing the findings. To mitigate this, reminders were used to encourage participation from a broader group of staff. Social desirability bias was minimized as survey responses were collected anonymously, allowing participants to provide more honest and candid feedback without fear of judgment or repercussion. A leadership peer participated in the development and evaluation of the IPAC competency framework, which may have introduced social desirability bias if the interview responses were not completely frank. However, the role in the interview was as a note-taker and not an interviewer, and negative comments were forthcoming during the interview.
This study had several strengths, starting with the use of the RE-AIM framework, which provided a structured evaluation across multiple domains (Holtrop, 2021). RE-AIM is well-suited for real-world program evaluations where contextual factors, such as leadership involvement, organizational goals, and staff perceptions, can influence success. The 60% response rate was a strength of the study, as it included diverse perspectives from different roles within the IPAC program, reducing the impact of response bias. The combination of staff surveys and leadership interviews ensured that both top-down and bottom-up perspectives were captured, making the results more actionable and applicable for future program improvements.
The transferability of this study’s findings is influenced by the specific context of the Alberta Health Services provincial IPAC program. The unique structure, size, and resources of Alberta Health Services may differ from those of smaller or differently organized health systems. IPAC programs in other regions may encounter distinct cultural, regulatory, and operational challenges that were not addressed in this study. However, the diversity of IPAC roles, ranging from infection control professionals to administrative and support staff, broadens the relevance of the IPAC competency framework and this evaluation. This diversity increases the potential for other healthcare organizations to adapt this approach. As a quality improvement project, the focus was on internal process enhancement, so broader application to different healthcare systems may require further validation.
Conclusions
The RE-AIM framework effectively identifies successes in the introduction of an intervention and potential risks to its sustainability. The IPAC competency framework supports professional development and is instrumental in operationalizing published IPAC competencies. Leadership barriers included challenges related to implementing the framework with larger teams, while staff/individual barriers focused on recognizing professional development as a priority within their roles. To ensure the sustainability of the intervention and the long-term professional success of IPAC staff, it is crucial to maintain adoption drivers, such as ongoing training, the perceived value of the tools, and clear leadership expectations.
REFERENCES
Alberta Health Services. (n.d.). About AHS. Retrieved January 25, 2025, from https://www.albertahealthservices.ca/about/about.aspx
Atkins, L., Francis, J., Islam, R., O’Connor, D., Patey, A., Ivers, N., Foy, R., Duncan, E. M., Colquhoun, H., Grimshaw, J. M., Lawton, R., & Michie, S. (2017). A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implementation Science, 12(77). https://doi.org/10.1186/s13012-017-0605-9
Bernard, H., Hackbarth, D., Olmsted, R. N., & Murphy, D. (2018). Creation of a competency-based professional development program for infection preventionists guided by the APIC Competency Model: Steps in the process. American Journal of Infection Control, 46, 1202–1210. https://doi.org/10.1016/j.ajic.2018.04.214
Billings, C., Bernard, H., Caffery, L., Dolan, S. A., Donaldson, J., Kalp, E., & Mueller, A. (2019). Advancing the profession: An updated future-oriented competency model for professional development in infection prevention and control. American Journal of Infection Control, 47, 602–614. https://doi.org/10.1016/j.ajic.2018.11.005
Bush, K., Leal, J., Acorn, L., Cordoviz, M., Cundict, F., Devine, A., Edwards, C. L., Fletcher, P., Gable, Y., Gagnon, H., Gallinger, S., Gill, V., Heinrichs, B., McFerran, B., Pearce, C., Pfister, T., & Meyers, G. (2022). Developing a competency framework for all staff roles in an infection prevention and control program. Canadian Journal of Infection Control, 37, 184–188.
Dekker, M., Jongerden, I. P., Caris, M. C., Bruijne, M. C., Vandenbroucke-Grauls, C. M. J. E., & van Mansfeld, R. (2023). Evaluation of an infection control link nurse program: An analysis using the RE-AIM framework. BMC Health Services Research, 23, 140.
https://doi.org/10.1186/s12913-023-09111-5
Fraser-Thill, R., & Gopal, S. (2023). How to talk to your team about their career development. Harvard Business Review.
https://hbr.org/2023/03/how-to-talk-to-your-team-about-their-career-development
Glasgow, R. E., Harden, S. M., Gaglio, B., Rabin, B., Smith, M. L., Porter, G. C., Ory, M. G., & Estabrooks, P. A. (2019). RE-AIM planning and evaluation framework: Adapting to new science and practice with a 20-year review. Frontiers in Public Health, 7, 1–9.
https://doi.org/10.3389/fpubh.2019.00064
Gilmartin, H., Smathers, S., & Reese, S. M. (2021). Infection preventionist retention and professional development strategies: Insights from a national survey. American Journal of Infection Control, 49(8), 960–962. https://doi.org/10.1016/j.ajic.2020.12.005
Holtrop, J. S., Estabrooks, P. A., Gaglio, B., Harden, S. M., Kessler, R. S., King, D. K., Kwean, B. M., Ory, M. G., Rabin, B. A., Shelton, R. C., & Glasgow, R. E. (2021). Understanding and applying the RE-AIM framework: Clarifications and resources. Journal of Clinical and Translational Science, 5, e126, 1–10. https://doi.org/10.1017/cts.2021.789
IPAC Canada. (2022, September). Core competencies for infection control professionals. https://IPAC-canada.org/photos/custom/pdf/IPAC_CoreCompetencies_ICPs_2022_revised.pdf
Jalali, M. S., Rahmandad, H., Bullock, S. L., Lee-Kwan, S. H., Gittelsohn, J., & Ammerman, A. (2019). Dynamics of intervention adoption, implementation, and maintenance inside organizations: The case of an obesity prevention initiative. Social Science & Medicine, 327, 115937. https://doi.org/10.1016/j.socscimed.2018.115937
Karpen, S. C. (2018). The social psychology of biased self-assessment. American Journal of Pharmacy Education, 82(5), 6299. https://doi.org/10.5688/ajpe6299
King, D. K., Shoup, J. A., Raebel, M. A., Anderson, C. B., Wagner, N. M., Ritzwoller, D. P., & Bender, B. G. (2020). Planning for implementation success using RE-AIM and CFIR frameworks: A qualitative study. Frontiers in Public Health, 8, 1-14. https://doi.org/10.3389/fpubh.2020.00014
Kromrei, H. (2015). Enhancing the annual performance appraisal process: Reducing biases and engaging employees through self-assessment. Performance Improvement Quarterly, 28(2), 53-64. https://doi.org/10.1002/piq.21120
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134. https://doi.org/10.1037/0022-3514.77.6.1121
Nkosi, B. E., & Sibanda, S. (2021). Evaluating an antimicrobial stewardship programme implemented in an intensive care unit of a large academic hospital, using the RE-AIM framework. South African Medical Journal, 111(9), 777-782. https://doi.org/10.7196/SAMJ.2021.v111i9.15585
Trivedi, K. K., Schaffzine, J. K., Deloney, V. M., Aureden, K., Carrico, R., Garcia-Houchins, S., Garrett, J. H., Glowicz, J., Lee, G. M., Maragakis, L. L., Moody, J., Pettis, A. M., Saint, S., Schweizer, M. L., Yokoe, D. S., & Berenholtz, S. (2023). Implementing strategies to prevent infections in acute-care settings. Infection Control and Hospital Epidemiology, 44(9), 1232-1246. https://doi.org/10.1017/ice.2023.171