External Evaluation
The Xcalibur Evaluation Team is comprised of trained and experienced evaluators who have been invited to present at conferences across the country.
The Xcalibur Evaluation Team is comprised of trained and experienced evaluators who have been invited to present at conferences across the country including GEAR UP conferences, Capacity Building Workshops, Data Research and Evaluation Institutes, American Educational Research Association, and GEAR UP West on many occasions. Xcalibur embraces the American Evaluation Association’s (2018) guiding principles for evaluators, which outline standards for professional ethics among evaluators. Developed by a committee of evaluators, these guidelines outline standards for conducting systematic inquiry, building capacity among stakeholders, employing honesty and transparency, evaluation practices rooted in equity and dignity, and leveraging evaluation to contribute to a just society.
We bring a team of seasoned evaluators with a wealth of expertise evaluating GEAR UP programs, externally and internally. Currently, the evaluation team at Xcalibur supports thirteen partnership GEAR UP grants and six state GEAR UP grants with robust evaluation services. Our evaluation team incorporates several evaluation approaches to ensure relevant findings that inform practice. We utilize strategies from mixed methods evaluation, participatory evaluation, utilization-focused evaluation, community-based participatory research, evaluative thinking, and evidence-based practice providing a deep reservoir of expertise for Xcalibur to draw upon when facilitating evaluation.
Mixed Methods Evaluation
To balance quantitative, qualitative, and mixed methods approaches, Xcalibur utilizes Crewell and Guetterman (2021) framework for a mixed methods approach. This framework draws on examples from multiple fields of research including program evaluation, multicultural research, counseling, and psychology.
Participatory Evaluation
The foundation of participatory evaluation is embedding local program staff and stakeholders in the evaluation work. Participatory evaluation is driven by local context and emphasizes reflection for program improvement, data-informed decisions, and building capacity of project staff and stakeholders. This inclusive framework assists programs to strengthen their leadership and expertise, sustain programming, and embed data into practice (Cousins & Choinard, 2012; Zukoski, A., & Luluquisen, M. 2002). Xcalibur brings independent judgment, evaluation expertise, and national context to the team, all important components to successful evaluation. Participatory evaluation ensures that local context and knowledge are integrated into all aspects of the evaluation.
Utilization-Focused Evaluation
Xcalibur uses this framework to prioritize actual use of evaluation findings and recommendations to improve programming and addresses challenges in the field (Patton, 2008). According to Patton (2008), “Intended users are more likely to use evaluations if they understand and feel ownership of the evaluation process and findings [and that] [t]hey are more likely to understand and feel ownership if they’ve been actively involved. By actively involving primary intended users, the evaluator is preparing the groundwork for use.” This framework is at the core of Xcalibur’s evaluation approach.
Community-Based Participatory Research
To incorporate the voices and perspectives of all stakeholders as partners, Xcalibur utilizes Community-Based Participatory Research (CBPR; Israel, 2001). CBPR leverages strengths and resources from the community through an iterative and collaborative process that bridges cultural differences and fosters trust and understanding among participants (Israel, 2001).
Evaluative Thinking
The emphasis of Evaluative Thinking is to strengthen critical thinking. All humans view results through the lenses of their own experience and knowledge (Archibald et al, 2016) Critical thinkers seek deeper and broader understanding and set aside assumptions and biases to look with fresh eyes at challenges, barriers, assets, and/or deficits. Evaluative thinking builds capacity for analysis and insight (Buckley et al, 2015). It is a skill which allows us to question and reflect on information, our own knowledge and experience, and the experience of others (Vo et al, 2018). Xcalibur works to build Evaluative Thinking among the evaluation team. It teaches the team members to focus on data, things we can see and prove, instead of what we think we know. Evaluative thinking enhances collaboration within the evaluation team and the result is a more proactive approach.
Evidence-Based Management
Evidence-Based Management (EBM) is a strategy used for decision-making that emphasizes collecting and critically judging evidence focused on increasing the likelihood of a favorable outcome. It is adapted from Barends, E., Rousseau, D.M., & Brinner, R. B (2014) and Barends, E., Rousseau, D.M (2018). Evidence is often central in any decision-making process; however, little consideration is given to the quality of the evidence which can result in unintended outcomes and mistakes. Integration of EBM into evaluation provides a systematic strategy for collecting evidence and critically identifying the best available evidence on which to base the programmatic decisions. The latter improves the way decisions are made using evidence that can be trusted.
