Reflective Conversations and Collaborative Sense-Making Processes

Submitted by Alison Homer on December 22, 2016 - 6:57am
Sharing learning from the Evaluation Community of Practice

Tammy Horne, Well Quest Consulting Ltd.

On our most recent Evaluation Community of Practice call, Tammy Horne, Principal of Well Quest Consulting Ltd. shared her recommendations on using reflective conversations to draw out practice-based learning from our successes and challenges, and on using collaborative sense-making sessions to involve a range of stakeholders in interpreting data. She emphasized the potential of these two approaches, as well as highlighted strategies for incorporating them into our work, in order to further evaluative thinking and an appreciation for learning. Key learning from this call is summarized below.

Reflective conversations

Reflective conversations emphasize meaning, can be either practice- or outcome-focused (often both), and can be intensive learning-focused engagement tools.

Tips for hosting successful reflective conversations

  1. Start with the positive and use 'broad strokes', such as opening the conversation in a way that focuses on exceptional moments (e.g. “Tell me about at time when you know you really made a difference”).
  2. Frame challenges as positive learning (e.g. “What advice would you give others who want to host a similar project somewhere else based on what you have learned here?”).
  3. Incorporate specific focus areas into the conversation for deeper learning. For example:
    • Accomplishing successes through partnerships that could not be achieved by any one partner working alone.
    • Balancing local community interests with the requirements of a provincial organization or other type of central body.
    • Developing community leadership.
    • Addressing changes in working relationships over time (positive or challenging).
    • Fostering sustainability of an initiative or its legacy.
  4. As the conversation takes shape, continue to circle around to specific examples of successes, hopes, and concerns, particularly those that are tied to solutions.
  5. Reflective conversations are usually in-person, but can be virtual. Online tools to deliver reflective sessions need to be interactive (e.g. webconferencing that incorporate verbal discussion, text chat, whiteboard visuals).

Collaborative sense-making

Collaborative sense-making could be considered a broader type of reflective conversation that helps us to move from “What” to “So what”. In a collaborative sense-making process, stakeholders convene to co-create meaning about evaluation findings. If the process allows participants to see how findings can further inform their practice, it can foster stronger evaluative thinking, and a culture of evaluation.

Tips for hosting successful collaborative sense-making processes

  1. Incorporate a focus area (e.g. Why are some of our clients making progress while others struggle? How is our current practice different than what we have done in the past, and what does this mean?).
  2. Choose a small number of key findings to discuss, and draft a session outline and 1-page summary with guiding questions. 
  3. Determine which facilitation process will work well, based on the group size and participant mix (e.g. World Café).
  4. Identify number and types of stakeholders; pre-assign people to groups that contain a variety of types of people.
  5. Focus on the deeper meaning of evaluation findings. What contributes to change and what does that change mean for practice?
  6. Close with a verbal summary of the facilitated discussion, and send out a post-session summary document.

Group vs. 1-to-1 reflection

Engaging a group containing a variety of stakeholders can provide diverse perspectives and rich interactions, and can allow participants to build on each other’s stories. However, a large group may not allow enough time to achieve the level of detail desired, or to get into deep reflections on meaning. Groups work well when presented with a few broad questions, followed by prompting and probing to bring out more depth. 1-to-1 conversations allow for more in-depth exploration, and may be especially useful for learning more about very specific types of change and related issues and challenges. 1-to-1 conversations can also draw out quieter people who seldom speak up in a group setting.

Recommendations from Evaluation CoP Group Members

  1. Embed reflective practice across activities in different ways (e.g. build reflection sessions into staff meetings and AGMs, or start each internal meeting with a reflective check-in).
  2. Include definitions. Even people who have a strong understanding of the data may not see it the same way as the evaluator does, and typically appreciate a review of what the data show, before discussing what findings mean.
  3. Account for the target audience and present data that is relevant to each type of stakeholder.
  4. Divide the sense-making work up into multiple sessions. People need time to reflect on data presented. Present information in smaller nuggets, then tackle a second set on another day.
  5. Monthly data sharing in small groups (e.g. with games, rewards and activities) can teach participants how to look at data.
  6. When reflecting on sustainability of a program, consider what would be most important to sustain – the program itself or its legacy.
  7. Learning Labs are helpful for sharing back and for diving deeper into learning
  8. Try the Focused Conversation Method which asks four types of questions: objective, reflective, interpretive, and decisional.
  9. Use online and/or distance technologies (e.g. Adobe Connect) to connect teams that are spread out. 
  10. Incorporate outcome mapping and outcome journals into reflective sessions.

NOTE: The fit of each of the above suggestions for a given project will depend on factors such as purpose of the conversation, group size, group commitment to spending time on reflection, available time and resources, and types of data available for discussion. 

Tammy Horne has been the Principal of WellQuest Consulting Ltd. since 1994. She has 30 years of experience that span the areas of evaluation, research, evidence-informed planning and design, and professional development. WellQuest works across sectors - including social services, justice, recreation, health, and education – and on cross-sector system-focused initiatives. She has a strong focus on individual and organizational capacity building, a commitment to working collaboratively, and the ability to balance rigour with practicality. Tammy has a Credentialed Evaluator (CE) designation from the Canadian Evaluation Society, a Ph.D. in Kinesiology from the University of Waterloo (with a social and cognitive psychology focus). She is presently pursuing a Certificate in Adult and Continuing Education.

Resources to Get Started 

~~~ Join the Evaluating Community Impact: Measuring Poverty Reduction Community of Practice ~~~