EIA News

Creativity, Flexibility, and Courage: How I am adapting my program evaluation practice during the global pandemic

May 15th, 2020

Since the global coronavirus pandemic turned the world upside down, I can’t help but wonder about how all of this will impact current and future program evaluation work.

Here are just a few of the questions I have heard from my clients, colleagues, nonprofits, foundations, and the evaluation sector at large:

• Should I keep working on evaluation work right now?
• Is it even possible to conduct evaluation right now, considering travel restrictions, office closures, and social distancing?
• What does the future of program evaluation look like? For nonprofits, for funders, for program participants?

While I certainly don’t have all the answers to these questions, here’s what I do know:

• The need to measure program impact, to collect and analyze program data, and to share findings widely isn’t going away
• We need to be willing to embrace this “new normal” and think out of the box to find creative solutions and a path forward. Here’s just one great resource I’ve found that’s full of practical and creative ideas for these strange times: “Contactless Evaluation in Times of Covid-19” by Vardhani Ratnala on the AEA blog
• Organizations must continue program evaluation work to improve and strengthen program outcomes, and ensure future program excellence and growth

My current work with Metro, the regional government agency for the greater Portland, Oregon area, is an example of how to move forward with program evaluation during these unusual times. The project is to evaluate Metro’s Investment and Innovation Program. This three-year pilot grant program is designed to reduce waste in the Portland Metro Area through nonprofit and business grants.

Instead of letting the pandemic halt progress, we have quickly adapted how we worked together.

Instead of in-person working sessions and my trusty box of meeting facilitation tools (my go-tos include flip charts, markers, and hands on mini-activities!), we quickly shifted to online meetings and email.

Here are just a few things we’ve found that work well:

All our meetings have strong, clear agendas. We use video calls to maintain a more personal working environment. We are meeting more frequently now than before, which helps us all stay on the same page. We stick to our meeting start and end times, which means we stay on task, productive, and focused. We include some “check in” time at the beginning of each meeting so we can share any personal updates and stay connected on a social level. When looking at materials, one person shares their screen so we’re all looking at the same thing at the same time (instead of each of us opening our own document on our computers). There’s always a designated note taker, who captures ideas in real-time, so the meeting feels dynamic.

While of course I miss being able to connect and collaborate in person, the team is embracing the changes and keeping the work moving forward.

Because we all have a shared willingness to embrace a new virtual style of working together, I’m confident we’ll all find answers to all of the functional questions we have about our evaluation plan in the coming weeks.

There is no “one size fits all” answer to how best to continue doing evaluation work in these uncertain times – but it’s clear that without creativity, flexibility, and courage, discovering solutions for these difficult questions would be impossible.

Here’s the big takeaway: while the pandemic has changed how we do program evaluation, it doesn’t change the fact that program evaluation is still important for organizations to make informed decisions about current and future programs.

Organizations owe it not only to current program participants (or grantees, in Metro’s case!), but also to future program participants to always remain dedicated and focused on continual program improvements.

Before I go, remember these 3 things:

• We’re all in this together. No one needs to feel alone through this – and that includes nonprofits, foundations, and other organizations working in the program evaluation space.
• With the right attitude, we can harness the power of collaboration and teamwork. Together, we can bravely face an uncertain future with hope and solidarity.
• I’m here to help! If you ever want to discuss your program evaluation needs – either during or after the coronavirus pandemic – please reach out! I’d love to meet with you and see how I can help… virtually for now, of course!

A special thank you to the Metro Investment and Innovation team Suzanne Piluso and Laura van der Veer for your commitment to measuring program impact. Your dedication to collecting and using data to understand your program’s successes, challenges, and opportunities is powerful. I’m proud to work alongside of you to learn all we can about how the program has been working and to give you the tools to keep growing and improving!

To all those organizations and my fellow evaluators who remain committed to data driven decision making through these trying times… keep up the great work! We’ve got this!

 

Resources I’ve found especially helpful lately:

• Managing & leading remotely
  • Nonprofit leadership guru Joan Garry gives 6 tips to help organizational leaders be successful in managing and leading teams in this new era of social distancing. Whether you’re a nonprofit executive director, a foundation CEO, or a board chair – I highly recommend taking a few quick minutes to read “A Priest and a Therapist Walk Into A Bar: 6 Tips For Leading From a Distance“.
• Tips for avoiding virtual meeting fatigue & facilitating effective virtual meetings
  • Beth Kanter, an internationally recognized thought-leader on all things nonprofit, has some fresh ideas about how to avoid virtual meeting fatigue, plus some advice for virtual meeting facilitation for more effective remote meetings.
• Ideas for conducting contactless program evaluation
  • I am constantly impressed by the guest bloggers featured on The American Evaluation Association 365 Blog, and this is no exception! Vardhani Ratnala’s recent article has some great ideas for how evaluators can conduct “Contactless Evaluation in Times of Covid-19”.
• Encouragement regarding qualitative program evaluation during the pandemic
  • New Zealand evaluator Isabelle Collins encourages those in our sector to embrace the opportunity in front of us to think outside the box and to overcome your “inner technophobe” in her AEA 365 Blog article, “Evaluation in the time of Covid-19“.