I spent about two years working on measures in the non-profit world. Looking back on it, it was a very rewarding and–at times–annoying struggle. I thought I’d share some of my high-level takeaways from that period of my life, in case you’re interested in measurement or self abuse.

Measures in the for-profit world seem hard enough from my limited experience, but in the non-profit world they can be downright difficult. Obviously, on the business side of things you have the luxury of a bottom line. That, my friends, is a huge plus.  But, I’m here to tell you that a bottom line only gives you so much comfort.  Measures are difficult, no matter what the situation.  Is it easy to measure the marginal impact of one person on a 7-person, highly integrated team?  Is it easy to measure the effectiveness of a brand?  Is it easy to measure the customer satisfaction of an internal service provider?

No.

A few of my favorite tips, tricks and ways of thinking are quickly sketched out below.  Leave questions in the comments if you want more detail, or feel free to add to the list of best practices!

• Kirpatrick Framework–if you do any sort of training, the Kirkpatrick Framework is a great way to (a) make sense of what you’re trying to do and (b) evaluate it.  It suggests that you measure the following things (which get harder as they get more important): what were the participants’ reactions to the training, what did they conceptually learn, what behavior change is a result of that learning, and what results were achieved after the behavior changed?
• Success Case Method–Let’s say you have something you want to measure that doesn’t have a lot of hard numbers easily accessible.  As Skyler pointed out in this great post, the Success Case Method may be of use to you.  It relies on interviews with participants as a way to get at whether your program had a positive ROI or not.
• “Directionally Correct”–One of my favorite phrases in measurement, “directionally correct” helps me (and often overly wound-up stakeholders) keep in mind that “good enough” is good enough.  In the world of measures, we tend to fall head over heals for numbers, stats, standard deviations, and any “hard” numbers. It might be blasphemous, but sometimes–sometimes–you don’t need the statistically perfect sample size to get good enough information to act on, especially for small or medium-sized projects.  Relying on anecdotes and imperfect measures can be a cheaper way to move the ball forward.
• Constellations–Sometimes your measures just aren’t too precise, even for those of us comfortable with the “directionally correct” standard.  Another little mental model that I like to use is the idea of combining a few imperfect measures into “constellations” of measures.  If all of them are forming the same basic pattern, or all are heading in the same general direction, then I feel better trusting the measures than putting all my faith into one.  Now, if you’re about to make a multi-million dollar investment based on this constellation of imperfect measures, you might think twice; but if you’re trying to figure out a go/no-go decision for an in-house project, this might be good enough to help you act.
• Benchmarking–Don’t forget to look around to see if there’s anyone to compare yourself to.  I bet that for most projects, there’s another internal team, a competitor, or a “best in the world” example that could illuminate your relative success.  This might help you identify measures that are pertinent, and may help you get a sense of your key characteristics (quality, reliability, speed, etc.) ranked up next to others.  The classic example is Southwest Airlines: they wanted to improve turn-around times for baggage handling, moving planes in and out, and loading and unloading passengers.  Instead of looking at other airlines to compare themselves to, they looked at practices used by pit crews in NASCAR.
• Program Theory/Logic Model–One of my favorite measurement tools, the program theory or logic model is a great conceptual model to help you (a) better plan out your projects so that they achieve results you actually care about and (b) better measure the key elements of your plan.  The basic structure can be seen in the link above, but it goes something like this: INPUTS > ACTIVITIES > OUTPUTS > OUTCOMES > ULTIMATE GOAL.  By laying out each element, the program theory helps you isolate “chunks” of your project to measure, as opposed to trying to measure the whole thing in one (nonexistent) super measure.

Kirkpatrick, Logic Model, Measures, NASCAR, Program Theory, Success Case Method