When is Less More?
January 5, 2024
Kevin Schulman, Founder, DonorVoice and DVCanvass
Facebook is nothing if not obsessed with users using the app more frequently and for longer durations. App notifications are like crack for this hamster wheel and so the idea of sending fewer seems anathema.
But this is exactly what their data science team did because user satisfaction surveys showed a preference for fewer notifications.
The result? It was a usage train wreck. Sending fewer notifications resulted in a significant decline in app usage. In most testing paradigms this would result in two knee-jerk reactions,
- People don’t know what they want or say what they do – hence surveys are mostly noisy and misleading
- The only thing that matters is in-market testing – hence case closed, fire up the notification machine
But the Facebook team had more confidence in their survey data, their users and the extended thinking on why sending fewer notifications might still work despite the early results.
There test was baked in from the beginning to be a year-long experiment. They hypothesized that fewer, but more relevant, notifications might enhance user satisfaction without compromising app usage.
- They used customer satisfaction data to only send the notifications that users rated as a 5 (on five pt scale) in import, instead of the usual 4 or 5 scores.
- They theorized that it may take a while for users to notice and appreciate fewer notifications.
- This would in turn breed higher user satisfaction overall and higher retention.
But the initial results were scary to look at and absent a well-thought-out, longer-term test with rationale it’s easy to imagine lots of pressure to abandon ship.
Look at retention (as measured by usage) in the sea of red in the early days, yikes. This is akin to sending fewer solicitations but only running the test for one month. You’ll never learn anything, but you will reinforce the false, too simplistic by half belief that ask more = make more.
The green rising tide tells the rest of the story at Facebook. Their key finding, “After a year we saw that in the fewer notifications experience, users were using Facebook more — it just took a long time for user behavior to shift and less disruption led to high organic usage, which increased both user satisfaction and app usage.”
Lessons for fundraisers?
- People don’t give because we ask. But if you believe that “ask more = make more” then you may disregard the first statement.
- People only become “good” donors (i.e. repeat givers) if they are psychologically satisfied, which is measured by measuring their sense of Autonomy, Competence and Relatedness
- Key driver analysis of these satisfaction measures combined with actual giving data will tell you what causes donors to be psychologically satisified
- Nowhere in this list of key drivers will you find “send more solicitations”
- If you overall communication and interaction with donors makes them feel satisfied, they’ll keep giving but mostly on their own terms. How one asks and what one asks for does tactically influence how much and how often people give.
- Any material change in donor/user experience may take a bit of time to get noticed and adjusted to
- Long-term effects may be different from short-term effects, or even the opposite