When it comes to funnel metrics, what matters is not the pirate metrics themselves but the funnel mindset. I want to share a case (still in the loop of hypothesis-experiment-improvement-hypothesis-experiment...) where, with limited internal staff and resources, we apply a funnel mindset to service operations planning using log data accumulated in the admin.
Situation.
1. An MVP service built and released in a very short window.
2. Refactoring was needed, but the Flutter developer left and no one took over the handover.
3. We were unable to collect log data from the app's main CTAs and key pages via tools like GA.
4. For a stretch of time, we had to operate the service without any front-end developer.
Task.
With only back-end staff, create funnels that support operations and decision-making.
Exploration 1.
Using logs accumulated on key events, work backwards to figure out how users actually move through the app service, then build the matching funnels.
Approach.
1. Gather the scattered logs
1) Using the signup date as the anchor, list the times (min, max, avg) to reach (complete) the main features such as create, edit, delete, etc.
2) Sort the listed items in descending order.
2. Check the human-road based on reach times
1) List the order in which members use the app's main features after signup.
2) Review whether that order actually matches the MVP and MLP conditions of our service.
5/2. Define user groups
1) Split users into groups by comparing their reach priorities.
2) Split groups using attributes from the membership info.
3) Match the two sets of groups and redefine the effective groups.
3. Compare the designed path with the human-road, and devise responses and improvements
*If, at 5/2, you were able to define user groups, the rest explores responses per group
1) If the priorities differ, come up with a plan to adjust them.
2) If they match expectations, come up with a plan to shorten the gaps between each step.
