Global online learning platform, 2026
Making product data trustworthy at scale
For a global learning platform, I audited 13M+ monthly product events and a 25K-table warehouse. I found tracking breaks that were silently inflating MAU and conversion, and turned the findings into a measurable, revenue-ranked roadmap.
The system
- Sources
- Capture
- Identity
- Definitions
- Reconciliation
- Baselines
Disciplines I coordinated
Engineering, data, business
- 13M+product events a month, audited
- 25Ktables in the data warehouse
- 11fixes prioritized, several with no app release
Context
Rootstrap's team was working on the client's mobile apps. The mobile roadmap had five priorities: better recommendations, opening the app on the learner's course in progress, fewer video freezes, more reliable store purchases and closing security gaps. I prepared a data assessment alongside that work, as a complementary contribution.
The problem
Product decisions relied on two sources, the analytics tool and the data warehouse, that disagreed in places. Nobody could say how confidently each priority could be measured.
My role
I led the assessment end to end: scope, method, hands-on analysis in Amplitude and Databricks, findings, fixes and the client report.
What I did
- A layered method. Sources, then capture, identity, definitions, reconciliation and baselines. Every figure in the report traces back to a saved chart or query.
- Found the purchase event wasn't a purchase. It logged entitlements, so most of its volume was zero-dollar. I defined a clean payment and trial-start signal that needs no app release.
- Found the gap in attribution. No source recorded the screen where an enrollment or payment started, and most first payments from app users closed on the web.
- Caught silent tracking breaks. One Android release mislabelled tens of thousands of learners a week. A background audio event inflated logged-out users and MAU. One iOS event fired many times per device.
- Reconciled the two sources to within half a percent on the first check.
- Rated each priority as high, medium or low measurability today, then ranked eleven fixes.
- Built the initiative portfolio. Each initiative has the metric it moves, guardrails, an app versus web baseline and a directional revenue range net of store fees.
Outcome
The report and the revenue-ranked initiative portfolio were delivered to the client.
Stack and tools
- Amplitude
- Databricks SQL
- Unity Catalog
- Google Sheets
- Claude