Platform product · Analytics · Data quality
Analytics clients could trust through a platform change
- Role
- Product Manager, Obsess
- Employer
- Obsess
- Period
- May 2022-January 2024 (employment)
In one minute
I product-managed a reusable analytics product at Obsess, from requirements and visualizations through testing, rollout, and handoff. I kept old and new analytics separate so clients would not compare measures that were not statistically equivalent.
What I owned
I wrote the PRD and worked on features, metrics, and visualizations. I participated in validation, rolled the product out internally, handled the first external rollouts, and then handed subsequent rollouts to the data team.
The analytics team contributed substantially to testing and delivery. My responsibility was product definition, decisions, initial rollout, and handoff.
What held up
Spatial heat maps helped client teams decide which rooms to reduce or remodel. Separating old and new analytics avoided presenting incompatible measures as a comparable series.
The analytics decision, step by step
The comparison risk
A platform change introduced measures that were not statistically equivalent to the old analytics.
My decision
Keep the old and new analytics separate so clients would not read them as one comparable series.
Validation with analytics
Test clicks, time, and heat-map accuracy. Resolve a refresh issue; identify incorrect formulas in some API metrics.
From data to use
Roll out internally and to the first clients, then hand off to the data team. Heat maps helped clients decide which rooms to reduce or remodel.
Explanatory reconstruction of my product decision. This is not the original dashboard or a measurement of business impact.
Choice + alternatives
The consequential choice
I decided to keep old and new analytics separate. That preserved the distinction between measures instead of encouraging clients to read a false before-and-after comparison.
I worked with the analytics team to test clicks, time, and heat-map accuracy. We resolved a UX/UI refresh issue. Testing also revealed incorrect formulas behind some API metrics.
Stakes
What was at stake
Client teams needed to understand what people did inside a 3D experience and use that information to improve it. An attractive dashboard would not help if its measures were inaccurate or misleading.
Constraint
The hidden constraint
The work coincided with a Google Analytics transition. Old and new measures were not statistically equivalent, and the incoming product should not be designed around a system that was being retired.
Delivery
What we built
A reusable analytics product with spatial heat maps that made low-activity rooms and products visible to client teams.
A rollout path from internal use to the first client deployments, followed by a data-team handoff.
Supporting material
Inspect the work
- Analytics decision walkthrough
View artifact · Explanatory reconstruction
A labeled reconstruction of the comparison risk, product decision, validation, and rollout, shown above.
What this proves I can do for you
I make measurement quality, client interpretation, rollout, and handoff part of the product decision.
Relevant problem?