When reporting to the rest of the company, you have to take into consideration statistical significance.
The reader may not always take it into account.
The reader may then make decisions based on insufficient data, coming to the wrong conclusions.
This may make whole teams waste time on features that are irrelevant or even harmful to the product.
Analysts can mitigate this issue by adding notes to their reports.
Indicating an interesting trend in the data but always noting that it’s not significant and may warrant further study.
– Samy
Component Analytics daily
Every weekday, I publish about Component Analytics and analytics collection strategies.