Category Archives: Android

Ray Casting Algorithm

If you’ve been struggling with the problem of determining whether a point lies inside or outside a polygon, perhaps in google maps, read on. This post is specific to Android and Node.js but the algorithm can easily be implemented in … Continue reading →

Read more at the source

Custom Accessibility Services

In the first post, I talked about making apps accessible to all. In this second and final post I will explain how to create an accessibility service and what can be the intentions behind creating one. How to create a … Continue reading →

Read more at the source

Accessibility: Increasing app reachability

I always thought of accessibility services in Android meant to help people with visual or physical disabilities in accessing apps, until recently, when I came across few apps which requests user to turn on their custom accessibility service in order … Continue reading →

Read more at the source

Droidcon India 2014 Experience

Last week we attended Droidcon 2014, held in Bengaluru. First thing we noticed was that we had underestimated it a lot. The event was humongous. With around 250-300 attendees, most of which were either cream Android devs or entrepreneurs. The … Continue reading →

Read more at the source

Tips for Designers: from a Developer

Android is a versatile OS with more than 1000 device manufacturers and more than 18000 distinct devices. Screen size of android phones vary from 2.6” – 6” and the resolution of screen ranges from 240 X 320 to 1440 X … Continue reading →

Read more at the source

Communication patterns for application components

Activities, services, fragments, helper classes etc. are main components of Android applications but its tricky to establish communication between these components. It’s tricky when one cares about writing reusable code – loosely coupled, plug-n-play-able. The goal here is to avoid … Continue reading →

Read more at the source

Fragment view state retention: A dirty solution

This is the last part of this 6 part series about Fragment Oriented Architecture in Android applications. In the previous post I talked about managing sessions in fragment oriented application. In this post I am going to talk about retaining … Continue reading →

Read more at the source

Session Management

This is the fifth part of a 6 posts series on Fragment oriented application architecture. In the previous post I talked about efficiently handling back button press inside fragment. In this part I am going to talk about session management … Continue reading →

Read more at the source

Handling back button press Inside Fragments

This is the fourth part of a 6 posts series on Fragment oriented application architecture. In the previous post I talked about Inter-Fragment Communication. In this part I am going discuss about elegantly handling back button press inside fragments in … Continue reading →

Read more at the source

Inter-Fragment Communication

This is the third part of a 6 posts series on Fragment oriented application architecture. In the previous post I talked about Transaction BackStack and its management. In this part I am going to talk about Inter-Fragment Communication. It’s a … Continue reading →

Read more at the source

When to A/B Test

March 11, 2013

Questions:

We’ve heard that A/B tests are awesome and all, but should we use them for everything?
And if not, then when should we run an A/B test?

Answers:

We should treat A/B testing just like any other decision making tool – it has costs and benefits and we should use A/B testing when the benefits outweigh the costs
Like all other tools it might yield incorrect results (false positives or false negatives) and it carries costs (implementation complexity and time, waiting for results).
When deciding whether or not to run an A/B test we should consider the tradeoffs versus alternative weighs to make a decision.

Decision making tools

Here’s a quick summary of a handful of decision making tools and when to use each. At TaskRabbit, we generally like to use gut calls, qualitative research, and A/B tests to make most business and product decisions.

Couple of observations:

As a startup, we’re OK with trading off decision-making confidence for speed (i.e., if an A/B test is looking particularly hairy to set up and run, then we’re perfectly fine with making the call based on gut or small-sample qualitative research).
The folks who work at TaskRabbit generally start to develop pretty good intuition about where problems/opportunities are in our space because we use our own service a lot. So, if making a gut call can instantaneously yield a ~80% confidence decision, then running an A/B test for a week to get an incremental 10% more confidence is a pretty high price to pay.

* = our preferred decision-making methods

Coin Flip	Gut Call*	Majority Vote	Qualitative Research*	Survey	A/B Test*
Pros: Instaneous decisions	Pros: Fast decisions You could be the next Steve Jobs if your gut calls are right 90% of the time	Pros: Team buy-in Access and summarize an entire group’s expertise	Pros: Get to know users and build empathy with their concerns Observing users is a great way to discover and surface unknown problems/opportunities Very difficult to argue with video footage or direct quotes from real users	Pros: Good way to gauge sentiment of a large sample of people Medium-speed decisions; it’s fairly easy/fast to recruit and drive traffic to a survey	Pros: Statistically significant results can be tuned to make them as confident as you want them to be (most teams use a 90%+ confidence interval) Collects very reliable data on real users interacting with a live product Very difficult to argue with statistically significant results
Cons: 50/50 chance of correct decision Only handles binary-choices, not so good with multi-variate permutations	Cons: Difficult to justify decisions to a skeptical audience Incorrect decisions made this way get attributed to you, personally	Cons: Can turn into a slow, political decision making process Groupthink can de-rail rational decision making	Cons: The research subjects you choose may not be representative of the entire userbase Recruiting research subjects and running tests takes time and budget	Cons: What people say they would do in a situation often turns out to be different than what they’ll actually do when faced with a real, working product or a real, live human being A skeptical audience will argue with your survey design	Cons: Takes time, code, and sometimes marketing coordination to set up an A/B test Once a test is running, takes a while (a week or more) to reach a large enough sample size Can only run a limited number of A/B tests at one time It’s a bit like panning for gold – many/most tests will be inconclusive, i.e., will not yield statistically significant differences

Hope that helps! Please feel free to comment…

[…]

The Ruby Rails Aggregator