Beta Testing: Alpha Beta Theta (Always Be Testing)

18.09.2017 |

Episode #9 of the course Bringing your app idea to life by Kara Ntumy


Welcome to the penultimate lesson! You may have been wondering all this time how to answer the most important question after you build an app: Do my users like my app? In the previous lesson, we talked about gathering user feedback and using this to influence the changes made to the app in each iteration. But how do you go about gathering and evaluating this feedback? Today, we will delve into the testing phase, take a look at some testing tools, and finally answer all these questions!

Now, after you have spent countless hours sketching, animating, prototyping, documenting, and (possibly) coding your app, it makes sense to test it out with real users to find out whether they enjoy using it. Enter the testing phase, a period spent for ensuring that the app is ready for launch. This phase is split into two—alpha testing and beta testing—and if you have been validating often, your app would have already been run through some tests (most likely by you) to help find and fix bugs (code-based issues).


Alpha Testing

Much like this testing, alpha testing is done toward the end of the development process, when the app (still an MVP) is in a near fully-usable state, and focuses on 1) improving the product and 2) ensuring beta readiness. In organizations, alpha testing is typically run internally, so only team members and other stakeholders who have been involved in the development would normally participate. For you, that would be any developers or test engineers (and you!) simulating what it would like to have a large user base using the app. This would entail fixing critical issues and the modification of features, depending on early feedback. Once these issues are addressed and the app is performing as expected, the next stage of testing ensues.


Beta Testing

Beta testing involves specifying the test devices, choosing the right segment of testers, the distribution tool, and method of feedback collection and evaluation. The main distinction between the two types of testing is that while alpha is done hidden away from the eyes of the public, beta testing is tested in the “real world” with some “real customers” (~100 or more) who can provide feedback on every element in the product. Beta testing only has to run a few weeks long, as the expectation is that there will be few bugs, fewer crashes, and even fewer changes. Much of the feedback collected here will be saved for future versions of the product. At the end of these tests, you will have a better idea of what the users think about the product and what their user experience will be like.


Testing Toolbox

Also, there are numerous testing tools in the world of iOS; the most popular ones are Apple’s TestFlight, which offers some basic information about crashes and sessions, or HockeyApp, which has a better offering of user metrics (e.g. devices on which your app crashed). In the Android world, you have Fabric Beta and the Google Play Console, which allows for beta and alpha testing.

When collecting feedback, you have the option of creating a questionnaire, creating a forum, or simply providing an email address where users can send their feedback. You can record and evaluate this using a spreadsheet. Alternatively, there are services such as Beta Family that help you find beta testers (for both iOS and Android apps) and create, distribute, and collect test reports. It may be a little tricky to execute a sound testing strategy, but it is extremely important. And guess what comes next: the app release party!


Recommended reading

Beta Testing: The Ultimate How-to Guide


Share with friends