By Kirk Franzen, CTO of MyCrowd
You’ve spent months developing an app that know your users will love and now it’s time to test your app amongst real users before you release your app in the App Store. Testflight is one of the leading iOS beta testing platforms to test an app and with good reason — it’s highly effective at providing detailed user feedback, crash reports and analytics to app developers and ultimately, helps developers ship a more polished product.
In this post, we outline three possible routes for beta testing an app through Testflight:
1) Beta testing with external testers
2) Ad-hoc distribution
3) Testing with internal testers
We’ll start with the first route of beta testing with external testers.
Beta testing with external testers
Through TestFlight, you can invite up to 2000 users per app to be external testers. Before you can add and invite testers, you have to do some special setup in iTunes Connect. Note that this process is different than the normal app build and distribution steps.
1) Enable TestFlight beta testing through your Apple developer account.
2) Then, create a Distribution Profile with the beta entitlement to distribute builds via TestFlight.
3) Upload a new build.
4) Before submitting the build for approval you must add a Description and information on What to Test. This is required for beta approval.
Note: At this point, you can distribute the build to internal testers. You can invite up to 25 people who are members of your iTunes Connect team. To test with external testers, continue reading step five below. To learn more about beta testing with internal testers, check out “Testing With Internal Testers” towards the end of this post.
5) Submit the app for beta review.
After beta approval from Apple, you can then distribute to external testers. To do this, you need to first add external users and then invite them to test your app. Make sure you have the email address for each tester and their first and last name (optional). Your build will available for up to 60 days to your external testers after they receive their invite from you.
I listed the detailed steps below. First, you need to add testers to your external testing list:
1. On TestFlight, select External Testing from the left pane.
There is also the option of importing a .CSV list of testers by selecting ‘Import File’ and then choosing a CSV file with the following format: first name, last name and email address. You can also organize users within specific groups. Alternatively, you can add existing testers by choosing Add Existing Testers from the Add (+) pop-up menu and selecting testers from the list.
Once you have created an external testing list, you need to invite those testers to access and test your beta app:
- From the External Testing pane, under TestFlight, click Add Build to Test next to the platform you want to test.
- Select the version from the dropdown and then the specific build you want to test and click Add.
- If you have not already added your Test Information, you will be prompted to do so. Then, click Next.
- Answer the Export Compliance questions and click Submit.
Your build will then be marked as Waiting for Beta App Review. Until there is at least one build approved, the version will show as Not Available for Testing.
Once your build is approved, you will be able to choose it from the External Testing pane and then click Start Testing to send invites to your testers.
From there, the build status will change to Testing and display the number of days that your testers have to test the app. The external testers will then get an email directing them to accept the invitation using the TestFlight app on their iOS device or via a redemption code.
To get testers outside of your team and inner circle, you can use MyCrowd QA, which has over 35,000 testers from the U.S. and around the world.
In addition to beta testing with external testers, you can also take an ad-hoc approach to testing apps.
This requires having specific device information and creating a build of the app that includes it. The app can then only be installed on those specific devices. The detailed steps on how to take an ad-hoc approach to testing can be found in the iOS developer library. As well, Mike Rettew provides a helpful post on how to create ad hoc distribution provisioning profiles.
Testing With Internal Testers
Testing with internal testers can be done fairly easily with just a list of names and email addresses. The testers included are simply emailed an invite to download and from there, can install and test the app in depth. But there are a couple of gotchas:
1) The emails must be added to the App owner’s iTunes connect account as technical resources. This gives app developers some ability to view the iTunes Connect account. Not an ideal scenario if you would like to get feedback through a diversity of users and test your app through a crowdtesting platform like MyCrowd’s studio.
2) Another caveat: if the email provided corresponds to an existing Apple account, it will create some hiccups. It is best to get a list of emails that do not already have Apple accounts. In this case, testers can have multiple Apple IDs. Check out the full details for setting up internal testers here.
Using Testflight sometimes isn’t as easy as it looks. In fact, we often have even experienced developers come to us with questions about using Testflight to beta test their pre-release apps.
Mat Cegiela writes a helpful need-to-know guide on using Testflight and we’ll cover taking an ad-hoc approach to testing and testing with internal testers in more depth in future posts.
Questions about using testflight? Shoot us an email at firstname.lastname@example.org and we’d be happy to help.