As i’ve been digging deeper into Soft Launching for my upcoming GDC talk, I’ve been asking, When is a Soft Launch complete? How do you know when you’re ready?

These are the hardest questions a Product Owner can ask and the truthful answer is, you will never really know. There are no clear answers, there are no strict redlines or clear benchmarks, at the end of the day it’s a call based on the data you have. Soft launching is all about data. Data comes in from the market and as a PO you have to judge whether you have strong game/market fit. I feel you have four key questions your data must try to answer:

• Are players stuck anywhere?
• Do my players stick at the game?
• Have I stacked some cash?
• Can it scale?

You may not have a clear answer to any of these questions, but you should still be in a position to make a call. Whatever numbers you have you cannot expect them to change by more than 20 percent in either direction.

Stuck

This is the easiest of all the questions to answer initially, but hardest of all to call before launching. This simply requires progression tracking of players based on something that gradually increases as a player plays, such as XP, Levels or Battles. You want to track cohorts of players who are at each stage and see of those players what percentage churn out. You’re looking to see:

• The XP level after tutorial with the biggest churn
• The single point in your funnel with the biggest loss of users
• Any gameplay feature that is used 10 times less frequently than your core loop

Churn is not directly related to any one feature, but spikes can signify bugs, difficulty issues, lack of clear objectives, or a game that’s devoid of challenge.  Keeping a track of your funnels and setting yourself a baseline which you want to stay under will allow you to optimize your game development to areas that are losing the most people.

Stick

If being stuck is all about Churn, then sticking is more about retention.  Rather than really looking at your game in terms of a progress, you’re looking at your game in terms of time. All F2P Games need to be playable for long periods of time, and it’s preferable to try to setup a habitual format that players get used to on an hourly, daily, weekly and even monthly timetable.

Retention graphs provide the best measure of this. Rather than obsessing about the metric you are at right now, focus on improving it from week to week by analysing weekly cohorts.  Each week take the D7, D14, D21, D28 and compare it to the previous weeks during development – the question is, have you seen any improvements? The later day retention figures are much more important, but also very difficult to clear quantify.  If you’re game is not improving steadily week on week, you’re missing a feature or the core loop is not holding up well after hundred of playthroughs.  This could be down to the core game not having enough Luck, Stats or Skill and requires deeper analysis. You want to see subtle but gradual improvements throughout your soft launch periods.

Stack

Stacking Cash. Quite simply, is your game ROI positive or getting towards ROI positive? You need to look into your marketing spend here and see how the LTV > CPI balance is looking.  During soft launch you may not have the data to accurately estimate purchases, or your retention curve and so you amplify those inaccuracies with an LTV calculation. If you’re only just ROI positive when you run a 365d LTV calculation, I would assume you’re wrong.  

What you do have at your disposal and with greater accuracy are:

• Total spend Vs Total Earnt (ROI)
• Conversion Rate + ARPPU
• Gems spent in game

Your ROI will almost certainly be negative. It’s likely to be extremely negative perhaps at 10 percent or less in 30d period.  Dependent on how your game is intending to monetize and how your retention curve does look you will need to set a benchmark with what you as a team are happy with and see if it can be improved with each build.

When observing conversion rate and ARPPU look at them as averages for each build.  This will give you a fair indicator of what’s likely to happen if your game could scale to more and more people.  Use a spreadsheet to work out, what’s the total number of installs we need to keep the lights on? Does this seem feasible with a feature/strong press coverage?

Finally, use your Premium Currency in game to see how quickly or how much spend is happening with your users and what they are spending it on.  As very quick reference, the gem spend per user per build should be increasing. Also making sure that the average number of gems in someone’s wallet is low means that people are not hoarding but need to spend them to progress. You’ll have to think about it in your games context but, the more gem spend the better.

Scale

The hardest of all the questions is “will my game scale?” Scale changes lots of things in lots of ways and is a true reflections of a well executed game is one that grows in fun as this scale, e.g. Pokemon Go. So without truly knowing, you have to make gut calls. In my mind, there are are four key piece that your team needs to be aware of:

Marketing – You can test this, it just requires money and it requires you to spend it faster on the marketing platforms.  As you increase your bid prices and your spend limit you will speed up installs, hopefully without changing your CPI as virality and organics help. Without going into detail here, you want to try to keep your CPI flat as the rest of your costs and spends increase.

Mechanics – The core game might work for the first 100 levels, but is it simply a matter of adding 100 more? Do you need to add an evolution or twist to the mechanic to keep it fresh? If you do make that change does it break your previous mechanics?

Economy – Almost all economies blow up at some point.  You will likely have an imbalance of power or items in the game, but can you detect them? This is what a soft launch will help you to decide?

Content – Is the team ready to handle the technical challenge of doubling or tripling the game? How much time does it take to create a level? How much time does it take for players to complete a level? Can the servers handle it? These are practical questions that if the game is a large success, you are not shocked by the complexity of growing your content.

Conclusion

All the decisions in the world won’t prepare you for abject failure or viral success, but in most cases your game will launch to some acclaim and then face the uphill battle of fixing, smoothing and perfecting the mechanics that work for your audience. The more harsh you are during the early stages of a game’s development, the better you can scale it later on, however, killing a creative endeavour that never hits the app store can be just as costly to your company and mental state. At the end of the day, launching games is why you started game development in the first place sometimes you should go with your gut. Fuck it, ship it.