Machine Learning masterclass #1 – How Coda uses ML to predict Lifetime Value

Machine Learning masterclass #1 – How Coda uses ML to predict LTV

At Coda we use machine learning (ML) in three different areas, and over the coming weeks we are going to explain these processes in a little more depth.

The first area in which we use ML is to predict the success of a game. We have developed a Lifetime Value (LTV) model which enables us to calculate how much a game is likely to make for its creator. As soon as the game is published on the app store we monitor how well it’s doing and, using machine learning, we are able to quickly gauge how successful it will be.

The second use for ML in is game annotation. We curate images of games on the app stores and label them with attributes such as genre, artistic style, game mechanic, 2D or 3D etc. Up until recently this had to be done manually. Now a deep learning model has taken over and uses screenshots of untagged games, and adds those tags automatically.

The last one, which is currently in development, is the SDK client model. Even though it’s in its early stages we are very excited about its huge potential to optimise games. We will talk more about this in a few weeks.

We wanted to share with you a bit more about the role that machine learning plays in determining games LTV , and how we use this information. So here Muhammed Miah, ML Engineer at Coda explains the process in more detail.

One of the ways Coda uses machine learning is to assess the potential value of a game. What process do you go through to set this up?

So ‘value’ here refers to the total advertising revenue that we expect a game to be able to generate. Two key components are necessary to make that assessment possible, namely, data and the algorithm.

Starting with user acquisition data, we look at how much it costs to get a user to play the game and couple it to actual gameplay behaviour. The moment that a game is published on the app store it starts generating interesting metrics. For example, we get early clues as to its revenue potential by how long users spend in the first session and whether they come back to the game.

One thing that is worth mentioning is that we have insights for all games historically. We are in fact comparing the behaviour of users for a specific game with that of all Coda games in the past, and we are able to see where it ranks against other games while making the prediction.

The prediction itself is made by a machine learning algorithm called a ‘Random Forest’. It is fed the above-mentioned data and calculates how long users are likely to play the game for, and how many ads they are likely to see in that time. This then enables the algorithm to come up with a figure for how much the game will make.

How will this evolve in the future?

We want to predict more than just lifetime value. We want to gauge players’ total playtime as well as engagement per level. This will provide us a better understanding of our users and on exactly what parts of our games that they enjoy the most.

Also, we will be rolling out models to help us understand a game’s revenue potential at all stages of its lifecycle. With this in place, we will have an idea of what to expect from even just the adverts that bring in users for the game. I believe that that will have tremendous value.

So once you have the data and LTV what happens next?

One of the key reasons for ascertaining the lifetime value is to help us make a decision about whether to go forward with the game or not. 

The question boils down to ‘is the lifetime value bigger than the user acquisition costs?’ If the LTV prediction is projected to be much higher than the user acquisition cost then we go forward with the game. If it’s not much higher, or perhaps is actually lower than the user acquisition cost then it’s clear that we have to make changes. The worse case scenario here of course is having to cancel the game.

So far we have put over 1,500 games through the Coda pipeline. This has provided us an enormous amount of data in terms of what makes a game successful, and I am glad to be able to leverage this when new games enter the Coda ecosystem.

So it is about balancing the cost to acquire users with the LTV?

Correct. What we want is for the machine learning algorithm to suggest that a game will make more than we spend on it. When that is true, we can then push that game forward much faster and acquire users more aggressively.

What is interesting is that our machine learning system is self-optimising and as more data comes in, it automatically retrains and produces better predictions. In essence, this is why we chose to add machine learning to this step of the process. Now that this is automated, we are able to pursue promising games much faster and help many more developers.


How machine learning can help you perfect your game

Machine learning is the process of teaching a computer system how to make accurate predictions when fed data. The technology, which incidentally is a subset of Artificial Intelligence, is ubiquitous. Whether it be algorithms that are powering recommendations on Netflix, Spotify, or Google determining whether an email we have just received is spam, we can really  take for granted the outsized role machine learning really does play in our lives.

Machine learning has been used sporadically in game development for several years now, but now has a key role in the creative process in enabling developers to work out what type of games to produce.

Machine learning underpins a lot of what we do at Coda from our Market Intelligence tool through to the SDK kit that enables developers to build and commercialise games. Here is a little more detail about how we use machine learning and the benefits it brings.

Assessing the market

Many game developers start their process with an idea. This might be to make a game in a specific type of genre or one that will attract a certain type of player. Some of their decisions are based upon what they enjoy about games developing and what they feel confident in producing. For others it may be driven from a commercial perspective. What type of games are going to ultimately be successful and make them money?

There are thousands of people in the world who can make games. Yet only a small minority of the games that are actually produced are a sustainable commercial success. The balance is already tipped in favour of the big studios who can back their bold, innovative ideas with large amounts of cash to acquire users and ultimately achieve virality. 

So games developers need a helping hand in understanding what games are likely to be successful and why. This is the mission of our Market Intelligence tool which uses machine learning in two very effective ways. Firstly it uses computer vision technology to tag various elements of the games that are already in the app stores. So it will be able to tell, for example, if a game is 2D or 3D or if it is cartoon-ish or a puzzle game but most importantly it can accurately determine which more mechanics and secondary mechanics the successful games are using. 

Among the parameters it looks at are;

Game Mechanics: IO, Idle, Puzzle, Rising Falling, Swerve or Tap Timing

Controls: Drag & Drop, Hold & Release, Swipe or Tap

View: Back, Isometric, Side, Top

Style: Abstract, Cartoon, Low Poly, Pixel or Realistic

Theme: City, Nature, Sports

This then feeds into the Market Intelligence tool to show the developer who is creating what type of games. It also powers our trending tool which can pinpoint the type of games that are proving most popular.

Suppose the game developer now knows what genre of game to work on. How do they know if it is going to be successful? The machine learning tool which powers Market Intelligence also looks at a host of datasets to predict what the game’s lifetime value (LTV) is likely to be.. We need to know how much money the developer will make in advertising and IAP from each player. In particular it looks at the behaviour of the users that have actually started playing the game in its early stage (including the time of day they play, how often and how long for). By scraping all the data about successful games and analysing them, we can then draw a picture of how much money the game is likely to generate from each user.

This is so useful for the developer as they can then go back to the drawing board and tweak the game, changing its format, parameters and more to optimise its chance of success. We then sometimes test the games again using this process and in the past have discovered that we can increase the LTV of the user which has ultimately made the games more successful and lucrative.

Coda is very proud of our prediction system as it has proved to be highly accurate in choosing which of the games on the platform is going to be most successful.

Optimising games in real time

Once a developer chooses a game and is confident that it has every chance of being a success they can then use our SDK which not only helps them create the game but saves them vast amounts of time and enables them to simply add layers and features which previously would have been complex to incorporate. The SDK is also smart. It can use machine learning to ensure that the game is as successful as possible by extending user engagement times and delivering subtle but effective monetisation options.

As the gamer starts to play so machine learning technology amasses information about the player and then in real time begins to optimise the game. So, for example, if they are a really good gamer it won’t necessarily show them games tutorials. It can also help with ad placement. If the player only typically plays the game for a few minutes at a time it will ensure that an ad is served quickly. If they generally tend to play for extended periods it might wait to show them an ad.

Ultimately Coda uses machine learning to keep the player engaged while making sure that ads are served at an appropriate time. Coda can also work with third parties, for example Facebook to optimise when the ads are shown, ensuring gamers are shown the right ads at their optimum time encouraging click throughs.

One thing that we are working on is identifying areas of the game that would be most worthwhile tweaking. For example, there may be particular levels that users don’t like or specific demographics that are hyper-engaged with it.

This is just the start of the process of optimising games using machine learning. There are many other options that we are currently considering and these will be rolled out to the platform in the coming months and years. 

We will be delving deeper into machine learning in the coming weeks at the Coda blog. Please let us know if there are specific machine learning related topics that you would like us to cover.


So your game bombed… What next?

The last few years have been really exciting ones for independent games makers. We have witnessed a wave of enterprising individuals deliver games that have captured the imagination of global audiences. Some games, like, Roller Splat and Archero and even Coda-backed examples like Rope Rescue, Balls Master and Juicy Stack have gone on to achieve massive downloads and revenue.

Anyone who has tried to get their mobile game published knows it can be a cut throat industry and only a few games will be a success. Most games won’t earn a cent, some games will make a small amount of money for their developers, others are more successful, some might go stratospheric.

What if the game you have produced is not a creative or financial success?

If that’s the case, it is time to have a re-think. We’ve noticed that most studios that approach us already have on average five games in their account and are still looking for a hit, so persistence is key. Analyse why your game didn’t meet your expectations and apply some of the lessons you learn next time around. 

There are several places you can go to get useful advice. GameAnalytics is a free service that will enable you to get a deeper insight into the performance of your game and the people who play it. There’s also  Deconstructor of Fun, a blog and a podcast that specialises in free-to-play games and is loaded with advice for games creators. Other useful podcasts include GameDev Loadout and Game Dev Advice

If you want more information now, here are some pointers on things to consider before you launch your next game.

1 Have a marketing strategy

This seems an odd place to start given that we are discussing creative failures, yet one of the reasons why games aren’t successful has little to do with the quality of the games, but rather the way they are marketed. 

Many game creators are brilliant at conjuring up innovative, addictive games. Yet when it comes to pushing them out they are often novices. The days when developers could launch a game out without a marketing strategy and become successful are long gone.

So here are a few tips from our team:

  • Don’t just assume you need to produce one ad. Mobile game marketing is all about the creative. Come up with a host of different versions and see what kind of CPIs you get. Change the words and the ‘call to action’ too so you know what works best.
  • When you produce your ads opt for videos as they generally work much better than images. See what rival companies, and indeed the big studios are doing. They have spent large amounts of money optimising their marketing, so it makes sense to take a lead from them in the way that they create video.
  • Wrap your head around Facebook ads. Of all the various platforms, Coda has found Facebook to be generally the most effective channel for CPI type campaigns. Spend time refining your target audience and check in on a daily basis to see what is working and what isn’t
  • Reach out to YouTube gamers and see if they will feature your game. There are many influencers you can choose from, though bear in mind that those with the largest audience will invariably want payment. The sweet spot is arguably mid sector influencers who have a growing audience but may still feature your game for free.
  • If you think your game is really good, try and get it featured on the app stores. This might sound like a lot of work, but it could have a transformative impact on the fortunes of your game.
  • Test Android vs iOS and see where you get the best CPIs and the best ARPUs. Coda generally tests and launches on iOS. The CPIs may be higher but generally we get better ARPU on iOS.
  • The US is not always the best market to launch in. It might be worth experimenting with other countries to begin with and then building up to target the US later. From a marketing perspective targeting US consumers is often more expensive than those in other territories. Also momentum is so important. If you can get a good start elsewhere you will have a lot of data about your game and how to market it effectively before you attempt to crack the US

You need a marketing burst to act as a catalyst so that enough people  know about your game, then if it is addictive as you hope it will be, the game will be successful. An effective user acquisition strategy is key to any successful game as is an understanding of how to optimise a game for the app stores (ASO).

If you’ve got a good game or two already under your belt but you’re finding all of this marketing a bit too overwhelming, Coda may be able to help. When you submit a game to us, we have a team of acquisition experts who can produce creatives and test your game for you.

2 Don’t skimp on market research

Even if you spend big on pushing your game out, there is a chance that it might be doomed from the start because there simply isn’t a market for it. 

There are many obvious reasons why your game didn’t find an audience, Some developers, for example, make the mistake of creating a game that is too close to one that they created before that was successful. Others might find that the niche in which their game is based simply isn’t popular enough.

That’s why we believe that market intelligence is so important when deciding what kind of game to create, being able to see what type of casual games are successful at the moment and which mechanics, elements and design styles are trending is essential for any game developer. 

To support game creators Coda has developed a Market Intelligence dashboard, which is a great way to see what is trending in the word of mobile games, particularly hyper casual games. It can provide insights that might make you rethink the type of game you are creating, or conversely validate your existing hunch. You can even see what games are being tested pre-launch  helping you spot any upcoming trends. 

3 Test and iterate

Another area in which games fall down is because they haven’t been properly tested. There are either glitches in the game or the user experience is a poor one. This then leads to gamer frustration, bad reviews and fewer downloads. 

Again this is something Coda seeks to address with our SDK. Whether you want to test your game designs or delve deeper into user journey our SDK can help. The key is to create a user experience that is seamless. Our always-on machine learning system can help, from analysing data from your in-game events to suggesting the best timing for interstitial ads.

4 Find ways to make your game more sticky

No matter how enticing your game is you need a strategy that ensures that players keep playing it on a regular basis. So many games fail because the game creator didn’t give stickiness enough attention.

One smart way of doing this is by creating an in-game currency, which enables players to buy more lives, skip levels, acquire new skins, upgrades and more. Currencies do require some general knowledge of gaming economics to make it work effectively. This can be really interesting research for new game developers and an interesting maths challenge but it can also be very daunting if you’ve never done it before.

For game creators that prefer to focus on designing great games, the Coda SDK is a great solution as it takes care of all currency and gaming economy considerations for you.. The  in-game currency module is so advanced it even allows developers to build a connection between in-game currency and shop currency almost immediately.

5 Focus on monetisation

Ok, so your game might be a creative success, but can you really call it a success if it doesn’t reward you for your endeavours?

One area that a lot of developers find difficult is monetisation. There are many different avenues to explore when it comes to making money from a game and some developers find dealing with multiple partners tedious and don’t pay revenue acquisition as much attention as they should.

For most casual games advertising is still the number one source of revenue, so games creators need to focus on building relationships with networks. There are however so many ad networks to choose from, so consider mediation. 

We can’t help talking about our SDK as it’s really useful for creators in this position but yes, our SDK can take care of all your monetisation issues in one simple integration.


Ultimately there are many reasons why games don’t succeed. Poor marketing is a major one, but choosing the wrong mechanic, not properly researching the market and not maximising revenue opportunities can all play a part. At Coda we encourage all game developers to explore these areas as best they can but if you ever find all of the added knowledge required to launch a successful game to be overwhelming, come and chat to us and let’s see if we can help you get your best games off the ground.


Will we see the Netflix or Spotify of mobile games?

Assemble a group of game developers and ask them about the future of gaming and it’s a solid bet that one of them will mention the impending arrival of the Netflix or Spotify of mobile games.

For several years now gaming pundits have predicted that a subscription-based service, where for a monthly fee gamers can gorge on whatever  mobile games are on the platform, is just months away. Yet still we wait.

Subscription based services are not new to gaming. Sony and Microsoft have offered a subscription service, where users have access to a wide spectrum of console games for a monthly fee, for several years. More recently, September 2019 saw the launch of Apple Arcade. Users currently pay $4.99 per month to access a growing number of games (around 100 at the time of writing) which can be played on Apple PC’s and TVs as well as iPhones and iPads.

Is the missing jigsaw piece in the gaming world an OS-agonistic, multi platform (mobile/PC/TV) service that allows its user to enjoy a broad range of ad-free mobile game titles  wherever they are for a small monthly fee?

The rise of subscription services

Subscription services have become a key way that consumers pay for their content. Netflix is the obvious example, yet Spotify started disrupting the music industry back in 2008 and now boasts the back catalogue of almost all the world’s biggest artists. In Scroll there is even a paid for service for websites, which charges a monthly fee to deliver up to 300 titles to devices and computers with the advertising stripped out. Add Hulu, BritBox, Disney and Amazon Prime to that list too and it is clear that subscriptions are king.

There are several reasons why a Netflix style service for gaming would be beneficial for both gamers and developers.

A subscription based service would enable developers to create games that worked seamlessly on a number of platforms. This could potentially mean that developers are able to reach a wider audience with their games.

There could be other benefits for games publishers. Game discovery wouldn’t have to rely as much on the app stores meaning a broader range of titles will get played. This would mean that some excellent games which maybe would have remained below the radar on the app store might acquire significant audiences. 

In theory a subscription service would allow more developers to make more money from more of their games, which in turn would mean less risk in developing games. With an increased likelihood of games making revenue, this reduces the hit or miss risk of getting a game published. Reduced risk for game developers might mean they are able to be more innovative and experimental with the games they produce, improving the diversity and creativity of game types and game play. Another positive of the reduced risk could be that investors are more willing to back games as they see a clear path to returns.

The way that the platform cross-promoted games might also prove beneficial to developers. Both Netflix and Spotify are built around very smart machine learning algorithms which constantly suggest content that it thinks users will like, this could help more games from indie developers to be discovered.

So for the gamer a platform like this would present a fantastic choice, and for the games developer it has some serious upsides too.

So why has this not happened yet?

The need for deep pockets

The most obvious reason is that building out this type of platform would be hugely expensive and beyond the reach of independent gaming studios and even the larger hits-focused mobile game publishers . Creating the platform would require a company with very deep pockets, but the hardest and most expensive part might be acquiring and keeping users. The platform would have to offer both the latest, most high profile games as well as a long tail of smaller, older games. Enticing all those developers onto the platform would be a very tricky job. It could take years. The Beatles held out until 2015 before licensing their music to Spotify, if several of the big games studios adopted a similar stance it could seriously impact on the popularity of the nascent platform.

Also one of the ways that Netflix has kept ahead of its rivals is by constantly adding new and popular content that’s not available elsewhere. The deals it hatches with content studios are often hugely expensive, and this could be replicated in a subscription-based gaming platform.

When you look at the large players that could pull something like this off, there doesn’t seem to be any one player who is best suited to launch a platform-agnostic mobile games subscription service.It could be an interesting brand extension for Netflix, but the company is clearly generating fantastic revenues in its own TV/ film core competency which it relies heavily on debt instruments to produce so getting distracted from it’s core competency to produce a gaming platform would be way too risky.

Facebook could in theory unite the industry behind such a  platform. Yet the company is facing impending antitrust examinations which some pundits have predicted will lead to its breakup as governments seek to dilute its power and ubiquity. These days, Facebook is also light on innovation, and the trust of the public required to launch and test innovative things. Their last big attempt at something new, libra, seems doomed to never get off the ground, mainly because nobody trusts Facebook to control a global currency.

Everyone will ask but what about Google and Apple? For the time being both seem perfectly content with the games ecosystems they have generated. Apple Arcade in particular has strengthened the company’s position in gaming as it demands that its games are unique to its platform. Google is fumbling along with Stadia which nobody is quite 

Blockbuster was huge once

It’s sometimes hard to remember that Netflix was once a scrappy startup and that people went to stores to pick up these disc things to watch movies on. Blockbuster was huge and unrivalled at the time and had the deep pockets we mentioned above, yet they failed to respond to the Netflix threat, a strategy turned a giant brand into a timeless warning about innovation and embedded culture. In the most pivotal moment in Blockbuster’s demise, the CEO’s plans to launch a Netflix-rivalling digital service were halted in favour of sticking to what they knew best and continuing to monetise off of penalising users rather than delighting them. This was their corporate culture, they didn’t have the willingness, knowledge, talent and culture to face this new threat. 

There are many incumbents in the mobile game industry who technically could launch a subscription service but their embedded monetization model, much like Blockbuster, comes at the expense of the user experience and lack the willingness, culture and relevant talent to pull it off properly. Perhaps the industry is waiting for it’s Netflix and the disruption will come from left field as it did with Steam for PC games and Fortnite for multi-platform games.

We also need to talk about developer revenue

Some game developers are fearful of a platform they believe will undermine their revenues. In the short term a platform could be good news for the big games studios, as it seeks to tie up lucrative, unique deals. Yet if the Netflix and Spotify business models are any example, as time goes by the fees paid to developers starts to drop as the platform establishes itself. 

A subscription service could mean the end of paid and IAP revenue for individual games. Developers can cite many musicians who were making a decent living through selling physical products like CDs and even downloads, but have since been poorly served by Spotify’s revenue distribution.

They would lose advertising revenue too, and it is questionable whether the royalties they receive from the platform would ever be as much as they currently get from in-game ads. 

“For some small studios or some individuals, there are likely to be benefits from a ‘Netflix for games’ type platform emerging,” argues Andrew French, COO of Coda Platform. “Yet disrupting the existing business models would be very complex, and probably not good financially for developers in the long run.”

“The distribution issue is already solved in games through Apple and Google Play. And ultimately, while there’s an awareness issue that can be solved through user acquisition, the best games and the best content will always do well,” he adds.

With game studios and smaller developers not entirely on board, creating a subscription based gaming platform might prove to be an uphill struggle for whoever was adventurous enough to attempt to pull it off. Still it would be foolish to rule out it ever happening, but for the foreseeable future the odds are stacked against it.