A perspective of Lidwine Sauer, Digital Trends and Insight at Ubisoft

In the week that the European Commission publishes its White Paper on AI, learn more about the exciting area of R&D in the video games industry.  This sector is the largest and fastest growing segment of the creative sector with an economy greater than music and film combined.  Most video game studios have their own R&D facilities in Europe working on blockchain, AI, virtual reality and augmented reality, developing many cross-over applications that are beneficial to wider society.  In this interview with Lidwine Sauer, Director of Trends and Insight at Ubisoft Innovation Lab, we gain some insight into innovation in the video games industry and its role in pushing the boundaries of technological advances.

Ubisoft is a leading creator, publisher and distributor of interactive entertainment and services with a rich portfolio of world-renowned brands including Assassin’s Creed, Just Dance, Tom Clancy’s video game series, Rayman, Far Cry and Watch Dogs. The teams throughout Ubisoft’s worldwide network of studios and business offices are committed to delivering original and memorable video gaming experiences across all popular platforms, including consoles, mobile phones, tablets and PCs.

“Programming interesting AIs has always been part of the fun and challenge of developing video games”

Tell us about your job and how you ended up in the video games industry?

Lidwine Sauer: I started working in the video game industry almost immediately after school. While I saw my friends join the ranks of big established firms and banks, I wanted my professional life to be a joyful adventure – so I chose Ubisoft. I wasn’t disappointed: I am still having a blast twenty years later.

My first job was product manager on “edutainment” games that helped kids learn while having fun. After various marketing positions, I took on change management and innovation projects to support the transformation of the company into a global entertainment leader.

Today I am Director of Insights and Trends for Ubisoft’s Strategic Innovation Lab. I strive to map out the technology, social and business trends that could impact the entertainment industry

What makes the Ubisoft Innovation Lab unique? 

(L.S.): The Strategic Innovation Lab’s mission is to help Ubisoft anticipate the future. It is a unique cross over between a think tank and a do tank. We analyse technological, social and business trends to identify opportunities for innovation; we then explore and probe these opportunities through prototypes and different types of projects with internal and external partners. Exploring, experimenting, learning, and preparing Ubisoft teams for change is at the core of what we do.

AI is one of the technologies that the EU and national governments are looking at from a strategic perspective in terms of how they can best support and transform our society. How is the video games industry working with AI and how does it benefit other areas of society?

(L.S.): There are basically two types of video games: the ones where you play against other human players, and those where you play against the game itself. In the latter, your opponents are AIs programmed to behave ‘intelligently’, create a sense of realism, and make the game immersive and fun to play. Programming well-balanced AIs has always been part of the challenge of developing games.

With the coming of age of machine learning and deep learning, new opportunities for innovation have opened up. One example is to use AI to improve game development processes while at the same time improving the gaming experiences we create.

To make our games accessible to as many players as possible, we release them in many different countries, which often implies that we localize the language spoken by in-game characters. One of the projects that our R&D team ‘Ubisoft La Forge’ has worked on, is an AI-powered solution called ‘Sound Matching’: it uses AI to animate the lips of in-game characters and create facial animations, based on the audio files directly from the recorded actors. This means that their lips automatically move according to the language they speak. What this results in is an increased quality of the experience and an increased sense of immersion for the player, but it also helps saving a significant amount of time and makes the whole localization process easier.

This Sound Matching AI was first used in our games Assassin’s Creed Odyssey and For Honor, but what is really interesting is that the work done with this project can also prove useful in fields beyond gaming. Researchers and doctors have found that they share the same challenges in medicine where, for instance, some treatments for anxiety require patients to have discussions with virtual characters. They told us that our technology could help create more immersive medical simulations.

The video games industry has been involved in AI for some years now.  What kind of research is taking place in this area today? Does the video games sector participate in any cross-sectoral R&D projects?

(L.S.): There are various ways we work with external partners to advance research in AI.

Ubisoft has developed a number of partnerships with the academic world. The objective of these research programs is to apply fundamental academic research to real industry problems, such as the lip-sync challenge mentioned earlier.

Academic research also benefits from the fact Ubisoft can share unique sets of data to train the AI algorithms. For an AI-coding assistant called ‘Commit Assistant’, Ubisoft shared 10 years’ worth of programming code with researchers. The result is a program that helps programmers spot bugs before the code goes into the testing phase. Academic papers were published to share the research results and encourage other teams to try for themselves. Now not only is it implemented in some of our production pipelines, where it catches up to 70% of bugs before they are implemented, but it has raised the bar for the entire software industry. Last year, we announced a partnership with Mozilla to bring the development of Commit-Assistant even further.

Another way we contribute to the advancement of AI is by supporting start-ups in the AI field. We invite them to join our Ubisoft Entrepreneurs Lab program, hosted in Paris at STATION F, the biggest startup campus in the world, and Singapore, at Infocomm Media Development Authority’s innovation space, PIXEL. As part of the program, we offer them a unique collaboration model based on mutual contribution with no equity involved, where startups benefit from the mentorship of Ubisoft executives and gain access to a wide range of experts across the entire company.

How do you see AI transform the video games sector and other entertainment sectors in the years to come?

(L.S.): The latest developments in AI are going to help video game publishers not only to improve the efficiency of game development processes, but they will also allow us develop better, richer and new types of entertainment experiences.

Task automation is often highlighted, since it allows teams to improve in various fields, but AI will allow creative people to extend their own field of exploration, finding new inspiration and assisting them with creation tasks so they can focus on higher added value activities.

Improving the efficiency of game development will allow our creative teams to focus on creativity, allowing them to build what is not even imaginable today.

If you follow the development of video games in the next years, be prepared to be surprised!

Interview conducted during February 2020 by Ann Becker, Video Games Europe