NC

I studied computer graphics. I liked painting since I was little but had no talent. I thought about what I had to do, since I liked to paint but had no talent. So I decided to study computer graphics. Computer graphics are divided into three major areas, including modeling, rendering, and animation, and I was one of the members that studied animation when the field just started to burgeon.

At first, I thought about ways to create new things based on motion-capture data and how human beings walking with two feet would not fall down and walk while maintaining balance. After then, I studied deeply about human models and started to dig deeper into their anatomical structure. I studied the signal that moved the muscles and bones starting from the brain and rendered complex movements on computer to walk, run, and move. I have carried out diverse research for the last 30 years, and their essence lies in the theme of human movement.

I once studied the relationship between the anatomical structure of a human being and their gait as part of a research project. Through an anatomical modeling of a human being and analyzing the length and the angle of each bone and the length and the available power of each muscle, we could represent them in a very lengthy vector space with parameters with thousands of dimensions.

In this case, deep learning learns about the mapping between the aforementioned dimensions and a person’s gait. Through repeating an endlessly large number of learnings, we could create a large-sized network that was trained people movements. Using this pre-trained network, people that are not machine learning professionals, including doctors and people in the field of rehabilitation medicine and bio-mechanics, may realize limitless potentials.

Simply put, we simulated the movement of muscles in each human movement in a virtual environment. The muscles that people use when they walk, run, or lift a thing would vary, and we could visually show how each muscle would move in what strength in what movement. It was important to handle the immense amount of computation, and we could solve the issue using deep learning.

The project concerned a difficult issue that had not been resolved for 20 years despite the efforts of many people. When I said that I would start the project, everyone around me tried to talk me out of it. However, I started the project, and ‘courage’ was one of the most important reason that prompted me to make such a challenge. Of course, I had an insight telling me that I had a decent chance to succeed in the project. Even though I feared that the result might not be so successful despite the fact that the project would require my concentration and efforts for at least ten years and wondered what the project would eventually offer me, I had the courage to start. Working as a researcher, I could feel that even though insights and knowledges would matter, the courage that would help you not give up despite failures and to make endless attempts was even more essential.

I just mentioned that computer graphics could be largely divided into three areas, and the quality of modeling and rendering in games has constantly improved, thanks to the new technologies that have been consistently introduced for the last few decades. Meanwhile, technology of the field of animation has not seen much improvement in the recent 20 years. Animation enables movement of each character and is directly linked to the main logic. Hence, it is impossible to use the backup data after making an attempt to change. Therefore, it was difficult to try new attempts for technological development.

However, the AI technology including deep learning has developed, and the animation technology applied to games is faced with a major change. At the moment, animators connect individual motion-capture data to render movements. However, we will be able to improve efficiency in the entire process where data are collected and rendered through computer using the machine learning technology. The innovation is imminent. It matters who would take challenges first at this turning point, and since I was appointed the CRO of NCSOFT, NC would become a pioneer to take lead in such a change.

Since physics simulation of main characters would become possible, we will see new visuals that previously did not exist. Physics simulation is to render animation based on physics rules. It is to apply various physics rules that we experience in reality to virtual environment. Until now, physics simulation in games were mainly used to simulate the surrounding objects, including collapsing walls and rolling stuff, and were not utilized in main characters. However, since the deep learning technology is evolving, we will be able to see game characters and their movements completely based on physics simulation in the near future.

There are many people that are surprised by my choice to move to NC. It is probably because they believe that the research that I have carried out is irrelevant to games, but the game industry did not seem so new to me, since I believe that the two, my previous research and the game industry, are very much related. I thought that I had tried everything that I could do in academia as a researcher and wanted to try new things in the future, and therefore, it was not so difficult for me to make a decision to join NC.

I have studied computer graphics for the last 29 years, and the dream and question that I have had in mind are related to how human beings could be expressed and reproduced using computers. AI Center of NC shared the same dream and goal with me as a researcher, and therefore, I could resonate with the vision and make the decision.

The scope of work that I could do has been expanded compared to the time when I served as a college professor. In university labs, people sharing similar academic background would gather to carry out research. In NC, AI Center is carrying out research in various fields, including game AI, speech, and vision AI and is composed of diverse members, including the people that carry out research and the ones that convert research results into actual outputs. In college, I generated ideas and proved their possibilities. However, in NC, I would strive to produce a single completed output like ‘digital humans’ through combining capabilities of multiple persons and countless ideas.

Games are realistic reproduction of reality in virtual environment to which our creative imaginations are added. For the last 20 years, the most important issue in games was to model the actual world in a way that is as realistic as possible, using reproduction technologies like capturing and scanning. However, I believe that in the next 20 years, it will be more important to model the ‘interaction’ between the actual world reproduced in virtual environment and the world of imagination that would be added to it.

Objects will stop existing only on the screens; they will become an object that we can talk to and have emotional and physical interaction with. Also, combination between reality and imagination would become important. Interaction make take place between virtual beings or between actual and virtual beings. The essence of interaction lies in the natural response to ‘uncertainties’. A natural response to countless, unexpected variables is the definition of interaction that NC is looking for. Materialization of such an interaction itself would be considered an exciting achievement.

For instance, ‘digital humans’ are considered as the crème de la crème of such interactions. In essence, I believe that digital humans should not only exist as a being that resembles the exterior of human beings appearing on the computer screen, but also be able to communicate with ‘me’, read and react to my facial expressions, remember me, and interact with me.

In creation of digital humans, the systematic way of thinking to organically connect diverse technologies would be considered the most important, instead of simply focusing on one or two technologies. The process to combine diverse technologies like deep learning, physics simulation, computer vision, voice synthesis, voice recognition, and chatbot and to establish a single system would pose technological challenges. In the recent few years, the components required to create digital humans have developed dramatically, thanks to the impact from the deep learning technology. I believe that we have reached the time when we could create digital humans that could communicate and exchange emotion with ‘me’.

Instead of discussing the images of digital humans to be created by the Company, I would like to say that the digital human technology itself has significant implication inside NC. For instance, in case actors appearing in a drama have great capabilities, the quality of the entire show would improve and the expenses and efforts required for production would be reduced. The same logic applies to the relationship between a game and its characters. In other words, creation of sophisticated digital humans would improve the quality of all services provided by NC and reduce expenses and efforts. The digital human technology would provide a future vision for NC and offer an important technological foundation.

I used to enjoy playing games since I was little; I first started using computers, thanks to games. The first class I taught after being appointed as a professor in Seoul National University was related to computer games. Since I study computer graphics, I have paid much attention to game content and made countless attempts to apply my research to games.

Games are the content that would never disappear in the future; I believe they are indispensable in our community. The potential of game content is limitless. As the CRO of NC, I believe that it is AI Center’s role to constantly think how much we could expand the scope of game content and to push its boundaries.

Sometimes, R&D teams seem to be the place to try weird things. However, ultimately, they give inspiration to other teams. Improvement of efficiency in game development processes is not its main goal. I believe that it is R&D organization’s role to realize the things that were considered impossible during the game development processes, to constantly show new visions, and to improve the level of imagination of development functions. There are not many game companies that maintain the R&D organization in a proper manner. I believe NC is the only local game company that does so, and it was one of the reason that I joined NC.

To take a great leap forward, there is no other option but to repeat countless trials and errors. In that regard, the most important thing required to the people in R&D organization is patience. I believe that it is the CRO’s duty to encourage their attempts and to tell them to keep doing what they are doing because it is okay if something unexpected happens. I try to cheer on my team to make them relax and have courage so that they could see longer and farther.

One of the limitations I felt while working as a professor in Seoul National University was that new ideas and research would not go much farther than publications. Since I could make challenges beyond the stage of publication here, I have high hopes for the time that I would spend in NC. I want to go all the way from the stage to address the problems in reality to the one to generate values.