Researchers in South Korea are developing a model for an AI-powered in-game esports observer

Image via: Shutterstock

Researchers at South Korea’s Gwangju Institute of Science and Technology (GIST) have developed a framework for an AI-powered in-game observer tailored for esports tournaments.

The framework, proposed in an article in Expert Systems with Applications journal, uses an object detection method and human observation data to determine the area of ​​most interest to viewers.

Read More IJESPORTS Launches Esports-focused Academic Research Journal What Science Says About Esports Advertising

In-game observers are tasked with deciding which players fans should watch and what camera angles to use during esports matches, relying on extensive game knowledge to follow key gameplay.

Although automatic in-game observers are included in many games, they usually rely on predefined rules and events. This means, according to the researchers, that the feature lacks the ability to independently assess the importance of in-game actions

Researchers at GIST led by Associate Professor Dr. However, Kyung-Jong Kim have proposed an approach that they say can overcome these limitations.

The innovation of the framework lies in treating the area viewed by viewers as an object rather than in-game objects themselves such as characters or buildings.

In their study, the researchers collected in-game human observation data in StarCraft from 25 participants. The areas viewed by viewers were identified and marked as “Ones”, while the rest of the screen was filled with “Zeros”.

This data was fed into a neural network to detect patterns between areas to find the “Region of Common Interest” (ROCI) – essentially the most exciting area for viewers to see.

The researchers then compared the result to other existing methods and found that the network’s predicted viewpoints were similar to the human observational data. The paper claims that the model outperforms other models over the long term in general tests across various matchups and StarCraft cards in the game.

A video released by the researchers compares their model to a rule-based alternative

More importantly, the authors claim that the proposed framework “can be applied to various games that can preserve game state, such as a mini-map, and the area viewed by a spectator is part of the game state”. League of Legends and Dota 2 were mentioned as examples.

The research could prove valuable and provide a cheaper option for smaller tournament organizers who lack the budget to pay for a dedicated observer.

Should the model prove successful in practice, it could also potentially offer improvements over human observers, who may miss important events occurring simultaneously on multiple screens, GIST said.

Principal Investigator and Associate GIST Professor Dr. Kyung-Jong Kim said in a press release: “We have created an automatic observer that uses [an] Object detection algorithm, Mask R-CNN to learn human viewer data.

“The framework can be applied to other games that represent part of the overall game state, not just StarCraft. As services like multi-screen broadcasting continue to grow in esports, the proposed automatic observer will play a part in these outcomes. It will also be actively used in more content to be developed in the future.”

In addition to Kyung-Joong Kim, the paper was co-authored by Ho-Taek Joo, Sung-Ha Lee, and Cheong-mok Bae. Gwangju Institute of Science and Technology is a research-oriented university in Gwangju, a city in southwest South Korea.

Jake Northland

email icon

Jake is a features and trending news editor for Esports Insider. He has been part of the ESI team since early 2021 and is interested in politics, education and sustainability in sport.