Sony Working On Communication Between Players Using Gestures

Rundown:

• Sony Interactive Entertainment recently published a patent that discusses a way for “facilitating secret communication between players” using “gestures” in multiplayer video games.
• The company desires to use gesture recognition and machine learning to predict what players may want to communicate to other players through gestures and send the appropriate response to the desired players accordingly.
• According to the claims of the patent, the technology will function through a camera, such as a webcam, that will capture the gestures made by a player through more different body parts and send that as image data to a machine learning algorithm that will identify the movements made.
• The recommendation(s) generated by the machine learning algorithm may include haptic feedback data or message data for a hand-held controller or audio data for a headset connected with the player whom the communication is intended for.
• The company desires to implement this video game technology primarily in e-sports, where players are unable to communicate openly with each other for fear of being overheard by their adversaries.

Sony Interactive Entertainment is one of the most widely recognised video game and digital entertainment companies in the world, with some of the most critically and commercially adored video game franchises under its name, like God of War and Marvel’s Spider-Man. Being one of, if not the most, successful video game companies in the world, the company is constantly trying to improve its video game development technologies to enhance the playability and accessibility of its video game franchises.

Only recently, Sony Interactive Entertainment published a patent for a video game development technology that uses “natural language input” for creating and modifying in-game assets through machine learning. Hence, it’s clear that the company desires to make video game development not only more convenient for video game developers but also more accessible for players by significantly reducing the time it takes to develop such video games. As of recent, it seems like the company may be working on another revolutionary video game technology that uses gesture recognition for players to secretly communicate with each other through “gestures” in multiplayer video games.

While communication in multiplayer video games, particularly competitive ones, only allows for players in the same team to directly communicate with each other through voice chat and/or messages, it becomes somewhat of a concern when the adversaries are in the same vicinity as the players themselves, like in an e-sports event. As such, players may be able to overhear the communication between their adversaries, and this can give them a significant advantage over the other team. However, Sony Interactive Entertainment may have found a solution to this, and one that the company claims to be almost completely accurate.

Earlier today, we came across a recently published patent from Sony Interactive Entertainment under the name of Sony Interactive Entertainment Inc. titled “SYSTEMS AND METHODS FOR FACILITATING SECRET COMMUNICATION BETWEEN PLAYERS DURING GAME PLAY,” which was filed in July and published only a couple of days go. The abstract for the patent reads, “Systems and methods for facilitating secret communication between players are described. One of the methods includes receiving image data of a first player to identify one or more gestures and receiving input data indicating a response of a second player to the one or more gestures. The method further includes training a model using the image data and the input data to generate an inference of communication between the first user and the second user.”

Hence, the company desires to use gesture recognition and machine learning to predict what players may want to communicate to other players through gestures and send the appropriate response to the desired players accordingly. “The method also includes generating additional inferences of communication between the first user and the second user. The method includes generating a recommendation to the second user based on the inference of communication and the additional inferences of communication,” it further reads. As such, the generated response will only be a recommendation that may or may not be completely accurate since artificial intelligence in itself isn’t completely accurate.

According to the claims of the patent, the technology will function through a camera, such as a webcam, that will capture the gestures made by a player through more different body parts and send that as image data to a machine learning algorithm that will identify the movements made. “The method of claim 1, wherein the client device is a first client device, wherein the image data is received from a second client device, wherein the first client device is coupled to the second client device via a computer network,” it explains. Furthermore, the machine learning algorithm may distinguish between the body parts using pre-determined characteristics, such as the shape and size, of pre-stored data based on different body parts.

Additionally, the machine learning algorithm may also contain a pre-determined amount that the gesture(s) may need to exceed for the movement to be accurately identified and processed. Furthermore, there may also be a certain time associated with the gesture(s) that may determine whether the movement has been executed for communication with another player or normally. As a result of the processed gesture recognition data, the player for whom the communication is intended may receive the recommendation made by the machine learning algorithm through one of their connected peripherals, like a headset or hand-held controller. As such, the recommendation may include haptic feedback data or message data for the hand-held controller or audio data for the headset.

In addition to this, the patent also discusses a server system for “facilitating secret communication between players” and connecting the players with whom the communication is being achieved through a computer network. The purpose of the server system is to “receive image data regarding a first player during a play of a game,” and “identify, from the image data, one or more gestures made by the first player, wherein the one or more gestures indicate a manner of playing the game to a second player,” and “provide the recommendation to a client device operated by the second player to inform the second player to engage in the gameplay.”

While “secret communication” isn’t a necessity in online multiplayer video games where players aren’t present in the same vicinity as the adversaries, Sony Interactive Entertainment, it seems, desires to implement this video game technology primarily in e-sports, where players are unable to communicate openly with each other for fear of being overheard by their adversaries. This is further corroborated by one of the figures mentioned in the patent that clearly illustrates an e-sports competition where players from either side team are sitting close to each other. In such a situation, the claimed “secret communication” becomes necessary.

While it’s uncertain whether what Sony Interactive Entertainment aspires to achieve with the claims made in the patent will actually come to fruition or not, as not all patents are eventually implemented, the company does see a need for such a video game technology, particularly since it has started investing into e-sports, with the recent acquisition of Repeat, an online tournament platform. Exactly how this video game technology is implemented, if at all, only time will, but it does seem rather promising.

What do you think about this? Do tell us your opinions in the comments below!

Similar Reads: Sony May Start Tracking History Of In-Game Digital Assets

Jace

Jace loves both video games and writing, so he thought the best career was to write about video games. He attended Full Sail University and graduated with a Bachelor’s in Creative Writing. He joined Gamsual in 2022, and when not covering video game news, he is often building and strategizing in simulation and RTS games. Xbox: Alchemist5966