Sony Patents Learning Unfamiliar In-Game Skills Through Gestures

Rundown:

• Sony Interactive Entertainment has recently published a patent for using gestures to search for in-game skills in a virtual environment.
• The system involves capturing data about the player’s real-world actions, identifying their progress level in the virtual environment, and matching the captured data to available in-game actions.
• The system can filter the available in-game actions based on conditions associated with a search trigger or the player’s progress level.
• The stored map of in-game actions and associated data can be specific to various aspects of the virtual environment, such as video game, genre, console, in-game environment, or activity.
• The patent aims to enhance the player’s virtual experience by providing personalised and intuitive access to in-game actions based on their real-world actions and progress level.

Sony Interactive Entertainment has recently received a patent titled “Gesture-Based Skill Search,” filed in Augsut 2021 under the name of Sony Interactive Entertainment LLC. The patent, published earlier this month, describes a system for searching for in-game skills using gestures to help players find guidance when introduced to unfamiliar video games. The system involves storing a map of virtual actions associated with real-world actions, capturing data of a player’s actions, identifying the player’s progress level in the video game, and matching the captured data to an available virtual action. Once the virtual action is identified, the system searches for instructions on how to perform it.

“A map regarding one or more virtual actions may be stored in memory. The virtual actions may be associated with a set of data associated with performing a corresponding action in a real-world environment. Data regarding a action by a user in the real-world environment may be captured. A current progress level of the user within the virtual environment is identified. The current identified progress level is associated with one or more available virtual actions,” reads the abstract for the patent. “The captured data may correspond to an identified one of the available virtual actions based on a match between the captured data and the set of data associated with the identified virtual action as indicated by the map. A search of one or more databases for instructions corresponding to the identified virtual action is initiated.”

Since video games, particularly multiplayer ones, constantly evolve, many players may struggle to keep up with the changing in-game actions and controls that could mean victory or defeat. For instance, in a video game like Fortnite, players always find new and improved ways to gain an edge over other players, but many of these ways are intricate to perform, particularly for more casual players. Hence, it seems like Sony Interactive Entertainment aspires to alleviate this issue with an automated system that could supposedly guide players on performing certain in-game actions and improving their skills in particular video games using gestures. The patent aims to enhance the gaming experience and increase player engagement and satisfaction by solving this problem.

According to the patent’s claims, the system described is a method for using gestures to search for skills in a virtual environment. Simply put, the method involves keeping track of all the different things you can do in the virtual world and watching what hand gestures you make in the real world. Then, it determines what you’re trying to do in the virtual world based on your gestures and tells you what to do next. “The method of claim 1, wherein the stored map further stores data regarding the instructions corresponding to the identified virtual action,” it elaborates. Hence, the instructions regarding a particular in-game action are generated based on the information stored in the map. Furthermore, the data about the player’s actions may be captured in the real world through various sensors like cameras, accelerometers, gyroscopes, or inertial sensors.

Addtionally, the patent mentions that the identification of the virtual action matching the captured data is “based on a difference between the captured data and the set of data associated with the identified virtual action being within a predetermined threshold,” so the identification depends entirely on the data associated with in-game actions, which means that the system may not be able to correctly identify actions that are not widely recognised or are more obscure. Furthermore, the system works by filtering the in-game actions available to the player based on certain conditions associated with a search trigger or the player’s progress level (associated with activity files). This means that when the player searches for a particular skill using gestures, the method will only display in-game actions relevant to the player’s progress level or meet the specific search criteria. This makes the search process more efficient and effective for the player.

The patent also states, “the stored map is specific to at least one of a game title, set of game titles, game genre, game developer, game console, game controller, set of controller modifications, game environment, and in-game activity,” which means that the system can be personalised to different aspects of the virtual environment, which would allow for a more tailored experience for each player. By storing maps specific to various aspects of the virtual environment, the system can provide more accurate matches between the player’s real-world actions and the available virtual actions, thereby improving the overall experience. This personalisation could enhance the player’s engagement with the virtual environment, making it more enjoyable and immersive. Lastly, the stored map can be one of many maps, and the appropriate map is identified based on the player’s virtual environment.

The system is designed to improve the learning experience for players, making it easier and more intuitive to learn new in-game skills and master existing ones. The system displays instructions corresponding to the identified virtual action, allowing players to quickly and easily learn new skills through a natural and intuitive process. This patent could significantly impact the video game industry, as it could potentially revolutionise the way players learn and master skills. The system could also have applications beyond gaming, potentially used in education and training. However, it’s important to note that this is just a patent for now and whether this system will be implemented by Sony Interactive Entertainment in the future only time will.

What do you think about this? Do tell us your opinions in the comments below!

Similar Reads: Sony Patents Foot-Operated Touchpad Controller For VR Games

Huzaifa Khan

Started out penning short stories in my room, then stumbled upon my real passion for video game and technology journalism. Now, I’m all about playing and writing about them. With experience from VeryAliGaming, eXputer.com, and TheNerdMag, I’m currently dishing out news on Gamesual from my place.