Sony's New AI Patent: Predicting Button Presses with Finger-Tracking Camera

Sony has filed a new patent, WO2025010132, titled “TIMED INPUT/ACTION RELEASE,” which outlines a potential solution to reduce latency in future gaming hardware. This innovative approach utilizes an AI model alongside additional sensors to predict and streamline user commands, potentially revolutionizing the gaming experience on PlayStation consoles.

The introduction of PlayStation Spectral Super Resolution (PSSR) with the PlayStation 5 Pro has allowed for upscaling to 4K resolutions. However, newer graphics technologies like frame generation can introduce additional latency, making games feel less responsive despite the increased frame rate. This issue is not unique to Sony; GPU manufacturers like AMD and Nvidia have introduced Radeon Anti-Lag and Nvidia Reflex, respectively, to address similar concerns.

Sony's patent aims to mitigate this latency by predicting user inputs. The patent describes a system where a machine-learning AI model works in tandem with external sensors, such as a camera focused on the controller, to anticipate the next button press. Sony's rationale emphasizes the importance of reducing the delay between a user's action and the system's execution of that command, which can lead to unintended consequences in gameplay.

The method outlined in the patent includes using camera input as an input to the ML model to predict the first user command. Additionally, the patent suggests that the sensor could be integrated into the controller itself, hinting at potential advancements in next-generation controllers, especially considering Sony's history with analog buttons.

While it's uncertain whether this exact technology will be implemented in the PlayStation 6, the patent clearly indicates Sony's commitment to reducing latency without compromising game responsiveness. This is particularly relevant given the widespread adoption of rendering technologies like FSR 3 and DLSS 3, which can introduce additional frame latency. Such advancements could significantly benefit genres like twitch shooters, which demand both high framerates and minimal latency. Whether this patent will translate into tangible improvements in future hardware remains to be seen, but it certainly points to an exciting direction for gaming technology.