PlayStation VR Facial Tracking Patent Released
Patents for a few of Sony’s new future inventions have been released one in particular grabbed attention and it was the PlayStation VR facial tracking patent, it is thought tracking facial features will provide a more realistic experience for it’s players.
The tracking will include eyes and mouth movements and in the document below it shows an avatar on-screen mirroring the movements.
A method for rendering a virtual reality (VR) scene viewable via a head-mounted display (HMD) is provided. The method includes detecting eye gaze of a user using one or more eye gaze sensors disposed in a display housing of the HMD. And, capturing images of a mouth of the user using one or more cameras disposed on the HMD, wherein the images of the mouth include movements of the mouth. Then, the method includes generating a virtual face of the user. The virtual face includes virtual eye movement obtained from the eye gaze of the user and virtual mouth movement obtained from said captured images of the mouth. The method includes presenting an avatar of the user in the VR scene with the virtual face. The avatar of the user is viewable by another user having access to view the VR scene from a perspective that enables viewing of the avatar having the virtual face of the user. Facial expressions and movements of the mouth of the user wearing the HMD are viewable by said other user, and the virtual face of the user is presented without the HMD.
The following implementations of the present disclosure provide methods, systems, computer-readable media, and cloud systems, for rendering virtual reality (VR) views into VR scenes for presentation to a head-mounted display (HMD). One method includes sensing the position of a nose of the user when the HMD is worn by the user. The method includes identifying a model of the nose of the user based on the position that is sensed. The model of the nose having a dimension that is based on the position of the nose of the user when the HMD is worn. The method further includes rendering images to a screen of the HMD to present the VR scenes. The images being augmented to include nose image data from the model of the nose. In one example, the HMD is configured to capture facial feature expressions which are used to generate avatar faces of the user, and convey facial expressions and/or emotion.
The embodiment’s described herein, therefore, enable systems of the HMD to track features of the user space, while wearing the HMD. The features can include identifying characteristics of the user’s nose and using that information to render at least a portion of the user’s nose in images presented to the screens of the HMD for viewing the virtual-reality content. By presenting images of the user’s nose in the images of the scenes being viewed, the images or partial images of the user’s nose will resemble the views that a user typically sees when not wearing an HMD. As such, the user’s brain viewing images will be accustomed to seeing at least part of the user’s own nose in the scenes, and the user’s brain will automatically filter out the presence of the nose.
However, by providing the user’s nose or partial views of the user’s nose in the scenes, it is possible for the user to feel a more realistic experience when viewing the virtual-reality content. If the user’s nose is not provided, at least partially in the images, the user’s appearance or perception of the virtual-reality scenes will be less than natural. As such, by providing this integration into the images, in locations where the user is expecting to see at least part of the user’s nose, the views into the virtual-reality content will be more natural, and as expected by the user’s natural brain reactions to filter out the nose. Embodiment’s described herein present methods for identifying the nose, characterising the nose physical characteristics, and then generating models of the nose for presentation at least partially in the scenes.
Below are some more images this time showing how the VR headset will work in-game with the center point being your nose. The headset looks to be designed slightly different from the original PSVR headset and we can only imagine this is what the headset will look like for the PS5.
Please don’t forget to like our Facebook page and follow us on Twitter it is really appreciated