There are already numerous platforms allowing anyone with a smartphone to create a virtual avatar and become a Virtual Youtuber. However, while some apps allow users to make gestures or poses, these are all preset and required the user to tap or swipe the screen while looking at the camera, which can be cumbersome and does not allow for natural movement.

Until now, full body motion capture has only been possible through specialized PC-based software applications, typically requiring multiple cameras and additional hardware attached to the person performing the motions.

Single 3D: Full body motion capture with only one camera

Now, all that is about to change. Fukuoka-based NEXT-SYSTEM Co., Ltd. has developed something called "Single 3D," as a part of their VisionPose motion capturing series. With no additional equipment to attach to your body, and only one camera, Single 3D can allow you to capture your face and body's motions which can be instantly reflected in a 3D model for broadcasting on YouTube or other video platforms.

App coming in June 2019

Even more exciting for those who have already taken the plunge into virtual broadcasting or those who are considering to, NEXT-SYSTEM will be releasing a Single 3D app in June, 2019 (iOS for now), effectively allowing anyone to express themselves with their entire bodies just like the most popular Virtual Youtubers, and all that without even needing a PC.

The advance is made possible through the use of an AI deep learning algorithm which does not rely on a depth sensor to track the 3D coordinates of the subject's body frame. Here is a diagram of the frame points which can be tracked with Single 3D:

The technology will be made available as an SDK for PC (C# or C++ library) and iOS (Swift library).

Videos

Here is a demo video, using a 3D model of NEXT-SYSTEM's Unidentified Virtual PR Rep Michiko 未確認仮想広報担当ミチコ, who was actually appointed as the company's PR rep:

---

For more information

If you'd like to know more about VisionPose and the company behind it, please visit their website here and follow them on Twitter to get the latest updates on the app's release (in Japanese).