Cosplay photo and video shoots seem to be getting more sophisticated year by year. While various video editing and specialized software applications have become easier to use and more accessible to the general public, adding special effects to your cosplay shoot is still out of reach for most cosplayers due to the technical expertise required.

But what if you could realize cosplay shoots that not only convey the fictional world of your characters through the costumes and props you are confident in making, but also with backgrounds and sophisticated special effects that react to your movements?

Gensōkyō System

Such is the potential offered by 幻想郷システム Gensōkyō System. Developed by 五味 Gomi (@GomiHgy), a programmer and DIY maker who puts the latest technologies to use in projects including decorative illumination and other types of cosplay tech, and realized through the support of a team of volunteers, Gensōkyō System uses projection mapping and motion tracking AI to enable next-level cosplay performance.

As you can see in the promotion video above, Gensōkyō System can not only create effects tailored to the character you're portraying but even works with cosplay of Gundam and other mecha.

An excellent opportunity to demonstrate its potential was recently afforded in a Touhou Project-themed promotion video by videographer LostLink on the "Rescue the Princess!" YouTube channel.

In addition to video, you can use Gensōkyō System to create amazing cosplay photography, like this:

Gensōkyō System provides several effects such as sword slashes, beams, magic circles, etc., and the color of each effect can be changed. They can also customize and provide exclusive effects for events.

---

Background

Gensōkyō System's volunteer team of engineers was already active in creating various works, mainly related to cosplay. They share a common goal to make cosplay expressions more interesting and to help realize the world of games and anime through technology.

For example, Gomi and fellow Gensōkyō System collaborator 静丘 Siz_Oka (@Siz_oka) created a highly detailed costume and tech solution for an illuminated cosplay of Laala Manaka from the PriPra series using voice recognition to change the pattern of her Twinkle Ribbon Cyalume Coord idol costume with the "Cyalume Change" vocal command.

The team also focused on projection mapping technology as an effective way to bring the world of games and anime into reality. They developed a system that can detect the position of a sword prop embedded with a gyroscope sensor and a camera to trigger a visual effect. Their first test case was a cosplay of Saber from the Fate/stay night series, unleashing the Noble Phantasm Excalibur. However, the system required a "weapon" for detection, so it was not very flexible.

To improve this, they came up with the idea of using Google's AI library, Tensorflow, to identify human figures and their poses, thereby allowing anyone to perform special moves at will without the use of a specially-designed weapon prop.

This was the final technical innovation that brought Gensōkyō System to life.

So far, Gensōkyō System has been shown at the Cosnavi Photo Exhibition Legacy 2022 (see the Tweet below) and other cosplay-related events where the general public can experience it and cosplayers have used the system for filming.

Future plans

The team intends to continue exhibiting at events in Japan and other countries if they have the opportunity to do so. By improving detection accuracy and further increasing the types of effects available, they plan on making it compatible with various games and anime.

Gensōkyō System requires technical skill and knowledge to set up projection mapping. Although they have received inquiries, for the time being, it's not available as a commercial product that you can buy and set up independently. To use it, you need to contact them or find out where they will be exhibiting next.

They're open to collaborating on interesting projects with those who'd like to join them in expressing the world of anime and games.

Contact

If you're interested in Gensōkyō System, you can contact Siz_Oka (@Siz_Oka) by direct message (English OK).