Website powered by

Meet the Digital Human Fred | Digital Human Showcase

With the digital human, the only limit is your creativity! As a digital assistant, moderator, or virtual companion, the digital human can do almost anything you can imagine. The digital human Fred is the perfect link to create natural interactions between humans and machines. Whether you are talking to a virtual assistant or getting details on your car inspection from your digital mechanic, with the digital human, the only limit is your creativity. Find out more about our digital human Fred:

https://digitalhuman.io

My part in this project was project lead, photogrammetry lead, and 3D rigger.
Leading a team of five people was a first for me, especially at the start of the coronavirus crisis with all the team members at home. It was my responsibility to establish a good home-office working relationship and good communication. First, we started with a prototype project with a three-month time frame. After the success of the prototype, we started casting our digital human Fred and the journey began.
My part as photogrammetry lead was to work on the technical process of scanning highly detailed faces and operating the scanner. For the digital human, the existing scanner had to be modified to produce the highest possible scan quality. Also, to accommodate scanning roughly 700GB RAW images in two hours, the infrastructure of the scan system had to be modified. In the end, seeing all the captured data at the final high-fidelity reconstruction was an amazing view.
After processing all the images and reconstructing them, I helped a bit with the scan cleanup. I helped with sculpting, fixing mistakes in the raw scanning data, and fixing only as little as possible to the point that we could extract the FACS blend shape.
After that, the majority of my time I had to dedicate to rigging Fred's body and face. This was a huge challenge for me as well, always trying to work closely with the animation department to fulfill their requirements for a detailed facial motion capture. Constantly going back and forth adjusting the rig. The biggest challenge was to get everything working in an Unreal Engine.
After my work on the rigging was done, I started helping the art department get all the assets finished in the Unreal Engine.

It was a huge challenge for us as a young, ambitious team, and it took a lot to get it done. But in the end, it was all worth it. This was an amazing opportunity for us as a team to tackle such a complex project. It was an amazing opportunity for me personally to work on improving the scan system and improving my rigging skills. But also the first time leading a small team and a project.

A huge thanks I have to give to all the team members who did an amazing job:

https://timkluthe.artstation.com
https://mroutback.artstation.com
https://david_tomicic.artstation.com

Digital Human showcase video

From left to right: 
uncleaned scan, cleaned scan, enhanced scan with details from the captured texture, projected head mesh

From left to right:
uncleaned scan, cleaned scan, enhanced scan with details from the captured texture, projected head mesh

different FACS headscans

different FACS headscans

Behind the scenes video

first prototype, we did in three month

facial joint setup for an extra flexibility for the animation department

facial joint setup for an extra flexibility for the animation department

facial controller for moving the joints

facial controller for moving the joints

facial controller setup for controlling the blendshapes

facial controller setup for controlling the blendshapes

extra joints for deforming grooming in unreal engine

extra joints for deforming grooming in unreal engine

extra joints for deforming grooming in unreal engine + final grooming

extra joints for deforming grooming in unreal engine + final grooming