Avatars

How to integrate CC4 avatars

Create and export the avatar in Reallusion Character Creator CC4

1. Create and configure your character in CC4. CC is installed on stvr1.informatik.uni-bremen.de. You need to add Expression Wrinkles. For the latter, select any of the available Actor>Expression Wrinkles from the Template item hierarchy in Content on the left-hand side of the CC4 editor and drag and drop it onto your character.
2. Export your character as FBX. Select Export>FBX>Clothed Character from the menu. In the export dialog, select Unity 3D as the Target Tool Preset. Choose Mesh from FBX Options. Select Delete Hidden Faces and Use Subdivided Mesh from Other Settings. Deleting hidden faces improves rendering time as the hidden faces will not be seen anyway. According to Reallusion: "The subdivision feature with Character Creator 4 allows you to enhance the topology of your characters for smoother and more detailed final renders." Then all you have to do is to add the Merged_Open_Mouth blendshape, which is used for facial animation and allows you to open your mouth properly. To do this, click on the settings icon in the top right corner, which looks like a sun. Another window with settings will appear on the right. There, you must activate the checkbox for Mouth Open as Morph. Press button Export when done. Another dialog will show up, informing you that an exported character cannot be imported back. Press button OK. Save the exported data in the SEE/Unity in a newly created folder Assets/Resources/Materials/CC4/<Name> where <Name> is the name of your character. You will have the following new items in the Assets/Resources/Materials/CC4/<Name>: <Name>.Fbx, <Name>.json, <Name>_Importinfo.txt, and a subfolder textures containing the textures of the character.

You should add your Reallusion Character Creator (CC4) project file to the top-level directory Reallusion of our Git repository. This allows us to make changes later if necessary. These files will be stored in our Git LFS.

Additional ressources:

• Auto Setup for Unity provided by Reallusion
• Video on how to export to Unity and Auto Setup

Import the avatar in SEE/Unity

The Unity Reallusion plugin for importing CC4 characters is already installed in SEE.

When Unity detects the newly added FBX data, it will open a dialog notifying you that it must mark textures as a normal map. Press button Fix now.

1. Import the character via menu Reallusion>Import Characters. The new character will then appear in grey in window CC/iC Importer 3D.
2. Select the new grey icon for the newly imported character. Then select the different quality features: High Quality Materials, Parallel Eyes, Two Pass Hair, and WrinkleMaps, Enable Cloth Physics, and Enable Hair Physics in Features as shown in the screenshot below. Use Bake Custom Shaders and Bake Separate Prefab. Then press Build Materials. This will create the following new subfolders in Assets/Resources/Materials/CC4/<Name>: Baked, <Name>.fbm, Materials, Meshes, Prefabs, and Settings. A new prefab should appear in Assets/Resources/Materials/CC4/<Name>/Prefabs/<Name>.prefab where <Name> is the name of your character.

The imported files should be checked into our Git repository.

Create prefab variant for player

1. Right-click Assets/Resources/Prefabs/CC4/BaseAvatar.prefab and select Create>Prefab Variant from the mouse menu. Give this new prefab variant the name <Name>.
2. In the window CC/iC Importer 3D, press button Preview, which is the top-most button on the right-hand side with an eye enclosed in a kind of box. Then a new scene will be opened with the new character. In the corresponding game-object hierarchy, copy all children of the top-most game object representing the character altogether (named <Name>).
3. Then open the newly created prefab variant Assets/Resources/Prefabs/CC4/<Name>.prefab and paste the copied game objects as direct children of the top-most game object in the hierarchy using menu Edit>Paste Special/Paste As Child (Keep Local Transform) (or use key shortcut Ctrl-Shift-V). The prefab variant will inherit all necessary game objects and their components from the base avatar (the behaviour). The newly created and added game objects define the avatar's appearance.

Configure the prefab variant for player

Open the prefab variant for the new player avatar and select the top-most game object in the hierarchy. Then configure the avatar using the Inspector window as follows:

• The Avatar in Animator component must be <Name>Avatar, which is contained in Assets/Resources/Materials/CC4/<Name>/Prefabs/<Name>.Fbx

• Configure component Look At IK by adding game object <Name>/CC_Base_BoneRoot/CC_Base_Hip/CC_Base_Waist/CC_Base_Spine01/CC_Base_Spine02/CC_Base_NeckTwist01/CC_Base_NeckTwist02/CC_Base_Head to attribute Head.

• Add a new empty game object AimTransform to the game-object hierarchy for the avatar under the path <Name>/CC_Base_BoneRoot/CC_Base_Hip/CC_Base_Waist/CC_Base_Spine01/CC_Base_Spine02/CC_Base_R_Clavicle/CC_Base_R_Upperarm/CC_Base_R_Forearm/CC_Base_R_Hand/AimTransform with local position = (0.0153701, 0.1400991, 0.03830782), local scale = (0.008820007, 0.008820003, 0.008820006), local rotation = (-86.34, 51.421, -51.463)`. This game object will be the Aim Transform in component Aim IK. You could simply copy and paste it from an existing avatar prefab.
• Configure component Aim IK by adding game object AimTransform create above to attribute Aim Transform. The Bones are set as follows:

1. CC_Base_Spine01
2. CC_Base_Spine02
3. CC_Base_R_Clavicle
4. CC_Base_R_Upperarm
5. CC_Base_R_Forearm

• Turn off the toggle Is Head in material Std_Skin_Head of the game object CC_Base_Body. Otherwise the face skin will not be rendered in the Unity editor play mode under Linux.

Configure SALSA Lipsync

SALSA LipSync provides facial animation features for our avatars. In particular, it integrates with our voice chat Dissonance such that the avatar's lips are synchronized with the spoken words. The following SALSA components are attached to the avatar:

• SALSA (Simple Automated LipSync Aproximation) is a realtime system for creating lip synchronization from audio input.
• EmoteR is an emote randomizer utility.
• Eyes is an eye and head tracking and random generation system.
• SilenceAnalyzer aims to increase the perceived accuracy of SALSA's automated lipsync processing.
• QueueProcessor coordinates the SALSA LipSync suite. It will keep track of the activating and deactivating animations and resolve any conflicts to prevent jittery fighting over what an animation should be doing.
• SalsaDissonanceLink connects the Dissonance audio system to the lip sync system of SALSA to provide real-time lip synchronisation for other speakers in the VoIP session.

There are video tutorials on SALSA.

Luckily, there is a one-click configuration of all settings.

• Select the root game object of the avatar.
• Then select the menu GameObject>Grazy Minnow Studio>SALSA LipSync>One-Clicks>Reallusion>CC4.

After the one-click setup of SALSA, you need to make the following further adjustments:

• In component SALSA under References, set Audio Source to None.
• In component SALSA under References, toggle on Use External Analysis.
• Remove component Audio Source from the avatar.

Configure FACSvatar

For FACSvatar to work correctly, specific components must be added to the avatar.

FACSnimator Script

The FACSnimator script must be added to the avatar. The script is located under Assets/Plugins/FACSvatar/FACSnimator.cs.

It should be attached to the CC_BaseBody GameObject of the avatar.

Head Rotator Bone Script

The Head Rotator Bone script must be added to the avatar. The script is located under Assets/Plugins/FACSvatar/HeadRotatorBone.cs.

It should be attached to CC_Base_BoneRoot GameObject of the avatar.

The following GameObjects must be referenced:

• Joint Obj_head: CC_Base_Head
• Joint Obj_neck: CC_Base_NeckTwist02

Configure the scenes

Scene SEEStart

1. Open scene SEEStart.
2. Select game object NetworkManager in the hierarchy.
3. Double click attribute Network Prefabs List in the component Network Manager.
4. Add a new entry NetworkPrefabs for the new player prefab variant Assets/Resources/Prefabs/CC4/<Name>.prefab. We order the characters first by gender: first males, then females. At the second level (within a gender group), we order by name.

Scene SEENewWorld

1. Open scene SEENewWorld.
2. Select game object Server in the hierarchy.
3. Add a new element for Player Spawns in component Player Spawner for the new player prefab variant Assets/Resources/Prefabs/CC4/<Name>.prefab. Adjust Position and Rotation as needed. We order the characters first by gender: first males, then females. At the second level (within a gender group), we order by name.

Important note: The order of the player prefabs in the Network Prefabs List of scene SEEStart and the Player Spawns in scene SEENewWorld must be identical.

Adjust the source code

• Add the name of the prefab <Name> to list Prefabs in script PlayerSpawner.cs.
• Add an entry to the assignment to playerSpawns in Awake() must be added. The number of elements in Prefabs must be identical to the number of elements on the right-hand side of the assignment to playerSpawns in Awake().

These settings become effective only if component Player Spawner of game object Server in scene SEENewWorld gets reset.

Important note: The order of the player prefabs in this list and the Network Prefabs List of scene SEEStart must be identical.