Components

Initial Generation of Scene and Character Motion We use the subgraph of the parsed text to match SSGs in the scene dataset for scene object retrieval. We represent character motion as locomotion and action motion, where the locomotion is generated by a pre-trained NSM model and serves to perform basic motions and determine character positioning in the scene, for instance, to walk, stand, run, etc. Action motions are mocap clips retrieved from dataset, and can be performed during locomotion. We fuse locomotion and action motion to produce diverse combinations of character motions.

Scene Layout and Character Motion Optimization

In order to ensure compatibility between scene layout and character motion, especially for character-object-interaction, we propose a method that takes the locomotion, action motion and scene layout as inputs, and iteratively optimizes the scene layout. In each iteration, our method tries to avoid unnecessary collisions between the character body and virtual objects in the scene, meanwhile ensuring body parts (i.e., finger tips) are close enough to the points of objects (i.e., keyboard). We formulate cost functions and employ the simulated annealing algorithm for iterative optimization of scene layout and character animation generation.

We have been able to isolate systems that generate animations from natural language input, focusing on those that perform some natural language understanding and are not purely data-driven

The part-of-speech (POS) of input text is first tagged by CoreNLP [MSB∗14] and transformed into a dependency tree, including all entities and their corresponding attributes and relationships which correspond to human actions, as shown in Fig.

The entity represents characters, the attribute corresponds to an action, and relationships record interactive models of the action. Then, the dependency tree is converted into an SSG, which saves descriptions of each object in the input texts and relationships between objects. In order to facilitate processing, we assume the user appends a “$” sign at the end of a text to indicate a scene description sentence. Moreover, our system also supports verbal commands. We provide a set of weather, virtual camera and motion control commands to facilitate animation creation. For example, the command “camera left” sets the virtual camera to the left of the character. The user can iteratively write and edit texts to update the results.

Initial Generation of Scene and Character Motion

We use the subgraph of the parsed text to match SSGs in the scene dataset for scene object retrieval. We represent character motion as locomotion and action motion, where the locomotion is generated by a pre-trained NSM model and serves to perform basic motions and determine character positioning in the scene, for instance, to walk, stand, run, etc. Action motions are mocap clips retrieved from dataset, and can be performed during locomotion. We fuse locomotion and action motion to produce diverse combinations of character motions.