When a screenplay is finished, several more steps must follow before the actual shooting can begin: The preproduction phase includes visualising the scenes, creating the set and casting the actors. During this creative and agile process, members of the production team and actors develop and exchange ideas regarding the story, graphic design and time sequences. New solutions in the area of digital visualisation and interaction can open up the process for non-technical staff and simultaneously make it affordable for small production enterprises.
Scientists at the University of Bremen’s Center for Computing and Communication Technologies (TZI) are now developing computer-based tools for digital pre-production in order to support the work of production teams. The main focus lies on natural user interfaces that can recognise the actors’ and producers’ hand gestures and body movements. These input options will make it easier to animate characters and create special effects. In addition, they will support the designing of scenes, e.g. by allowing the placing, positioning and altering of landscapes and objects in an intuitive way.
Applications in film, theatre and animationThe project “first stage – Fast and easy previsualization for creative industries”, led by TZI’s digital media research group under Prof. Dr. Rainer Malaka, focuses not only on conventional film productions, but also on projects in the areas of theatre, animation films and visual effects. Each of these can benefit from interactive technologies that allow a highly realistic preview of the finished work without requiring significant training for their users.
Film directors, for example, can use “first stage” to express their ideas more directly in the early stages of the production process. Also, actors of a theatre production might not have to travel to the first rehearsals all at the same time anymore – they can capture their scenes digitally. The performances can then be reviewed, adapted and rearranged on the screen in order to find the best form of presentation.
Artistic expression is combined with technologyThe system can receive input in various ways. For example, inexpensive standard hardware is already capable of recording and analysing movements. People – or individual body parts – displayed on a screen can therefore be animated using intuitive hand gestures. Speech recognition and touchscreens are among the other options.
One possibility to record movements of the entire body in order to generate lifelike imagery is the use of suits that are equipped with a multitude of sensors. “The actors’ artistic expression can be combined with the opportunities that new technologies offer”, says Benjamin Walther-Franks, who coordinates “first stage” at TZI. “Some powerful previz tools are already available, but they are very complicated to use. We want to create a system that everyone can operate.”
EU funding within “Horizon 2020”The three-year project, which started on 1st June, 2016, is funded within “Horizon 2020”, the EU’s framework programme for research and innovation. Small European production enterprises and cultural institutions are supposed to benefit most from the results. In contrast to large American entertainment corporations they are usually unable to maintain their own research and development departments.
In addition to TZI, the project consortium consists of seven more organisations. Technolgy partners are Moviestorm (UK), Rokoko (Denmark) and Next Limit Technologies (Spain). The State Theatre Linz, Vogel Audiovision and Arx Anima (all in Austria) participate as application partners. InfoConsult (Germany) supports the project management.