Research Publications

Does It Feel Real?: Using Tangibles with Different Fidelities to Build and Explore Scenes in Virtual Reality (2019) CHI '19

Thomas Muender, Anke V. Reinschluessel, Sean Drewes, Dirk Wenig, Tanja Döring, and Rainer Malaka

Professionals in domains like film, theater, or architecture often rely on physical models to visualize spaces. With virtual reality (VR) new tools are available providing immersive experiences with correct perceptions of depth and scale. However, these lack the tangibility of physical models. Using tangible objects in VR can close this gap but creates the challenges of producing suitable objects and interacting with them with only the virtual objects visible. This work addresses these challenges by evaluating tangibles with three haptic fidelities: equal disc-shaped tangibles for all virtual objects, Lego-built tangibles, and 3D-printed tangibles resembling the virtual shapes. We present results from a comparative study on immersion, performance, and intuitive interaction and interviews with domain experts. The results show that 3D-printed objects perform best, but Lego offers a good trade-off between fast creation of tangibles and sufficient fidelity. The experts rate our approach as useful and would use all three versions.

Download the paper
Analysis of Previsualization Tasks for Animation, Film and Theater (2019) CHI '19 EA

Thomas Muender, Georg Volkmar, Dirk Wenig, and Rainer Malaka

Previsualization (previs) is an essential phase in the visual design process of narrative media such as film, animation, and stage plays. In digital previs complex 3D tools are used that are not specifically designed for the previs process making it hard to use for creative persons without much technical knowledge. To enable building dedicated previs software, we analyze the tasks performed in digital previs based on interviews with domain experts. In order to support creative persons in their previs work we propose the use of natural user interfaces and discuss which are suited for the specific previs tasks.

Download the paper
Demonstrating VRBox: A Virtual Reality Augmented Sandbox (2019) CHI '19 EA

Dmitry Alexandrovsky, Tanja Döring, Susanne Putze, Thomas Fröhlich, Timo Stabbert and Rainer Malaka

We present VRBox--an interactive sandbox for playful and immersive terraforming that combines the approach of augmented sandboxes with virtual reality technology and mid-air gestures. Our interactive demonstration offers a virtual reality (VR) environment containing a landscape, which the user designs via interacting with real sand while wearing a VR head-mounted display (HMD). Whereas real sandboxes have been used with augmented reality before, our approach using sand in VR offers novel and original interactive features such as exploring the sand landscape from a first person perspective. In this demo, users can experience our VR-sandbox system consisting of a box with sand, multiple Kinect depth sensing, an HMD, and hand tracking, as well as an interactive world simulation.

Download the paper
Get a Grip! Introducing Variable Grip for Controller-Based VR Systems (2019) IEEE '19

Michael Bonfert, Robert Porzel and Rainer Malaka

We propose an approach to facilitate adjustable grip for object interaction in virtual reality. It enables the user to handle objects with loose and firm grip using conventional controllers. Pivotal design properties were identified and evaluated in a qualitative pilot study. Two revised interaction designs with variable grip were compared to the status quo of invariable grip in a quantitative study. The users performed placing actions with all interaction modes. Performance, clutching, task load, and usability were measured. While the handling time increased slightly using variable grip, the usability score was significantly higher. No substantial differences were measured in positioning accuracy. The results lead to the conclusion that variable grip can be useful and improve realism depending on tasks, goals, and user preference.

Download the paper
VRBox: A Virtual Reality Augmented Sandbox for Immersive Playfulness, Creativity and Exploration (2018) CHI PLAY '18

Thomas Fröhlich, Dmitry Alexandrovsky, Timo Stabbert, Tanja Döring, and Rainer Malaka

Augmented sandboxes have been used as playful and educative tools to create, explore and understand complex models. However, current solutions lack interactive capabilities, missing more immersive experiences such as exploring the sand landscape from a first person perspective. We extend the interaction space of augmented sandboxes into virtual reality (VR) to offer a VR-environment that contains a landscape, which the user designs via interacting with real sand while wearing a virtual reality head-mounted display (HMD). In this paper, we present our current VR-sandbox system consisting of a box with sand, triple Kinect depth sensing, a virtual reality HMD, and hand tracking, as well as an interactive world simulation use case for exploration and evaluation. Our work explores the important and timely topics how to integrate rich haptic interaction with natural materials into VR and how to track and present real physical materials in VR. In a qualitative evaluation with nine experts from computer graphics, game design, and didactics we identified potentials, limitations as well as future application scenarios.

This paper received the Honorable Mentions award at the conference.

Download the paper
Empowering Creative People: Virtual Reality for Previsualization (2018) CHI '18 EA

Thomas Muender, Thomas Fröhlich, and Rainer Malaka

Previsualization (previs) is an essential phase in the design process of narrative media such as film, animation, and stage plays. Digital previs can involve complex technical tasks, e.g. 3D scene creation, animation and camera work, which require trained skills that are not available to all personnel involved in creative decisions for the production. Interaction techniques such as virtual reality (VR) enables users to interact with 3D content in a natural way compared to classical 2D interfaces. As a first step, we developed VR based prototypes and performed an exploratory user study to evaluate how non-technical professionals from the film, animation and theater domain assess the use of VR for previs. Our results show that users were able to interact with complex 3D scenes after a short phase of familiarization and rated VR for previs as useful for their professional work.

Download the paper
Extending Augmented Sandboxes with Virtual Reality Interaction (2017) MuC '17 Workshopband

Timo Stabbert, Thomas Fröhlich, Dmitry Alexandrovsky, and Rainer Malaka

Augmented sandboxes are often used as educative tools to create, explore and understand complex models. For the use case of a water cycle simulation, we extend the interaction space of augmented sandboxes into virtual reality to overcome limitations of current systems that include non-interactive 2D projections and shadow problems. We present our ongoing research and the prototypical setup of our VR sandbox consisting of a triple Kinect setup, depth sensing, VR, and hand tracking using Leap Motion. The setup shall help us to explore the space of haptic redirection. Further, we discuss our water cycle simulation use case and interaction scenarios that facilitate VR interaction and visualization.

Download the paper
Play While You Work: Productive Play for Digital Content Creation (2017) CHI PLAY '17 Doctoral Consortium

Thomas Fröhlich

In this paper, I present my ongoing dissertation project in- vestigating playful interaction techniques for productive pur- poses. The objective to find out how users can effectively create digital content while actually playing. Building upon and extending the notions of gamification, serious games and games with a purpose, I focus on productive aspects of play where not a task is extended with playful interaction, instead I aim to create playful environments where play is the main activity, while at the same time, productivity can be achieved as a direct result of the game or play activity. A first user study revealed that the approach has a high hedo- nistic value, low learning curve and creative self-perception. At the same time, the work results are comparable to stan- dard tools in a reproduction task.

Download the paper
Productivity & Play: A First-Person Shooter for Fast and Easy Scene Design (2017) CHI PLAY '17

Thomas Fröhlich, Jan von Oehsen, and Rainer Malaka

Building 3D scenes for games can be a difficult task, especially for beginners and other non-experts wanting to contribute their own ideas in an easy way. Tools for map or world creation are often complex to understand and use, don’t support the creative process of building a scene, and decouple design from player experience. To overcome these problems, we present a First-Person Shooter (FPS) game for level creation using game mechanics for all tasks, making the process more easy and enjoyable while also providing direct experience of the work from a player’s point of view. We evaluated our approach in a preliminary study where we compared our game to a simplified version of the Unity editor, representing generic world building tools, and found that users enjoy the our game, find it more usable, and feel more creative.

Download the paper

All publications of the first.stage project can be found on OpenAIRE.


Newsletter & Press

first.stage Newsletter 2019

first.stage Partners, May 2019

Dear Reader, the third first.stage newsletter presented here contains what happened in our project since our last newsletter. We are very happy about your interest in our project. The consortium has been working together even closer to realise our ideas. After nearly one year of user evaluation within the consortium the previsualisation prototype we have implemented is near to post project commercialisation and will appear on the market soon. The project result is a user-friendly system that fulfils the needs of professional creatives e.g. directors of theatre productions, previsualising their ideas in a novel way utilizing virtual reality presentations and motion capturing.

Read the whole Newsletter
Filmen beginnt im Kopf – und in 3D

negative space, 25. September 2017

We were happy to present first.stage to the nice people of negative space who visited University of Bremen in September 2017. They produced a nice video showcasing the project vision and status. Follow the link to the full story.

Read the Article
Schnelle Rohfassung für große Effekte, 9. September 2017

Interview with Kai Erenli from our project partner Arx Anima about previsualisation and the first.stage project. Follow the link to the full story.

Read the Article
Newspaper Article about first.stage

Weserkurier Bremen, 3. September 2017

Article about the first.stage project in the local newspaper of Bremen, explaining the idea and goals of our project to the broader public. Our goal to make previs accessible to smaller, low cost productions as well as the focus on natural user interfaces is described.

Read the Article
Second slide
Press Release University of Bremen

University of Bremen, 25. August 2017

When a screenplay is finished, several more steps must follow before the actual shooting can begin: The preproduction phase includes visualising the scenes, creating the set and casting the actors. During this creative and agile process, members of the production team and actors develop and exchange ideas regarding the story, graphic design and time sequences. New solutions in the area of digital visualisation and interaction can open up the process for non-technical staff and simultaneously make it affordable for small production enterprises.

Read the whole Press Release
first.stage Newsletter 2018

first.stage Partners, May 2018

Dear reader, we would like to introduce you to the second first.stage newsletter. A lot has happened in our project since the first one, and we appreciate your interest. The whole first.stage consortium is pleased to say that the vision everyone has set out to achieve is coming ever closer. A first working prototype, so to say a physical manifestation of the ideas that led to the conception of the project in the first place, is almost ready. This, in conjunction with the in-depth testing conducted, will lead to a unique system, allowing professionals creatives e.g. directors of film productions to previsualise their ideas in a way that was not possible before. The feedback received, especially by our test-case partners Vogel Audiovision, Landestheater Linz, and arx anima was very promising. In this issue of the newsletter, we will endeavour to provide you with a short overview about visible activities of the first.stage project.

Read the whole Newsletter
first.stage Newsletter 2017

first.stage Partners, June 2017

“Fast and easy previsualisation for creative industries“ is an H2020 project funded by the European Commission and planned with a duration of 36 months with 8 partners from 5 countries.

During the last three decades information technology has facilitated a paradigm shift in almost all areas of human life, from manufacture, technology, science to sociology and art. Cutting edge computer animation has become a staple in almost all high budget triple-A productions. But information technology has yet to extend its reach beyond the movie blockbusters to smaller productions, and to media that are only slowly adapting to new technologies. One of the areas in which information technology would be very helpful, but no efficient solutions are available is previsualization. Previs, for short, allows directors and other creative people to develop ideas, break boundaries and convert their artistic vision into reality. This is not to say that there are no tools available for previs at the moment. Their main drawback is the high degree of technical expertise required to operate them. This, in effect, means that any production needs to hire an expert in this area specifically for this purpose. This is not feasible due to monetary restraints and the fact that this abolishes one of the major benefits generated by previs, saving valuable time. The first.stage project has taken up the challenge to develop new solutions to allow creative people utilise previs for their project.

Read the whole Newsletter