Our research and innovation work focuses on how technology can help buildings become “intelligent” places that engage and interact meaningfully with humans and respond to COVID challenges. The 12-month long project encompasses the ideation, design, and development of four “proof of concept” intelligent building installations that can sense and understand human activities and support them with timely information and suggestions. Embedding the concept of “building as a living organism” and responding to COVID-19, these installations offer new experiences and redefine the GAC building’s services and operations. They have been trailed and evaluated, and we have analysed the feasibility of each installation for future work and implementation.
GAC Spatial Installation
1. GAC Spatial Hirer
GAC Spatial Hirer is an online platform which allows virtual navigation within a 360-degree volumetric virtual replica of the GAC building. The conceptual design and technical development of the application has been influenced by the experience we aim to create for a very specific type of user: people who aim to explore the building with the intention of hiring a particular space. Users can remotely navigate spaces and activate information about the different spaces inside the building via digital tags which have been attached to surfaces, such as walls, ceilings, and doors. The information refers to technical specifications as well as other useful information about different facilities and features of studio and performance spaces, as well as labels for accessible toilets and room locations at 81 Ryrie Street building. The online platform has been designed to provide as many navigational options as possible within the building to help users make an informed decision without the need to visit the building.
Limitations: Although the application is portable enough to be embedded into any website (e.g., GAC website), the experience of the installation can be affected by the user’s (hirers’) ability to access a desktop or an IPAD. The use of the application on a smart phone is also possible but the mobile phone screen’s size can limit the experience.
Future Directions: The current application can be expanded to provide direct access to a feature that will allow the user to send a direct query for hiring a particular space immediately after the navigation through virtual GAC space. This important functionality must be designed separately, and one way to easily achieve that is to embed a ‘query’ web-link into the current application.
The current prototype can be explored and experienced through the following website: https://gdom.mindlab.cloud/gac/
Username: guest
Password: domSentMeHere
2. GAC SPATIAL KIOSK
GAC Spatial Kiosk is a large touchscreen which uses an interactive 3D model to display useful real-time information for visitors. For the specific purpose of this prototype, we selected three different types of real-time information to extract and visualise, and these are directly related to the use of space and ease of circulation within the building: what kind of activities and performances are taking place in different spaces inside the building at any given time; how busy certain spaces are with an indication of average waiting times; and what is the easiest and most convenient route to any particular location inside the building. The GAC Spatial Kiosk aims to contribute to both the operational efficiency of the building, by providing an overview of the various activities connected to space, and the experiential aspect, by helping visitors avoid crowded areas and directing them to their desired location to minimise existing crowding.
Limitations: While the prototype uses only sample data for demonstration purposes, it can be updated with additional data easily. Live sensor data connection to the spatial model was lab-tested but was not included in the user evaluation because there were no people using the indicated areas at the time of the testing.
Future Directions: Several staff members highlighted the usefulness of the Kiosk for back-of-house operations to guide all necessary activities of theatre companies that are using different spaces around the building. One recommendation was to develop a content management system that will enable faster management of real-time spatial data by GAC staff. The content could be introduced through multiple touch-screens placed at different locations (and floors) in the building for the convenience and comfort of employees and others involved in the process.
Talking Screens Installation
This project addressed the central theme of a spatially intelligent GAC by providing visitors with a digital intelligent building manager that responds to their questions. The project envisaged a virtual avatar that can be summoned by visitors everywhere in the building via wall projection or digital screens and provide them with answers to frequently asked questions. The service would be like other digital agents, such as SIRI or Alexa, but personified by a humanoid virtual avatar to facilitate natural face to face interactions. Conversational video is a new way to provide users with interactive content enabled by natural language processing and artificial intelligence. The system facilitates the playback of video snippets and creates a question-and-answer dynamic that approximates face-to-face communication. When asked a question, the system can sift through all corresponding video clips about a topic and find an intuitive conversational response.
Limitations: The initial prototype exhibited a number of limitations such as relatively low video quality and the voice recognition feature was also not at full performance. The testing revealed important clues for the project team to improve, especially how the interface could be better linked to voice recognition feature. Another limitation was with regards to ability of the system to play-back only pre-recorded video snippets. This somehow hampered the user experience during the first round user-testing.
Future Directions: A future version of Talking Screens will be redesigned from the ground up. Instead of using a filmed footage of a real actor and playing back pre-recorded video only, we plan to use a game engine to create a digital character, (e.g. metahuman). The use of a system as such will help overcome several limitations such as streaming and resolution issues and would reduce the production cost. It will also allow more flexibility to diversity responses that can be generated through AI.
Memory Lane Installation
This installation addressed the idea that the GAC has a ‘memory’ inspired by the overarching theme of a building as a living organism. The conceptualised arts centre memory consists of a visualisation of shows and events on a large touch screen through which ‘the memories’ can be accessed and interrogated by GAC staff and visitors. Each memory was defined as a collection of images, video snippets, with a brief textual description. Like human thought processes, the system facilitates a flow between a current memory and related memories by providing links to related shows. When the system is not in use by visitors, it will show that it is alive by visualising a thinking process which involves (selecting) a series of shows and events that light up whilst all other shows are dimmed in brightness. A white highlight behind the tiles of the shows visualises the firing of synapses and the thought process.
Limitations: The first prototype did not focus on the distribution of content, therefore only a sample image/video content have been linked to the images. Since the current project focused mainly on the design of the interface and interaction with its users, the initial prototype development focused on those aspects. Sorting, filtering, categorising, structuring and updating the content as new information becomes available is an equally important process which are planned to run parallel to the development task, in the future versions of the prototype.
Future Directions: Following the results of the user test, future versions of Memory Lane will focus more on the interactive features of the system which will require a larger display screen (for better multi-user interaction). Another important feature will be utilisation of sound and music to assist users in reminiscing about past performances. The interactive nature of the screen can be further improved via screen hints, clever environment design, such as the demarcation of an active zone, and signage around the screen. The Memory Lane experience could be further improved by providing an attractive system sound loop with interesting interaction sounds, and by playing sound and/or video snippets from the shows.
Thought Bubbles Installation
The Thought Bubbles installation prototype represents a post-performance audience voting experience. The collection mechanism uses a QR code-activated mobile device voting interface. The visitor is presented with questions and voting options. The voting options (Figure 9) are presented in textual and visual form (descriptions and emojis) and the possible response range from positive to negative. Thought Bubbles display mechanism (Figure 10) builds on the idea of the individual and the collective. Each vote is represented as a bubble, and bubbles of the same category (voting option) merge and form larger bubbles. Each bubble of various size contains the same visual element as the voting options from the visitor’s feedback collection. The bubble reactions gather and animate on the shared display for the audience to share how they feel before departing. The reactions are stored and visible to GAC staff via a private dashboard data display. The dashboard allows the comparison of results across the sessions of a single show and across different shows. The ability to immediately visualise their own vote in the context of total votes provides participants with some assurance that their vote matters and that they are being heard.
Limitations: The visitor experience of this installation can be affected where there are limits on the accessibility of voting devices. For example, children under certain ages may not own mobile phones and, consequently, this younger audience could be excluded from the experiences and GAC staff will not hear their voice.
Future Directions: Following the suggestions of GAC staff who participated in the latest evaluation, one future direction is to use this installation to leverage digital promotional opportunities. Thought Bubbles results could be shared on the GAC website and on digital poster displays at GAC venues to show activity across different venues. Additionally, feedback results captured via the voting tool could be directly distributed to GAC staff in real-time for better understanding of visitors’ experience when in GAC venues.
