Mixed Reality Maps to help convey disaster information

In safe and industrialized nations most people have only been sporadically in contact with disaster information. But nevertheless, suitable cartographic products are needed to support first aid responders and help teams trying to save as many lives as possible in disaster events. The spatial information on which these teams have to rely on are currently maps without any interactivity and flexibility which was detected by a requirements analysis asking stakeholders working in the field of disaster management. Based on this analysis a tabletop mixed reality application was developed. Within the application two case study scenarios were integrated. Both scenarios together showed the flexibility of the mixed reality approach for representing disaster information in a suitable and feasible way. The first scenario illustrates a large fire event as well as several additional information sources which can assist response and management teams in various directions. The second scenario displays an earthquake event visualizing damaged and destroyed houses and streets. This scenario as well integrates many different information sources and spans from small scale area visualizations to large scale indoor maps for highlighting e.g. injured people in distinct rooms or floors. The resulting application was evaluated by experts. First the application was presented and explained to the experts. Afterwards the experts were asked in a structured interview how they would evaluate the usability and willingness to use such an application in real scenarios. The experts were impressed by the available possibilities in representing and integrating disaster information via the tabletop mixed reality approach yet mentioned that a head mounted device compared to a smartphone would be more beneficial in practice.


Introduction
Communication is one of the inherent tasks in cartography. The media on which the spatial information is presented has changed in the last decade. Paper maps are omnipresent and are used in many domains. The making of paper maps has a long tradition with well-established cartographic design rules and principles. With the upcoming accessibility of digital spatial information, the availability of web maps has increased and is particular important in daily life. Therefore, web mapping has become an everyday tool and is nowadays extended by new and again upcoming visualization trends like 3D maps, mixed and augmented reality which are already established. Nearly every consumer smartphone has the built-in capabilities of using augmented reality on a very easy entry level 1 . Recent developments show the potential of this up-to date cartographic visualization approaches. In particular, augmented reality approaches can be found in touristic tour guides (Nawshin, 2021), archeological visualization of excavation sites (Cannella, 2019;Boboc et 1 https://developers.google.com/ar; https://developer.apple.com/augmented-reality/ al., 2019). Besides, these holographic information displays are developed and have found their way into the public. Displays like Microsoft HoloLens 2 are useful to give the user a complete immersive impression of their surroundings. Different models (Khamsim, 2019) and data (Jahnke et al., 2019) as well as wayfinding hints (Cron et al., 2019) can be visualized in different degrees of freedom.
In safe environments and industrialized nations most people have sporadically been in contact with disaster information via media news or fundraising campaigns for affected people. But nevertheless, people affected, help teams and the general public are calling for valid, accurate and well visualized spatial information (Kremser, 2020) to secure as many lives as possible. Maps are in many situations the means of choice (Copernicus Emergency Service 3 ) for the communication of disaster information to the different stakeholders involved. The ability to integrate interactivity and different levels of information in such a map is difficult and if once produced the procedure to update the map is time consuming. Augmented or mixed reality could be a tool which features new insights and makes the map and data exploration more convenient. Therefore, the questions which have to be answered with regard to the available technologies like augmented and mixed reality are as follows: • Are the current visualization techniques (printed maps) the adequate medium and able to facilitate an effective management of disasters?
• Does it make sense to incorporate additional dimensions into the communication to support the multidimensionality of such data?
• Can the visualization of this multidimensionality help decision makers to grasp more quickly the desired information?
The above-mentioned research questions are merged into two hypotheses which should be answered by this work to support future developments in this domain.
Communication of disaster information as well as the management of disasters are in need of visualizations which go beyond current conventional cartographic products.
Augmented and mixed reality as well as 3D maps are useful tools for the communication of disaster information to distinct user groups like people managing help activities.

State of the Art in Disaster Information Communication
There is a wide range of freely available or commercial disaster response software products on the market. Disaster management tools are currently capable of importing and editing vector-based GIS-layers as well as displaying raster layers either as client-side geodata or as WMS/WFS layers. Up to now, responders rely on 2D maps to depict the situation on the ground. This is certainly the most robust way of presenting situations, not least because the broad mass of users is familiar with it, and the information is presented in such a way that it can be easily grasped by the general public, regardless of whether the situation is presented in the form of a classical map or as an orthophoto.
The base maps can be enriched with relevant information such as collection points, shelters, hydrants, water tapping points as well as with topographic information like contours and height points. These products provide GIS functionalities like the demarcation of danger zones, for example, after the discovery of a World War II bomb, giving the number of people affected or the representation of a dispersion area of a smoke or poison cloud.
On the other hand, disaster response services such as the Copernicus Emergency Service 3 , Sentinel Asia 4 or International Charter Space and Major Disasters of UN SPIDER 5 (space-based information for disaster management and emergency response) provide decision makers both with vector data and printable maps. These services working operational since many years and provide map products like reference maps showing the situation before an event, delineation maps showing the extent of an event like flooded or burned areas ( Figure 1) and grading maps giving the severity of an event. These maps are independent from the required map scale, they can be produced both for overview maps covering huge areas and for detailed analysis for examples after technical accidents showing the situation of small areas. Due to the time-critical approach of these services, timeconsuming and labor-intensive 3D representations have not yet become established. The Copernicus EMS risk and recovery service 6 , however, offers the possibility to provide up-to-date terrain models which can be used e.g. for 3D applications.
In order to find their way around buildings, emergency forces are provided with so-called object or area detail maps for larger buildings. These maps show emergency exits, elevators, smoke protection doors, fire extinguishers, electricity ports or smoke detectors in addition to the floor plan. These are usually carried as a printed map and are displayed in buildings.
The project TT-GSAT 7 (Test-and Trial-Centre for Geoinformation-and Satellite based Rescue & Emergency Services) has developed a concept for a "Centre for testing and trial of satellite-based services for rescue and emergency organizations" in cooperation with rescue and emergency organizations, other public authorities and industry stakeholders, which will be implemented now in a demonstrator project.
The project intends to achieve better networking between all participating organizations in order to support useroriented development and enable emergency forces to apply new and innovative satellite-based technologies as quickly as possible.
Cartographic products are one of the state-of-the-art communication media in disaster management. Based on the disaster type and management, the media has to fulfill different purposes. The most recent products are delineation ( Figure 1) and grading maps, resources maps as well as detailed object and area maps. The transition to a fully featured digital workflow is in a beginning phase supported by different disaster GIS applications.
3D maps are an emerging field to support the user in understanding and decision-making while providing or mimicking a more natural way of perception. 3D data supports rescue forces in deciding whether the terrain is accessible, for example, or whether there is a risk of further hazards, e.g. due to landslides. Dusse et al. (2015) investigated the question "How are Information Visualization tools supporting Emergency Management?" Apart from the differentiated nature of the possible incidents, mobile and web-based forms of presentation were found to be replacing desktop applications and gaining in importance.
Lanfranchi and Ireson (2009) differentiated the user requirements in terms of functional requirements like multimodal interfaces, content upload, information enrichment and non-functional requirements like trust, privacy, resilience, reliability for emergency response personnel and community citizens. The authors of the study investigated the importance of the different points for both groups. One result of the study was that the two groups differed primarily in the area of non-functional requirements. while almost all requirements were rated highly in the emergency response staff group, only the items speed and ease of use were given a high priority in the community citizen group. 7 https://business.esa.int/projects/tt-gsat

New Visualization Approach for Disaster Information
The current visualization approaches in disaster information communication are lacking flexibility and interactivity. Both aspects are inherently available by using up-to-date representation methods. As augmented and mixed reality is available for modern mobile devices the degrees of freedom in interactivity and flexibility can bring a new visualization experience to the stakeholders. The research workflow ( Figure 2) consists of four steps: (1) the description of the current situation, (2) a requirements analysis, (3) the development of a mixed reality app and (4) expert interviews concerning the developed mixed reality app.

Figure 2: Research Workflow
The chosen approach for the cartographic application was a combination of the 3D digital twin concept and mixed reality. Mixed reality technology as a standalone concept creates immersive environments and visualizations which are characterized by the presence of both physical and virtual items (Intel, 2021). The extra benefit of mixed reality is that the digital content is able to co-exist with objects in the real world (Valendu, 2018) and thus making it possible for the user to interact and manipulate these objects in a natural and realistic way. All of the mentioned key features in mixed reality give the opportunity to enhance the situational awareness in emergency management processes.

Requirements Analysis
To get a full picture of the everyday routine in emergency management, an online questionnaire was set up with the participants being able to answer questions over a 2months period. Giving credit to the various actors that can be associated with the domain, the survey population was chosen from a broad selection of professions. These included mostly public authorities, e.g. federal disaster and Consisting of four question blocks the online survey tried to gather information about how practitioners work, what types of maps and information technology they are using, what they consider the biggest problems and challenges at the moment and what future improvements they would benefit from the most. The structure of the survey aimed at a comprehensive understanding of the domain and its prerequisites. From all the responses, 30 were selected on the grounds of completeness with 95% or more answered questions.
One main takeaway from the survey highlighted the fact that there is a high improvement potential for cartographic products. The current overall quality of emergency management maps was rated 'very bad' by 87% of the population and the current information quality of those maps was evaluated with the same result by 83%. A dominant request by many participants was access to realtime disaster information (e.g. flood waves) and live data about objects (e.g. critical infrastructures). Pictures and videos from the scene were also often missing. Another main flaw perceived by many participants was the lack of trustworthiness and completeness of disaster information. Figure 3 shows the 5 main obstacles and problem areas named by practitioners in emergency management.

Case Study
As the practical output of this research, a disaster management application (DiMAN) was created by utilizing the emergency management practitioners' responses. The application consisted of two disaster event scenarios. One is located in Castelo Branco, Portugal, and shows large-area forest fires whereas the other one presents a city-scale earthquake situation in Istanbul, Turkey. Both scenarios were chosen to visualize the wide range of interactivity, flexibility and different LoDs (level 8 https://devblogs.microsoft.com/dotnet/c-9-0-on-the-record/ of detail) disaster maps could effectively address. Some details are not visible on all scales because of their limited scope but might turn out to be crucial for an effective response coordination. As a consequence, multiple LODs should be given the outmost attention when designing disaster maps. Figure 4 shows the main menu of the developed app to choose one of the two scenarios.

DiMAN Application
DiMAN is based on a tabletop mixed reality approach that visualizes the map on top of a table or other rectangular flat surface (e.g. floor) using the device's camera. The map is being generated by mixed reality technologies as a hologram that is anchored to the table. The primary goal of the application is to enhance the situational awareness of disaster management practitioners, specifically those working in control or crisis rooms.
Multiple technical elements were integrated and merged into a disaster application. The application is written in the programming language C# 8 and was tested and built in the Unity 9 environment. Essentially a game development platform Unity provides the functionality and experimental playground to develop mixed reality applications. Moreover, two main SDKs were utilized for the development: the Mapbox Maps SDK (Mapbox, 2021) for the tabletop mixed reality creation and the WRLD 3D SDK (WRLD3D, 2021) for the crafting of indoor maps. Finally, the application was wrapped up into an Android application to be executed on a smartphone device (Google Pixel 3).  information about the event. Below a small-scale overview map is placed. A home button in the bottom-left corner takes the user back to the main menu. On the top of the screen is the map legend and layer list where the user can hide and display the desired content. In the top-right corner the base map or a satellite-imagery can be exchanged via a drop-down menu. The L-shaped layout leaves a rectangular part of the screen in the bottom-right corner as the main window for the tabletop-mixed reality experience. All POIs in the map are clickable and ready to display pictures or video-livestreams from the scene plus additional relevant information. The weather button down right takes the user to the exact scenario coordinates in a professional forecast website.

Second Scenario: Istanbul, Turkey
On bigger scales, scenario II shows a fictional earthquake event that impacts on some of the core quarters in Istanbul, Turkey ( Figure. 6). The "city scale" allows for a visualization of point events that occur near each other. This map depicts buildings that have been affected by the earthquake and are damaged or destroyed. Besides, the map shows hospitals and safe zones for civilians to gather safely. The same functionality with clickable POIs like in scenario I is also accessible here. With various finger commands the user can navigate around the scene, pan and zoom to a certain location and retrieve information.

Results and Discussion
In the following paragraphs the conducted research will be summarized, the benefits as well as the limitations will be discussed with domain experts and put into perspective for possible future implementations of mixed reality in the field of emergency management.

Results
Both literature research as well as the requirements analysis indicate the unsatisfactory mapping solutions currently present in emergency mapping. Real-time topographic and thematic information needs to displayed in an organized and intuitive way to properly enhance the situational awareness of decision-makers when time is running. Multiple people should be able to interact with the map at the same time to facilitate problem-solving and the coordination of an effective response. The developed mixed reality tabletop application provides a solution based on the shortcomings of current disaster maps. It shows a dynamic, organized display of relevant geo-spatial data that the user can more intuitively interact with. Livedata, pictures or video-streams can easily be integrated and retrieved through user commands. Significant decisionmaking information can be presented in a more natural way and in different scales or LoDs. Finally, it is possible to extend the application and tailor it to the needs of a specific situation.

Discussion
For a more qualitative evaluation three different experts (Federal Office of Civil Protection and Disaster Assistance (BBK), Munich fire brigade, THW) were interviewed and asked to give their opinion about the application and the assumed benefits of mixed reality in the emergency management domain. All of the experts agreed that mixed reality substantially enhances the perception of space and therefore has the potential to raise the situational awareness of decision-makers. Interactive features like the retrieval of critical information through finger taps and the concept of real-time data integration in the 3D map was very much appreciated. Moreover, the simultaneous cooperation possibility that mixed reality provides when multiple actors stand around the tabletop map and are able to make edits or draw sketches was considered very useful. As for possible use-cases of the DiMAN application the most suitable ones emerging during the interviews were a scenario in crisis rooms ( Figure 8) and for education purposes, the situation training of disaster responders in particular. Many more potential use-cases for mixed reality applications in the domain were mentioned yet the current state of the emergency mapping infrastructure appears to be years away from the proper implementation of sophisticated cartographic models.  (2019) However, despite the many potential benefits some problems and limitations exist. On the technical side practical experiments point to the fact that a smartphone is not the best choice for a mixed reality tabletop map because it presents itself a bit bulky and unhandy. A headmounted device should give the user a more comfortable experience as he is not required to hold the device with his hands. He would be able to move around the table freely and use gestures to communicate with the application. In addition, the lighting conditions play a major role for the proper finding of a plane (here: table) and the correct visualization of the geospatial data on top of it.
Software improvements are necessary for a less errorprone application. Another problem might be spatial data availability as the degree of accuracy and up-to-date-ness of relevant spatial information is in most cases unsatisfying for a certain use-case scenario (e.g. outdated or not-digitized city plans). Sources are also often rare and usually don't match with each other. This leads to a significant amount of time used for data preparation.
On the political front the emergency management domain suffers from a general lack of geo-specialists and occasional funding, as a crisis does simply not happen every day. In Germany crisis teams are assembled nonpermanently only for a specific disaster event and are dissolved afterwards. Eventually, it should be mentioned that not all use-cases require a very detailed modelling.

Outlook
Due to its complexity, disaster management requires interdisciplinary cooperation in the fields of geoinformation, league representation, data acquisition, data transmission, communication and many other areas. Today's decision-makers demand a situation display that allows them to track the emergency forces in real time, query their readiness mode and also display the operational scenario as up-to-date as possible. This requires new ways of representing reality. Mixed reality and 3D approaches can therefore be a useful visualization tool for disaster management, as they simplify and enhance the representation of reality and allow the dynamic integration of data from different sources, allowing the viewer to more easily understand the data. A standardised mixed reality tabletop disaster model which can be easily adjusted to new scenarios and allows for smart mapping could boost the efficiency and the quality of response activities. The current trend towards combining maps and game engines (like Unity) will bring many more features and UI | UX improvements in the near future. All of this will not only magnify our perception of reality when interacting with cartographic mediums but also provide stunning new management and planning solutions for many domains. Future research should be trying to evaluate the many technical advances in the field of cartography and mixed reality of the last decade in order to implement practical solutions for real-world problems.