An Era in Jinling: Game Interactive Heritage Digital Art Exhibition

1. Introduction

Central to the museum industry and the overall audience experience lies the pivotal role of exhibitions.1 The various forms of exhibitions encompass diverse experiences and evoke distinct emotions within audiences. When visitors step into a museum or art gallery, they do so with a sense of anticipation, seeking a tangible encounter that immerses them in the present moment and physical space. They yearn to be inspired, intrigued, and intellectually stimulated, open to embracing novel perspectives and fresh ways of perceiving the world around them.2The advancement of digital technology has signicantly facilitated the preservation of cultural heritage and the emergence of digital exhibitions within museums, thereby transforming the exhibition landscape and narrative practices of museums and cultural institutions over the past two  decades. Traditionally, museum exhibitions have predominantly embraced an encyclopedic approach, relying on curatorial authority and physical objects to fulfill their educational mission  and provide visitors with a enriching learning experience.3The emergence of new digital narrative experiences is reshaping our under- standing of cultural heritage exhibitions and redefining the dynamics between cultural heritage, information, technology, and human consciousness. This expansion of possibilities has significantly enriched the narrative realm within thematic exhibitions. Through these innovative approaches, museums and art galleries have the opportunity to seamlessly blend the virtual and digital elements with physical artifacts, creating immersive and interconnected exhibition spaces.4 This emerging trend is accompanied by a growing recognition that museum life is embedded in the  audiences’ life of our digital ecosystem.5 This will allow the curatorial role to expand from the individual to the curatorial  team, including digital technologies, conservation, historical narratives, visual arts, and other interdisciplinary of collaborative approaches to curation. Designing a collaborative immersive digital art exhibition offers a captivating and eagerly anticipated approach to elevate the viewer’s digital exhibition experience. By actively involving the viewer in the core essence of the artwork and its whimsical gameplay, a dynamic sense of curiosity is ignited. This curiosity is driven by the desire to explore and unravel layers of meaning embedded within the artwork, whether it be in pursuit of deciphering the exhibition narrative or simply being present within the artwork  itself.6 The advantages offered by digital technologies in exhibition activities should be duly acknowledged and effectively integrated into museum exhibition practices to cater to the evolving needs of audiences in the digital era.7 Located in Jiangsu Province, China, the Deji Art Museum is a private, non-profit art institution founded by the Deji Group, with 9,ooo m²of  exhibition space. This study focuses on one of the permanent digital art exhibitions, “An Era in Jinling”Heritage Digital Art Exhibition, which occupies nearly 1,ooo m² of exhibition space and features a digital representation of the Chinese Song Dynasty The theme of the exhibition is a  digital representation of the city of Nanjing as it was more than 1,ooo years ago, as depicted in  the Chinese Song Dynasty scroll“Jinling” (Fig. 6). The exhibition’s curatorial team brings together professionals from multiple disciplines, including curators, historians,digital  technology engineers, and sound and visual designers. Current museum digital exhibitions are built on digitizing cultural heritage and combining it with physical cultural heritage to form a unified exhibition narrative space. Therefore, the curatorial team first used scanning technology to create artworks as digital archives and invited historians to delve into the architecture, humanities and costume styles of the Song Dynasty more than 1,ooo years ago through analysis of artworks from the same period. At the same time, the team collaborated with experts in 3D modeling and motion capture technology to digitally model the architecture and environment of the artworks in order to complete the visual image design of the city’s dynamic landscape, creating an immersive virtual space for the digital art exhibition of cultural relics. The exhibition highlights real-time interaction by simulating the dynamic, visual, and auditory experience of reality, exploring the playfulness of the exhibition and gaining historical knowledge. The innovation of“An Era in Jinling”Heritage Digital Art Exhibition is that visitors can create personal digital characters in the exhibition like a game, and let the digital characters and the viewer’s body dynamically synergize, stimulating the viewer’s dynamic curiosity through a game exploration The exhibition transforms art viewing into an active experience through the illusion of immersion8,which is the first time such a format has been used in a heritage exhibition to provoke reflection.

2. Background

Storytelling has been posited as a co-creative tool to increase visitor engagement and experience memorability.9Over the course of several decades, exhibitions have commonly employed encyclopedic approaches as a means to fulfill their educational objectives and offer audiences a comprehensive learning experience.10These exhibitions fail to acknowledge the significance of the viewer’s experience and their acquisition of information throughout the exhibition, thus restricting audiences to a passive observation of the artwork being displayed.11An increasing number of audiences are expressing criticism towards the conventional narrative approach that relies on textual explanations and audio aids, which tend to dominate exhibitions featuring static displays of artifacts.The digital age has had a profound impact on all aspects of society, and cultural heritage,as an important part of human history and heritage, has been equally affected by these changes.12The advent of digital technology and the abundance of cultural information have significantly transformed the approach to exhibition design in museums and galleries. This paradigm shift has given rise to new methodologies that embrace digital technology and multi-sensory experience design as innovative ways of narrating exhibitions.13 The emerging trend of technology-enabled narrative experiences incorporates a diverse range of visual and aural elements, including pictures, objects, sound, light, and advanced hardware such as holographic projections or touchscreens. These innovative advancements are transforming traditional storytelling by integrating immersive and interactive components,providing audiences with multi- sensory  engagements that enhance their overall experience,14 Facilitating the integration of digital artifacts with virtual technologies, this enables exhibition  spaces in museums and art galleries to seamlessly blend the physical and digital realms.15 These  alterations possess the potential to redfine the role of the viewer, enabling them to assume a more active position and engage in the co-creation or co-production of exhibitions by actively interacting with the exhibition environment and assuming a central role in the event.16 By  enabling audiences to transcend the monotonous and traditional encyclopedic narrative approach, the integration of digital technology opens up avenues for immersive storytelling, thereby transforming the viewer experience in heritage displays.

The exhibition holds a central position in museums, art galleries, and audience experiences, serving as the primary focal point. It is within the exhibition space that the curatorial team strives to create a multi-dimensional environment, collaborating to select imaginative solutions that effectively showcase the displayed pieces, convey the narrative of the exhibition, and seamlessly integrate art and technology. Moreover, with the advent of internet platforms, information and images related to exhibitions have become easily accessible to users, who can view, share, and perpetuate their impact on a broader audience. The exhibition hall serves as a communal space that bridges the gap between museums or art galleries and their audiences, enabling  visits and fostering interactions through diverse digital technologies. In the realm of heritage art exhibitions, an increasing number of digital exhibition approaches are being employed, incorporating elements such as video, virtual reality, technological tools, lighting, and sound. As these new technologies continue to deepen the viewer’s experience and reshape conventional under- standings of traditional heritage exhibition spaces, exhibitions are designed to provide immersive encounters. Nonetheless, amidst these evolving exhibition curation practices, the presentation of artists and their works remains a cornerstone of the twenty-first-century exhibition, redefining the exhibition experience space and reaffirming its significance.

The Covid-19 pandemic has accelerated the digital technology revolution, prompting individuals to progressively embrace a digital lifestyle. This lifestyle emerges from the convergence of daily living and digital technology, culminating in the creation of a novel digital art exhibition paradigm. At the core of such exhibitions lies the development of multi-sensory and immersive digital art show- cases, establishing a solid foundation for the integration of artistic expression with digital technology. Moreover, this phenomenon has led to the incorporation of digital art exhibitions  into people’s digital lifestyles. These exhibitions of digital art deviate from the traditional heritage exhibitions by transforming the standard textbook narrative into an immersive narrative. In addition to digitally capturing the physical appearance of artifacts, heritage digital art exhibitions provide diverse avenues for experiencing the conventional cultural relics exhibition. This includes engaging visitors through visual, auditory, tactile, and other sensory modalities, thus enabling a comprehensive understanding of the profound historical and cultural information associated with the artifacts. The audience is immersed in a multi-dimensional experience through the utilization of digital simulations. This paper aims to explore, through the analysis of case studies, the transformative impact of digital technology on the audience’s role within heritage exhibitions. By shifting the focus towards the audience experience, the introduction of digital technology has disrupted the traditional narrative and design framework of heritage exhibitions. Adopting a game interactive approach, the exhibition enhances the audience’s enjoyment and fascination by unveiling the aesthetic and historical aspects concealed beneath the surface of the artifacts. By seamlessly integrating digital technology, these elements become an integral part of the  immersive experience offered to the audience. Consequently, this work makes three contributions to research:(1) By examining the development of game- based interactive heritage digital art displays, it adds to our knowledge of heritage exhibi- tion design. (2) It provides an overview of the curatorial team’s exhibition design concept and concludes the practice of exhibit- ing heritage digital art in an interdisciplinary setting. (3) It offers fresh viewpoints on how heritage exhibits are changing in the context of the digital ecosphere.

Fig. 1. Conceptual framework of the experience.

 

3. Methodology

The embodied human experience can be conceptualized as a dynamic interaction that takes place between our physical body and the surrounding environments, which can be characterized by four essential dimensions: time, space, actors, and the presence of various entities,   including the natural world.17 Digital technology thus provides the opportunity to display a full or incomplete recreation of the four dimensions of the human life experience. A conceptual frame- work for the experience brought about by a subject’s interaction with the world is shown in Fig. 1, which is itself defined in four dimensions (time, space, actors, and artifacts).

Based on the conceptual model of experience outlined above, the integration of serious education with entertainment, inter- action, play, and participation has the potential to transform the role of museums and art galleries. This integration extends the four dimensions of the experiential framework to digital art exhibitions of heritage, fostering a dynamic and engaging environment. By combining elements of education and entertainment, facilitating interaction and play, and encouraging active participation, museums and art galleries can redefine their purpose and enhance visitor experiences in the digital age. When applying the previously mentioned conceptual framework of experience to the exhibition conceptual framework of a digital art exhibition focused on heritage, certain modifications are necessary. This innovative exhibition concept comprises four essential components: exhibition space, artwork (artifacts), interaction, and a digital hardware system, all of which revolve around the central element of the audience’s experience.Within the exhibition space, which encompasses the area where the art-works are showcased, the audience shares a common experiential realm with the museum or art gallery. The primary element of the exhibition presentation is the artwork, which serves as the interface through which the audience engages with the exhibition environment. In the case of the interactive heritage digital art exhibition, the objective is to transform the traditional experience of consuming an encyclopedic narrative. This innovative exhibition employs a multitude of interconnected systems, which operate in collaboration with the digital hardware system. This system serves as the foundation for the exhibition management system, which operates behind the scenes in the exhibition area. Fig. 2 depicts a conceptual framework in four dimensions,with a central focus on enhancing the audience experience.The framework presented here emerges from the interconnectedness between the audience and exhibition experiences within the context of a heritage digital art exhibition. This framework encompasses four key dimensions, namely the exhibition space, artifacts, interaction, and digital hardware system.  The ‘An Era in Jinling’ Heritage Digital Art Exhibition centers around the viewer experience,  driving the curatorial intention to create a multi-dimensional immersive virtual environment. By seamlessly integrating the exhibition space, artifacts, interaction, and digital hardware system, this exhibition strives to deliver a traditional exhibition that not only imparts a profound educational impact but also takes into consideration the aspects of entertainment and interactive comprehensive exhibition design. The collaborative efforts of the team involved have resulted in the development of a conceptual model for the heritage digital art exhibition.

The collaboration dynamics between museums and art galleries have undergone a transformative shift, transitioning from an authoritative curatorial approach to a focus on immersive techniques that actively engage audiences through an experience-cen-tered paradigm. However, this transition presents signficant challenges for curators, as they grapple with technical intricacies related to exhibition design, such as lighting considerations for video and digital art, spatial arrangements, and the integration of interactive components. Particularly, the curator’s role becomes increasingly complex when dealing with the convergence of digital visual art, mixed media art, and exhibition hardware systems.18

4. Design solutions

A multimodal immersive exhibition experience makes interdisciplinary cooperation inevitable. This set of research also extensively acknowledge it.19 The core curatorial team has not grown as much as anticipated, but the interdisciplinary tasks necessary for exhibition design and production can already be completed with the assistance of other teams. At various phases, experts from many fields work as a curatorial team.20 To establish an immersive exhibition environment encompassing both digital and physical elements, a meticulous design approach was imperative. This encompassed various aspects, including the hardware system, light- ing arrangements, visual imagery, and spatial acoustics. Furthermore, a digital exhibition intelligent management platform was implemented to facilitate data analysis and oversee the exhibition’s operations comprehensively. The collaborative involvement of external professionals throughout the creative process not only materialized the curatorial team’s vision but also afforded them  an active role in diverse aspects of design and production. Consequently, this collaborative approach catalyzed a paradigm shift in the curatorial team’s thinking, fostering innovation and creativity.21 The curatorial team demonstrated innovation by integrating virtual digital characters into the digital exhibition, deviating from traditional approaches to digital exhibition design. Taking inspiration from game designers, the concept enables audiences to independently  create virtual digital characters using their portable smartphones, thus immersing themselves within the digital exhibition space and receiving an intelligent tour simultaneously. To enhance this experience, audiences are provided with UWB positioning bracelets as part of the exhibition, enabling the virtual digital characters to move in real-time synchrony with the audience’s  location. This unique approach allows audiences to engage with the exhibition as if participating in an online game, while simultaneously gaining insights into the historical significance of the featured works. The curatorial team meticulously devised intricate design solutions centered around the exhibition’s objects, drawing inspiration from the conceptual model presented in Fig.2. of the heritage digital art exhibition. Aimed at fostering audience engagement, the ‘An Era in Jinling’Heritage Digital Art Exhibition was developed across four dimensions: exhibition space,interaction, digital hardware system, and artwork (artifacts). Through this process, the curatorial team dedicated considerable effort to refine the exhibition’s design. Consequently, they advanced the theoretical and conceptual framework for exhibition design,ultimately proposing the exhibition design model illustrated in Fig.3.

Fig. 2. Conceptual model of Heritage Digital Art Exhibition.

 

4.1. Exhibition space

The audience’s experiential journey is profoundly shaped by the exhibition space, as it acts as a communal realm where interactions occur between the audience and the museum or art gallery. Fig.4illustrates the layout of the exhibition area, which has been delineated by the curatorial team into on-site and off-site sections for this specific exhibition. The offsite design encompasses the wall adorned with Jinling characters, serving as an extension of the exhibition space. On the other hand, the on-site section is further divided into three distinct areas: visual images, soundstage, and lighting, each catering to specific aspects of the immersive experience.The curatorial team drew up the floor plan of ‘An Era in Jinling’Heritage Digital Art Exhibition (Fig.5 ) based on the design of the exhibition space, with five areas: (1)Wall of Jinling characters, (2) Samrt wristband, (3) Locker, (4) Replica display or‘An Era in Jinling’, (5) Restroom. Wall of Jinling characters is off-site design of exhibition space; Replica display or‘An Era in Jinling’is the display area for artifacts;Samrt wristband is the area for audiences to receive UWB positioning bracelets;Locker and Restroom is a functional area for audiences.

Similar to a movie poster in a window, the space beyond the exhibition area serves as a tantalizing glimpse into the contents, arousing the curiosity of the audience. The design of this off-site space is illustrated by a collection of figures displayed on a wall. To develop the exhibition’s narrative, the curatorial team collaborated with a group of experts and drew inspiration from an ancient painting from Jinling. Through a meticulous comparative analysis of contemporaneous heirloom paintings, unearthed artifacts, and documentary materials, the narrative was meticulously crafted. “An Era in Jinling”is an exceptional historical artwork dating back to the Song Dynasty, created by a Chinese artist, showcasing the city of Nanjing over a thousand years ago. Fig.6. showcases the only surviving replica, painted by a court painter of the Qing Dynasty in 1794.

The 533 figures The figures are shown in Fig. 6, which is a miniature model of ancient figures placed at the entrance of the exhibition hall. The miniature figures of various shapes and sizes are taken from ancient paintings, modeled in D and then D printed. The audiences can see the wall of figures built by hundreds of models to enhance their interest in the exhibition.

The spatial design of the in-site encompasses a harmonious integration of lighting,soundstage, and visual imagery. Particularly,the visual images take the form of a captivating Moving Scroll titled ‘An Era in Jinling’(Fig. 10), which can be likened to a dynamic digital panorama capturing the essence of the city. Leveraging motion capture technology, the curatorial team has  transformed 533 figures from the original painting into animated sequences that faithfully  represent their respective roles. Consequently, the in-site’s space design becomes an amalgamation of visually immersive imagery, enveloping soundscapes, and carefully orchestrated light- ing. The captivating portrayal of ‘An Era in Jinling’ Moving Scroll (Fig. 1o) serves as a vivid depiction of the city’s historical era in a digitally animated format. From the electronic  questionnaire Q3,  we found that 82.4% of the audience were satisfied with the design of the dynamic virtual city scenery, which changed from a flat at painting to a vivid digital image, allowing the audience to feel the urban appearance of Nanjing more than 1,ooo years ago from  the dynamic relationship between people and objects, vividly restoring the ancient people’s  life, and many of them felt they were in the historical time and space to feel the atmosphere of the city. The curatorial team based on the professional status and period etiquette of the characters in the paintings are shown in Fig. 7. The team used motion capture technology to recreate a moving image of the 533 characters in the painting as real person.

Fig. 3. Exhibition theory design model.

 

In the subsequent phase, the architectural elements, surrounding environment, and narrative of the paintings will undergo digitization, alongside the development of a visually captivating representation of the city’s dynamic terrain by proficient technical experts specializing in D modeling and motion capture. It is important to note that the digitization process employed in the‘An Era in Jinling’Heritage Digital Art Exhibition distinguishes itself from conventional heritage exhibitions through its ability to translate the artistic and historical narratives of the showcased exhibits into immersive digital visual outputs,thus affording each visitor a personalized and tailor-made viewing experience. Instead, the project endeavors to recreate the authentic and dynamic living environments of characters from the Song Dynasty. It aims to construct an interactive and immersive intelligent heritage digital art exhibition that vividly portrays the flourishing and vibrant urban landscape of that era.Esteemed experts and scholars have undertaken comprehensive comparative studies encompassing diverse disciplines such asancient Chinese painting and calligraphy,local history of Nanjing, economic history,agricultural and handicraft history, architectural history, transportation history, costumes, music, and period etiquette.Additionally, contemporary heirloom paintings have been meticulously integrated intothe exhibition, culminating in a visually captivating experience that allows the audience to immerse themselves in the bustling city of Nanjing over a millennium ago during the Song Dynasty. The exhibition’s narrative is enriched through the discovery of numerous historical details, which include the depiction of Nanjing’s urban environment during the Song Dynasty, the identification of professional roles of each figure depicted in the image, and the representation of the commercial activities taking place within hundreds of stores. The digital landscape of the city is divided into three main regions: the countryside, the bustling market, and areas removed from the central action. To create a dynamic digital representation of this landscape, the elements from the three sections of the antique paintings are meticulously extracted and reconstructed in a three-dimensional format (see Fig.8).

Fig. 4. Design of the exhibition space.

 

Fig. 5. Exhibition Plan.

 

Viewers of the‘An Era in Jinling’Moving Scroll (Fig.9) will immediately observe the dynamic portrayal of countryside vistas, bustling markets, and serene retreats depicted in the artwork, all brought to life through the captivating motion displayed on the screen.This immersive experience unfolds across a massive LED screen, measuring meters in length and.meters in height, presenting an entire city’s vibrant digital landscape.Engaging with the installation, the audiencebecomes immersed in a vivid urban panorama, witnessing animated scenes of shopkeepers engaging with customers, city dwellers going about their daily lives,skilled craftsmen diligently plying their trades, horse-drawn carriages gracefully traversing the streets, and breathtaking outdoor landscapes that evoke a sense of grandeur and awe.

Fig. 6. An Era in Jinling by Feng Ning (Replica) Qing Dynasty, dated 1794 A.D. Handscroll, ink and color on paper
(Length: 35 cm, Width: 1050 cm).

 

The“An Era in Jinling”Heritage Digital Art Exhibition aims to create an immersive environment resembling an ancient city,providing audiences with a panoramic, multisensory, and multi-dimensional sensory experience (Fig.10). With the realization of the interactive feature of“characters into the paintings, real-time interaction,”the inclusion of an immersive sound field area within the exhibition hall became crucial.The implementation of a surround sound system with fixed duration and content,while considering the audience’s physical location, enables sound to be precisely transmitted to specific positions. However, due to the dynamic nature of the exhibition where the audience’s perspective constantly changes,the conventional concept of “sound finding people” is impractical. Therefore, it was necessary to innovate and achieve a concept of “people finding sound”through technological advancements. By employing a revolutionary positioning technique and spatial audio design, Ultra-Wideband (UWB) positioning bracelets can be utilized to track the movements of digital characters and activate the corresponding sound matrix, ensuring that the audience remains in the optimal listening position throughout the exhibition.

Fig. 7. Sculptural models of 533 3D figures in the exhibition.

Fig. 8. Capturing real person 3D modeling.

The exhibition offers the audience the freedom to explore at their own pace, and as their path evolves,the digital characters within the exhibition also adapt their movements and decision-making processes accordingly. Consequently, each visitor’s experience in the exhibition hall becomes a blend of shared encounters and distinct, personalized moments. It became apparent that a singular,prerecorded audio loop was inadequate to fulfill this requirement, as it failed to cater to the diverse and evolving needs of individual audience members.

In response to these specific requirements,the curatorial team devised a distinctive spatial audio model, effectively dividing the exhibition hall into sound matrices,thereby transcending the constraints imposed by conventional digital exhibition sound field design principles (Fig.11). The background music of the exhibition is composed of these sound matrices, while the exhibition’s intelligent management platform generates real-time sound effects as digital characters interact with non-player characters (NPCs). This dynamic interplay of sound components allows the audience to fully immerse themselves in a captivating and ever-evolving auditory backdrop, enabling them to concentrate on a clear and immersive audio-visual experience.

For the audience’s experience from the immersive sound field space, we did experience data collection in the electronic survey questionnaire Q1. 87.6% of the audience held a positive evaluation of the immersive sound field experience, while12.3% of the audience held a negative evaluation, the audience did not have a strong enough feeling of sound switching when moving through differentexhibition spaces, due to the need to wear headphones at the same time to obtain the digital game character’s Because of the need to wear headphones at the same time to obtain the sound cues of the digital game characters, thus ignoring the background sound effects in the exhibition space, resulting in the interference of the exhibition sound experience.

Fig. 9. The 3D model of people, buildings and environment in the dynamic digital view of the city

The lighting design of the exhibition space incorporates a sophisticated lighting system consisting of a multitude of softly illuminated lamps, as depicted in Fig. 12. To recreate the ambiance of an ancient city, vibrant yellow paper lanterns shaped like traditional lanterns were employed. These luminaires are strategically suspended throughout the exhibition space, encompassing the entirety of the audience’s path. To ensure optimal focus on the exhibited images, the top portion of the roof is coated with a dark, light-absorbing material, while the overall illumination from the luminaires is intentionally dimmed.97.9% of the audience hold positive comments on the lighting effect design of the exhibition space, but the audience tend to ignore the design of the exhibition lighting effect in the exhibition, but the lighting system has a key role in the perfect presentation of the exhibition effect, so it cannot be ignored by the curatorial team.

Fig. 10.‘An Era in Jinling’ Moving Scroll.

Fig. 11. Sound matrix design scheme.

 

4.2. Interaction

The ‘An Era in Jinling’ Heritage Digital Art Exhibition is to provide audiences a game- interactive viewing experience, turning the encyclopedic narrative of a museum or gallery into an  entertaining and engaging format. The ‘An Era in Jinling’ Heritage Digital Art Exhibition uses digital technology to recreate the urban environment and social history of Nanjing during the Song Dynasty in China 1,ooo years ago, with the goal of interpreting the cultural significance behind heritage. Audiences to the exhibition can create digital avatars on their cellphones without using VR equipment, enjoy a first-person viewpoint in a historically accurate exhibition environment, and play a game interactive to have an immersive digital viewing experience (Fig. 13). To complete the immersive game interactive experience, a virtual digital character enters the virtual exhibition space in place of the audience members’ actual bodies.

Audiences can build a virtual digital character and join the dynamic digital landscape of the city by scanning the official QR code of the exhibition with the smartphone they are carrying and entering the exhibition’s theme APP (Fig. 14). From the Q5 of the electronic questionnaire, it was found that 91.7% of the audience held a positive view of approval. The viewers think that such a game way of viewing the exhibition is not experienced in other digital exhibitions, which is much more vivid than the passive feeling of textbook style. It is much more vivid than the passive experience of textbook style. Letting the exhibition proceed in a game experience mode and designing many hidden narrative experiences in the plot for the audience to find out by themselves adds to the fun of the exhibition. To provide a real-time gaming interactive experience, a virtual character serving as the audience’s digital carrier would enter a dynamic scroll and allow the audience’s ‘embodiment’ to enjoy the sights within the painting.

The exhibition utilizes Unity engine technology for real-time rendering for the first time, which is a digital technology extensively utilized in game creation, VR/AR, and other sectors. This allows the audience to truly ‘Walking into the scroll’. With the advent of this feature, users will be able to generate and choose characters for their own virtual video games. In order to increase the fun and interactivity of the exhibition, 25 NPCs are included to the dynamic digital characters, and when the audience roams in the digital art exhibition and meets these 25 ‘mysterious characters’ carrying hidden knowledge through the smart guide app, the audience can listen to the ‘Living people in the paintings’. This interactive, self-determined way of viewing the content of the exhibition is like a game to get the exhibition narrative and novel experience.

Fig. 12. Lighting design scheme.

To achieve the immersive experience of the audience “walking into the scroll” and subsequently engaging in a first-person “roaming”experience, it is crucial to accurately record and position  the physical location of the audience and their viewing path in real time. This necessitates the implementation of precise and real-time indoor positioning technology.

The curatorial team adopted the UWB high-precision positioning technology after several failed efforts. Radio frequency (RF) technology called UWB Positioning Systems is now employed for accurate indoor positioning. The virtual digital avatar can move in real time with the audience’s  physical location by donning a UWB positioning bracelet (Fig. 14). When the audience moves to a new area, the system automatically recognizes the positioning bracelet and determines its coordinates to update the digital character’s position.Four signal points are enough to track hundreds of positioning bracelets in a space with line-of-sight (LOS) conditions (about 6 × 4 square meters). It is possible to accommodate numerous audiences at once and use algorithms like multi-point positioning to determine their locations for in-the- moment engagement after  enough signal points have been set up. UWB indoor positioning systems are usually employed  in sizable warehouses. Originally utilized in mine exploration and military missile place- ment, this technology can attain single-digit centimeter precision on-site.Through the electronic questionnaire Q4 found that the audience for wearing electronic bracelets at first feel new. At the same time, 72.1% of the audience think that the comfort is better than the previous wearing   immersion helmet, this design is positive attitude. However, 27.8% of the audience thought that the UWB electronic positioning bracelet was not sensitive enough, especially when the audience gathered in a certain exhibition area at the same time, data transmission and positioning delays and lags, resulting in a negative feeling of the viewing experience.

Fig. 13. Audience using UWB positioning bracelet.

Fig. 14. Audiences use their smartphones to create virtual characters.

Upon entering the exhibition premises, visitors can obtain a UWB electronic positioning wristband by scanning the official QR code with their smartphones. This initiates a  synchronized data connection between the visitor’s phone, the positioning bracelet, and the exhibition system. Through the exhibition’s dedicated mobile application, accessed by scanning a QR code with their smartphones, visitors can create personalized digital avatars within the exhibition’s digital realm, reminiscent of a gaming experience.

As visitors approach the designated area in front of the exhibition screen where the data link induction takes place, their digital avatars generated on their mobile devices seamlessly integrate into the exhibition screen.This is made possible by the real- time positioning capabilities of the positioning bracelet and the exhibition system’s synchronized data transmission. Consequently, the digital avatar on the exhibition screen mirrors the movements of the visitor in real- time, ensuring a dynamic and immersive experience.

Over 20 non-player characters (NPCs) engage the audience in a first-person narrative, sharing insights into Nanjing’s history and the grandeur of the Song Dynasty depicted in the painting. These NPCs serve as direct sources of information about Nanjing when engaged by the audience. Upon interacting with these characters, audiences are rewarded with “Nanjing Coins” in their accounts, signifying the unlocking of knowledge points. As a way to celebrate their acquisition of fascinating knowledge, audiences have the ability to set off fireworks any-where on the display screen using their “Nanjing Coins”. Both the audience and digital characters navigate through the digital exhibition area concurrently, creating a multidimensional immersive experience that transcends time and space.

4.3. Digital exhibition intelligent management platform

The ‘An Era in Jinling’ Heritage Digital Art Exhibition system is effectively administered by an intelligent management platform specifically designed for digital exhibitions. This platform oversees the seamless operation of five additional systems, namely: (1) the sound matrix system, (2) the urban dynamic digital graphics system, (3) the smart phone running exhibition APP system, (4)the UWB positioning bracelet system, and (5) the Virtual digital character interaction system. Serving as the ‘brain’ behind the Heritage Digital Art Exhibition, the digital exhibition intelligent management platform assumes the vital role of aggregating data and information generated by the exhibition audience. Through the implementation of various visual, auditory, and other perceptible techniques, this platform is responsible for creating a multi-dimensional immersive exhibition environment, wherein visitors can effectively immerse themselves and experience a vivid sense of ‘living’.

4.4. Artifacts

In the realm of artworks, a replica display titled ‘An Era in Jinling’ (Fig. 15) is prominently  featured. Following an immersive exploration of the complete digital scroll, the audience is led  towards a conventional display case thoughtfully positioned near the exhibition space’s exit. Within this case, Fig.15 provides a close-up view of a paper replica depicting ‘An Era in Jinling’. This exquisite artwork transports the audience back in time, offering a glimpse into the rich history of Nanjing, China, spanning over a thousand years. By transitioning from the digital realm to this tangible representation, audiences are  granted the opportunity to revisit this ancient painting. This shift liberates them from the confines of dry and abstract textual explanations commonly found in heritage digital art exhibitions. Instead, it allows for a more profound comprehension and reflection the artistic and historical significance of the work itself, fostering an immersive exhibition experience.

5. Results and discussion

Transforming an exhibition from a traditional passive, textbook narrative to one that engages audiences and places them at the center of attention, using an immersive approach and interactive storytelling through games, can create memorable experiences and redefine the audience experience.22 To assess the satisfaction of the exhibition, 2oo randomly interviewed audiences at the exhibition exit after viewing the exhibition live and using a Likert-type scale from 1 to5: 1 (strongly disagree), 2 (disagree), 3 (neutral), 4 (agree) and 5 (strongly agree) to collect audience feedback. The questionnaire was designed with 30 questions that were based on the exhibition satisfaction feedback. This study invited 6 experts in the field to discuss and select 10 key questions out of the 3o questions to be used in the questionnaire. These 10 questions are: Q1 (How do you feel about the background sound of the exhibition?), Q2 (How do you feel about the lighting effect of the exhibition space?), Q3 (How do you feel about the visual effect of the dynamic digital images of the city on the exhibition screens?),Q4 (What is your impact on the use of devices such as location bracelets and cell phones in exhibitions?),Q5(What is your assessment of the game like creation of digital bodies in the exhibition inter- acting with the exhibition?), Q6 (How do you feel about the immersive experience in viewing the exhibition?), Q7 (Do you think the audience,s experience is the most important in digital art exhibitions?), Q8 (Do you feel that the experience of visiting a digital art exhibition enhances the space for dialogue between the viewer and history and culture?), Q9 (Do you feel that digital art exhibitions provide a deeper knowledge of history than traditional heritage exhibitions?), Q10(Your comment on the overall design of this digital art exhibition?).

Fig. 15. Replia Display of ‘An Era in Jinling’.

After the questionnaires were designed, the audience who had seen the exhibition on site were randomly surveyed. In total, 2oo audiences were interviewed, respondents met three inclusion criteria: (1) they had viewed a digital exhibition in a museum or gallery within the last year; (2) they had a basic understanding of the differences between digital and traditional exhibitions; (3) they completed the transfer questionnaire. With the three inclusion criteria met, 97 valid questionnaires were collected, for a total of 97o valid data. The satisfaction data for each question is presented in turn in Table 1.

From the results of the 1o questions presented in Table 1, it is clear that experiencing the digital art exhibition in a playful and inter- active manner allowed these audiences to rate the exhibition as ‘amazing,, ‘I feel like I have traveled back in time, (Table1, question eight), ‘It,s like the artifacts are alive, and ‘So much historical knowledge is hidden in a painting, (Table 1, question nine),  some people will have Some people felt that they could not finish the exhibition. Many audiences felt like they were using their smart- phones to create their own virtual digital characters like a  game (Table 1, question five), which moved with them and viewed the exhibition as if they were physically entering a virtual digital world. The audience feel svery surprised when they talk to the 25 NPCs in the paintings, and these conversations cover a wealth of historical knowledge. The background sound of the exhibition sounded very realistic (Table 1, question one), making the audience feel like they were in the city of Nanjing in the Song Dynasty. When audiences are confronted with the paper work again after viewing the digital exhibition, they will reinterpret the painting from a different perspective, attempting to uncover details from the painting that match the digital exhibition. Five of these ten questions, Q3, Q5, Q6, Q7, and Q9, did not have negative ratings, and these five questions responded to the visual experience of the digital moving image of the city covered by the digital exhibition, the interactive experience of the game characters, the immersive experience, the importance of the viewer experience, and the effect of gaining historical knowledge compared to the traditional exhibition. Q8 and Q1o focus on collecting feed- back on the overall exhibition experience. Since most museum exhibitions are still presented in narrative modes such as physical objects and photographic materials, it takes some time for the audience to adapt to the acceptance of digital exhibitions. As the narrative object of this exhibition is a single long axis scroll, it cannot be compared with a comprehensive thematic exhibition in terms of narrative volume, which limits the information volume of the exhibition narrative, and the deep excavation of the narrative construction requires the joint efforts of the curatorial team and different professionals. At the same time, as the cost and curatorial workload of digital exhibition is much higher than that of a single physical exhibition, the audience is confronted with a rich sensory experience in a small volume of digital  exhibition to test the collaborative efforts of the curatorial design team.

Table I. Audience perceptions of digital art exhibitions.

 

Through a survey, 97o valid assessment data were gathered, which is a compilation of 97 valid surveys in which the audience rated the 1o questions in Table 1. The 97o valid data were  compiled and analyzed using the Sankey diagram format. The audience’s overall judgment of the topics covered in the exhibition can be seen from the situation shown in Table 2: strongly dis- agree value (14), disagree value (34), neutral value (2o9), agree value (488) and strongly agree value (225). The three components of ‘neutral’, ‘agree’, and ‘strongly agree’ received the majority of attention in the evaluation data.

Following the gathering of audience feed- back through a questionnaire, six experts from related fields were once more gathered for interviews. These interviews were based on the participants’ perceptions on the ‘An Era in Jinling’ Heritage Digital Art Exhibition. All names have been changed to pseudonyms in order to increase the informational value of the conclusions drawn from the  interviews and safeguard the respondents’ privacy.One Chinese expert on heritage conservation (Li) expressed support for the exhibition among the expert opinions gathered and emphasized the necessity for modern museums and art galleries to adopt an integrated and inclusive exhibition approach. The exhibition’s story has to be integrated into and become a part of people’s digital life given the rapid advancement of virtual reality technology and the expansion of human digital activity under the impact of COVID-19. Jin, a researcher from Korea, also said that whether an exhibition is at a museum or an art gallery, its educational components cannot be disregarded. The usage of game interaction is a novel invention that needs to replace the serious teaching approach used in conventional displays. The necessity to combine and express multidisciplinary projects like museum space, heritage conservation,digital technology,and exhibition hardware system in the same exhibition space, which places more demands on curators and curatorial teams, was also brought up by a scholar from Japan.

6. Conclusion

Today, museums and art galleries must reevaluate how to design an exhibition from an audience-centered one to one that is object- centered. As digital technology continues to advance, it is becoming increasingly common to employ it for historical exhibition activities at museums. In the current dynamic environment, however,museums face the challenge of innovating and adopting technology and integrating arts education  with market trends.23

It is preferable to establish digital art shows that prioritize an audience-centered approach, placing emphasis on providing a three-dimen-sional immersive experience and utilizing digital technology that enables independent audience engagement. Through the integration of immersive technologies, these exhibitions have the ability to fully immerse spectators within their unique settings and narratives. Rather than relying solely on technology, digital art exhibitions leverage multidisciplinary approaches to create a multi-dimensional exhibition experience. By actively participating in this sensorially rich display, the audience is encouraged to contribute their insights, further enriching the significance of the exhibition.

Table II. Comprehensive analysis of the data returned by the audience.

 

 

The contribution of this paper lies in its examination of the ‘An Era in Jinling’ Heritage Digital Art Exhibition as a case study. It introduces an interactive approach to experiencing heritage by  incorporating games, placing emphasis on the audience, and enabling them to actively engage with the digital exhibition through the creation of digital virtual characters, all while avoiding the  constraints of head-mounted display devices. By employing these digital characters, the exhibition moves beyond the conventional encyclopedic narrative approach typically found in traditional heritage exhibitions, offering an exciting and highly anticipated means of enhancing the audience’s digital exhibition experience. These digital virtual characters seamlessly integrate into the digital exhibition space, providing an intuitive and immersive interaction that is not hindered by the limitations imposed by head-mounted devices.

It can be inferred from audience comments and expert conversations that digital art exhibitions have become a staple of modern life and that the future of curation in museums and art galleries will be audience-centered and audience-involved digital exhibition design for heritage. Reference-worthy digital art exhibits include audience participation, game interactives, and immersive transdisciplinary history.

 

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1.Tula Giannini,Jonathan P.Bowen, “Rethinking Museum Exhibitions:Merging Physical and Digital Culture-Past to Present”,Museums and Digital Culture:New Perspectives and Research,2019,pp.163–193.https://link.springer.com/chapter/10.1007/978-3-319-97457-6_8.

2.Tula Giannini,Jonathan P. Bowen,“Curating Digital Life and Culture:Art and Information”,Electronic Visualisation and the Arts,2016,pp.237–244.https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2016.46.

3.Lizzie Muller,Ernest Edmonds,Matthew Connell,“Living Laboratories for Interactive Art”,CoDesign, No 4,2006,pp.195-207.https://www.tandfonline.com/doi/abs/10.1080/15710880601008109;Qi Wang,Yanhui Lei,“Minds on for the Wise:Rethinking the Contemporary Interactive Exhibition”,Museum Management and Curatorship,No.4,2016,pp.331-348.https://www.tandfonline.com/doi/abs/10.1080/09647775.2016.1173575.

4.F.Irace,G.L.Ciagà,Design&Cultural Heritage,Milano,2013.

5.Giannini and Bowen,2019, pp.163–193.

6.Josephine Machon,ed. Immersive Theatres:Intimacy and Immediacy in Contemporary Performance,Bloomsbury Publishing,2017.

7.Yuebo Yu,“Big Data Technology in Museum Exhibition Digitization”,Journal of Physics:Conference Series,No. 4, 2020. https://iopscience.iop.org/article/10.1088/1742-6596/1648/4/042044/meta.

8.Edwina Bartlem,“Reshaping Spectatorship: Immersive and Distributed Aesthetics”,Fibreculture,7,2005.http://www.immersence.com/publications/2005/2005-EBartlem.html;Marianne Foss Mortensen,“Designing Immersion Exhibits as Border-Crossing Environments”,Museum Management and Curatorship,No3,2010,pp.323–336.https://www.tandfonline.com/doi/abs/10.1080/09647775.2010.498990.

9.Maria Manuela Guerreiro Ana Cláudia Campos, Meghan C.Beevor,“Storytelling in Heritage Tourism:An Explorationof Co-Creative Experiences from a Tourist Perspective”, Museum Management and Curatorship,2023.https://www.tandfonline.com/doi/full/10.1080/09647775.2023.2230194.

10.Lizzie Muller,Ernest Edmonds,Matthew Connell,2006,pp.195–207; Qi Wang, Yanhui Lei,2016,pp.331–348.

11.Marcello Carrozzino, Massimo Bergamasco, “Beyond Virtual Museums: Experiencing Immersive Virtual

Reality in Real Museums”, Journal of Cultural Heritage, accessed 4 November 2010. https://www.sciencedirect.com/science/article/abs/pii/ S1296207410000543.

12.Alberto Garlandini, “Museums and Heritage in the Digital Age. The Challenge of Cultural Change and Technological Innovation”, SCIRES-IT-SCIentific

RESearch and Information Technology, No 1, 2021,  pp. 11–18. http://www.sciresit.it/article/view/13568.

13.Federica Dal Falco, Stavros Vassos, “Museum Experience Design: A Modern Storytelling Methodology”, The

Design Journal, 6 September 2017. https:// www.tandfonline.com/doi/abs/10.1080/

14606925.2017.1352900.

14.Qi Wang, Yanhui Lei, 2016, pp. 331–348.

15.F.Irace, G. L. Ciagà, 2013.

16.Lizzie Muller, Ernest Edmonds, and Matthew Connell, 2006, pp. 195–207; Pamela Barnes, Gayle McPherson,  “Co-creating, Co-producing and Connecting: Museum Practice Today”, Curator: The Museum Journal, No 2, 2019, pp. 257–267.

17.J. Dewey, Art as Experience, New York, 1934;M. Merleau-Ponty, Phenomenology of Perception, New York, 1962.

18.Tula Giannini, Jonathan P. Bowen, 2019, pp. 163–193.

19.Sue M. Davies, “The Co-production of Temporary Museum Exhibitions”, Museum Management and   Curatorship, No 3 (2010), pp. 305–321. https://www.tandfonline.com/doi/abs/10.1080/09647775.2010.498988; Lærke Mygind, Anne KahrHällman, Peter Bentsen, “Bridging Gaps between Intentions and Realities: A Review of Participatory Exhibition Development in Museums”,Museum Management and Curatorship, No 2 (2015), pp. 117–137. https://www.tandfonline.com/doi/abs/10.1080/09647775.2015.1022903; Anne Rørbæk Olesen, Nanna Holdgaard, Ditte Laursen, “Challenges of Practicing Digital Imaginaires in Collaborative Museum Design”, CoDesign, No 3, 2020, pp. 189–201. https://www.tandfonline.com/doi/abs/10.1080/ 15710882.2018.1539109.

20.Zoi Popoli, Izabela Derda, “Developing Experiences:Creative Process Behind the Design and Production of Immersive Exhibitions”, Museum Management and Curatorship, No 4, 2021, pp. 384–402. https://www.tandfonline.com/doi/full/10.1080/09647775.2021.1909491.

21. Zoi Popoli, Izabela Derda, 2021, pp. 384–402.

22. Zoi Popoli, Izabela Derda, 2021, pp. 384–402.

23. Indria Handoko, Adi Nugroho, “The Adoption of Virtual Technology in Museums: The Case of a Modern and Contemporary Museum in Indonesia”, Museum Management and Curatorship, 2023. https://www.tandfonline.com/doi/full/10.1080/ 09647775.2023.2225167.

 

Summary
The ‘An Era in Jinling’ Heritage Digital Art Exhibition, which focuses on the interactive virtual reality game environment created by
the audience’s experience, is an immersive digital art exhibition composed of the audience, the hardware system of the digital exhibition environment and the virtual digital environment presented. As a game-interactive immersive digital art exhibition in a heritage exhibition, it invites the audience to take apersonal digital role in the exhibition and use the game-interactive method to enjoy an immersive digital exhibition. This paper aims to engage audiences through the ‘audiencecentric’ exhibition of digital art, allowing them to use their senses to enjoy immersive exhibitions of digital art. This paper also demonstrates how digital art exhibitions of history can provide new insights to the general public in the preservation and exploration of cultural heritage.