Rendering a new vision of the Grand River

Janet Shuh, Dean of Continuing Education, Academic Quality and the Centre for Teaching and Learning Innovation at Mohawk College and Dr. Dawn Martin-Hill, professor and founders of the Indigenous Studies program at McMaster University, in front of digital XR display.

The XR Innovation Studio works with the Indigenous water research initiative at McMaster University to create immersive digital world.

The XR Innovation Studio at Mohawk College is bringing cutting-edge technology to traditional storytelling in partnership with Ohneganos Ohnegahdę:gyo, an Indigenous water research initiative led by Dr. Dawn Martin-Hill, professor and founder of the Indigenous Studies program at McMaster University. 

The result is an immersive virtual depiction of the Haudenosaunee creation narrative using extended reality (XR). Users are transported on an odyssey down the Grand River by canoe, their journey marked by stops at a longhouse and knowledge centres, before travelling forward in time to the present and witnessing the world around them change. 

“The goal is to share how the water was prior to colonialism and industrialization and show how it can be clean again if we can motivate younger generations to make better decisions and choices to bring her back to health,” says Martin-Hill. 

A committee of elders, including clan mothers and language speakers, offered cultural guidance as the project unfolded over the past three years. 

The broader Ohneganos Ohnegahdę:gyo initiative provides scientific findings of water quality and biology, including plants and fish, through data collected through sensors monitoring by the research team, along with actionable solutions. 

“Gathering data is useless if the public cannot understand or access the information, Ohneganos was dedicated to knowledge transmission and training,” says Martin-Hill. 

Alongside the science, Haudenosaunee elders shared knowledge about how the Grand River has changed through colonization and what it will take to clean it up. 

“The hope is this leads young people back to the water and gets youth of all backgrounds to understand this is a resource on which all life depends. We are trying to transmit the importance of freshwater and how little of it there is around the planet. So just getting some of those really important facts into young minds. That’s what the elders wanted.” 

The technical skill in VR at Mohawk College allowed the McMaster vision to communicate and engage around water research and Indigenous knowledge to be realized, says Janet Shuh, who is Dean of Continuing Education, Academic Quality and the Centre for Teaching and Learning Innovation.

Mohawk College’s XR Innovation Studio is dedicated to the development of virtual learning experiences for education, training and professional development using XR technologies. XR (extended reality) is an umbrella term that encompasses all immersive technologies: virtual reality (VR), augmented reality (AR), and mixed reality (MR). 

XR technology allows for the creation of realistic environments for training and education. Learners, both students and those undertaking workplace training, can engage in hands-on learning experiences without risk or constraint and without the need for expensive physical equipment. 

The College’s XR program is comprised of three primary pillars that include: integrating the technology into curriculum (12 courses so far integrate XR technologies), undertaking applied research with industry partners, and lastly providing co-op students with hands-on experience and build competencies in XR development. 

“Mohawk College is at the forefront of academic technology innovation and experimentation,” says Shuh. This investment, in part, is being driven by the College’s three-year Digital Learning Strategy launched in 2022. 

“Creating an innovation engine is a key aspect of our Digital Learning Strategy. XR is where we’re investing our energy in terms of innovation, so partnerships such as the one with McMaster are critical to driving that pedagogically driven academic technology innovation agenda. At the same time, we’re training students who will be the future talent that will fuel the growth that is coming in this cutting-edge sector.” 

Fiona Leigh, a Software Development student who will enter her fourth semester in September, has completed a four-month co-op, along with an eight-month work term in the XR Innovation Studio. For much of that time, she’s been working on the Grand River project. 

One of the challenges was to match the visual experience to the narration in both English and Mohawk languages, says Leigh. The audio could not be changed. 

“It was a lot of making sure the timing was right, along with making sure there are no collision issues. Inside the experience, you're resting on top of a lot of objects, and you're moving from between environments. There's a lot that has the potential to go wrong where you just fall into the middle of nowhere in the VR world.” 

In addition to all that, the creation of densely wooded terrain with lots of textures means the experience takes time to render within the VR headset. 

“That requires a lot of optimization and changing lighting values to make sure it runs smoothly.” 

Working on the project confirmed that she’d like to work in XR development, likely in gaming, says Leigh. 

“It was great to be able to get more experience in the field to solidify my choices, because now I can continue to do that if I want, because I have the experience.” 

The collaborative nature of the project was important for Leigh’s skill-building, too. 

“It was great to get experience talking to people about software on a technological level, as well as have conversations where I was gathering social and cultural information, and trying to understand their point of view and how they experienced things rather than just talking code language.” 

Other work completed by the XR Innovation Studio include: 

• An application that allows Power Engineering students to practise high-risk scenarios in a safe and controlled environment; 
• A simulation for learners in the Aerial Tower Communication Specialist micro-credential where learners go through the steps of donning personal protective equipment and safety checks before climbing a communication tower and experiencing an authentic sensation of working at great heights; 
• XR escape rooms for nursing, construction, and math programs in which students tackle a series of educational puzzles in order to escape; and 
• The re-creation of an in-person fluid mechanics lab for Building Construction Sciences through the use of state-of-the-art scanning tools to capture the physical lab and its equipment and then creating digital replicas through 3D modelling.  

This story originally appeared in Momentum Magazine.