Skip to content

A Mixed Reality Solar System

Jump to bottom
Mike Caprio edited this page Mar 6, 2019 · 39 revisions

Experiment with Multi-User Mixed Reality Experiences to Educate Others About the Solar System

Hackathon Findings

  • tk

Hackathon Projects

Background

The Science Visualization Group at AMNH (SVG - formerly known as Science Bulletins) is responsible for the production of a great deal of content for education, outreach, and science communication (including podcasts, graphics, interactives, videos, and space shows played in the Hayden Planetarium and around the world); for the administration of various programs and grants related to science visualization (including the OpenSpace project); and also for the exploration of new tools and technologies to improve and enhance exhibits or other Museum visitor experiences.

SVG is collaborating with educators from the Museum's Education department on this challenge in order to explore the implementation of so-called "mixed reality" experiences. As the toolkits for virtual and augmented reality has been developing, museums and institutions of all kinds have been experimenting with these burgeoning technologies to see what advantages and disadvantages they confer. Museums especially have their own unique constraints, as they must strike a balance between providing a captivating user experience and communicating vital information - all while maintaining scientific accuracy!

An example of multi-user mixed reality.

Mixed reality prototypes and other types of interactive projects are being made at the Museum, but SVG would like to see what ideas your team has when it comes to creating multi-user interactive experiences and suggesting how they would be used in an educational capacity. One consideration that immediately comes to mind is peer learning - it has been shown that learning and sharing information with peers is a great way to attain educational goals, especially among younger students. A multi-user mixed reality experience could enable peer learning by giving the users different pieces of information that they have to communicate to others and require all users to work together to achieve goals.

What Should We Teach Others About The Solar System?

When educators approach the topic of the solar system, they traditionally focus on comparison and contrast of the various characteristics of the objects residing in it, and on comprehension of the scale of objects or their distances in relation to each other. As an example, when comparing the various planets in the solar system, educators often compare and contrast:

  • planetary landscapes and surface features
  • planetary core features
  • planetary atmospheres (if a planet has one!)

Educators also need to make use of context and points of reference, especially in terms of comparing issues of scale and distance. For example, they could try to compare the size of Olympus Mons (the biggest volcano in Solar System) to the size of Mt. Hood (an Earth volcano) - but what other object or point of reference could they use to truly comprehend the differences in scale?

This image combines a photograph of Los Angeles with comet 67P Churyumov-Gerasimenko (the comet is around 3.5 km by 4 km).

Solutions

SVG has a model of the solar system in Unity that could be used in your solution. You are free to work with whatever libraries or technologies you like to create your prototype experience, keeping in mind that the most key aspect is that it must be a multi-user experience in some way. We are primarily looking for solutions based on interaction between multiple mobile devices, like smartphones. Consider: how will bluetooth figure in? There are a vast array of bluetooth beacons all around the Museum, mostly used by the AMNH Explorer app for location detection. What about geolocation generally, how could that be a part of this experience?

You may find some resources or projects from the first AMNH hackathon, Hack The Universe, to be useful. Here are a few challenges of interest - you can find past projects linked at each page below:

Some possible solutions might be:

  • "Angry birds" in the solar system. Can you create some kind of "gravity assist" interactive experience where multiple users have to work together towards a particular goal? Something like a mission simulation where you have to launch a craft and use gravitational fields to get to a certain location? What if different users could act as different systems or instruments on the spacecraft, receiving different information they have to share with each other?

  • Viewing different wavelengths of light. Our eyes aren't that great! What if we could "see" solar system objects differently, the way that instruments on spacecraft do? Kind of like those examples of how a bee sees a flower differently that humans do? What if one user could wave a mobile device in front of an object and see it one way, and another user could see it a totally different way?

  • SUB-CHALLENGE: SPICE converter for Unity - NASA uses a file format called SPICE to store data about the telemetry of spacecraft; with SPICE data, you can determine a very precise position and orientation of a spacecraft in space and time, even down to the positioning of all of its instruments at particular times. This allows us to create visualizations of past, present, and future space missions! OpenSpace uses portions of SPICE files for its visualization, but as far as we know there is no such tool to do the same thing in an engine like Unity. This would be a super valuable tool for the Museum but also for anyone else who wants to do any kind of space modeling or scientific visualization in Unity or other similar game engines. It could also be useful for you to create a new kind of file format, as there are past missions which don't have SPICE data (like old Apollo missions) that we want to visualize. You may want to consider making something like a JSON style file format or some such for the relevant portions of SPICE files needed for mission visualization.

  • Anything else you can think of!

Resources

Challenge owner: Nick Bartzokas and Yvonne De La Pena

Challenges --|-- Online Resources And Data Sets --|-- Code of Conduct --|-- Home

Challenges

Online Resources And Data Sets

Museum Stakeholders

Home

Clone this wiki locally