Hear Neuroscience: IPad App Demonstrating the Hearing Process

Client: Genetic Science Learning Center. University of Utah.

Genetic Science Learning Center is part of the University of Utah and creates visually engaging products that bring science ideas and processes alive in addition to delivering science and health ideas to teachers, students, and the public.

Working with the Genetic Science Learning Center team of artists, programmer and science experts, I was commissioned to program a prototype of the inner ear processing sound. 

Programming

Interactive programming with the Unity3D game engine and Javascript and C#.

Prototyping

Iterating through each stage of the prototype in a collaborative environment.

UI Design

UI design using Illustrator.

Hear Neuroscience: Screen Recording

Working with the team at Genetic Science Learning Center, I programmed this application along with the talented creative Pete Anderson. The IPad app demonstrates how the inner ear processes sounds using the game design software Unity3D and Javascript programming language.

On the app store for IPad. Free to download.

See the App on the App Store

Process.

Research: I spent a couple months researching how the hearing process works. It was really exciting to see how the inner ear processes sounds. I created charts and graphs of the different parts of the inner ear and then mapped out how that might translate to animation and programming for each part.

Programming & Animation.

The main parts I programmed and animated included:

  1. Sound frequencies effecting the speed of an animation of the ossicles: malleus, incus and stapes ( 3 bones of the middle ear ). The higher the frequency, the faster the animation. Animation was done in Cinema 4D and then put into Unity 3D for programming.
  2. The different frequencies lighting up the correct location in the cochlear.
  3. Accessing the microphone of the IPad to allow recording and input of the human voice.
  4. Playing sounds loaded on the application.

After creating a working prototype, the Genetic Science Learning Center team created the final 3D models and created some of the sounds in addition to fine tuning the programming and uploading to the app store.