Hear Neuroscience: IPad App Demonstrating the Hearing Process

Client: University of Utah: I was commissioned to program a prototype in Unity of the inner ear processing sound. This is the final product.

Programming

Interactive programming with the Unity3D game engine and Javascript and C#.

Prototyping

Iterating through each stage of the prototype in a collaborative environment.

UI Design

UI design using Illustrator.

Hear Neuroscience: Screen Recording

The IPad app demonstrates how the inner ear processes sounds using the game design software Unity3D and Javascript programming language.

On the app store for IPad. Free to download.

See the App on the App Store

Process.

Research: I spent a couple months researching how the hearing process works. It was really exciting to see how the inner ear processes sounds. I created charts and graphs of the different parts of the inner ear and then mapped out how that might translate to animation and programming for each part.

Programming & Animation.

The main parts I programmed and animated included:

  1. Sound frequencies effecting the speed of an animation of the ossicles: malleus, incus and stapes ( 3 bones of the middle ear ). The higher the frequency, the faster the animation. Animation was done in Cinema 4D and then put into Unity 3D for programming.
  2. The different frequencies lighting up the correct location in the cochlear.
  3. Accessing the microphone of the IPad to allow recording and input of the human voice.
  4. Playing sounds loaded on the application.

After creating a working prototype, the Genetic Science Learning Center team created the final 3D models and created some of the sounds in addition to fine tuning the programming and uploading to the app store.