source


Air Flow and Sound

This applet works in the same way a voice synthesizer would. It parses the user’s input looking for units that potentially form the syllables in a word. After deciding which units are syllables it looks for audio files that match those units and concatenates them to form words.

The “cool” button does the same thing but, instead of matching the syllables with speech files, it looks for musical files (at this time it only works with the second line of text on the choice button). The sounds are created by instruments which, in one way or another, resemble the movements the mouth makes to produce sound. The objective here is to map the different kinds of consonants (stops, fricatives, nasals, etc.) in musical terms. In this applet I explore two kinds of sounds:

Stops – “staccato” sounds; beats, sudden interruptions; represented by drums
Fricatives – flowing high-frequency sounds; represented by a saxophone

Because the applet recognizes not only consonants but also vowels, all instrumental sound is mixed with piano notes that represent the vowels.