This page was automatically generated by NetLogo 5.0.2.
The applet requires Java 5 or higher. Java must be enabled in your browser settings. Mac users must have Mac OS X 10.4 or higher. Windows and Linux users may obtain the latest Java from Oracle's Java site.
In order for this to work, this file, your model file (AudioTexturesISEA2012_nourl.nlogo), and the files NetLogoLite.jar and NetLogoLite.jar.pack.gz must all be in the same directory. (You can copy NetLogoLite.jar and NetLogoLite.jar.pack.gz from the directory where you installed NetLogo.)
The applet will also need access to the sound bitmap and url extensions. Copy the entire directory for each required extension into the same directory as the applet.
On some systems, you can test the applet locally on your computer before uploading it to a web server. It doesn't work on all systems, though, so if it doesn't work from your hard drive, please try uploading it to a web server.
You don't need to include everything in this file in your page. If you want, you can just take the HTML code beginning with <applet> and ending with </applet>, and paste it into any HTML file you want. It's even OK to put multiple <applet> tags on a single page.
If the NetLogoLite files and your model are in different directories, you must modify the archive= and value= lines in the HTML code to point to their actual locations. (For example, if you have multiple applets in different directories on the same web server, you may want to put a single copy of the NetLogoLite files in one central place and change the archive= lines of all the HTML files to point to that one central copy. This will save disk space for you and download time for your users.)
powered by NetLogo
view/download model file: AudioTexturesISEA2012_nourl.nlogo
Audio Textures.
Have you ever wanted to listen to anything around you?
Now you can!
Audio Textures uses computer programming to map visual aspects of hue and brightness to pitch and timbre of sound so you can create cool textures out of anything you see!
Basically, this app analyzes image and translates it to sound based on four components:
x (height) - Pitch
y (length) - Duration
hue (color) - Timbre/Instrument
brightness - Velocity
By using these characteristics of the two arts, we can interpret the image as sound.
Before starting, be sure to set the speed (top slider with blue ball) to slower than the middle, that way it sounds better and the program doesn’t crash!
Then drag the red slider labeled “score” to any of the 11 choices*.
Next hit “setup” or “load-score” to view the image.
Once you find the one you want, it’s time to hear that sucker.
Here’s where you have options:
By hitting “go” you can listen to the piece in a chromatic scale (very dense/atonal)
“Tonalize” plays the piece in a major scale (still dense but tonal)
“Randomize” plays random pitches but still uses hue-based instruments
“Equalize” plays all the pitches as one cool chord (based off the golden ratio)
You can also use “chordalize” to subtract out every other note of the scale used in either “go” or “tonalize” if it is too dense for your tastes.
*These are in order of how we interpreted the variables of sound/image.
The first few deal with black and white images with some shading and the last couple deal with complex textures and real art pieces (don’t sue us Van Gogh).
The instruments change with different color patches!
Also, notice how the dominate color of an image seems “subtracted out”?
It is! To clean up image each one removes its most common color on a scale of the color variance within the image: it makes it sound better!
The model is now moving in the direction of edge detection and cognitive science.
Soon we will use the app to detect edges in images and incorporate that into the usic translation portion.
Also, soon the app will incorporate live motion, so it can understand change in an image!
As far as cognitive science goes, we’re looking at adapting the app to guide blind people through rooms by describing the objects around them with sound!
Stay tuned!
Literally!
Heavy use of bitmaps, url, and sound extensions.
Pixound app for iPhone does similair things to this app.
Special thanks to Stephen Guerin and Acacia McCombs for being inspiring and innovative with art, science, and teaching!
patches-own [ tonalizer equalizer ]
extensions [sound bitmap]
globals [instruments backgroundcolor colorvariance ]
to setup
reset-ticks
set instruments [""]
ct
ask patches [set equalizer (1 / ( 5 ^ .5 ) * ((( (1 + ( 5 ^ .5 )) / 2 ) ^ pycor ) - ( 1 - ( 5 ^ .5 )) / 2 ) ^ pycor )]
ask patches with [pxcor = 0 ][ sprout 1 [set heading 90 set shape "circle" set color [255 0 0 128]]]
ask patches with [pycor >= 3][ set tonalizer 1 ]
ask patches with [pycor >= 7][ set tonalizer 2 ]
ask patches with [pycor >= 10][ set tonalizer 3 ]
ask patches with [pycor >= 14][ set tonalizer 4 ]
ask patches with [pycor >= 17][ set tonalizer 5 ]
ask patches with [pycor >= 21][ set tonalizer 6 ]
ask patches with [pycor >= 24][ set tonalizer 7 ]
load-score
end
to load-score
import-pcolors (word "score_" score ".png")
set backgroundcolor item 0 modes [pcolor] of patches
set colorvariance mean [pcolor] of patches / 24
end
to go
tick
ask turtles [
fd 1
if pcolor != [pcolor] of patch-ahead -1 and pcolor > backgroundcolor + colorvariance or pcolor < backgroundcolor - colorvariance or patch mouse-xcor mouse-ycor = patch-here [
let velocity 0.5 * item 2 extract-hsb pcolor
sound:play-note item instrument sound:instruments ( pycor + 48 ) velocity .5
if [pcolor] of patch-here = [pcolor] of patch-ahead 1 [ sound:play-note item instrument sound:instruments ( pycor + 60 ) velocity 1
]
set instrument (item 0 extract-hsb pcolor) / 255 * 127
]
]
end
to tonalize
tick
ask turtles [
fd 1
if pcolor != [pcolor] of patch-ahead -1 and pcolor > backgroundcolor + colorvariance or pcolor < backgroundcolor - colorvariance or patch mouse-xcor mouse-ycor = patch-here [
let velocity 0.5 * item 2 extract-hsb pcolor
sound:play-note item instrument sound:instruments ( pycor + ( pycor - tonalizer ) + 48 ) velocity .5
if [pcolor] of patch-here = [pcolor] of patch-ahead 1 [ sound:play-note item instrument sound:instruments ( pycor + ( pycor - tonalizer ) + 36 ) velocity 1
]
set instrument (item 0 extract-hsb pcolor) / 255 * 127
]]
end
to equalize
tick
ask turtles [
fd 1
if pcolor != [pcolor] of patch-ahead -1 and pcolor > backgroundcolor + colorvariance or pcolor < backgroundcolor - colorvariance or patch mouse-xcor mouse-ycor = patch-here [
let velocity 0.5 * item 2 extract-hsb pcolor
sound:play-note item instrument sound:instruments ( pycor + equalizer + 48 ) velocity .5
set instrument (item 0 extract-hsb pcolor) / 255 * 127
if [pcolor] of patch-here = [pcolor] of patch-ahead 1 [ sound:play-note item instrument sound:instruments ( pycor + equalizer + 60 ) velocity 1
]
]
]
end
to randomize
tick
ask turtles [
fd 1
if pcolor != [pcolor] of patch-ahead -1 and pcolor > backgroundcolor + colorvariance or pcolor < backgroundcolor - colorvariance or patch mouse-xcor mouse-ycor = patch-here [
let velocity 0.5 * item 2 extract-hsb pcolor
sound:play-note item instrument sound:instruments ( pycor + random 60 + equalizer ) velocity .5
set instrument floor (item 0 extract-hsb pcolor) / 255 * 127
if [pcolor] of patch-here = [pcolor] of patch-ahead 1 [ sound:play-note item instrument sound:instruments ( pycor + random 60 + equalizer ) velocity 1 ]
]
]
end
to chordalize
tick
ask patches with [ pycor mod 2 != 0] [ set pcolor black ]
end