Comp Sound

Sound

p5.js Sound Library

In addition to the basic drawing API, p5.js includes add-on libraries. The parameters chapter introduces the p5.dom library, and the examples in this chapter use the p5.sound library. The p5.sound library builds on the Web Audio API and provides functions for generating tones, playing recorded sounds, and visualizing the waveform and spectrum of sounds. I highly suggest taking a look at all of the sound examples to get an idea of what the sound library can do.

This chapter will focus on playing and visualizing pre-recorded sound assets. The comp music chapter talks more about sound synthesis.

• p5.sound API Reference
• p5.sound Examples

let myImg;

function preload() {
  myImg = loadImage("images/world.jpg");
}

function setup() {
  createCanvas(500, 500);
}

function draw() {
  image(myImg, 0, 0);
}

let mySound;

function preload() {
  mySound = loadSound("sounds/hack-comp.wav");
}

function setup() {
  mySound.loop(0, 1, 1, 0, 4);
}

Computers, Light, Sound, and People

Light and sound flow through our environment as electromagnetic and air pressure waves. Our eyes and our ears collect data about these waves and our visual and auditory cortexes process that data to create information. Though they work in different ways, both of these sensory systems are powerful. We can take advantage of these systems by choosing the types of forms we create.

Light and Sound Lecture Notes

Playing Sound with p5.js

Playing Recorded Sounds

p5.js makes it pretty easy to play sound assets, like this audio clip from Hackers(1995).

Changing Rate + Pitch

The third parameter to play() controlls the rate of playback. If you play a sound faster the pitch will go up; play it slower and the pitch will go down. p5 cannot change the pitch of a sound without changing its length.

Looping

Drum Kit

This example uses frameCount to keep time and play a beat with drum samples. This works for a quick sketch, but frameCount isn’t a great timekeeping source, so you might notice hitches in your rhythm. The Tone.js lets you precisely schedule when samples are played, and might be a better choice if you want to make music.

Bounce

This example doesn’t have any sound in it… yet.

Capturing Output

p5.sound provides methods for recording the sound generated by your program and saving it to a file. The following function records length milliseconds of audio and saves it as output.wav.

Your audio may sound fine at run time but clipped when you play back the .wav. If this happens, try reducing the volume of the sounds you generate.

If you can’t get good quality audio capture from p5.SoundRecorder, you might get better results using a screen recording program like Quicktime Player.

// uses the p5 SoundRecorder and SoundFile classes to record the audio output.
// begins recording when called. records for _length_ time in milliseconds.
function record(length) {
  var soundRecorder = new p5.SoundRecorder();
  var soundFile = new p5.SoundFile();
  soundRecorder.record(soundFile);
  setTimeout(function() {
    console.log("Recording Complete");
    soundRecorder.stop();
    save(soundFile, "output.wav");
  }, length);
}

Visualizing Sound with p5.js

Cuepoints

Volume

Wave

FFT

The Fast Fourier Transform transform a signal from the time domain to the frequency domain. For audio analysis that means the FFT can tell the strenght of different frequencies in an audio buffer.

For a visual exploration of FFT, see this excelent video by 3Blue1Brown But what is the Fourier Transform? A visual introduction.

Making Sounds from Scratch

You can create an empty soundFile object with new p5.SoundFile() and generate the sound data yourself with Javascript. To do so you will create and fill a Float32Array and then attach it to the SoundFile object with setBuffer().

Creating sounds this way lets you work at the lowest possible level: individual samples. This can be fun, and it gives you complete control, but you will probably need to reinvent some basics.

P5.js has functions for working with oscillators, envelopes, and effects if you want to work at a little higher level. You might also consider using a dedicated Javascript sound synthesis library like Tone.js, which is a little more powerful and better documented.

Reference Links

Jamie Wong: Color from Hexcodes to Eyeballs

Jamie Wong describes the each step of how color is expressed and interpreted.

Compform '16 Comp Music Notes

Compform '16 week on computational music.

Compform '16 Comp Music Notes

Examples from Compform '16 week on computational music.

WebMidiAPIShim

A javascript shim for accessing midi devices.

3Blue1Brown: Fourier Transform

A great visual introduction to the Fourier transform.

An Interactive Introduction to Fourier Transforms

Visual and interactive demos of Fourier Transforms for both sound and drawing.