Atau Tanaka's BioMuse system is interesting. In this video he controls wind type sounds by moving his arms and hands. A simplified version of this with Wiimotes would not be too difficult to design.
https://vimeo.com/2483259
Tuesday, 9 July 2013
Wednesday, 3 July 2013
Wii Footsteps Update
This is the same patch as mentioned in the previous post but with an additional ability to adjust sounds dependent on speed.
A wiimote attached to the leg of the user measures their walking speed, when this reaches a certain value (currently a MIDI CC value of 30 but this needs to be adjusted) the MIDI channel is switched from 1 to 2 to control another sampler on MIDI channel 2 (it may be better to just add 12 to the MIDI value and store the running samples in the same sampler instance in Ableton).
This is a good improvement on the previous patch although sticking a wiimote in a sock is hardly the best way to do it.
Footstep samples for specific surfaces at different walking/ running speeds now need to be found to demonstrate this properly.
Still need to find a way to adjust velocity effectively (does a hard footstep have more high or low frequency content than a soft footstep? if so adjustment of high or low pass filters relative to the velocity info from the balance board could be worth experimenting with).
PD patch (new section highlighted blue).
A wiimote attached to the leg of the user measures their walking speed, when this reaches a certain value (currently a MIDI CC value of 30 but this needs to be adjusted) the MIDI channel is switched from 1 to 2 to control another sampler on MIDI channel 2 (it may be better to just add 12 to the MIDI value and store the running samples in the same sampler instance in Ableton).
This is a good improvement on the previous patch although sticking a wiimote in a sock is hardly the best way to do it.
Footstep samples for specific surfaces at different walking/ running speeds now need to be found to demonstrate this properly.
Still need to find a way to adjust velocity effectively (does a hard footstep have more high or low frequency content than a soft footstep? if so adjustment of high or low pass filters relative to the velocity info from the balance board could be worth experimenting with).
PD patch (new section highlighted blue).
Wii Fit Footsteps
In this configuration the wii fit board triggers recordings of footsteps in Ableton Live via Pure Data.
12 footstep samples (taken from one wav from the sony sound effects library) were loaded into Live's drum rack sampler.
Each footstep on the wii fit board triggers one of the 12 samples at random. At the moment there is a chance that the same sample will be triggered twice in a row, this is less than ideal as that could sound unnatural, I'm looking into ways to rectify that.
Although the PD patch is sending velocity information it is currently inadequate. Footsteps sampled at different velocities may be required but I'm not sure they are attainable. Sorting the samples into perceived velocity order and then triggering them based on the force applied to the wii fit board may work but I doubt that the board
Andy Farnell discusses the superioriy procedural audio over recorded audio for footsteps in Designing Sound. Although his argument is probably true this an attempt at a compromise. Working with audio samples does not take into account the different biomechanical movements made by the foot when moving at different speeds (striking with the heel or ball of the foot when running or walking). I cannot think of a way to rectify this but the ability to input speed information with a wiimote (either by a button press or by pitch or yaw info). [An idea - would attaching a wiimote or nunchuk to the users calf/ shin be able to determine speed information via the accelerometers and then switch to a running/ walking bank of samples when appropriate?]
Screenshot of the PD patch -
12 footstep samples (taken from one wav from the sony sound effects library) were loaded into Live's drum rack sampler.
Each footstep on the wii fit board triggers one of the 12 samples at random. At the moment there is a chance that the same sample will be triggered twice in a row, this is less than ideal as that could sound unnatural, I'm looking into ways to rectify that.
Although the PD patch is sending velocity information it is currently inadequate. Footsteps sampled at different velocities may be required but I'm not sure they are attainable. Sorting the samples into perceived velocity order and then triggering them based on the force applied to the wii fit board may work but I doubt that the board
Andy Farnell discusses the superioriy procedural audio over recorded audio for footsteps in Designing Sound. Although his argument is probably true this an attempt at a compromise. Working with audio samples does not take into account the different biomechanical movements made by the foot when moving at different speeds (striking with the heel or ball of the foot when running or walking). I cannot think of a way to rectify this but the ability to input speed information with a wiimote (either by a button press or by pitch or yaw info). [An idea - would attaching a wiimote or nunchuk to the users calf/ shin be able to determine speed information via the accelerometers and then switch to a running/ walking bank of samples when appropriate?]
Screenshot of the PD patch -
I'm finding that the possibilities of the wii equipment are more extensive than I first thought. I'm wondering if this should take priority over the Arduino experiments.
Thursday, 27 June 2013
Wiimote Gunshots
A wiimote was used to trigger gunshot samples using PD/ a sampler in Live.
After hours of experimenting with various expressions the [threshold] object in PD solved the problems I was having with false triggering.
Pressing the B button on the wiimote changes the midi channel - Single shot samples are on channel 1, rapid gunfire on channel 2.
I may create a number of variants of the gunshot sounds, associate them with different MIDI notes in the sampler and introduce an element of random selection in PD, this could add variation but would also potentially eliminate the issue of re-triggering a sample before it completes if only non-repeating MIDI notes are allowed.
The left to right gesture isn't the most intuitive for this sound, up and down may be more so. However, using the pitch values would mean that the threshold values would have to be closer resulting in more false triggers (due to the smaller range of motion in the wrist in the up/down direction compared to left/ right). It's likely that the best solution would be to control gun 1 with the nunchuk and gun 2 with the wiimote, though I was hoping to use the nunchuk to control other parameters.
I'm not sure how accurate the sync would be when using this method with video footage.
PD screenshot:
Tuesday, 25 June 2013
wiimote controlling helicopter sounds
Osculator, pure data, a wiimote and a wii nunchuk were used to control certain elements of a helicopter sound.
In the screenshot below the x axis of the joystick controls the distance parameter while the pitch value of the wiimote controls the engine speed.
Andy Farnell's PD patches are starting to feel quite limiting. A set of multisampled recordings of a vehicle would be useful now with various speeds, gear changes, brake noises etc. These could be triggered from within a PD patch but it may be too complex to do in the time available.
This is a good example of what I have been trying to achieve, though I have no idea how they did it as I can't get hold of the relevant paper - http://www.youtube.com/watch?v=OknJyGV6jwY
In the screenshot below the x axis of the joystick controls the distance parameter while the pitch value of the wiimote controls the engine speed.
Andy Farnell's PD patches are starting to feel quite limiting. A set of multisampled recordings of a vehicle would be useful now with various speeds, gear changes, brake noises etc. These could be triggered from within a PD patch but it may be too complex to do in the time available.
This is a good example of what I have been trying to achieve, though I have no idea how they did it as I can't get hold of the relevant paper - http://www.youtube.com/watch?v=OknJyGV6jwY
Footstep control
I've been trying to adapt the Andy Farnell footstep PD patch to be controlled by the wii fit balance board. This is proving to be difficult because the PD patch isn't designed to be controlled step by step but has a built in system to simulate walking.
Using the wii board to control simple footstep samples is inadequate because this does not take in to account the force/ pressure applied by the foot, or which part of the foot strikes first.
I have found an interesting paper on the subject -
http://vbn.aau.dk/files/37619504/Turchet_Serafin_Dimitrov_Nordahl_DAFx10_P50.pdf
If anything this may just prove that this is too complex and would probably deserve a project of its own.
This paper provides another approach using sensors built into shoes -
http://www.speech.kth.se/prod/publications/files/3441.pdf
This paper provides another approach using sensors built into shoes -
http://www.speech.kth.se/prod/publications/files/3441.pdf
Thursday, 13 June 2013
IR Sensor
A sharp IR sensor was used to output MIDI CC messages.
The results were erratic without smoothing, so this was applied taking an average from 5 readings (10 was too unresponsive).
At present the IR sensor only gives readings of approximately 20 - 80 (once converted to MIDI friendly numbers) when 0 - 127 is expected and wanted. I assume that the IR sensor needs to be calibrated, but I'm not sure how to do this yet.
The sensor was connected to analog input 0.
#include <MIDI.h>
const int numReadings = 5; //the readings from the analog input
int readings[numReadings]; //the index of the current reading
int index = 0; //the index of the current reading
int total = 0; //the running total
int average = 0; //the average
int midiout = 0; //midi out
int inputPin = 0; // analog pin for reading the IR sensor
void setup() {
MIDI.begin(); // Launch MIDI with default options
for (int thisReading = 0; thisReading < numReadings; thisReading++)
readings [thisReading] = 0;
}
void loop() {
float volts = analogRead(inputPin)*0.0048828125; // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
float distance = 65*pow(volts, -1.10); // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
delay(100); // arbitary wait time.
total= total - readings[index]; //subtract the last reading
readings[index] = analogRead(inputPin); //read from the sensor
total= total + readings[index]; //advance to the next position in the array
index = index +1;
if (index >= numReadings) //if we're at the end of the array...
index = 0; //...wrap around to the beginning.
average = total / numReadings; //calculate the average
MIDI.sendControlChange(16, average /8, 1);
delay (1);
}
This code was used to control the distance parameter of a helicopter PD patch taken from Designing Sound by Andy Farnell:
Blue highlighted elements were newly added and consist of a simple ctlin object listening for CC messages which then control the distance slider. As it stands the distance increases as the object is moved closer to the sensor which is obviously counterintuitive, simply switching the slider settings from 0-1 to 1-0 did not rectify this.
Ultimately I get the impression that an IR sensor may not be of too much value to this project due to its inaccuracy. Perhaps an ultrasonic distance sensor would be of more use.
The results were erratic without smoothing, so this was applied taking an average from 5 readings (10 was too unresponsive).
At present the IR sensor only gives readings of approximately 20 - 80 (once converted to MIDI friendly numbers) when 0 - 127 is expected and wanted. I assume that the IR sensor needs to be calibrated, but I'm not sure how to do this yet.
The sensor was connected to analog input 0.
Arduino code is below. It makes use of elements taken from the smoothing example sketch, and an IR sketch found at http://luckylarry.co.uk/arduino-projects/arduino-using-a-sharp-ir-sensor-for-distance-calculation/ as well as some original elements and the Arduino MIDI library.
#include <MIDI.h>
const int numReadings = 5; //the readings from the analog input
int readings[numReadings]; //the index of the current reading
int index = 0; //the index of the current reading
int total = 0; //the running total
int average = 0; //the average
int midiout = 0; //midi out
int inputPin = 0; // analog pin for reading the IR sensor
void setup() {
MIDI.begin(); // Launch MIDI with default options
for (int thisReading = 0; thisReading < numReadings; thisReading++)
readings [thisReading] = 0;
}
void loop() {
float volts = analogRead(inputPin)*0.0048828125; // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
float distance = 65*pow(volts, -1.10); // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
delay(100); // arbitary wait time.
total= total - readings[index]; //subtract the last reading
readings[index] = analogRead(inputPin); //read from the sensor
total= total + readings[index]; //advance to the next position in the array
index = index +1;
if (index >= numReadings) //if we're at the end of the array...
index = 0; //...wrap around to the beginning.
average = total / numReadings; //calculate the average
MIDI.sendControlChange(16, average /8, 1);
delay (1);
}
This code was used to control the distance parameter of a helicopter PD patch taken from Designing Sound by Andy Farnell:
Blue highlighted elements were newly added and consist of a simple ctlin object listening for CC messages which then control the distance slider. As it stands the distance increases as the object is moved closer to the sensor which is obviously counterintuitive, simply switching the slider settings from 0-1 to 1-0 did not rectify this.
Ultimately I get the impression that an IR sensor may not be of too much value to this project due to its inaccuracy. Perhaps an ultrasonic distance sensor would be of more use.
5 Piezo Trigger
Five piezos are used to trigger a thunder sound in Pure Data in an attempt to control both panning and volume at once. This could be potentially useful when live control of a thunderstorm with lightning strikes at various distances and sizes, it could also be used in a war scene with many explosions of varying intensity.
There seems to be no real advantage of using this setup over a traditional set of drum pads/ triggers but it's helped me get to grips with some basic Arduino concepts. It may be worth experimenting with arranging the piezos differently before moving on to more complex sensors.
The piezos were connected to analog inputs 0-4, a 1M Ohm resistor was wired in parallel with the piezos to function as a pull down resistor.
Below is the Arduino code, it makes use of the Arduino MIDI library. It's modified from the example "knock" sketch that comes with the Arduino IDE so there may be a few needless vestiges of that in there.
#include <MIDI.h>
const int ledPin = 13; // led connected to digital pin 13
const int piezo1 = A0; // the piezo is connected to analog pin 0
const int piezo2 = A1;
const int piezo3 = A2;
const int piezo4 = A3;
const int piezo5 = A4;
const int threshold = 100; // threshold value
int sensorReading1 = 0; // variable to store the value read from the sensor pin A0
int sensorReading2 = 0;
int sensorReading3 = 0;
int sensorReading4 = 0;
int sensorReading5 = 0; // variable to store the value read from sensor pin A1
int ledState = LOW; // variable used to store the last LED status, to toggle the light
void setup() {
pinMode(ledPin, OUTPUT); // declare the ledPin as as OUTPUT
MIDI.begin(); // Launch MIDI with default options
}
void loop() {
// read the sensors and store it in the variable sensorReadingx:
sensorReading1 = analogRead(piezo1);
sensorReading2 = analogRead(piezo2);
sensorReading3 = analogRead(piezo3);
sensorReading4 = analogRead(piezo4);
sensorReading5 = analogRead(piezo5);
// if the sensor 1 reading is greater than the threshold:
if (sensorReading1 >= threshold) {
// send noteon
MIDI.sendNoteOn(100, analogRead(piezo1) / 8, 1);
delay(100);
MIDI.sendNoteOff(100, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 2 reading is greater than the threshold:
if (sensorReading2 >= threshold) {
// send noteon
MIDI.sendNoteOn(75, analogRead(piezo2) / 8, 1);
delay(100);
MIDI.sendNoteOff(75, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 3 reading is greater than the threshold:
if (sensorReading3 >= threshold) {
// send noteon
MIDI.sendNoteOn(50, analogRead(piezo3) / 8, 1);
delay(100);
MIDI.sendNoteOff(50, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 4 reading is greater than the threshold:
if (sensorReading4 >= threshold) {
// send noteon
MIDI.sendNoteOn(25, analogRead(piezo4) / 8, 1);
delay(100);
MIDI.sendNoteOff(25, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 5 reading is greater than the threshold:
if (sensorReading5 >= threshold) {
// send noteon
MIDI.sendNoteOn(1, analogRead(piezo4) / 8, 1);
delay(100);
MIDI.sendNoteOff(1, analogRead(piezo1) / 8, 1);
delay(5);
}
}
The piezos were attached to a hardback notebook with electrical tape as shown in the image below.
There seems to be no real advantage of using this setup over a traditional set of drum pads/ triggers but it's helped me get to grips with some basic Arduino concepts. It may be worth experimenting with arranging the piezos differently before moving on to more complex sensors.
The piezos were connected to analog inputs 0-4, a 1M Ohm resistor was wired in parallel with the piezos to function as a pull down resistor.
Below is the Arduino code, it makes use of the Arduino MIDI library. It's modified from the example "knock" sketch that comes with the Arduino IDE so there may be a few needless vestiges of that in there.
#include <MIDI.h>
const int ledPin = 13; // led connected to digital pin 13
const int piezo1 = A0; // the piezo is connected to analog pin 0
const int piezo2 = A1;
const int piezo3 = A2;
const int piezo4 = A3;
const int piezo5 = A4;
const int threshold = 100; // threshold value
int sensorReading1 = 0; // variable to store the value read from the sensor pin A0
int sensorReading2 = 0;
int sensorReading3 = 0;
int sensorReading4 = 0;
int sensorReading5 = 0; // variable to store the value read from sensor pin A1
int ledState = LOW; // variable used to store the last LED status, to toggle the light
void setup() {
pinMode(ledPin, OUTPUT); // declare the ledPin as as OUTPUT
MIDI.begin(); // Launch MIDI with default options
}
void loop() {
// read the sensors and store it in the variable sensorReadingx:
sensorReading1 = analogRead(piezo1);
sensorReading2 = analogRead(piezo2);
sensorReading3 = analogRead(piezo3);
sensorReading4 = analogRead(piezo4);
sensorReading5 = analogRead(piezo5);
// if the sensor 1 reading is greater than the threshold:
if (sensorReading1 >= threshold) {
// send noteon
MIDI.sendNoteOn(100, analogRead(piezo1) / 8, 1);
delay(100);
MIDI.sendNoteOff(100, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 2 reading is greater than the threshold:
if (sensorReading2 >= threshold) {
// send noteon
MIDI.sendNoteOn(75, analogRead(piezo2) / 8, 1);
delay(100);
MIDI.sendNoteOff(75, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 3 reading is greater than the threshold:
if (sensorReading3 >= threshold) {
// send noteon
MIDI.sendNoteOn(50, analogRead(piezo3) / 8, 1);
delay(100);
MIDI.sendNoteOff(50, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 4 reading is greater than the threshold:
if (sensorReading4 >= threshold) {
// send noteon
MIDI.sendNoteOn(25, analogRead(piezo4) / 8, 1);
delay(100);
MIDI.sendNoteOff(25, analogRead(piezo1) / 8, 1);
delay(5);
}
// if the seonsor 5 reading is greater than the threshold:
if (sensorReading5 >= threshold) {
// send noteon
MIDI.sendNoteOn(1, analogRead(piezo4) / 8, 1);
delay(100);
MIDI.sendNoteOff(1, analogRead(piezo1) / 8, 1);
delay(5);
}
}
The piezos were attached to a hardback notebook with electrical tape as shown in the image below.
The piezos control a modified version of an example patch from Andy Farnell's Designing Sound as shown below. Pan and gain controls were added (highlighted in blue). The left outlet of the notein object triggers the "go" object and plays the thunder sound when any value is received.
The middle outlet of the notein object is the velocity value which controls the output volume, a spigot object is used to store the velocity value for longer than the duration of contact with the piezo, maintaining a consistent volume for each hit.
As the midi notes used are 1 for the leftmost piezo, 25, 50, 75 for the middle piezos and 100 for the rightmost these values are divided by 100 in order to control the pan_position fader.
Triggering a different instance of the original patch for each piezo would probably be more useful as the current setup doesn't sound great when the lightning/ thunder strikes overlap.
Tuesday, 11 June 2013
Simple MIDI Pot
This blog is really just a way to keep track of my Arduino experiments for my MSc project. It will probably be incredibly boring to anyone other than myself.
After a week or more I've finally managed to get the Arduino talking with Pure Data. Attempts at using the USB port to send serial messages that were then intended to be converted to MIDI using a MIDI-serial bridge (http://projectgus.github.io/hairless-midiserial/) were frustrating and fruitless. I'm now using the Arduino MIDI library (http://playground.arduino.cc/Main/MIDILibrary), I'm unsure if this is the best way to proceed but it's making things easier for now.
Currently I've only got a potentiometer sending CC messages to PD which isn't much use to the project but it's a start. There is some instability with the MIDI value flickering now and then when the pot isn't in use, I'm hoping that the use of data smoothing when using more complex sensors will rectify this. A distance sensor and piezos are the next sensors I intend to experiment with.
Below is the code and a diagram made in fritzing for the simple potentiometer patch. Hopefully the code doesn't get too mangled by blogger.
#include <MIDI.h>
void setup() {
MIDI.begin(); // Launch MIDI with default options
}
void loop() {
MIDI.sendControlChange(16, analogRead(0) / 8, 1); // (Control number, pot reading divided by 8, channel)
delay(5); //delay of 5ms before repeating loop
}
The pot is 50k and the resistor is 220ohm
After a week or more I've finally managed to get the Arduino talking with Pure Data. Attempts at using the USB port to send serial messages that were then intended to be converted to MIDI using a MIDI-serial bridge (http://projectgus.github.io/hairless-midiserial/) were frustrating and fruitless. I'm now using the Arduino MIDI library (http://playground.arduino.cc/Main/MIDILibrary), I'm unsure if this is the best way to proceed but it's making things easier for now.
Currently I've only got a potentiometer sending CC messages to PD which isn't much use to the project but it's a start. There is some instability with the MIDI value flickering now and then when the pot isn't in use, I'm hoping that the use of data smoothing when using more complex sensors will rectify this. A distance sensor and piezos are the next sensors I intend to experiment with.
Below is the code and a diagram made in fritzing for the simple potentiometer patch. Hopefully the code doesn't get too mangled by blogger.
#include <MIDI.h>
void setup() {
MIDI.begin(); // Launch MIDI with default options
}
void loop() {
MIDI.sendControlChange(16, analogRead(0) / 8, 1); // (Control number, pot reading divided by 8, channel)
delay(5); //delay of 5ms before repeating loop
}
The pot is 50k and the resistor is 220ohm
Subscribe to:
Posts (Atom)