Atau Tanaka's BioMuse system is interesting. In this video he controls wind type sounds by moving his arms and hands. A simplified version of this with Wiimotes would not be too difficult to design.
https://vimeo.com/2483259
Tuesday, 9 July 2013
Wednesday, 3 July 2013
Wii Footsteps Update
This is the same patch as mentioned in the previous post but with an additional ability to adjust sounds dependent on speed.
A wiimote attached to the leg of the user measures their walking speed, when this reaches a certain value (currently a MIDI CC value of 30 but this needs to be adjusted) the MIDI channel is switched from 1 to 2 to control another sampler on MIDI channel 2 (it may be better to just add 12 to the MIDI value and store the running samples in the same sampler instance in Ableton).
This is a good improvement on the previous patch although sticking a wiimote in a sock is hardly the best way to do it.
Footstep samples for specific surfaces at different walking/ running speeds now need to be found to demonstrate this properly.
Still need to find a way to adjust velocity effectively (does a hard footstep have more high or low frequency content than a soft footstep? if so adjustment of high or low pass filters relative to the velocity info from the balance board could be worth experimenting with).
PD patch (new section highlighted blue).
A wiimote attached to the leg of the user measures their walking speed, when this reaches a certain value (currently a MIDI CC value of 30 but this needs to be adjusted) the MIDI channel is switched from 1 to 2 to control another sampler on MIDI channel 2 (it may be better to just add 12 to the MIDI value and store the running samples in the same sampler instance in Ableton).
This is a good improvement on the previous patch although sticking a wiimote in a sock is hardly the best way to do it.
Footstep samples for specific surfaces at different walking/ running speeds now need to be found to demonstrate this properly.
Still need to find a way to adjust velocity effectively (does a hard footstep have more high or low frequency content than a soft footstep? if so adjustment of high or low pass filters relative to the velocity info from the balance board could be worth experimenting with).
PD patch (new section highlighted blue).
Wii Fit Footsteps
In this configuration the wii fit board triggers recordings of footsteps in Ableton Live via Pure Data.
12 footstep samples (taken from one wav from the sony sound effects library) were loaded into Live's drum rack sampler.
Each footstep on the wii fit board triggers one of the 12 samples at random. At the moment there is a chance that the same sample will be triggered twice in a row, this is less than ideal as that could sound unnatural, I'm looking into ways to rectify that.
Although the PD patch is sending velocity information it is currently inadequate. Footsteps sampled at different velocities may be required but I'm not sure they are attainable. Sorting the samples into perceived velocity order and then triggering them based on the force applied to the wii fit board may work but I doubt that the board
Andy Farnell discusses the superioriy procedural audio over recorded audio for footsteps in Designing Sound. Although his argument is probably true this an attempt at a compromise. Working with audio samples does not take into account the different biomechanical movements made by the foot when moving at different speeds (striking with the heel or ball of the foot when running or walking). I cannot think of a way to rectify this but the ability to input speed information with a wiimote (either by a button press or by pitch or yaw info). [An idea - would attaching a wiimote or nunchuk to the users calf/ shin be able to determine speed information via the accelerometers and then switch to a running/ walking bank of samples when appropriate?]
Screenshot of the PD patch -
12 footstep samples (taken from one wav from the sony sound effects library) were loaded into Live's drum rack sampler.
Each footstep on the wii fit board triggers one of the 12 samples at random. At the moment there is a chance that the same sample will be triggered twice in a row, this is less than ideal as that could sound unnatural, I'm looking into ways to rectify that.
Although the PD patch is sending velocity information it is currently inadequate. Footsteps sampled at different velocities may be required but I'm not sure they are attainable. Sorting the samples into perceived velocity order and then triggering them based on the force applied to the wii fit board may work but I doubt that the board
Andy Farnell discusses the superioriy procedural audio over recorded audio for footsteps in Designing Sound. Although his argument is probably true this an attempt at a compromise. Working with audio samples does not take into account the different biomechanical movements made by the foot when moving at different speeds (striking with the heel or ball of the foot when running or walking). I cannot think of a way to rectify this but the ability to input speed information with a wiimote (either by a button press or by pitch or yaw info). [An idea - would attaching a wiimote or nunchuk to the users calf/ shin be able to determine speed information via the accelerometers and then switch to a running/ walking bank of samples when appropriate?]
Screenshot of the PD patch -
I'm finding that the possibilities of the wii equipment are more extensive than I first thought. I'm wondering if this should take priority over the Arduino experiments.
Thursday, 27 June 2013
Wiimote Gunshots
A wiimote was used to trigger gunshot samples using PD/ a sampler in Live.
After hours of experimenting with various expressions the [threshold] object in PD solved the problems I was having with false triggering.
Pressing the B button on the wiimote changes the midi channel - Single shot samples are on channel 1, rapid gunfire on channel 2.
I may create a number of variants of the gunshot sounds, associate them with different MIDI notes in the sampler and introduce an element of random selection in PD, this could add variation but would also potentially eliminate the issue of re-triggering a sample before it completes if only non-repeating MIDI notes are allowed.
The left to right gesture isn't the most intuitive for this sound, up and down may be more so. However, using the pitch values would mean that the threshold values would have to be closer resulting in more false triggers (due to the smaller range of motion in the wrist in the up/down direction compared to left/ right). It's likely that the best solution would be to control gun 1 with the nunchuk and gun 2 with the wiimote, though I was hoping to use the nunchuk to control other parameters.
I'm not sure how accurate the sync would be when using this method with video footage.
PD screenshot:
Tuesday, 25 June 2013
wiimote controlling helicopter sounds
Osculator, pure data, a wiimote and a wii nunchuk were used to control certain elements of a helicopter sound.
In the screenshot below the x axis of the joystick controls the distance parameter while the pitch value of the wiimote controls the engine speed.
Andy Farnell's PD patches are starting to feel quite limiting. A set of multisampled recordings of a vehicle would be useful now with various speeds, gear changes, brake noises etc. These could be triggered from within a PD patch but it may be too complex to do in the time available.
This is a good example of what I have been trying to achieve, though I have no idea how they did it as I can't get hold of the relevant paper - http://www.youtube.com/watch?v=OknJyGV6jwY
In the screenshot below the x axis of the joystick controls the distance parameter while the pitch value of the wiimote controls the engine speed.
Andy Farnell's PD patches are starting to feel quite limiting. A set of multisampled recordings of a vehicle would be useful now with various speeds, gear changes, brake noises etc. These could be triggered from within a PD patch but it may be too complex to do in the time available.
This is a good example of what I have been trying to achieve, though I have no idea how they did it as I can't get hold of the relevant paper - http://www.youtube.com/watch?v=OknJyGV6jwY
Footstep control
I've been trying to adapt the Andy Farnell footstep PD patch to be controlled by the wii fit balance board. This is proving to be difficult because the PD patch isn't designed to be controlled step by step but has a built in system to simulate walking.
Using the wii board to control simple footstep samples is inadequate because this does not take in to account the force/ pressure applied by the foot, or which part of the foot strikes first.
I have found an interesting paper on the subject -
http://vbn.aau.dk/files/37619504/Turchet_Serafin_Dimitrov_Nordahl_DAFx10_P50.pdf
If anything this may just prove that this is too complex and would probably deserve a project of its own.
This paper provides another approach using sensors built into shoes -
http://www.speech.kth.se/prod/publications/files/3441.pdf
This paper provides another approach using sensors built into shoes -
http://www.speech.kth.se/prod/publications/files/3441.pdf
Thursday, 13 June 2013
IR Sensor
A sharp IR sensor was used to output MIDI CC messages.
The results were erratic without smoothing, so this was applied taking an average from 5 readings (10 was too unresponsive).
At present the IR sensor only gives readings of approximately 20 - 80 (once converted to MIDI friendly numbers) when 0 - 127 is expected and wanted. I assume that the IR sensor needs to be calibrated, but I'm not sure how to do this yet.
The sensor was connected to analog input 0.
#include <MIDI.h>
const int numReadings = 5; //the readings from the analog input
int readings[numReadings]; //the index of the current reading
int index = 0; //the index of the current reading
int total = 0; //the running total
int average = 0; //the average
int midiout = 0; //midi out
int inputPin = 0; // analog pin for reading the IR sensor
void setup() {
MIDI.begin(); // Launch MIDI with default options
for (int thisReading = 0; thisReading < numReadings; thisReading++)
readings [thisReading] = 0;
}
void loop() {
float volts = analogRead(inputPin)*0.0048828125; // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
float distance = 65*pow(volts, -1.10); // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
delay(100); // arbitary wait time.
total= total - readings[index]; //subtract the last reading
readings[index] = analogRead(inputPin); //read from the sensor
total= total + readings[index]; //advance to the next position in the array
index = index +1;
if (index >= numReadings) //if we're at the end of the array...
index = 0; //...wrap around to the beginning.
average = total / numReadings; //calculate the average
MIDI.sendControlChange(16, average /8, 1);
delay (1);
}
This code was used to control the distance parameter of a helicopter PD patch taken from Designing Sound by Andy Farnell:
Blue highlighted elements were newly added and consist of a simple ctlin object listening for CC messages which then control the distance slider. As it stands the distance increases as the object is moved closer to the sensor which is obviously counterintuitive, simply switching the slider settings from 0-1 to 1-0 did not rectify this.
Ultimately I get the impression that an IR sensor may not be of too much value to this project due to its inaccuracy. Perhaps an ultrasonic distance sensor would be of more use.
The results were erratic without smoothing, so this was applied taking an average from 5 readings (10 was too unresponsive).
At present the IR sensor only gives readings of approximately 20 - 80 (once converted to MIDI friendly numbers) when 0 - 127 is expected and wanted. I assume that the IR sensor needs to be calibrated, but I'm not sure how to do this yet.
The sensor was connected to analog input 0.
Arduino code is below. It makes use of elements taken from the smoothing example sketch, and an IR sketch found at http://luckylarry.co.uk/arduino-projects/arduino-using-a-sharp-ir-sensor-for-distance-calculation/ as well as some original elements and the Arduino MIDI library.
#include <MIDI.h>
const int numReadings = 5; //the readings from the analog input
int readings[numReadings]; //the index of the current reading
int index = 0; //the index of the current reading
int total = 0; //the running total
int average = 0; //the average
int midiout = 0; //midi out
int inputPin = 0; // analog pin for reading the IR sensor
void setup() {
MIDI.begin(); // Launch MIDI with default options
for (int thisReading = 0; thisReading < numReadings; thisReading++)
readings [thisReading] = 0;
}
void loop() {
float volts = analogRead(inputPin)*0.0048828125; // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
float distance = 65*pow(volts, -1.10); // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
delay(100); // arbitary wait time.
total= total - readings[index]; //subtract the last reading
readings[index] = analogRead(inputPin); //read from the sensor
total= total + readings[index]; //advance to the next position in the array
index = index +1;
if (index >= numReadings) //if we're at the end of the array...
index = 0; //...wrap around to the beginning.
average = total / numReadings; //calculate the average
MIDI.sendControlChange(16, average /8, 1);
delay (1);
}
This code was used to control the distance parameter of a helicopter PD patch taken from Designing Sound by Andy Farnell:
Blue highlighted elements were newly added and consist of a simple ctlin object listening for CC messages which then control the distance slider. As it stands the distance increases as the object is moved closer to the sensor which is obviously counterintuitive, simply switching the slider settings from 0-1 to 1-0 did not rectify this.
Ultimately I get the impression that an IR sensor may not be of too much value to this project due to its inaccuracy. Perhaps an ultrasonic distance sensor would be of more use.
Subscribe to:
Posts (Atom)