intelligent digital effects

project sound examples

2007-05-22T13:43:00.000+02:00

For now this project will rest a few months (exams). We made a few sound examples to let you hear some effects, and a few concrete things one can do with an intelligent effect patch as explained in the final written project report.
In the folder effects one finds several examples of DAFx.
In the folder iDAFx one finds the five basic tracks used to mix a song and then a showcase called song.

tap

2007-05-16T00:02:00.000+02:00

tap

This block outputs a bpm value when triggered by a bang that can come from a MIDI message (eg a drum pad). When it loses track because the trigger is not periodic enough, it restarts and for a moment (2 taps) holds the previous tempo.

version

v1.0: built 14/05/2007 with Pd v0.40-2 [pd][pdf]
based on http://www.parasitaere-kapazitaeten.net/Pd/pdpatches

interface

Input 1: the inlet that receives triggers, anything counts: a bang, a value,...
Input 2: a reset knob: resets the counting when tempo is lost. Also happens automatically.

Output 1: the tempo in beats per minute (bpm)
Output 2: a bang when tempo is lost. It's not necessary to connect this with the reset input. It's taken care of inside

implementation

To calculate the tempo, this algorithm uses three steps: it calculates the interval between the previous click and the current click, it calculates the mean inter-click interval time and it computes the bpm value of this. Then as an extra step the last value before losing the tempo and dropping to 0bpm is remembered by only updating the outlet if it's not 0 (0 can never be a tempo of a song, therefor it's allowed)

The time between the current and previous tap, beat or click is calculated by a timer object, which does just that. A simple counter is used to start this process only after two taps, each time the reset restarts the counter.

Then the mean is calculated by adding the new value to the sum of the previous values (this last sum is made by multiplying the last mean value by the last counter index, so a small backward recalculation is done). This is each time divided by the current counter value (indicating the number of terms of the sum).

Another step decides if the tempo is between the right values, that it doesn't deviate too much. If it does, the whole thing is reset.

round

2007-05-15T22:49:00.000+02:00

round

This block extends the integer calculations of Pure Data by allowing to use a rounding value of a float.

version

v1.0: built 10/05/2007 with Pd v0.40-2 [pd][pdf]

interface

Input 1: a float control signal input

Output 1: an integer control signal output signal

implementation

A floor operation is performed on the value. This floored value is subtracted from the original version, which gives a value between 0 and 1. If this value is greater than or equal to 0.5, then the int block which holds the floored value + 1 is banged, otherwise the int block with the floored value is banged. The last int block is just to collect signals, it's not necessary for the correct functioning of this algorithm.

dynamics

2007-05-15T16:05:00.000+02:00

dynamics

This block is a versatile dynamics processing block that, depending on the parameters on its inlets is a limiter, compressor, expander or noise gate. It can compress, suppress and expand at the same time. Four regions can be chosen, each with their own behaviour. One has to use an argument when using multiple instances of this block in the same patch.

version

v1.0: built 14/05/2007 with Pd v0.40-2 [pd][pdf]

interface

Input 1: the (mono) audio signal input
Input 2, 3, 4: The different thresholds expressed in a dB scale ranging from 0dB to 100dB. (this means that a threshold of -20dB equals 80dB in this scaling)
Input 5, 6, 7, 8: The different ratios R that indicate the behaviour of what happens if the signal level reaches that part.

limiter	R = infinity
compressor	1 < R < infinity
linear	R = 1
expander	0 < R < 1
noise gate	R = 0

Output1: the output audio signal

Display:
In the abstraction itself one can see the input signal level compared to the output signal level and compared to the unity level 100dB (or 0dB). Also g(k) is displayed against unity.

implementation

We use a scheme based on DAFx (p37). The level of the signal is measured to obtain the instantaneous RMS in dB with the env~ block. We take an RMS over 2048 samples. Some moses blocks route X(dB) the right way. Then the following calculation is done:

g(k) = [th + ( X(dB)-th )/R] / X(dB)

The float object f does nothing else but collecting the four routes.
The delay on the original signal is meant to align the samples but is now set on 0, because we haven't made a method yet to discover how long the delay in the other leg is. There is a certain drift anyway, because the gain factor g(k) is often greater than 1 even when all ratios R are 1. TODO

pitchShifter~

2007-05-09T17:36:00.000+02:00

pitchShifter~

This block shifts the pitch over some halftones, up or down according to the sign of the first inlet. There is a variable delay between the minimum delay,
corresponding to the third inlet and the the minimum delay + the size of the window, corresponding to the second inlet.

version

v1.0: built 8/05/2007 with Pd v0.40-2 [pd][pdf]

interface

Input1: incoming mono audio signal
Input2: number of halftones up or down depending on sign of the input
Input3: the size of the window in ms
Input4: the minimum delay in ms

Output 1: original signal shifted over a number of halftones specified by input2

implementation

The desired pitch shift in halftones(num) is converted into a transposition factor t= 2^(num/12)= e^(num*log(2)/12) = e^(0.05776*num).
In the tape head rotation speed part f is calculated with f = [(t-1)*Fs]/s (Fs= samplerate and s= size window). This f is used to calculate
the saw tooth signals and corresponding envelopes.
The minimum delay (input3) is added to both sawtooth signals to make the delay input for the vd~ blocks.
The fact that two copies are made (one is shifted in phase with respect to the other one --> 0.5 correspondig to 180 degrees)
is to avoid discontinuities due to sampling.
Finally both vd~ block outputs are then multiplied by the corresponding envelopes and summed.

multitapdelay~ / reversedelay~

2007-05-09T15:55:00.000+02:00

multitapdelay~ / reversedelay~

These blocks are derivatives of the very general stereomultitapdelay~ block. They represent specific cases that are nice to use in practice. The multitapdelay~ has 10 delay taps that each are DLY (delay time) delayed and the next tap is always FBK (feedback level) attenuated. One can choose which taps are played back. The new settings herefore work only when the FBK is touched again. An argument must be specified when one wants to use multiple delays in one patch. The reversedelay~ does exactly the same but each tap will incremently play louder.
For both effects one can choose to give a pingpong effect by adjusting the panning of all taps. The uneven taps evenly go left, the even taps evenly right for positive PAN values.

version

v1.0 [pdf]

pan~ v1.1 [pd]

v1.0: built 9/05/2007 with Pd v0.40-2 [multitapdelay~.pd] [reversedelay~.pd]

interface

Input 1: the signal you want to delay.
Input 2: the delay time parameter DLY. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec tot 3000msec (3 seconds). This is the length of the delay line used in the implementation and is thus an absolute maximum.
Input 3: the feedback parameter FBK. The first tap is attenuated (amplified) with this value divided by 100 (1000) for the multitapdelay~ (reversedelay~). In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of hundreds/thousands. The system can't get instable because there are no feedback loops involved. But when the FBK value is 100 (internally 0.1) for the reversedelay~, the last tap is at its maximum, being as loud as the original signal.
Input 4: the panorama parameter PAN. The value ranges between -127 and 127. They are applied evenly to each tap, but the uneven ones with a reversed value.

Output 1, 2, 3, 4, 5, 6, 7, 8, 9, 10: the delayed, wet taps
Output 11, 12: the delayed, wet stereo signal.

implementation

At the left the delay part is shown. Each tap will receive it's final DLY parameter here. The multiplication with 1 is a leftover from the general stereomultitapdelay~ implementation which also provided feedback loops here. At the bottom of this image the two main outlets are shown.
Everything is labeled with dollar arguments to ensure unique working when using multiple instance of delays.
In the image above the inlet is shown and also the section that calculates all the DLY and FBK parameters. For the multitapdelay~ here it is:

DLYi = SUMi(DLY)
FBKi = 1(or 0)*(FBK/100)^i

That means the DLY parameter is linearly raised and the FBK parameter is exponentially attenuated.
The next image shows this part for the reversedelay~ (it's the only section that differs for both effects). The feedback is now linearly raised.

DLYi = SUMi(DLY)
FBKi = 1(or 0)*SUMi(FBK/1000)

The next image shows the section that controls the on/off position of each tap and shows the position of each taps' parameters. The loadbang block and set 1 message are there to initialize the patch to all taps on.

The next section shows how the panning is controlled and shows also the rest of the taps.
The uneven taps get a reversed value of the PAN parameter. Each tap is connected to an outlet before entering the panning logic. After the panning logic the both channels are routed to the main outlets. For information of the pan~ function one must look at the respective documentation.
TO DO: put the outlets of each tap in a specific logical order

ringmod~

2007-05-09T10:44:00.000+02:00

ringmod~

A type of audio mixer combining two audio signals, and outputting their sum and difference. The frequencies found in the original signals are not passed through to the output. For example, if two sine waves (single frequency waveforms containing no overtones) are inputted, one with a frequency of 1000 Hz, and the second at 400 Hz, the ring modulator will output two frequencies: 600 Hz and 1400 Hz. With more complex waveforms (which contain many more overtone frequencies) ring modulators produce a clangorous, "metallic" result often used for special effects, in synth programming, and so on. One popular use has been to process vocals, which produces sci-fi sounding "robotic" voices. [sw]

version

v1.0: built 12/04/2007 with Pd v0.40-2 [pd][pdf]

interface

Input 1: the (mono) audio signal input
Input 2: the modulation frequency, it's got no real boundaries but typical values are presented in the image on top.
Input 3: a reset inlet. When this is banged, the modulating wave is reset to it's beginning. Look at the documentation of the waves block.

Output1: the ring-modulated output audio signal

Controls:
MIDI commands for choosing different oscillators for modulation. View the documentation of the waves block.

implementation

Digital ring modulation is nothing more than a multiplication of a signal with another signal. The modulating signals' frequency is very low when one wants to hear the modulating envelope. For ring modulation it's faster. The multiplication is an arbitrary scaling of the modulation frequency input and can be left away. The outlet is high pass filtered to forbid the lowest frequencies (inaudible 5Hz and less) to slip through.

octaver~

2007-05-08T20:08:00.000+02:00

octaver~

This block has two outputs, respectively the input signal one octave higher and the input signal one octave lower. The synthesised octave effect is obtained by doubling (octave-up) or halving (octave-down) the frequency of the original input signal. This is possible due to the simple two-to-one relationship between the frequencies of musical notes which are separated by an octave.

version

v1.0: built 8/05/2007 with Pd v0.40-2 [pd][pdf]

interface

Input1: incoming mono audio signal

Output 1: original signal one octave lower
Output 2: origininal signal one octave higher

implementation

For the left output (octave higher) the purpose is to take out the odd harmonics. The result is a signal with only the even ones left, which sounds an octave higher. The spectral envelope of the input will remain roughly unchanged. In this way we avoid the chipmunk effect resulting by using a speed change to obtain a pitch shift. To do so we use a variable comb filter without feedback (non recirculating) that combines two delayed copies of the incoming signal. One copy with a fixed delay (delread~) with a window size set to that one of the pitch following algorithm. The delay in ms is adjust to the output of fiddle~: 1000.2048/Fs with Fs= sample rate. (where mtof is used to translate MIDI pitch to frequency) The variable delay is the same, plus 1/2 of the measured period of the input signal, 1000/(2f) ms where f is the frequency in cycles per second. These two delay times are added and smoothed with line~ and then used as input in the variable delay line vd~. The resonant frequencies of the resulting comb filter are 2f,4f,6f,... because the difference between the two delays is 1/2f. The the response is zero for f,3f,5f.... Thus the odd harmonics have been filtered out. You can consider this operation in a different way. The incoming signal is output twice, a half cycle apart: odd harmonics are shifted 180 degrees and cancel eachother out. The even ones are in phase, so they don't.

For the right output (octave lower) we use ring modulation. The ring modulation itself happens in osc~ and with the multiplier block. Multiplying with 2 is done to get the original better balanced. Again we get the fundamental frequency from the fiddle~ block (after conversion from MIDI pitch). This is then of course multiplied with 1/2 to get one octave lower.
The moses is used to stick the most recent good guess, because when fiddle fails it outputs 0. Because fiddle~'s analyse is most closely aligned to the middle of the window(2048/2) a standard pd delay block of fixed 1024 samples of the original signal, can be used for synchronization. You can change this to get a different effect. It is also possible to change the factor 1/, by example a large integer (15) and you'll also get a completely different effect (introduction of formants).
In this implementation we use osc~ to get the ring modulation. You have to notice that osc~ only uses the sine wave to modulate the original signal.
We could use our own waves block to get more possibilities.

traditionaldelay~

2007-05-08T19:38:00.000+02:00

traditionaldelay~

This block delays an incoming signal (two first inlets to allow for mono and stereo signals) and outputs the wet, delayed signal(s) on one channel. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop. Users can vary them by using the two other inputs. In the display one can see the current position.

version

v1.0: built 8/05/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1, 2: the signal you want to delay. If only input 1 (or 2) is connected, only output 1 (or 2) will give a result.
Input 2: the delay time parameter DLY. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec tot 3000msec (3 seconds). This is not only the maximum value of the slider, but also the length of the delay line used in the implementation.
Input 3: the feedback parameter FBK. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands.

Output 1, 2: the delayed, wet signal

implementation

The delay itself is pretty simple: the incoming signal is read by a delread~ objects that write the signal into delay lines called $1-delayLeft and $1-delayRight and with length 3000ms. Although there's no visible loop, this is how the loop is formed: after a delay time DLY the associated delread~ object reads what's in the delay line, outputs it and sends it back into the delwrite~ object. Before outputting the delayed signals they are attenuated every time by a factor FBK ranging between 0 and 0.999 (with 1 clipping is to easy), in steps of thousands. That's why the control parameter lies between 0.001 and 999.
The DLY parameter of each channel is offset by the addition with the two last inlets.
Just before entering the outlet the signals are high pass filtered for very low frequencies to not reach the outlets.
TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline, as can be seen in the reverb blocks of this project.

control curves

2007-05-06T22:23:00.000+02:00

control curves

v1.0 [pdf]

As a last step for making iDAFx, when the effects and the feature extractions are already existing, counts the control curve. The best work already made on it is from the hand of Verfaille and Arfib.

The feature extraction stage ends with a curve of values that change. Before entering the parameter curve/values of the effect a few adaptations must be made:

combine
One can choose (it's optional) to combine several features. These can be added as a linear combination, each with their specific weight. Also must the signal be normalized. A block is made for this: combine.
map
Several mapping strategies have to be provided, depending on the features and the effect parameter one wants to control. A lot of experimenting may be done with all possibilities. Different blocks are made for this, they all have one input and one output, both normalized: mapLinear, mapSine, mapTrunc, mapTime,...

Also other mappings can be used. mapping is nothing more than applying a function. More window-like functions can be used too: hamming, hann, gauss,... and also non-symmetrical windows come in handy. View the documentation in the document mapWindow. Some other techniques are also presented is this documentation (eg bpm2time )
scale
In the last step the curve must be shifted and scaled to the appropriate range of the effect parameter: scale.

Sometimes it's better to couple the feature and the parameter directly (for example: bpm feature, bpm effect parameter). Sometimes it's nice to control waves (from eg the waves block) using features and let these waves control parameters of effects. View the document about waves. One must always pay special attention to interfaces.

Hereunder follows the documentation of the blocks used for the technique described in the above mentioned paper.

combine

This block makes a linear combination of the inlets. The factors can be chosen from the display. The inlets must be normalized before entering the block. The output is normalized too.

version

v1.0: built 29/04/2007 with Pd v0.40-2 [pd]

interface

Input 1, 2 and 3: control signals to combine. They have to be normalized before entering the block. One can also use only two inlets.

Output 1: a normalized control signal that is the linear combination as chosen from the display.

Controls:
number box 1, 2 and 3: The factors a, b and c in the equation y = ax1 + bx2 + cx3.

implementation

The equation used here is:

y = (ax1 + bx2 + cx3)/(a+b+c)

The bang blocks make sure that it doesn't matter which inlets are used or not used. Either way the sum will be updated, even though a cold inlet might be used (that normally would not update that sum). If the inlets are normalized (and this is assumed), the division by the sum of all coefficients will normalize the output too.

scale, scaleS

This block scales a normalized input to a range that is specified in the controls on a display or at the inlets.

version

v1.0: built 29/04/2007 with Pd v0.40-2 [scale.pd][scaleS.pd]

interface

Input 1: Input control signal ranging between 0 and 1
Input 2, number box m: lower boundary of the target range
Input 3, number box M: upper boundary of the target range

Output 1: a scaled version of the input signal

implementation

For both instances the following mathematical formula is performed:

y = (M-m)*x+m

mapLinear/mapLin

This block performs a linear mapping between a normalized input and a normalized output. That constraint is not even necessary, the only real constraint is that the curve starts at zero. But for mapping purposes the signal is mostly normalized. Include an argument number!

version

v1.0: built 29/04/2007 with Pd v0.40-2 [mapLinear][mapLin]

interface

mapLinear:
Input 1: control signal
Input 2, 3, 4: knee of the curve: x1, x2, x3. These knees correspond with thresholds in dynamics processing
Input 5, 6, 7, 8: slopes of each line: a1, a2, a3, a4. The slopes correspond eg with compression factors: 2:1 would give a slope of 1/2 or 0.5
Output 1: the normalized result of the linear curve

mapLin:
Input 1: control signal
Input 2: slope of the line a
Input 3: y-intercept of the line b

implementation

This patch does nothing more than performing the formulas you find in the graphs in the section interface. To check where x is situated, moses blocks are used.

mapSine

This block maps the normalized input according to a sinusoidal curve (as one can see in the figure in the preamble).

version

v1.0: built 29/04/2007 with Pd v0.40-2 [pd]

interface

Input 1: a normalized input control signal
Output 1: a mapped signal output according to the given curve. It ranges from 0 to 1.

implementation

The given formula is performed with Pure Data blocks. The sin~ block is a phase shifted version of the cos~ block. View the documentation of sin~ if needed. Because it outputs an audio signal, the bang object bangs a snapshot~ object that samples and holds the value of that moment. Because the bang block is connected with the output of the calculation, each change will result in a new sample at the output of the snapshot~ block.

mapTrunc/mapTruncS

This block maps a chosen part of the normalized input to the output (as one can see in the figure in the preamble). This mapping curve is a very basic limiter and noise gate. The threshold of the noise gate would be tm and the threshold of the limiter would be tM. Include an argument number!

version

v1.0: built 29/04/2007 with Pd v0.40-2 [mapTrunc][mapTruncS]

interface

Input 1: a normalized input control signal
Input 2, number box tm: lower boundary of truncation
Input 3, number box tM: upper boundary of truncation
Output 1: a mapped signal output according to the given curve. It ranges from 0 to 1.

implementation

In comparison with other implementations in this document, nothing new is done, only another formula. The formula performed here with Pure Data blocks is:

y = [tm (if x<tm) + tM (if x>tM) + x (if tm<x<tM)]/(tM-tm)

mapTime/mapTimeS

This block time stretches the original control signal. The time-stretched parameter alpha divides the original range into two parts: the lowest will be contracted to sm, the upper dilated to sM (as one can see in the figure in the preamble). Include an argument number!

version

v1.0: built 29/04/2007 with Pd v0.40-2 [mapTime.pd][mapTimeS.pd]

interface

Input 1: a normalized input control signal
Input 2, number box alpha: time-stretched parameter that realizes the boundary between contraction and dilatation.
Input 3, number box sm: contraction factor smaller than or equal to 1
Input 4, number box sM: dilatation factor greater than or equal to 1
Output 1: a mapped signal output according to the given curve. It ranges from 0 to 1.

implementation

As with the other blocks in this document a formula is calculated with PureData blocks. Care is taking to include dollar arguments to ensure multiple and correct use in the same patch. The formula calculated is:

y = sm^((alpha-x)/alpha) (if x < alpha) + sM^(x.alpha/(1-alpha)) (if x > alpha))

bpm2time

This block calculates the period between two beats or any division of it. It can be used to calculate a certain delay time parameter.

version

v1.0: built 29/04/2007 with Pd v0.40-2 [pd]

interface

Input 1: a bpm value. Best somewhere between 40bpm and 250bpm, but it works for every value given.
Input 2: the division (DIV) parameter specifies how to divide the period between two beats. Here's a list with values that can be used on instruments like the Korg Microkorg and Roland SH32. The ones with an asterisk are whole divisions of a period. The others need several periods to synchronise again with the first count of a measure. A block is made for this purpose: measure. It has buttons for every possibility.

1/1 DIV=1 *
3/4 DIV=0.75
2/3 DIV=0.66666
1/2 DIV=0.5 *
3/8 DIV=0.375
1/3 DIV=0.333333 *
1/4 DIV=0.25 *
3/16 DIV=0.1875
1/6 DIV=0.16666 *
1/8 DIV=0.125 *
3/32 DIV=0.09375
1/12 DIV=0.08333 *
1/16 DIV=0.0625 *
3/64 DIV=0.04687
1/24 DIV=0.04166 *
1/32 DIV=0.03125 *

Output 1: a time parameter in msec. 60bpm=1000msec

implementation

This is the measure block implementation. All in the red box is part of the display (by a graph-on-parent).
The mathematical formula for this calculation is:

DLY = DIV*60000/BPM

This formula come from 60sec/minute and 1000msec/second. The bang block is provided to make sure that the output is also updated when only the DIV inlet is changed, although it is attached to the cold inlet of the multiplication block.

chorus

2007-05-03T22:51:00.000+02:00

chorus~

This block simulates the very famous chorus effect. It adds several variants of the input signal to the input signal as if it was
several people playing at the same time. The imperfections of such a choir are simulated with variable delays, variable phase and variable volumes.

version

v1.0: built 28/04/2007 with Pd v0.40-2 [pd][pdf]

interface

Input 1: the mono input signal
Input 2: the frequency of the variation of the delays of the four outputs (all the same). This is expressed in Hertz. Because we're talking about a control signal or modulation here, it gives the best results to use small frequencies like 1Hz.
Input 3: The modulation level. This changes how great the variation is of the delays. 4 is a good value.
TO DO: add default values that load at start

Output 1, 2, 3, 4: These are four variably delayed output signals with a phase difference of Pi/2. They can be added to each other with or without specific weights.

implementation

The input audio signal is put in a delay line with a maximal length of 1000msec. This extract will be read by four identical variable delays whose delay time is controlled by the following logic: a phasor~ objects' frequency is taken from the inlet. When the frequency is changed at the given inlet, the line~ object makes this a smooth change with a default ramp of 50msec before reaching the new value. This sawtooth signal is then used as an input for several cos~ blocks. It's nothing more than a run through all values between 0 and 1, repeating with the specified frequency. Each cos~ block receives a sawtooth with a different phase, a difference of Pi/2. The delay times will thus be out of phase, giving different delay times. These cosine signals are shifted to positive values between 0 and 2 with the [+~1] block. The a modulation value (also smoothed when changed) is multiplied with the cosine making it more or less expressive by changing it's amplitude. With this final cosine signal a variable delay is driven and output via the outlets.

mapWindow

2007-04-29T16:21:00.000+02:00

mapWindow

v1.0 [pdf]
v1.1 [pdf]
In this document one can find everything about the mapping stage with windows, used for our iDAFx. Four possible windows are presented (but adding some others is a piece of cake). For each window a big one and a small one are possible. The big one is 1024-points large and the small one only 128, which agrees with the range of the MIDI message parameters. The effect is the same, but the latter is obviously less accurate.

windows.m is the m-file that is used to make these windows. Every table is then saved in a textfile and this one is used inside Pure Data.
[hammBig.txt] [hammSmall] [blackmanBig.txt] [blackmanSmall.txt] [gaussBig.txt] [gaussSmall.txt] [hannBig.txt] [hannSmall.txt] [windows.m]
We supply two blocks for applying a window to a control signal: windowBig and windowSmall. The documentation is given hereunder:

windowBig/windowSmall

This block applies a 1024-points/128-points window to a control signal. One can choose between different windows. One has to define an argument to use multiple instances of this block in the same patch without influencing each other.

version

v1.0: built 29/04/2007 with Pd v0.40-2 [pd]/[pd]
v1.1: built 30/04/2007 with Pd v0.40-2 [pd]/[pd]

interface

Input 1: a control signal ranging between 0 and 1024/128

Output 1: a control signal ranging between 0 and 1

Controls: a knob per window type

implementation

The values for each window are read from a .txt file (that have to be found in the same folder as the patch). Upon clicking a button, a bang is sent to the appropriate read message, and the right curve is loaded in the graph. The tabread block is then used to map the inlet to the outlet.
In v1.0 one could not use multiple instances of this block in one patch, because there was an unsolved problem with arguments. That is now solved with the help of Frank Barknecht. In two help-patches he explains all our oddities with dollarsigns. [dollars-help.pd] [dollars.pd] And his most beautiful solution was adopted, the one with the sends, which at the same time doesn't have the problem of difference between arguments in objects and arguments anyway.

weekly update

2007-04-24T18:52:00.000+02:00

What's been keeping us busy:

We're working on the beat detection block for pure data. There keep existing some problems in matlab but they might soon be solved after thoroughly skinning and optimizing the code.
As for me I implemented some extra blocks, mostly quite basic: the EARdetector (envelope, attack and rest), ring modulator (that also functions as tremolo), chorus and something more special. The documentation will follow.
Also we started writing our thesis by making a contents summary. This will happen in LaTeX, something new for us...
Also I've been and I still am trying to understand C for Pure Data externals.

mixer

2007-04-21T17:41:00.000+02:00

mixer

This is collection of a few patches we made to use in main patches for using our effects. It is more than a simple mixer, but it certainly hasn't got all the functionalities of a professional mixer. It has got all that might be useful for our effects project: gain control, mute control, external input or wave files, many channels that are all summed on a bus to the master,... A functionality like sends and returns is not implemented. The structure that is used (apart input and channel patches) implicitly has this function, because one can attach multiple signals to them.
This document documents on the patches.
v1.0 [pdf]
v1.1 [pdf]
new block added; channel bug temporarily fixed

inputmulti~

This block implements a multichannel input. One can choose ten different wave files or five stereo files. When using wave files one can loop them. One can internally adjust the input gain of the signal to the best non-clipping level. The gain, clip and instantaneous RMS level is displayed on the block. It's best to not use more than one instance of this block in the same patch.

version

v1.0: built 15/05/2007 with Pd v0.40-2 [pd]

interface

To choose other sound files, one must open the input patch, specify the soundfile in the right folder in a message and click on that message.

Output 1, 2, 3, 4, 5, 6, 7, 8, 9, 10: a levelled mono audio signal

Internal controls:
start: start the chosen wave file. ctlin message 48.
play: when the wave file is done it is immediately replayed. ctlin message 49.
TO DO: we might better call this a loop button or something.

implementation

The wave playback section: each tabplay~ block starts playing when it receives a bang. It then plays the wave file that is loaded in the array with its name, found in the graph at the bottom left. One can choose another one by specifying a new soundfile in a message block. When the tabplay~ block is finished playing a track, it bangs the right output. This bang is used to let the message 1 allow the multiplication (when it's zero or without banging the hot input of the multiplication, the bang that follows will not be reached). The play toggle has a value of 127 to be consistent with the MIDI message value. A select block selects this value and bangs the tabplay~ block to replay the track.
A simple section is provided to adjust single gain factors per channel. Just specify another factor in the right message and click on it.

This implementation leaves away many features of the other input. But we found a bug there: if one wants to play different tracks he cannot. Due to the dollar problem with arrays it would change the array of each input~ block.

input~

This block implements a dual input. One can choose to use wave files or to go live with an external input (adc~). Both are calculated real-time. When using wave files one can loop them. When using the adc~ one can choose which channel of the audio card is routed to left and right channel. One can adjust the input gain of the signal to the best non-clipping level. The gain, clip and instantaneous RMS level is displayed on the block. When using more than one input~ block in a patch be sure to include an argument after the block name.

version

v1.0: built 20/04/2007 with Pd v0.40-2 [pd]

interface

To choose other sound files, one must open the input patch and click on a specific message that contains the file

Output 1: The mono audio signal or the left channel audio signal
Output 2: The right channel audio signal

Controls:
source for left, right: One can choose a channel from 1 to 8 from the soundcard that is available. When another routing is chosen, you must first enter 0 too. Otherwise you will be adding different channels from your soundcard to the same outputchannel of this block. If you are playing wavefiles it's best to put these sources to 0.
start: start the chosen wave file. ctlin message 48.
play: when the wave file is done it is immediately replayed. ctlin message 49.
TO DO: we might better call this a loop button or something.
reset: reset the gain to 0
gain slider: adjust the gain. the number box next to it only shows the chosen gain value, it does not change it!
clip: When the RMs of the signal exceeds 100dB (the zero attenuation level) this knob lights up in red.
VU-meter: shows the level of the signal at any time. You can also use it to see if it's a stereo signal or not.
TO DO: implement peak meter

implementation

The gain system: when reset is pressed, a message is banged that will send a 0 to the gain slider. The gain slider is he first control in the gain system, the rest is linked from there. The number box thus only shows results and does not adapt them. The slider ranges from -100 to +20 dB. It is then shifted and recalculated to RMS values before being used as an attenuation factor at the output, equal for both channels. The clip bang lights up in red when the moses block detects a RMS value higher than 0dB on the left channel.
TO DO: also check the right channel for clipping.
The wave playback section: the tabplay~ block starts playing when it receives a bang. It then plays the wave file that is loaded in the arrays named leftInput and rightInput, both contained in the graphs at the top right. One can choose another one by clicking on the specific message bottom left. When the tabplay~ block is finished playing a track, it bangs the right output. This bang is used to let the message 1 allow the multiplication (when it's zero or without banging the hot input of the multiplication, the bang that follows will not be reached). The play toggle has a value of 127 to be consistent with the MIDI message value. A select block selects this value and bangs the tabplay~ block to replay the track.
The VU meter section: The VU meters have to receive control messages in dB values ranging from -100 to +112. The output signals are therefore sampled with the snapshot~ block and recalculated from RMS to dB. In the future we might replace this wit env~ blocks or EARdetection~ blocks.
The adc source selection: The signal that is input in a specific input of your soundcard is multiplied by a 0 or a 1 depending on which number you input in the number box in the display. If you select two times another valid number for the same audio channel, they will be both output. So be careful to also select another number (0) to mute all the inputs.

channelM2M~

This is one of the channel blocks. It's the bottom part of channels on a real mixer. It allows to make all kinds of routing in your patch. It's output is routed to a master~ block to make it audible. When multiple blocks of this kind are used, one must specify a unique argument. This is almost always the case. No outlet is visible, because the signal is put on the right (depending on the signal: left or right channel bus of...) master bus internally. Be sure to include a master~ block in your patch!
In this block one can mute this channel, reverse the signal and adjust the level. A VU-meter is also visible to follow what happens. This specific block is one that has no counterpart in a real mixer, because you cannot put it in a stereo image. But it is useful though as an inline amplification block.

version

v1.0: built 20/04/2007 with Pd v0.40-2 [pd]

interface

Input 1: The mono audio signal
Input 2: A control input for the dB level. It ranges from 0 to 127, the former meaning silence, the latter the loudest.

Controls:
solo: doesn't work yet
mute: when enabled, the signal is not throughput to the masterbus. When released again, the last value is taken again.
reverse: when enabled, the phase of the signal is reversed. This is not audible per se, only in comparison with other tracks or itself.
dB-slider: see Input 2.
VU-meter: shows the level of the signal at any time.
TO DO: implement peak meter

implementation

The dB-level is divided by 1.27 to make the input of the dbtorms range from 0 to 100. This is smoothed (line~) and multiplied with the high passed signal at the input of the block. This signal is not connected to the hot inlet as normal, but this works because the level control signal is turned into an audio signal in the line~ block.
The reverse logic is the same as in the rev~ block [v1.0 pd] [v1.0 pdf]. In the future it might be substituted here.
The logic for driving the input of the VU-meter is a dB calculation and a shift to the right scale. But this might be replaced by an env~ block in the future.
The mute logic: If mute is banged the message 0 is put as master-lvl while the previous value is stored in the bottom f(loat) object. Another bang releases this value again on the master-lvl.

channelM2S~

This is one of the channel blocks. It's the bottom part of channels on a real mixer. It allows to make all kinds of routing in your patch. It's output is routed to a master~ block to make it audible. When multiple blocks of this kind are used, one must specify a unique argument. This is almost always the case. No outlet is visible, because the signal is put on the left and right master bus internally. Be sure to include a master~ block in your patch!
In this block one can mute this channel, reverse each channel signal and adjust the level. A VU-meter is also visible to follow what happens. One can also change and reset the place of the signal in the stereospectrum with the PAN parameter.

version

v1.0: built 20/04/2007 with Pd v0.40-2 [pd]

interface

Input 1: The mono audio signal
Input 2: A control input for the dB level. It ranges from 0 to 127, the former meaning silence, the latter the loudest.
Input 3: A control input for the PAN position. -128 means left and 128 means right.

Controls:
pan: when clicking on the bang the PAN position is reset to zero. The number box can be used to change the position and the result of either a change with the number box, either a change at the corresponding inlet will be visible with the slider. Internally (in the block) one can choose between 4 panning laws, also look at the documentation of the monopanner~). The default law is a sine curve.
solo: doesn't work yet
mute: when enabled, the signal is not throughput to the masterbus. When released again, the last value is taken again.
reverseL, reverseR: when enabled, the phase of the signal is reversed. This is not audible per se, only in comparison with other tracks or itself.
dB-slider: see Input 2.
VU-meter: shows the level of the signal at any time.
TO DO: implement peak meter

implementation

channelS2S~

This is one of the channel blocks. It's the bottom part of channels on a real mixer. It allows to make all kinds of routing in your patch. It's output is routed to a master~ block to make it audible. When multiple blocks of this kind are used, one must specify a unique argument. This is almost always the case. No outlet is visible, because the signal is put on the left and right master bus internally. Be sure to include a master~ block in your patch!
In this block one can mute this channel, reverse each channel signal and adjust the level. A VU-meter is also visible to follow what happens. One can also change and reset the place of the signal in the stereospectrum with the PAN parameter.

version

v1.0: built 20/04/2007 with Pd v0.40-2 [pd]

interface

Input 1, 2: The two channels of the stereo audio signal
Input 3: A control input for the dB level. It ranges from 0 to 127, the former meaning silence, the latter the loudest.
Input 4: A control input for the PAN position. -128 means left and 128 means right.

Controls:
pan: when clicking on the bang the PAN position is reset to zero. The number box can be used to change the position and the result of either a change with the number box, either a change at the corresponding inlet will be visible with the slider. Internally (in the block) one can choose between 4 panning laws, also look at the documentation of the monopanner~). The default law is a sine curve.
solo: doesn't work yet
mute: when enabled, the signal is not throughput to the masterbus. When released again, the last value is taken again.
reverseL, reverseR: when enabled, the phase of the signal is reversed. This is not audible per se, only in comparison with other tracks or itself.
dB-slider: see Input 3.
VU-meter: shows the level of the signal at any time.
TO DO: implement peak meter

implementation

master~

Each analog mixer has an audio chain that ends with a master section. It represents the last step and is connected with the master outs. This block is also the last block in our chain. It puts the two master busses on the speaker outputs. There can be only one master~ block per patch.

version

v1.0: built 20/04/2007 with Pd v0.40-2 [pd]
TO DO: also implement a recording facility for recording examples of our results.

interface

Controls:
mute: when enabled, the signal is not throughput to the speakers. Also the DSP is stopped. One can use it as a panic! knob. When released again, the last value is taken again. When the MIDI value 40 is received mute is toggled.
dB-slider: controls the volume level of each channel. They are both controllable with the MIDI values 30 and 7 (two last sliders on a Korg MICROkorg)
VU-meter: shows the level of the signal at any time.
TO DO: implement peak meter

implementation

For the level and meter logic the same implementation is used as in the examples above.
The DSP is stopped when the mute knob has value 1, with the command pd dsp 0. When it's not 1 (but lower, thus being 0) the command pd dsp 1 puts the DSP (and the sound) back on.

pan~

2007-04-21T15:35:00.000+02:00

pan~

This block is a simple panorama block. It has an input to control the position and works according to the equal power idea, using a sine function.

version

v1.0: built 15/04/2007 with Pd v0.40-2 [pd][pdf]
TO DO: the sine function is implemented with a table. When pan~ is used several times in one patch, they'll all have to work with the same control function (sine in this case). I didn't yet find how to comprise the $ argument in the patch so that every pan~ block works with its own table.
v1.1: built 9/05/2007 with Pd v0.40-2 [pd][pdf]

interface

Input1 : the mono audio signal input
Input2 : the control input; ranging from -127 (LEFT) to 127 (RIGHT) where 0 marks the CENTER position

Output1: the left channel audio signal
Output2: the right channel audio signal

Controls: internally one can choose between 4 panning laws: sine, root, gogin and linear. (see the documentation of the monopanner~)

implementation

The inlet receives a control signal as specified in the interface section. This signal is shifted. It's range is then from 0 to 256, which is the same range as the table used. For every value an attenuation factor is looked up in the table. This is then multiplied with the signal itself.
The initialization is done by setting the control value to zero and initializing the table.
Dollar arguments are implemented to make sure multiple use of pan~ in the same patch allows for unique behaviour.

rev~

2007-04-21T14:56:00.000+02:00

rev~

This block implements signal reverse: a stereo signal enters the block and one can choose to reverse each channel or both by sending a 1 for reverse and a zero for normal.

version

v1.0: built 14/04/2007 with Pd v0.40-2 [pd][pdf]

interface

Input1 : the left channel audio signal input
Input2 : the left channel control input; 0 is normal, 1 is reverse signal
Input3 : the right channel control input; 0 is normal, 1 is reverse signal
Input4 : the right channel audio signal input

Output1: the left channel audio signal, normal or reversed
Output2: the right channel audio signal, normal or reversed

implementation

To reverse a signal a multiplication with -1 is done. A moses block bangs the right message for the multiplication by routing the control input. This message changes the factor of the multiplication.

EARdetector~

2007-04-20T15:38:00.000+02:00

EARdetector~

An envelope, attack and rest detector. This blocks tells us the instantaneous RMS and peak values of a signal, both in dB and amplitude scale. It also gives a bang when an attack is detected or a rest of more than a specified time constant is registered. The attack and rest detection is based on the envelope of the signal.

version

v1.0: built 20/04/2007 with Pd v0.40-2 [pd][pdf]

interface

Input 1: the signal you want to follow. The attack and rest detectors will only work nicely when this signal is amplified to the loudest level without clipping. This is like adjusting the gain on a mixer.

Output 1: in this output three results are packed: a smoothed RMS signal, the instantaneous RMS signal (both in dB) and the smoothed amplitude signal (between 0 and 1)
Output 2: in this output two results are packed: a smoothed peak signal in dB and the amplitude signal of it.
Output 3: receives a bang if an attack is detected. This happens when a threshold of 55dB is reached and the signal was lower than 45dB before it.
Output 4: receives a bang if a rest is detected. This happens whenever a threshold of 45dB is not reached for longer then 100ms

When loaded the rest outlet gives a bang after the specified delay.

implementation

The basic block used here is the env~ block. It's output is in dB. To calculate the RMS amplitude value a dbtorms block has to be inserted.
TO DO: test if different arguments give better results (the argument stands for the interval in msec used to calculate the RMS on)
This signal is smoothed by running it through three lowpass filters with a cutoff frequency of 1Hz. Because the filters work on audio signals, a snapshot~ block had to be attached.
In the help patch about the env~ block that is found in a standard version of Pd, one sees that the peak value is 3.01dB higher than the RMS value.
The different results are packed to two outlets, the first outputs all RMS values, the second all peak values.
The attack detection works as a state machine with two states. When the state is 0 and the signal reaches 55dB or higher, the state becomes 1. When the state is 1 and the signal gets lower than 45dB, the state becomes 0. It might be necessary to adapt these thresholds later.
Also the rest detector works a two-state state machine. If a signal is more than 100msec under the threshold of 45dB the state changes to 1. It might be necessary to adapt these thresholds later.

beat detection

2007-04-16T21:05:00.000+02:00

beat detection

development sheet

In this document blocks and thoughts will be collected that have anything to do with the development of a beat detector in Pure Data. This document (or post) will be updated as long as the first definitive version is not ready.
The basic implementation structure is as follows:
In a main file beat~ everything will be done.
Some smaller topics:

First a few links to previous posts about the topic:
- explanation of the idea
- about bonk~
An add block is made to be able to add more than two arguments at the same time. It receives a list from a pack command with up to 20 elements. When this pack block is banged at the first input, the addition is calculated, and only then. So be sure to put some kind of bang logic there! At the outlet appears the sum as a result of pairwise additions. This block tested and works when the bang is applied.
--->implementation--->

Oops: we are a bit banged by the complexity of using Pd to translate this algorithm. We're now skinning the matlab code we have (see link to beat it!) and learning how to make our own externals in C.

sin~

2007-04-12T19:18:00.000+02:00

sin~

This block calculates the sine for a audio signal (sample by sample). Only cos~ is included in PureData.
The input is the audio signal. The output its' sine. You can verify its working well with testcos_sin.pd

version

v1.0: built 12/04/2007 with Pd v0.40-2 [pd][pdf]
related testfile: testcos_sin.pd
v1.1: built 13/04/2007 with Pd v0.40-2 [pd][pdf]

interface

Input1 : the (mono) audio signal input

Output1: the sine for the audio signal

implementation

The implementation is based on the relationship between the cosine and the sine function. You have to be careful here because in PureData the arguments are normalised. This means 1 corresponds to 360 degrees or 2*Pi. The cosine of 0.25 gives 0, 0.5 gives -1, 0.75 gives 0 again. This means the sine of 0.25 has to be 1, 0.5 has to be 0, 0.75 has to be -1. So there is a translation of 0.25 between the 2 functions. This comes from sin(x) = cos (pi/2-x)

rev_mono~

2007-04-12T15:45:00.000+02:00

rev_mono~

This reverb block contains only 1 input (the mono signal).

The creation arguments (liveness, crossover frequency, HF damping) are supplied in three inlets as shown.

The "liveness" (actually the internal feedback percentage) should be 100 for infinite reverb, 90 for longish, and 80 for short and so on...
The crossover frequency and HF damping work together: at frequencies above crossover, the feedback is diminished by the "damping" as a percentage. So zero HF damping means equal reverb time at all frequencies, and 100% damping means almost nothing above the crossover frequency gets through. This is implemented to simulate a vintage analog reverb sound, because the analog circuitry used had also as an effect that the high frequencies were attenuated.

version

v1.0: built 11/04/2007 with Pd v0.40-2 [pd][pdf]
Final version; defaults are included, output level has to be controlled from outside (not like in rev3~.pd)

interface

Input1: mono input channel.

Outputs1-4: 4 different combinations of the internal delay taps. Possibly combine them into left and right output

Controls:
INLET1: set a value for the internal feedback percentage, called liveness, usually between 85 and 100. It ranges form 0 to 100.
INLET2: set a value here that indicates the crossover frequency.Once reached, damping occurs. It ranges from 0 to 20kHz. Default is 3000Hz.
INLET3: set a value for percentage of feedback damping after reaching the crossover frequency. It ranges from 0 to 100.

implementation

The input directly goes to the pd doit block, actually the reverberation calculation block. In this case there is only one mono channel input. We make a combination of the same input into this block.

The control signals are, if needed, translated to different range/values and are given a name. This is the case for INLET1 (gets name fb): the liveness input goes from 0 to 100 (default 85), but in reality has a maximum value of 25%. For INLET2 (gets name lop): the crossover frequency default is set to 3000 Hz. For INLET3 (gets name damp): HF damping is set to a percentage value (default is 50) and a linear transition is set to 50 ms in order to obtain a smooth change of value.

In the figure below you can see the actual filter bank for calculating the reverberation effect. You can see there are 16 different delay lines that are combined into this diagram and all transformed with respect to the give parameters. Lop~ is used to subtract the low frequency part of the signal from the original in order to obtain the high frequency part. After this operation damping with factor damp occurs. The level of each delay tab is influenced by fb. The actual combinatorial diagram is just one implementation method. Different methods are possible. This is the best found.

rev_stereo~

2007-04-12T15:24:00.000+02:00

rev_stereo~

Left/right input respectively for left/right stereo input.
The creation arguments (liveness, crossover frequency, HF damping) are suplied in three inlets as shown.

The "liveness" (actually the internal feedback percentage) should be 100 for infinite reverb, 90 for longish, and 80 for short and so on...
The crossover frequency and HF damping work together: at frequencies above crossover, the feedback is diminished by the "damping" as a percentage.So zero HF damping means equal reverb time at all frequencies, and 100% damping means almost nothing above the crossover frequency gets through.This is implemented to simulate a vintage analog reverb sound, because the analog circuitry used had also as an effect that the high frequencies were attenuated.

version

v1.0: built 11/04/2007 with Pd v0.40-2 [pd][pdf]
Final version; defaults are included included, output level has to be controlled from outside (not like in rev3~.pd)

interface

Input1 & 2: the left and right channel from stereo signal

Outputs1-4: 4 different combintions of the internal delay taps. Possibly combine them into left and right output

Controls:
INLET1: set a value for the internal feedback percentage, called liveness, usually between 85 and 100. It ranges form 0 to 100.
INLET2: set a value here that indicates the crossover frequency.Once reached, damping occures. It ranges from 0 to 20kHz. Default is 3000Hz.
INLET3: set a value for percentage of feedback damping after reaching the crossover frequency. It ranges from 0 to 100.

implementation

The input directly goes to the pd doit block, actually the reverbation calculation block. In this case there are 2 different inputs (left and right channel). It is possible to make a combination of the same input into this block.

The control signals are, if needed, translated to different range/values and are given a name. This is the case for INLET1 (gets name fb): the liveness input goes from 0 to 100 (default 85), but in reality has a maximum value of 25%. For INLET2 (gets name lop): the crossover frequency default is set to 3000 Hz. For INLET3 (gets name damp): HF damping is set to a percentage value (default is 50) and a linear transition is set to 50 ms in orde to obtain a smooth change of value.

In the figure below you can see the actual filterbank for calculating the reverbation effect. You can see there are 16 different delay lines that are combined into this diagram and all transformed with respect to the give parameters. Lop~ is used to substract the low frequency part of the signal from the origininal in order to obtain the high frequency part. After this operation damping with factor damp occures. The level of each delay tab is influenced by fb. The actual combinatorial diagram is just one implementation method. Different methods are possible. This is the best found.

waves

2007-04-10T15:44:00.000+02:00

waves

A block that implements some waves with a specific frequency and normalized output. It provides a (co)sine wave, (reverse) sawtooth, triangle and rectangular wave. Two blocks can be added or combined and rescaled in any other way to make new waves. One waves block connected to another one gives quadratic waves.

version

sin~ [pd]
v1.0: built with Pd v0.40-2 [pd] [pdf]
v1.1: adapted with Pd v0.40-2 [pd] [pdf]
sine wave added, RESET knob added
v2.0: built with Pd v0.40-2 [pd] [pdf]
new control logic
reverse sawtooth added

interface

Input 1: A value for the frequency of the wave. It is expressed in Hertz and thus must be positive.
Input 2: Reset knob. When it is pressed, the wave is restarted from the beginning. For a sine wave, square and sawtooth this is 0, for a cosine, reverse sawtooth and triangle it is 1
Input 3: MIDI control change number. This is used to select the wave: when MIDI control change message 52 sends a value of 127 (aka is pressed in), the different waveforms are cycled.

Output 1: A waveform with a specified frequency, specified form and an amplitude that ranges between 0 and 1.

implementation

fig: respectively prints of a reverse sawtooth, sine or cosine, sawtooth, triangle and quadratic wave

The frequency value one inputs is turned into a signal type with sig~. You get a signal with constant amplitude equal to the value specified. Then a phasor~ object runs through every value starting from 0 to 1 and again at a frequency specified. This already is a ready-made sawtooth signal.
The sine block is based on the relationship between sine and cosine function. Read more in documentation of sin~
The sawtooth wave is also used as table lookup signal for the other waves. The cos~ block calculates for each sample the cosine value. This sine is raised above 0 and normalized by dividing with two, or less expensive: multiplying with a half.
The triangle is calculated by shifting the sawtooth so it lies around zero, then taking the absolute value so that the negative part is mirrored to the positive side. This triangle wave has to be scaled by multiplying it with two.
To obtain a reverse sawtooth, the sawtooth is multiplied with -1 which results in a mirroring of the wave around the x-axis. To rescale the wave to a range of 0 to 1, a shift is performed by adding 1 to the wave.
The rectangular wave is made by selectively choosing a 1 or a 0 to output. This is done by the moses block. Because it takes only control lines or values, one has to add a snapshot~ block before it that is banged after each DSP cycle with the bang~ block. The sig~ block transforms this control signal into an audio signal again.
All waves can be seen in the graph when banged on the knob that is located next to the select block on top. It isn't automatically rewritten because that would slow down calculations. Every time a new wave is selected it will be drawn once.
The control logic is simple. When a specific MIDI control change number is received with a value of 127 a counter adds up. This addition is modulo the number of waveforms. An extra select block is added to select a number from the counter and then bang the right message. Each output in this patch has a multiplication with one or zero to turn it on or off.
-ronny 4/14/07 6:59 PM

phaser~

2007-04-06T10:44:00.000+02:00

phaser~

Adds a phase-shifted sound to the original sound, producing a swirling modulation that creates spaciousness and depth. Three different methods are selectable and one can control the frequency and range of the modulation, also providing different control wave forms.

version

v1.0: built 5/04/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input1: incoming mono audio signal
Input2: this inlet takes values between 0 and 127 that specify the frequency of the control waves. It stands for the RATE parameter.
Input3: this inlet takes values between 0 and 127 that specify the range of the control waves. It stands for the DEPTH parameter.

Output1: The phase shifted mono signal that is output.

Controls:
Different control curves: For a sine wave, sawtooth, triangle or square wave, currently the MIDI control messages are assigned this way: respectively 52, 53, 54, 55.
Different phaser implementations: Also assigned with MIDI control change messages: 4 allpass stages is 56, 3 times 4 stages is 57 and 6 stages is 58. The default is four allpass stages.

implementation

The basic building block of the phaser is a pd allpass~ block. As seen in the figure it is a real one-zero (non-recursive) 'reverse' filter followed by a real one-pole (recursive) filter, both with the same coëfficiënts. For visual reasons, the coëfficiënts are passed to an outlet. This combination thus presents one stage of allpass filtering. This leaves the gain of all frequencies unchanged, but it alters the phase response.
The coëfficiënts of these allpass filters are altered in time. The waves block outputs a specific control curve with a chosen frequency (RATE). This curve (that lies between 0 and 1) is then scaled with the DEPTH parameter and it is also offset to lie under one. The DEPTH parameter is divided by 131 because the maximum value of the curve is 0.97 and when a range is chosen it can maximally extend to zero. The maximum value that can be passed is 127, so the 131 divides the interval maximally.
With the select and ctlin block one can choose between three setups. The A-form (A has no special meaning) uses four stages and outputs the signal coming from the last stage. The B-form, that sounds more boosted, uses three times four stages and sums every block of four stages. The C-form is another widely spread form that uses 6 allpass stages.

beat detection in pure data

2007-04-05T23:15:00.000+02:00

Today I've also been working on a beat detection algorithm for pure data.
The basic idea behind the whole algorithm is outlined in beat this!.
It consists of four steps:

filterbank
smoothing
diff-rect
comb filtering

The first three steps (see website for more explanation) are done by the bonk~ block that is included in the standard Pd download. So as an output we get, for eleven parts of the frequency spectrum, the loudness of an attack.
The last step use a bunch of combfilters, each representing a specific tempo. All filters are convoluted with each signal and this should have to give higher energy values for comb filters that are close to the tempo of the signal.
I made a combfilter block that we can use afterwards. It uses three arguments: window, frame or blocksize firstly, then the resonant frequency and then the bandwidth. The bandwidth specifies how wide the teeth of the comb are: when it is close to zero, most of the signal is let through. When it's close to 0.5, the teeth are very wide and there is not much space between them.
I already realize that the convolution is better done in the frequency domain, because it is a multiplication there.
But the rest is for tomorrow or something.
Sleep tight

flanger~

2007-04-04T18:30:00.000+02:00

flanger~

This block applies a flanger effect on the incoming mono audio signal, that is the original signal delayed with a variable delay. It gives a metallic resonant effect like a jet plane taking off or landing. You can choose between several control curves and a free input for a control curve. You also have to include an argument otherwise the effect won't work.

version

waves v2.0: look at the documentation of this block to see what it does [pd]
v1.0: built 4/04/2007 with Pd v0.40-2 [pdf]
v1.1: built 5/04/2007 with Pd v0.40-2 [pdf]
waves block added for more choices in control curve
free control added
v1.2: built 13/04/2007 with Pd v0.40-2 [pd][pdf]
waves block is separated in development and documentation
thus no real differences with previous version, alone the documentation has changed.
v2.0: built 13/04/2007 with Pd v0.40-2 [pd][pdf]
a feedback loop is added and the level and feedback are unified into one control

interface

Input1: incoming mono audio signal
Input2: modulation frequency (MODFREQ) ranges from 0-127 (representing normalized values between 0 and 1). This parameter specifies how fast the delayed signal changes between leaping forward and lagging behind. It's modulation curve is selectable in the waves block: one can cycle through different waveforms by pushing a specific button (see documentation on waves). The amplitude is fixed to 10. If one wants to use a different amplitude for the control curve of the variable delay, the previous version should be used.
Input3: the feedback level (FBK) ranges from 0-127. This parameter specifies the range of the variable delay, how far it lags behind or leaps forward. It represents the maximum difference from no delay. When leaping forward the pitch gets higher and vice versa. If it's zero, the loop is off.
Input4: an inlet for a control signal that directly controls the variable delay time. Be careful to give a quite low frequency signal with an amplitude that is not too big.

Output1: outgoing mono audio signal: this is only the wet signal, the original signal delayed with a variable delay.

implementation

The input signal is written into a delay line.
The waves block outputs a normalized waveform with a frequency specified by the MODFREQ parameter.
The amplitude is fixed to 10 by a multiplication. This waveform is then used as the varying delay time for the vd~ block. When the values of the waveform are greater than zero, the delay will leap forward (with a maximum of one block size), when it's smaller it leaps behind (the delay time is greater).

This signal is put in the feedback loop, which feedback level is controlled by a simple amplitude logic consisting of scale block (/127) and a multiplication.

It is important to include an argument. This is needed to make, at the same time, 2 different flanger computations possible. Otherwise the delay-lines of several flanger effects used would have the same name. Now with the dollar sign, the name of the argument is inserted.

stereomultitapdelay~

2007-04-02T18:04:00.000+02:00

monomultitapdelay~

This is a versatile delay effect. It can simulate the traditional delay (as in the monotraditionaldelay~ block and others), and many more. One has ten taps at his disposal, each one is separately controllable: delay time DLY, volume level of that tap LVL and place in the stereo image PAN. The first five taps also have the feedback level FBK parameter (as for the traditional delay).
Use:

To simulate the monotraditionaldelay~ use one of the first five taps, set LVL and PAN to 0. All other parameters are also 0. Now choose your DLY and FBK.
Ping-pong delay: Look at the figure on top of this page. Every tap is routed to another output. Here you see a L-R style. Other styles are: L_C_R, dynamic and random.

Every tap is output separately, but the stereo signal is on the last two outputs. All these signals are wet (that means, the original signal is not included)

version

monotraditionaldelaysimple~ v1.0 [pd]
pan~ v1.0 [pd]
v1.0: built 2/04/2007 with Pd v0.40-2 [pd][pdf]
Temporary version; presets will be included, display will be less full, more inputs will be added to control the effect like other delay blocks (BPM, measure, an array of DLY,...)

interface

Input1: the mono signal you want to delay/ go crazy

Outputs1-10: the mono signal that is output by the corresponding tap
Outputs11-12: the left and right main output of the effect. All the taps are summed here after being placed in the stereo image.

Controls:
DLY1-10: set a value here that indicates when the delayed signal will be throughput. It ranges from 0 to 3000msec.
LVL1-10: set a value here that indicates the volume level of the corresponding tap. It ranges from 0 to 100, 100 meaning at the same level as the input signal.
FBK1-5: set a value here that indicates the attenuation of the inner delay loop of this tap. It ranges from 0 to 100, 100 meaning that the loop will keep on repeating infinitely and as loud as the input signal (instability)(TO DO)
PAN1-10: set a value here that indicates where the mono signal is placed in the stereo image. It ranges from -127 to 127, 0 meaning CENTER. The panning curve used is the equal power sine curve.

implementation

The input is directly written in a delay line delay with a maximum delay length of 3000 msec. It is also on a bus with the throw~ block for every tap to grab the signal from.
The main output section also consists of two buses, one for each channel of the stereospectrum.

From left to right: a single delay part with feedback control, a single delay part and a single tap part.

A delay in a tap is done by reading the delay line delay at a specific time. The delread~ object does this. For the five first taps also a simple mono traditional delay block is foreseen. It's nothing but a skinned version of the original monotraditionaldelay~ block. It also has no display. You can see the implementation in the figure hereunder.
Every tap is then attenuated by the LVL parameter before being sent to its output. Also this attenuated signal is panned with a pan~ block, also a simpler version of the original monopanner~ block. See the figure underneath. As you can see, also here the controls are left out and only the inlets are kept. It is then put on the buses outL and outR. There will be no documentation for these two blocks as long as their not used elsewhere.

For completeness also the implementation of the controls and display are given. Every parameter is sent or received apart, where a pack and unpack block could have been used. This is not done because the idea exists that it would slow down the algorithm. This because every time a parameter would change, a whole array must be re-sent, packed and unpacked. On the other side, it would look nicer.

monotraditionaldelay~

2007-04-01T15:56:00.000+02:00

monotraditionaldelaybpm~

This block implements a traditional delay effect with feedback loop. Every parameter can be changed by the inlets, except the measure. It handles a mono signal and outputs a mono signal too. It is based on the parameter bpm and works on the tempo of the song. You can choose how much times the delayed signal is repeated between two beats.

version

v1.0: built 30/03/2007 with Pd v0.40-2 [pd][pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the beats per minute parameter BPM
Input 3: the feedback parameter FBK

Input 4: the on/off switch for external control

Output 1: the delayed, wet mono signal

Controls:

params: These four number boxes show the current values of the respective parameters. There's also a small slider to display the DLY parameter physically (easier to compare the effect of the BPM on the DLY). The DLY value is the delay time between the current and the next signal samples. The BPM value is current tempo that is chosen. actFBK is the current feedback level, which indicates the attenuation of the next delayed signal compared to the current one. The memFBK is the value of FBK that will be used when the switch is turned to back on.

TO DO: find another name for this parameter, as the FBK is not really remembered. This is not a problem, because one has to be able to set a new value before turning it on again.
top number box and horizontal slider: adjust the beats per minute (BPM) parameter. One can choose a tempo for the delay. It ranges from 20bpm tot 250bpm. This equals a maximum delay value of 3000msec when a BPM value of 20bpm is chosen and the measure is 1/1.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands.
switch button: is used to set the FBK parameter to 0 and thus turning of the delayed signal. When the switch is turned back on, the last chosen value of FBK or the new value if the controls are changed, will be reset.

measure: One can specify how many times the signal is sent back into the delayloop between two beats. A fraction of 1/2 will divide the DLY time by two, so that two delays are heard between two beats.

TODO: maybe also some fractions with a numerator greater than the denominator can be added to the effect.

implementation

The delay itself is pretty simple: the incoming signal is read by a delread~ object that writes the signal into a delay line called delay1 and with length 3000msec. Although there's no visible loop, this is how the loop is formed: after a delay time DLY the associated delread~ object reads what's in the delay line, outputs it and sends it back into the delwrite~ object. Before outputting the delayed signals they are attenuated every time by a factor FBK ranging between 0 and 0.999 (with 1 clipping is to easy), in steps of thousands. That's why the control parameter lies between 0.001 and 999.
A possibility is left in the implementation to also mix the wet output with the original, dry signal, by connecting the unconnected throw~ and catch~ objects.
TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline.
A short calcultation is done to compute the delay time parameter from the BPM and the measure choice DIV. DLY=DIV*60000*BPM^-1. The bang and multiplication block have to be added to ensure direct reaction when another measure is chosen. Here also the default value of 1 is set by using it as a parameter in the multiplication lbock.

The initialisation is done mainly at the left of the canvas: The BPM is set to its maximum value 250 so that the DLY is as small as possible. FBK is set to zero and the switch is off too.

The switch functions because the value of the toggle (0 or 1) is used to multiply the current FBK with, resulting in a zero when the switch is off and the actual value if it is on.

PureData remarks 2

2007-03-30T15:23:00.000+02:00

As long as I don't find a better solution, I'll put the parameters for display and for adjusting both apartly in the display. If I just connect a number box and a slider together, stack overflow errors occur, sometimes slowing down the system.
I connected my korg microkorg midicontroller to my laptop to use it in PureData. This works fine. I keep a written chart with all the control change numbers of the controls at hand. Because MIDI data ranges from 0 to 127, I adjusted the inputs of some effects I already had. From now on this will be the unified way for ranges of control inlets.
Apparently when one reads into an array, an array must be added to the patch too. You can make it small by unchecking the graph on parent button, but it is still wondering around there.
It's important that when you will be using a block several times in the same patch (something we must do), every parameter name that can be sent without a line is named with a $-sign. send, receive, throw, catch, delread, delwrite and several others must have a name like $1-name. When you use the block, you give a unique number as argument. This number is filled where $1 is placed. This works only when the dollarsign is placed in front of the name.
The dollarsign has another meaning when used in messages. There exists no $0. The dollarsign is replaced by the n-th value of the inlet.

monotraditionaldelaybpm~ bad try

2007-03-29T21:18:00.000+02:00

monotraditionaldelaybpm~

bad try - discontinued

This block delays an incoming signal and outputs the wet signal on one channels. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop, actually one per channel. Users can change all parameters with the visible controls, or they can vary them by using the three other inputs.
When varying the beats pet minute (BPM) parameter, one can choose what the division is of the time between two beats. Only the grey buttons will keep the delayed signal synchronized with the original signal.

version

v1.0: built 28/03/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the beats per minute parameter
Input 3: the delay time parameter
Input 4: the feedback parameter

Output 1: the delayed, wet monosignal

Controls:
top number box and horizontal slider: adjust the beats per minute (BPM) parameter. It specifies the tempo that synchronizes the delayed signal. It ranges from 20bpm to 250bpm.
middle number box and horizontal slider: adjust the delay time (DLY) parameter. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec tot 3000msec (3 seconds). This is not only the maximum value of the slider and number box, but also the length of the delay line used in the implementation. When these are adjusted, the DLY controls are adjusted accordingly.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands. When these are adjusted, the value of the BPM controls are not adjusted accordingly (read the implementation to see why).
switch button: is used to reset the feedback parameter to 0 and thus turning of the delayed signal. When a cross is visible, the effect is turned on. The previous value is remembered and displayed in the number box above the FBK slider as long as one doesn't touch the number box. When turning the effect back on, the slider and the number box jump to the last value remembered.
16 measure buttons: choose the fraction of time between two beats. It resembles the measure of a musical piece, but it is not. It's more like a measure between two beats, as if between two beats a total measure takes places. The grey buttons will stay synchronized between two beats, but also the other ones can stay synchronized over different beats:

implementation

The implementation is the same as the monotraditionaldelay~ block. Only the new details will be described here.

For smoothing the signal when parameters are changed, a line block is implemented for the value.
Adjusting the BPM slider is the same as adjusting the delay time DLY parameter. The factor is: DLY = 60000*BPM^-1. This is then multiplied by a factor that expresses the measure as a fraction of time between two beats. The mechanism used for this is a message that is banged the moment you press the button on the display. This updates the factor (that has a default value of 1). To make sure it updates directly, a bang block is inserted to the hot input of the multiplication block. Inside the message the measure fractions (DIV for division) are set rounded to 5 numbers after the comma.

TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline.

TO DO: When adjusting DLY, the other one is not updated in the display. In the next version this is taken care of. For this update it would require an inverse recalculation. This consumes CPU time, so the effect of it must be researched first.

The switch toggle switches the effect off by setting the feedback level FBK to 0. The value is remembered and restored when turning the effect back on. In the number display you still see the remembered value. How this system functions can be seen in fig. 1: switch implementation above.

problems

There are some problems, although everything works.

When the output of the bpm slider is connected with the input of the number box above it, an error: stack overflow occurs and slows down the computation. Strange, because exactly the same thing is done with the delay_time controls. The same error shows up when it's adjusted although not connected to each other, but without really slowing things down. The latter happens also with the delay controls. It stops when the r dly block above the controls are uncoupled, for both controls: DLY and BPM.

When the feedback_level controls are adjusted an error: inlet: no method for 'list' occurs if the switch is turned of and an error: stack overflow occurs when the switch is turned on, without really slowing things down.

To make it possible to adjust the DLY control displays when changing the BPM controls the r dly block must stay attached to the delay_time number box. As a result these stack overflow errors keep appearing...

stereotraditionaldelay~

2007-03-28T17:09:00.000+02:00

stereotraditionaldelay~

This block delays an incoming signal and outputs the wet signal on two channels. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop, actually one per channel. Users can change all parameters with the visible controls, or they can vary them by using the two other inputs, except for the phase reverse and switch-off. The difference with the mono delay, except for the stereo part, is the variable delay time per channel and the choice for in-phase or reversed-phase delayed signals.

version

v1.0: built 28/03/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the delay time parameter
Input 3: the feedback parameter
Inputs 4 and 5: the left and right channel delay time offset

Output 1 and 2: the delayed, wet stereosignal

Controls:
top number box and horizontal slider: adjust the delay time (DLY) parameter. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec to 1500msec (1,5 seconds). This is not only the maximum value of the slider and number box, but also the length of the delay line used in the implementation.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands.
switch-off button: is used to reset all the parameters to 0 and thus turning of the delayed signal.
TO DO: It might be better to call the switch-off button a reset button, implying that after pressing it, the values will not be remembered.
phase-reverse buttons: toggle on and off to create slight difference due to cancelling out of parts of delayed signals.
TO CHECK: It might be possible that this function doesn't work properly because the difference with both switches off is really negligent.
L-offset and R-offset: adjust the delay time of each channel relative to the delay time DLY. It ranges from -200msec to 200msec.

implementation

The basic delay is implemented the same way as the mono traditional delay, but with two copies then. Both loops have the same FBK parameter in common, that is calculated once.
For smoothing the signal when parameters are changed, line blocks are implemented for the dlyLeft, dlyRight and fbk values. Also a one pole high pass filter is added at the outputs, with a very low cutoff frequency.
The dlyLeft and dlyRight values are calculated by adding the delay time parameter DLY to respectively the L-offset and R-offset parameter. Both values are being input in the left slot of the +, because that's the hot one: everytime a value at the left is changed, the block will execute an addition.
Also here a throw~ and catch~ block are left behind. If you want to add the dry signal to the wet one inside the block, it's done by connecting the unconnected blocks to the nearby slots.
The phase reverse logic works like this: when the button is toggled it transmits a 1 or a 0. The moses block sends a 0 to the left output and a 1 to the right, because x<1 is sent to the left, etc. The following messages contain the factor by which the signal must be multiplied. -1 for a phase reverse (as easily seen when you draw a signal on an x-axis with a phase circle next to it). The multiplication that follows (and begins with a 1) is banged by the message and the factor thus changes inside the *~ block.
TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline.

monotraditionaldelay~

2007-03-28T16:25:00.000+02:00

monotraditionaldelay~

This block delays an incoming signal and outputs the wet signal on one channel. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop. Users can change the two parameters with the visible controls, or they can vary them by using the two other inputs.

version

v1.0: built 28/03/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the delay time parameter
Input 3: the feedback parameter

Output 1: the delayed, wet mono signal

Controls:
top number box and horizontal slider: adjust the delay time (DLY) parameter. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec tot 3000msec (3 seconds). This is not only the maximum value of the slider and number box, but also the length of the delay line used in the implementation.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands.
switch-off button: is used to reset all the parameters to 0 and thus turning of the delayed signal.
TO DO: It might be better to call the switch-off button a reset button, implying that after pressing it, the values will not be remembered.

implementation

beat detection in PureData

2007-03-27T22:30:00.000+02:00

A block exists that tracks beats. I didn't yet read the details, but it's standard in Pd and is called bonk~. Obiwannabe wrote an article on it here. He also wrote other interesting stuff about music synthesis and all using Pd.
I also found a library that collects a bunch of feature extraction scripts written in C. I want to download it, but I don't know how I should compile it or at least make it work anyway. It's named aubio.

weekly update

2007-03-27T21:47:00.000+02:00

This week we both read through the tutorials of PureData as indicated last week. Also the book of Puckette M. is not virgin anymore.
For the next week we'll implement some concrete effects. Concrete means that they really work with a given input, from input to output. If there's time we might also try an extraction algorithm already. But we have to keep it simple so as to achieve something concretely this week.
Things we talked about during our meeting:

I tried already measuring the latency by just connecting the input of my external soundcard (EDIROL FA101) with the output through PureData. With a sample rate of 44,1kHz and quantizing at 24bit I reach a minimal latency of 37msec, these is too audible and 1msec less results in a big mess. This is not the case when using other profeesional audio software on my computer (adobe audition 2.0, Live 6.0.1) I didn't really search further, I ony tried it out once and it works at least. I know there exists a latency test patch for Pd on the internet. I'll search it the next time I do real input tests.
Anyway, Sam says that an audible latency is OK as long as we have a realtime working effect. The latency of the program would be rather normal because of the complexity of the code.

monopanner~

2007-03-27T21:32:00.000+02:00

monopanner~

discontinued

This block places a mono signal in a stereo panorama. You can choose the panning curve that is applied. An input is also provided to vary the pan parameter externally.
This block is discontinued but it's implementation is used in the series of channel blocks.

version

goginLeft.txt - goginRight.txt

linearLeft.txt - linearRight.txt

rootLeft.txt - rootRight.txt

sineLeft.txt - sineRight.txt

pancurves.m

v1.0: built 27/03/2007 with Pd v0.40-2 [pd]
v1.1: adjustment 28/03/2007 with Pd v0.40-2
the output of the horizontal slider is now connected to the number box above it. This way the value in the number box is updated also when you slide the slider.
v2.0: built 29/03/2007 with Pd v0.40-2 [pd]
extra input for reset
PAN parameter input rescaled for MIDI data: 0-127
separate controls and displays for PAN parameter
rescaling calculated once instead of twice

v2.1: adjustment 1/04/2007 with Pd v0.40-2 [pd][pdf]

Other connections made it able to leave some away, for the initialisation logic

The reset knob only resets the PAN to the center. The table is left untouched.

Initialisation sets the table too, to the root function, because it's a more normal pan control curve than the gogin curve that was the default before.

interface

Input 1: the mono signal you want to pan
Input 2: the pan parameter, that is also included in the front of the block. It's range goes from 0 to 127 in steps of 1, where 0 stands for LEFT and 127 stands for RIGHT. This results in a PAN parameter that changes with a step size of 2, zero is not reachable with this input.
TO DO: re-size the tables, so that the step size of the input as well as the slider is equal.
Input 3: reset the PAN parameter to CENTER. That is the value read on sample 128 of the curves

Output 1 & 2: the stereo output signal.

Controls:
number box and small slider PAN: This displays the current value of the PAN parameter on a scale of -127 for LEFT to 127 for RIGHT with 0 as CENTER (when controlling from the second inlet, the value 0 is not reachable. One can do it with the reset button. This display is included to view the changes while the effect is controlled externally, as you see on the top figure: the current PAN value is 127, apparently controlled from the outside, because the controls themselves are still centered.
reset: resets the pan parameter to 0 or CENTER
number box and horizontal slider: adjust the pan parameter
gogin, linear, root and sine: choose a panning curve. root is the default value after loading the patch.

implementation

For the pan control curves a table is used. Per curve type there are two tables: one for the left and one for the right channel. The vectors are calculated in matlab with the following m-file:

xMin = -128;
xMax = 127;
x = [0:1/(xMax-xMin):1];

% gogin
arg = x*pi-pi/2;
goginLeft = sqrt(2)/2*(cos(arg)-sin(arg));
goginRight = sqrt(2)/2*(cos(arg)+sin(arg));
figure;plot(goginLeft,'b');hold on;plot(goginRight,'r');

% square root
rootLeft = sqrt(1-x);
rootRight = sqrt(x);
figure;plot(rootLeft,'b');hold on;plot(rootRight,'r');

% sine
sineLeft = cos(x*pi/2);
sineRight = sin(x*pi/2);
figure;plot(sineLeft,'b');hold on;plot(sineRight,'r');

% linear
linearLeft = 1-x;
linearRight = x;
figure;plot(linearLeft,'b');hold on;plot(linearRight,'r');

As you might have already remarked a choice is made here about how many values the control curve has. This can be changed afterwards in this m-file. xMin en xMax are defined here the same way as they are defined in the block itself: ranging from -128 to 127. A larger range would result in a smoother transition when panning, but with these values there's already no audible step.
The plotted figures can be seen in the section interface.
After calculating the values, this vector is copy pasted into a .txt-file. This space separated list is directly readable by PureData through the command read placed inside a message box. The values are thus read and written away in an array called left Channel and right Channel.

The inlet accepts values between 0 and 127, the controls range from -127 to +127 and the control curves are written in tables with a width that ranges from 0 to 256. So the range of the inlet must be multiplied with two and a one has to be added. The controls just have to be shifted over 128 samples. Then the tabread block looks up the corresponding value, depending on the chosen control function. This value is used to attenuate the incoming audio signal.

The center values per type are:

gogin: L=0.693; R=0.720
linear: L=0.494; R=0.505 (-1 is a better center)
root: L=0.702; R=0.711 (-1 is a better center)
sine: L=0.700; R=0.713 (-1 is a better center)

The table that draws the array as a graph in the patch itself as an y-index that ranges from -1 to 1. The negative values are necessary for representing all the values of the gogin function.
TO DO: maybe I can change the range of the control curves to an odd number of indices, so as to provide a real centered center, with equal loudness for both the left and right channel.

PureData remarks

2007-03-27T19:12:00.000+02:00

Apparently Pd organizes the inlets and outlets of a subpatch according to their horizontal position. So no matter if they're placed at the top or at the bottom of the subpatch, you will find the leftmost inlet also as the left inlet of the blok in the parent patch.
The delete knob of the computer does NOT work...
I had problems moving a canvas around in a subpatch, but apparently this is done with the canvas properties box (right-click on an empty spot in the window). The margins stand for the distance form the left and top margins of the window.
Also the placement of the title of a box (knobs, etc.) is moveable by adjusting their x_off and y_off parameters.
Also something strange happens when I insert my monopanner~ block in the main file (just an input and an output). The array drawn in the subpatch appear at the far bottom of my parent patch main.pd... Any idea why?

And now for the monopanner~ object I implemented today. You can find the documentation in the folder monopanner~ or in the post with the same name. The latest document I keep as a google document on my google account.

weekly update and information resources to PureData

2007-03-23T13:18:00.001+01:00

Short update today. I showed some basic features of PureData I discovered last two days. Conclusion: try to work with PureData as fast as possible (as in next week).
That's what we are going to do. Firstly by running through all the tutorial patches.
I also discouvered the accompanying book (which might turn into a reference work) written by the author of PureData. It gives mathematical background to all the exercices and more: The techniques and theory of electronic music. I printed the book and now I'm looking in it and reading interesting parts.
Another essential work, but I'm putting it here more for private reasons than for this thesis is the book of Benson: Music, a mathematical offering.

2007-03-20T19:17:00.000+01:00

PureData session on beat controlled delay (pt1)

Today I start already a practical example as an extra means to try to learn PureData. I'm still very pleased by the program. I can run it from my cheap memorystick and it's so small I can even save all the soundfiles and pd files with it.
So I began with an abstraction for the output like the one given in the tutorial (only changing the control to a slider).
Then I added the delay loop. It has the control parameters DELAY TIME (DLY) and FEEDBACK LEVEL (FBK) which are now controlled by a slider. This slider can receive a number as input, so we can attach the result of the extraction algorithm or mapping to the slider itself without having to change anything else.
I now want to implement a multitap delay (where you can specify the number of delayed samples you hear) but using the delay with feedback loop (not with the easier but totally other way).

First I tried to calculate when the level of the nth delayed signal would be smaller than the fraction of the n-1th version. But after a while I realized that the only easy way to do this was by stopping the input of the delay loop after n * DLY.
But also this doesn't work because if you stop input for the loop, also the ongoing delays are not written anymore and only one delay tap rests.
Therefore I tried starting from a multitapdelay to join the two types of delay. But I don't find a way to implement it without having a number of delay reads equal to the max number you want to have. So I don't find a way to do it in a loop...

Then I also want to try to attenuate every feedback copy while it's playing, because now it is played with a constant attenuated level. This can be done sith an envelope maker in the loop or the same effect might be reached by giving an envelope to the overall delayed signal. wow I'm going too fast. This is a feature that migh be useful when using really long delays, otherwise it's really not needed. But in this case it might give some very surprising and creative results (eg by not attenuating but making the feedbacked signal louder as it is played). It would require a lot of control signalling at first sight, so I put it on a long trail. Maybe one evening, who knows...

When I draw it on a paper, it seems that it is equivalent to, or that the only possible thing is to apply a control curve to the feedback level parameter. Let's try that sometime!

I also implemented a panorama block with which you can choose different connections between the left and right channels. One problem I encounter here is that when I try to mix both signals in both outputs, it doesn't give the expected result. Any ideas why? See the image.

Installing and learning PureData

2007-03-19T10:45:00.000+01:00

For future reference:
Installing PureData in Windows XP is as simple as downloading a zipfile and saving the extracted files somewhere in a folder where you like. You find the program itself in the subfolder /bin/pd.exe .
From there you can run it, or you can create a shortcut to somewhere else.
I also checked if we could indeed add our own blocks in C and that's OK, it's even possible in C++ or FORTRAN (whatever).
Now I'm running over a bunch of examples to learn working with the platform.

a question that rises now is that for example a delay (even very pratical and complex ones) can be made in PureData without 'descending' to level of matlab language. Everything can be done at the higher level, they even got analysis blocks (but their complexity and functionality I must first research), so why bother with the matlab level?
This is a question for tomorrows weekly meeting.

matlab session on the signal routing for testing my delay and other effects

2007-03-15T17:18:00.000+01:00

I worked on the file effectSystem.m and all it needs still is the output part: playing and plotting the results of the effect. And also the part where the different frames are glued together again to form a soundfile and the mixer part.
After that I can start implementing and testing effect files I already have, or new ones.
I did not yet pay attention in effectSystem.m to the feature extraction part.

EFFECTS

2007-03-15T15:43:00.000+01:00

heb nog paar linkjes gevonden met implementaties voor effecten
moesten we er toch nog willen bijnemen:

REVERB

http://cnx.org/content/m14222/latest/

DISTORTION

http://cnx.org/content/m14221/latest/

http://www.cs.stir.ac.uk/~lss/lsspapers.html
http://www.cs.stir.ac.uk/~lss/recentpapers/IEEETransNNSept2004.pdf

using pitch detection
http://ismir2005.ismir.net/proceedings/1008.pdf

information:
http://www.cs.uvic.ca/~gtzan/work/publications.html

Onset detection

2007-03-15T15:37:00.000+01:00

veel links, veel lectuur, veel informatie beetje verwarrend...

http://www.music-ir.org/evaluation/mirex-results/articles/all/short/west.pdf
http://www.music-ir.org/evaluation/mirex-results/articles/onset/collins.pdf
http://www.speech.kth.se/prod/publications/files/875.pdf
http://ieeexplore.ieee.org/iel5/10376/33958/01621213.pdf?tp=&isnumber=33958&arnumber=1621213

Er bestaat zelfs zoiets als de onset detection Contest
Hadden we dit geweten....

http://www.lam.jussieu.fr/src/Membres/Leveau/SOL/SOL.htm
http://www.lam.jussieu.fr/src/Membres/Leveau/ressources/Leveau_ISMIR04.pdf

nog iet gevonden over Piano note detection
te vinden op http://cnx.org/content/m14196/latest/
dit zijn een aantal m-files, waarvan de onset detection
eigenlijk vrij goed algemeen werkt... momemteel
gaan we hiervan uit, indien achteraf blijkt dat dit niet voldoende is,
zullen we toch teurg op die papers moeten hervallen...

beat detection: the sequel

2007-03-15T15:09:00.000+01:00

Als vervolg op vorige topics over beat detection/extraction, zie hier:
http://idafx.blogspot.com/2007/02/some-links-for-beat-extraction.html
http://idafx.blogspot.com/2007/02/vandaag-beat-detection-algorithm-enkele.html

Na overleg met Toon besloten da de subbandinformatie moet gewogen worden.
Bij het herbekijken van de code beseft dat ik de subbandverdeling met parameter
Bandlimits had aangepast aan de sample frequentie wat feitelijk niet de bedoeling was,
omdat daardoor de hogeren frequentiebanden een hoger gewicht krijgen. Het is de
bedoeling om de lage frequentiebanden het meest te laten doorwegen.
Bandlimits = [200 400 800 1600 3200 samplefreq] zoals in de oorspronkelijk code
stond dus.

test results ---> geprobeerd met die wegingsfactoren
---> wegingsfactoren ??---> bandbreedte subbanden aanpassen
---> oorspronkelijke hersteld --> juiste results voor loopkes

AC_100A_kit1.wav ---> 100
AC_75_kit2.wav 76 acc,bpm:
AC_90_kit1.wav 90
BB_135_Lp10.wav 135
BB_95_K03_Lp03.wav 95
FHP_130_DRMLP_26.wav 130
FHP_130_DRMLP_48.wav 130
Jw_65_kit2.wav 65
Jw_70_full1.wav 70
Jw_80_full2.wav 80
Jw_95_perc3.wav 95

Sh_120 Rhodes Loop 7C.wav 120
Sh_120 Synth Loop 3C.wav acc,bpm 1,118 .5,120.5 .3 119.6 .2,119.6 .1,119.6
Sh_125 C Live Bass 2.wav 125
Sh_128 Drum Loop 1.wav 128

Toms_diner.wav 86, kan ook kloppen want heb ongeveer schatting gedaan op basis aantal tellen in die 4 seconden ---> 8/9 tellen

Conclusie: juiste resultaten, algoritme detecteert ook wanneer er geen beat aanwezig is
het gaat hier dus om RITME extractie.

Op zoek naar niet iteratieve methodes !!?
interessante links:
-->selfsimilarity matrix: http://www.fxpal.com/?p=similaritymatrixFAQ
-->http://www.cs.cmu.edu/~lebanon/IR-lab.htm

heel interessante site !!
http://mue.music.miami.edu/thesis/KurtJacobson/KJacobson_msmue_final.pdf
Hier staat op het einde van de pdf een heel uitgebreide matlab code voor het berekenen
van allerlei features... mss moeten we daar eens in gaan snuffelen...

Nog nen site met allerlei tools uitgewerkt in matlab code:
http://labrosa.ee.columbia.edu/matlab/

2007-03-13T18:49:00.000+01:00

Brainstorm session on our goal

After the presentation of last friday we decided to choose a few effects and a few features and to implement these in real-time. The idea is to write working code with matlab at first and we can use simulink for the inputs/outputs and signal routing. Then we'll translate it to c code, using PureData probably for the routing part.
For feature extraction we would surely like to try onset detection and the other two we already worked on (rhythm/beat and pitch extraction).
As for the effects: delay and vibrato for sure, and then maybe a wah effect, panning, (,...)
The basic ideas we want to try with these effects and features are:

the delay time parameter of the delay effect controlled by the tempo of the song (auto-adaptive as well as adaptive (master channel or drums channel, with a mic or with insert cables))
the rate and depth of the vibrato will be faster with high pitched notes and/or loudness
the wah effect and the panner can be controlled with onset and beat detection, but this already works this way
when a singer sings a high and loud note, the delay effect can be more distinct and with a nice long delay (to make it stronger. this is the same as the sound engineers do by hand now)
...

There might be added some other idea later on, but this are the starting blocks.
So the beat extraction will extract the tempo of the music andwe will use this to sync the other effects. This is one example of cross reference for our list of effects and features.

2007-03-06T18:43:00.000+01:00

delay in simulink with adjustable parameters and weekly update

Problems:

How do I make in simulink a vector with N zeros and thereafter a one, while the N is supplied by an input? I didn't find blocks for creating matrices, so I wanted to do it with a matlab code snippet. But this gave a lot of strange problems. delayVector = [zeros(N,1);1]; This was my function, but the debugger said that this N gives problems and that the matrix must be constant or non-empty. Has it got something to do with iterations? I don't know what to do with this, so temporally I will use a fixed matrix.

Non-constant expression or empty matrix.
This expression must be constant because its value determines the size or class of some expression.

This is still a problem after applying toons proposal (see comment). The identity matrix seems not to output a matrix. Here you see the error code simulink gives us:
Element 1 of port 2 in 'effectSystem/Delay1/Selector' has a value of 257. This value is not within the range of permissible values (1 through 1) that allows selection of a valid input data element.

I tried narrowing down all possible causes so far that I ended with another little document:
And even this won't work. So I ain't got no clue for what's happening.

Now I got another problem with our FIR filter: there would be a singularity problem.

Trouble solving algebraic loop containing 'delaySubSystem/Subsystem/ -N z' at time 0. Stopping simulation. There may be a singularity in the solution. If the model is correct, try reducing the step size (either by reducing the fixed step size or by tightening the error tolerances)

I don't understand what they mean, and even so I can't find a way to adjust the step size like they suggest.

weekly update

First we talked about the things we did and didn't understand during the last days.
And of course our discovery of extensive literature on the topic and what would be our place now. Is it changed or not?
For the presentation on friday:
Tell something general, about the effects, about the feature extractions, about the proposed mapping method of Verfaille. And ending with our future vision.
As we think now there are two ways to continue from here:

Or we try the general framework idea and after that we try all sorts of combinations
Or we try some specific new effects out, but then we don't have to generalize

Tomorrow we schedule the presentation making.

2007-03-06T13:31:00.000+01:00

important papers

These two papers show that there's already been some research on our topic.Ihaven't got much time to conclude them now, but by showing a few images fromthesecond paper, you already see how they made a general framework like we werethinking of.

I'll summarize the papers by summing up the things I marked:

Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p2:
In the class of loudness effects (perceptive classification) the first adaptive effects already exist: compressor and expander, limiter and noise gate. They're auto-adaptive effects.
Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p4:
Chorus and flanging are specific cases of phase modifications known as phase shifting or phasing.
Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p5:
Here they advise lowpass filtering the pitch extraction model (based on autocorrelation). I marked this because it could be useful for my pitch detection algorithm, they wrote this to say that it's might be enough to use not so good algorithms depending on what you're going to use the feature for.
Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p5:
Here they introduce the three layer mapping strategy using a perceptive layer. There are sound feature extraction, sound feature combination and signal conditioning.
The sound feature combination using warping (after normalizing and before combination). This can be used for different purposes: a truncation of the feature in order to select an interesting part, a low-pass filtering, a scale change (from linear to exponential or logarithmic).
What is meant by this?

Fourth, and finally, a warping function can also be applied to the feature combination output in order to symetrically provide modifications of features before and after combination.

Control signal conditioning is usually used to protect hardware from an input signal.
Long term features can be used as controls of the mapping (fi(n)), while short term features are useful as inputs.
Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p7:
Real-time implementations are performed in Max/MSP and non-real time in Matlab.
I've seen this programming environment Max/MSP being named a couple of times in papers about our subject. It looks a bit like simulink for musical purposes.
Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p3:
A beautiful classification scheme of audio extraction features is given:

>> still two other papers to summarize!

2007-03-03T19:07:00.000+01:00

simulink and some reading

I worked in simulink and things I encountered to remember are:

delay blocks are calculated this way:
I put the delay in frames. 1 frame has a default value of 256 samples. Because the sampling period (Ts = 1/Fs) equals 2,27E-5, one frame takes 5,8 milliseconds to end. So if the delay value is 10 frames, we have a delay time of 58 msec.
so the general formula could be stated as: delay time = Nsamples * Nframes / Fs with Nsamples the number of samples per frame and Nframes the number of frames.
Normal delay times are:

ROLAND SH32

stereo delay: 0-500 ms
long delay: 0-1000 ms
stereo and long delay sync type: (Thirty-second note), (Sixteenth-note triplet), (Dotted thirty-second note), (Sixty-fourth-note triplet), (Sixty-fourth note), (Thirty-second-note triplet), (Eighth note), (Quarter-note triplet), (Dotted eighth note), (Quarter note), (Sixteenth note), (Eighth-note triplet), (Dotted sixteenth note)
13 possibilities!
feedback : -98% - +98% (no mapping given)

YAMAHA MO8

0.1 - 1638.3 (with value range: 1-16383)
feedback level: -63 - +63 (0-127)
they also use a feedback time, which could mean

they use delay time parameter for the first delay and the feedback time for a new delay in the feedback loop, so for the rest of the delays. If so then it's better to rename the delay time parameter to initial delay time.
Another possibility is that it is used to specify a kind of gate time, like the number of copies you allow.

T.C.ELECTRONIC D2

feedback time in stereo mode is maximally 5 sec.

KORG MICROKORG

Here they don't really give a time but the values: 0-127
For the syncing they give as parameters: 1.32 - 1.1 (see the diagram) 15 possibilities!
feedback (delay depth): 0-127

So it seems that for our purpose, our delay must have a range which is the same as the range our bpm algorithm will work with. And there might have to be an ability to choose the sync type too, with a knob or something (=gesture control)
I designed two delay implementations in simulink. The only one I'm going to speak of is the one with the feedback line. The other one has a more flexibel, but too complex way of assigning each delay or feedback level a gain value.
For the feedback looped delay, I have set a feedback level of 50%. It thus has identically the same effect as the other implementation.

To see what a negative percentage for the feedback level could mean, I tried one out and also calculated the gain after every round through the loop: it gives the same values as for the positive counterpart, but alternating negative/positive. Is it true that this is just a phase reverse after every loop? And is that the reason why I don't hear a difference at first sight? And why do professional keyboards give this choice than? Or is there another meaning connected to this negative percentage? I just tried adding two delays: one with a positive feedback level of .5 or 50% and one with a negative one of -.5 or -50%. This gives a more beautiful sound, and on the image you can see that some delaypeaks filter each other out due to a the sum of opposite magnitudes. (in my thesis folder on ESAT, you can find an image an the model too). This is an image where the original signal is not shown!
Now I am trying to include a HF damp filter in the loop (because it reappears in many synths). For info on filters for audio applications, this website seems interesting: http://ccrma.stanford.edu/~jos/filters/.
The ranges are:

YAMAHA MO8: 0.1 - 1.0 kHz (1-10)
ROLAND SH32: 200Hz - 8kHz or bypass

Oh yes, sometimes you can choose how many times a sound is repeated, this is possible with a multitap delay. eg the T.C.ELECTRONIC D2 has a max delay time of 5000msec stereo or a total of 10000msec mono. When you want 5 taps you have a max delay of 2000msec on every tap, when you want 10 taps, you still have 1000msec available. I don't know if the implementation differs rather than the parameter calculation. Therefore I'll just remember the possibility but I won't implement it now.

-6/3/2007-

Another thing with simulink: a single line represents a scalar value per iteration. A double line represents a vector. In this case there are double lines because of the fact that the audio input is given in frames. There is a way to see how the signals look on every line: Format -> Port/Signal Displays -> Signal Dimensions
The simplest way to use and process a stereo signal is to split the two channels in two mono channels right after the input block and to merge them again right before you enter the output block. This is done with the following commands:

select columns (stereo > mono)
Simulink -> Signal Routing -> Selector
merge columns (mono > stereo)
Simulink -> Math Operations -> Matrix Concatenation

2007-03-03T16:00:00.000+01:00

some more reading on adaptive effects

Today is a reading day, because I found a lot of interesting articles about our subject.

First I read the article on 'an interdisciplinary appproach to audio effect classification'. The best things to remember: they explained some vocabulary (effect, sound/audio effect, sound tranformation) that make it somewhat easier to classify them in a first way. Then they state that the most general way to classify audio effects is by their perception, what is heard by the listener. They also made layered model of levels on which you can speak about effects: (low-to-high level) digital implementation technique, processing domain, applied processing, control type, perceptual attribute, semantic descriptors. What we can remember here is not how to exactly classify effects, but more a clear view on them.
Then I discovered the thesis of our friend V.Verfaille: Effets audionumeriques adaptatifs: theorie, mise en oeuvre et usage en creation musicale numerique. Over 300 pages about our subject. I won't read that now, because it's too much. But we must try to read as much of it as possible!

Also I wanted to try simulink again. (it's a long time ago). Look at the last weekly meeting post to see why I want to do this. I'll try some effects or so out. Probably enough problems to solve before I succeed in that.

signal processing blockset contains wave in and out blocks. You must be in the right working directory for the audio files to be found. When starting simulink the working directory is changed!
I also have a problem when trying to link more than one signal. I think I'm just using the wrong blocks. (mux or merge) Idea of sam is to use an adder block and maybe scaling blocks per signal. I'll try this out.

Reading at home and matlabsession in ESAT on HPS and a interface between effects

2007-02-28T15:58:00.000+01:00

I finally found an article on that specific thing we are working on. (uch, some articles):

Very interesting of course.
In the first article they speak of mapping functions and a control curve. Also the control scaling part is nice.

I'm going to read the other ones now too, because I tried some other things in matlab, but they wouldn't work well: a little bit more explanation:

I implemented the basics of the HPS algorithm as promised yesterday. Basically it's allright, but I don't know if it works because firstly I didn't yet implement the deciding code (which decides which peak is the current pitch), secondly because I didn't think about frames or blocks (Maybe this is not necessary, but I didn't yet think about this), thirdly because there's a seemingly stupid thing happening in matlab: it doesn't recognize the file I want to play. (Solved: I have to give the attributes as a string, so encapsulated with ' ' )
I also tried to make an applyEffect.m-like file that selects the extraction algorithm and effect, and then reads the file, applies the effect with the parameters from the extraction algorithm and plays the file. This is complexer than I thought, so I need to read or think about the control curve some more.

The official title of our thesis

2007-02-28T13:26:00.000+01:00

Our department asked to give them the official title of our work. They need it for the official webpages about thesises on the website of KUL.
Little brainstorm:

It's live music we're dealing with. Live as in real-time. Also as in 'with public' but that doesn't change anything for us.
It's music we're dealing with. If it's not necessary to specify which kind of music I wouldn't do it. But it might be appropriate to make a coarse distinction anyway. Something like: electro-acoustic music, or modern music (although this does not necessarily cancel out classical music). Maybe this isn't even necessary because the fact that you use audio effects already implies that you won't be doing classical music. Woops, not true: some effects might be useful for classical live conceerts too. What to do with this item?
Digital effects of course
Intelligent is a better name then adaptive if you ask me, although adaptive is already used in literature [verfaille, arfib]

Extensively comment on this post please!

2007-02-27T17:14:00.000+01:00

matlab session on pitch tracking using harmonic product spectrum

I'm hoping to implement a working matlab algorithm based on the HPS method.
Some information on it:

http://www-ccrma.stanford.edu/~pdelac/220c/Presentation/img1.htm
A short image-based explanation of the algorithm and some parameters of it
http://cnx.org/content/m11714/latest/
A post about pitch detection explaining HPS too.

Much time 'lost' on the weekly meeting.
I started implementing HPS, but I'll continue tomorrow, nothing worth to say for now.
I also tried the frame-by-frame idea. Have to study it, but I made an implementation in the thesis folder on ESAT.

In between the soup and potatoes (dutch saying) I added a mixer section to the applyEffect algorithm:

%% mixer
inputSignalMixer = outputSignalEffect(1:length(originalSignal)); % cut any extra samples away for the wet and dry signals to match
outputSignalMixer = wet * inputSignalMixer + (1 - wet) * originalSignal; % mix wet and dry signal

weekly meeting

2007-02-27T15:55:00.000+01:00

A very long meeting, many things are said. I'll try a conclusion:
In this stage we see too many possibilities, too many ways we can walk. That's normal.

A good idea is to keep on swimming in this chaos for a few days and then try to decide on some things and ordering everything in our minds.
We discussed a while on realtime issues and complexity issues:
Following Toons' opinion there are two extremes for us. I made a little figure of them:

The first one shows the most difficult one: extracting information from the signal upon which we want to apply the effect.

The second one is complexer, but easier. Here we have the whole spectrum of chosen tracks we want to use to extract different features from the music. Best is to use them all at once as an input, then the algorithm can choose the tracks it needs for the solution. This feature vector is then used as an input for effects that are applied on different tracks of choice (not necessarily the same tracks that are used as input for the feature extraction.)

-28feb2007-
There is another well defined extreme I thought of: It's like the second idea, but using only the master tracks as an input for the extraction algorithms. This means you have your 'intelligent effect box' and you plug into it a stereo mic or the outputs of the stereo masters of the mixer. The algorithms have to segment the music then or at least they have to do their thing without knowing specific knowledge about the input. They must be able to deal with noise from the public. This is a problem we won't handle.
I talked also a lot about the interface of the effect blocks and extraction blocks. Probably the best and only choice is to use a modular structure. But in that case I want to have as soon as possible a standard interface.
We will never reach the stage where we implement the effect in C or something like that, the farthest we'll get might be simulink (modular and realtime). It might be the right time to switch to simulink sometimes already.
Obviously I forgot a lot of details here... someone?

2007-02-26T14:00:00.000+01:00

time based pitch tracking in matlab

Today I'll try a basic algorithm based on long term prediction (LTP). See DAFX p348 and further. It is optimized for a singing voice. This optimization is done in the post-processing part. I have to change the main algorithm the same way I did with the frequency based algorithm of the previous post.
Some calculations were needed to make sure the matrix dimensions of the for-loop are not exceeded.

indexMax = ((blocks-1)*K) + N + pre
-> ((blocks-1)*K) < Nx
-> Nx = n1 - pre = n1 - lmax
< n1 - pre + N + pre
= n1 + N

So we need to pad the X-vector with N extra zeros:

X = [X,zeros(1,N)]; % zero-padding until new length

The algorithm, with indicated external functions and default values:

% M-file 9.23
% pitch_tracker_ltp.m
%
% main file for a pitch tracker based on long-term prediction in time domain
% (c) 2002 Florian Keiler

fname='x1.wav';
%n0=2000; %start index
%n1=210000;
[X,Fs]=wavread(fname);
n0=1;
n1=length(X);

K=200; % hop size for time resolution of pitch estimation
N=1024; %block length

% checked pitch range in Hz:
fmin=50;
fmax=800;
b0_thres=.2; % threshold for LTP coeff
p_fac_thres=1.05; % threshold for voiced detection
% deviation of pitch from mean value

%[xin,Fs]=wavread(fname,[n0 n0]); %get Fs
%lag range in samples:
lmin=floor(Fs/fmax);
lmax=ceil(Fs/fmin);
pre=lmax; %number of pre-samples
if n0-pre<1
n0=pre+1;
end
Nx=n1-n0+1; %signal length
blocks=floor(Nx/K);
% blocks=floor(length(X)/K);%number of blocks
Nx=(blocks-1)*K+N;
%[X,Fs]=wavread(fname,[n0-pre n0+Nx]);
X=X(:,1)';
X = [X,zeros(1,N)]; % zero-padding until new length

pitches=zeros(1,blocks);
for b=1:blocks
x=X((b-1)*K+(1:N+pre));
[M, Fp]=find_pitch_ltp(x, lmin, lmax, N, Fs, b0_thres);
if ~isempty(M)
pitches(b)=Fs/M(1); % take candidate with lowest pitch
else
pitches(b)=0;
end
end

%%%% post-processing:
L=9; % number of blocks for mean calculation
if mod(L,2)==0 % L is even
L=L+1;
end
D=(L-1)/2; %delay
h=ones(1,L)./L; % impulse response for mean calculation
% mirror beginning and end for "non-causal" filtering:
p=[pitches(D+1:-1:2), pitches, pitches(blocks-1:-1:blocks-D)]; % 2D samples longer
y=conv(p,h); % length: blocks+2D+2D
pm=y((1:blocks)+2*D); % cut result

Fac=zeros(1,blocks);
idx=find(pm~=0); % dont divide by zero
Fac(idx)=pitches(idx)./pm(idx);
ii=find(Fac<1 & Fac~=0);
Fac(ii)=1./Fac(ii); % all non-zero element are now > 1
% voiced/unvoiced detection:
voiced=Fac~=0 & Fac<p_fac_thres;

T=40; % time in ms for segment lengths
M=round(T/1000*Fs/K); % min. number of blocks in a row
[V,p2]=segmentation(voiced, M, pitches);
p2=V.*p2; % set pitches to zero for unvoiced

%plotting things
figure(1)
clf
time=(0:blocks-1)*K+1; %start sample of blocks
time=time/Fs; %time in seconds
t=(0:length(X)-1)/Fs; % time in sec for original
subplot(211)
plot(t, X)
title('original x(n)')
axis([0 max([t,time]) -1.1*max(abs(X)) 1.1*max(abs(X))])

subplot(212)
idx=find(p2~=0);
plot_split(idx,time, p2)
title('pitch in Hz')
xlabel('time/s rightarrow')
axis([0 max([t,time]) .9*min(p2(idx)) 1.1*max(p2(idx))])

% synthesize sinusoids:
y2=synth_sine_fade(p2,K,Fs);
soundsc(y2, Fs)
wavwrite(.99*y2,Fs,[fname, '_pitch_ltp'])

This is the result with default arguments:

n0 = 1
n1 = length(X)
K = 200
N = 1024
fmin = 50
fmax = 800
b0_thres = .2
p_fac_thres = 1.05

The parameter K has the same effect as in the previous post.
Already having a very low threshold value I assumed the algorithm would be able to track x2.wav (a musical piece with strong melody) too (as seen in the previous post). But this is not true here. Whether this is inherent to the algorithm itself or due to the post-processing part I cannot tell. With a quick look on the code it seems that for both algorithms tested until now, the post-processing part is exactly the same, so this suggests that the algorithm itself is not fit for tracking musical melodies.
The threshold value has another effect apparently, because when I enlarge it, the results for x2.wav are somewhat better. I might check the exact meaning of the threshold value, but first I'll try some other algorithms, necause it seems that this one is inherently not fitted for our purpose.

2007-02-23T15:38:00.000+01:00

matlab session on pitch tracking ctd.

I started analyzing the FFT based pitch tracking algorithm described in DAFX pages 337 and further.
First I did the find_pitch_fft.m file:

function [FFTidx, Fp_est, Fp_corr] = find_pitch_fft(x, win, Nfft, Fs, R, fmin, fmax, thres)

%[x,Fs] = wavread(soundFile);
%x = x(1:Nfft+R);
% [FFTidx, Fp_est, Fp_corr] = find_pitch_fft(x, win, Nfft, Fs, R, fmin, fmax, thres)
%
% M-file 9.15
% find_pitch_fft.m
%
% find pitch candidates of given signal block x
% x: input signal of length Nfft+R
% win: window for the FFT
% Nfft: FFT length
% Fs: sampling frequency
% R: FFT hop size
% fmin, fmax: minumum/ maximum pitch freqs to be detected
% thres: omit maxima more than thres dB below the main peak
%
% (c) 2002 Florian Keiler

FFTidx = []; % FFT indices
Fp_est = []; % FFT bin frequencies
Fp_corr = []; % corrected frequencies
dt = R/Fs; % time diff between FFTs
df = Fs/Nfft; % freq resolution
kp_min = round(fmin/df);
kp_max = round(fmax/df);
x1 = x(1:Nfft); % 1st block
x2 = x((1:Nfft)+R); % 2nd block with hop size R
[X1, Phi1] = fftdb(x1.*win,Nfft);
[X2, Phi2] = fftdb(x2.*win,Nfft);
X1 = X1(1:kp_max+1);
Phi1 = Phi1(1:kp_max+1);
X2 = X2(1:kp_max+1);
Phi2 = Phi2(1:kp_max+1);
idx = find_loc_max(X1);
Max = max(X1(idx));
ii = find(X1(idx)-Max>-thres);

%----- omit maxima more than thres dB below the main peak -----
idx = idx(ii);
Nidx = length(idx); % number of detected maxima
maxerr = R/Nfft; % max phase diff error/pi (pitch max. 0.5 bins wrong)
maxerr = maxerr*1.2; % some tolerance
for ii=1:Nidx
k = idx(ii) - 1; % FFT bin with maximum
phi1 = Phi1(k+1); % phase of x1 in [-pi,pi]
phi2_t = phi1 + 2*pi/Nfft*k*R; % expected target phase after hop size R
phi2 = Phi2(k+1); % phase of x2 in [-pi,pi]
phi2_err = princarg(phi2-phi2_t);
phi2_unwrap = phi2_t+phi2_err;
dphi = phi2_unwrap - phi1; % phase diff
if (k>kp_min) & (abs(phi2_err)/pi<maxerr)
Fp_corr = [Fp_corr; dphi/(2*pi*dt)];
FFTidx = [FFTidx; k];
Fp_est = [Fp_est; k*df];
end
end

Problems I encountered:

I erased Fs as an input argument, because I can get directly from the wavread command. This posed no problems until I used another function that calls this function. For both functions to use the same Fs, it's necessary to send it as an input argument.
When inputting the argument win I must describe it like this:
find_pitch_fft('La.wav',hann(1024),1024,1,20,15000,40)

Then after that I went on to the Pitch_Tracker_FFT_main.m file.

In the main function only I encountered some problems:

I had to correct the first line like this:
I had to add the apostrophes and the .wav extension
fname='x1.wav';
Next thing was that they used the algorithm optimized for a specific wavfile. To generalize this I had to adjust the n0 and n1 boundaries. I reasoned, what is the most general solution for this, and it would be to set n0 to zero and n1 to the number of samples the given piece contains.
%these are optimized for x1.wav
%n0=2000; %start index
%n1=210000;
%I'll try it in general
[X,Fs]=wavread(fname);
n0=0
n1=length(X)
This gave some problems where they do some changes on n0 and n1, because suddenly the new n1 would be bigger than the length of the piece. I solved this by just eliminating this part. The algorithm still works then, so I ask why this part was contained in the algorithm. (OK)
%Nx=n1-n0+1+R %signal length
%blocks=floor(Nx/K)
%Nx=(blocks-1)*K+Nfft+R
%n1=n0+Nx % new end index
But this gives a matrix dimensions exceede problem:
for b=1:blocks
x=X((b-1)*K+1+(1:Nfft+R));
Strange that matlab doesn't seem to complain about the fact that blocks isn't defined
These and other adjustments kept creating new problems. I tried it otherwise:
I followed the original algorithm step-by-step to see what it did.

Nx = n1 - n0 + 1 + R
%so it actually makes the chosen part longer
%in my case, as I want n0 to be 1 en n1 to be the length of the audio sample, it makes Nx greater than the total length of the wave file.
blocks = floor(Nx/K)
%As far is I understand this divides the total number of Nx samples in Nx/K blocks, each of length K (floored to the nearest smaller integer), and non-overlapping.
%so this must be something else than the blocks for the FFT who are overlapping in their length-R.
%anyone an idea what they're here for then? The algorithm talks about hop size for time resolution for pitch estimation (see lower for an answer (remarks after the algorithm))
Nx = (blocks-1)*K + Nfft + R
= ((floor(Nx/K))-1)*K + Nfft + R
< ((Nx/K+1)-1)*K + Nfft + R
= Nx + Nfft + R
= n1 - n0 + 2*R + 1 + Nfft
%the new end index becomes
n1 = n0 + Nx
< n0 + n1 - n0 + 2*R + 1 + Nfft
= n1 + 2R + 1 + Nfft
This is the maximum number of samples that are added to the original part of the audio sample, in my favored case this is the max number of samples added to the total audio sample.
This now gives us the choice:
- OR we do the wavread here. We must then care for the fact that the original n1 must be smaller than the total length of the audio sample minus 2*R + 1 + Nfft. Because now you will certainly not try to 'wavread' too many samples, samples that aren't there.
- OR we can zero-pad the wave vector here until it has the new length n1-n0
  This is what I'll do here. It might give an small mistake for the last blocks that comprise these zeros, but there will always be problems at the end of a sample. I think the zeros minimize thiese problems. (I can be wrong)

This brings us to the first part (that works generally):

% M-file 9.18
% pitch_tracker_fft_main.m
%
% main file for a pitch tracker based on FFT with phase vocoder approach
% (c) 2002 Florian Keiler

fname='La.wav';
[X,Fs]=wavread(fname);
n0=1
n1=length(X)

Nfft=1024;
R=1; % FFT hop size for pitch estimation
K=200; % hop size for time resolution of pitch estimation
thres=50; % threshold for FFT maxima
% checked pitch range in Hz:
fmin=50;
fmax=800;
p_fac_thres=1.05; % threshold for voiced detection
% deviation of pitch from mean value

win=hanning(Nfft)'; % window for FFT

Nx=n1-n0+1+R %signal length
blocks=floor(Nx/K)
Nx=(blocks-1)*K+Nfft+R
n1=n0+Nx % new end index
X=X(:,1)';
length_afterCalculatingNewLength = length(X)
for i=length(X):n1 this I changed to:
X(i)=0; X = [X,zeros(1,n1-length(X))];
end
new_length = length(X)

pitches=zeros(1,blocks);
for b=1:blocks
x=X((b-1)*K+1+(1:Nfft+R));
[FFTidx, Fp_est, Fp_corr]=find_pitch_fft(x, win, Nfft, Fs, R, fmin, fmax, thres);
if ~isempty(Fp_corr)
pitches(b)=Fp_corr(1); % take candidate with lowest pitch
else
pitches(b)=0;
end
end

Remarks on parameters:

If I enlarge K (1000) the algorithm works much faster, but, as you see, the time resolution is not so good.
If I make K smaller, the calculations take very long and the pitch tracking is much more fluent. Sometimes it has short faulty tones. Maybe we can filter these out assuming that the voice cannot change that fast and short. But this would stop us from using the algorithm for instruments...

K thus sets the nuber of times that the main for-loop is run.
A very big (1000) value for R results in long calculations and technically seen a worse pitch estimation, see fig9.27, p337 in DAFX.
For input x2.wav to give a meaningful result I set thres on 1 (which is very small compared to the preferred 30-50). This means that we leave the allowed deviation from zero for the phase error very small. I don't really understand why that gives better results here. If I let the thres value have its default value, then we get nothing but two low blobs of sound somewhere in time.

Next time I'll try the time-domain based pitch tracking algorithm from DAFX. For now I'm stopping here.

matlab session on pitch detection, but more about ring modulator etc.

2007-02-22T16:45:00.000+01:00

Also today I do a little warming up: I'd like to give a it a shot and implement a ring modulator (DAFX p76). The ring modulator of my yamaha MO8 has the following parameters:

oscillator frequency coarse (0.5 - 5 kHz)
oscillator frequency fine (0 - 127)
LFO wave (tri, sine)
LFO depth (0 - 127)
LFO speed (0.0 - 39.70 Hz)
HPF cutoff frequency
LPF cutoff frequency
dry/wet balance
EQ low frequency
EQ low gain
EQ high frequency
EQ high gain

Only the first five comprise the ring modulation, 6 and 7 are filter parameters, 8 is a mixer parameter and the rest are EQ parameters.
Some remarks:

I used
modSound = sin(2*pi*OSCfreq*[0:1/Fs:Nbits]);
but the Nbits indicates the quantization depth (16 bits when used with the wave file flute2.wav in my folder). Instead I have to use the length of the sound vector (as seen in the algorithm) when I want the modulating signal to be as long as the soundFile input.
What I still don't get is why I have to divide the length of the vector sound with Fs. It is a kind of normalization, and if I don't do it, it wouldn't work. But I don't know why.
solution: Try it with a ridiculous 2Hz sampling frequency and 10 seconds music sample. You'll see that if you want as many samples on the sine wave as on the input sample you'll have to divide the length of your input minus 1 through 2 to get to the 10 seconds.
I spent a reasonable amount of time in generating different signals to be used as modulators; sine wave, triangular wave (special case of:), sawtooth, square wave. Everything works allright and the explanations are to be found in the m-file.
The algorithm:
function [output] = ringmod(soundFile, OSCfreq, LFOwave, LFOdepth, LFOspeed)
% ring modulator
% y(n) = x(n).m(n)
% OSCfreq: oscillator frequency (best between )
% LFOwave: type of wave that modulates the audio signal (sine, triangle, square, sawtooth with standard width)

[sound,Fs,Nbits] = wavread(soundFile);

% defining the modulating wave type 'LFOwave'
% SINE
if (strcmp(LFOwave, 'sine')) % use strcmp instead of == to compare strings
% sampled taking steps of Ts or 1/Fs and this until length(sound)/Fs samples are
% calculated
% number of vector values 'length(sound)' must be normalized by Fs
% because ... (?)
modSound = sin(2*pi*OSCfreq*[0:1/Fs:(length(sound)-1)/Fs]);
end
% SAWTOOTH
if (strcmp(LFOwave, 'sawtooth'))
width = 0.9 % this defines where the max of the wave is situated in the interval between 0 and 2*pi
modSound = sawtooth(2*pi*OSCfreq*[0:1/Fs:(length(sound)-1)/Fs],width);
end
% TRIANGLE
if (strcmp(LFOwave, 'triangle'))
%triangle wave (width = 0.5)
modSound = sawtooth(2*pi*OSCfreq*[0:1/Fs:(length(sound)-1)/Fs],0.5);
end
% SQUARE
if (strcmp(LFOwave, 'square'))
modSound = square(2*pi*OSCfreq*[0:1/Fs:(length(sound)-1)/Fs]);
end

%wavplay(modSound, Fs); %for testing
%sound(1:100)
%modSound(1:100)
%length(sound)
%length(modSound)
output = sound .* modSound';

wavplay(output, Fs);
This effect is perceived as nice eg. when using a voice as input, sine or other wave as modulation and low frequencies (~50Hz)
I had some problems with the multiplication, but after trying it with small matrices I saw why:
- I had to use a dot-product for elementwise multiplication .*
- I had to have two columnmatrices or vectors. The modSound waves are on the contrary row matrices, so I had to transpose them: modSound'

OK, a warmup during 3 hours, typical...
So now a little bite and then: pitch extraction!

Hm, another two hours spent on surfing the internet and finding sites we could use. Look at the posts involved: Links, just links , some links for beat extraction.
Also I started a post that will contain ideas we have during our work. This way we have a leading trail when experimenting later on.
OK, another try towards pitch detection.

I'll talk about pages 336 and further in DAFX.
Pitch extraction is the same as estimating the fundamental frequency f0 and then possible postprocessing like pitch tracking and taking into account frequency relationships.
I'll be reading the interesting things and tomorrow I'll try the matlab code.

Links, just links

2007-02-22T15:16:00.000+01:00

http://www.cs.uvic.ca/~gtzan/work/publications.html
homepage and publications of george tzanetakis, main interest field retrieval sytems for audio
Representative music fragments extraction by using segmentation techniques
Melody extraction on polyphonic music using 'four measure based segmentation' and 'LBDM concatenation'
Is used for music classification, but mainly MIDI files. It uses the knwledge of different tracks, where we have one track without knowing the different tracks apartly.
Human perception and computer extraction of musical beat strength
Musical pattern extraction: from repetition to musical structure
Automatic drum track extraction from polyphonic music signals
Extraction of characteristic music textures (eigentextures) via graph spectra and eigenclusters
Independent Component Analysis for music similarity computation
Efficient pitch detection techniques for interactive music
An efficient pitch tracking algorithm using a combination of fourier transforms
Real-time temporal segmentation of note objects in music signals
real-time/ onset/ offset
Salient Feature Extraction of Musical Instrument Signals
Thesis of a student of the princeton department music
Robust Singing Detection in Speech/Music Discriminator Design
Speech/Music/Silence and Gender Detection Algorithm

Ideas, just ideas

2007-02-22T14:42:00.000+01:00

detect when a song is being played or not and use this information to turn on and off the reverb on the singers' voice.
At first I thought using the singers' mic as an input, but that's not enough. Practically the algorithm must know the difference between someone talking and someone singing. Is it enough to do this by detecting when music is playing? No, because when singing (parts) in a capella the reverb must remain switched on!
see http://ieeexplore.ieee.org/iel5/9248/29346/01326825.pdf
detect the tempo of a song and use it as the delay time parameter of a delay effect.
detect the pitch/tone at which a singer is singing and (when switched on) automatically pull the pitch to the closest existing note. Practically this might only be useful for specific notes, those notes that are difficult for the singer to reach. For the rest it must be turned off, because the importance of a singer as an instrumentalist is that he can form notes between existing ones, and not to forget glissandos too.
Might already exist: autotune (DAFX p336)
Another one that is metioned also in DAFX p336: detect amplitude and use it as a parameter for compression
detect no silence and let the compressor with high output gain only work then. This way when there is silence the noise will not have such a high gain and disturb the sound.
maybe a compressor that only compresses the track in the frequency bands (or even as detailed as the pitch itself) it's currently playing. Wait, I thought of this as means to not make the noise problems worse in silent passages, but this doesn't change. Better idea: same idea but with a noise gate.

some links for beat extraction

2007-02-22T13:52:00.000+01:00

http://www.fxpal.com/publications/FXPAL-PR-01-152.pdf
http://www.fxpal.com/publications/FXPAL-PR-01-022.pdf
http://www.csl.sony.fr/downloads/papers/2002/ZilsMusic.pdf automatic extraction of drum signals from polyphonic music signals

In a little chat with toon we talked about the problems so far with beat detection.
A possible part of the solution is to append weight factors to the frequency bands, so that the most important band has the biggest weight. The question is how to calculate these factors. The best idea is to let them be calculated automatically, by means of correlation values or something for every band.
Also we talked about bigger time windows, as in why don't we look for repeating riffs or measures. This might work better with more difficult pieces of music. This brought us to googling on "measure detection" and other stuff. But this is a TODO.

measure extraction music
This particular search term discovers loads of interesting websites. This is done in the post: Links, just links.

another idea:
why not use ICA but for audio then, to extract independent components like a drum line?

matlab session on pitch extraction

2007-02-21T18:59:00.000+01:00

+tony, ronny+

I wanted to try another effect before starting with pitch detection algorithms. The one I chose was guitar distortion effect.
I stumbled upon a very good link: www.musicdsp.org
There you can find an archive on analysis, effects and some more. It seems that most of the coding is done in C or C#. The distortion effect I tried here is translated to Matlab code.

function [x]=gdist(a,x)
%[Y] = GDIST(A, X) Guitar Distortion
%
% GDIST creates a distortion effect like that of
% an overdriven guitar amplifier. This is a Matlab
% implementation of an algorithm that was found on
% www.musicdsp.org.
%
% A = The amount of distortion. A
% should be chosen so that -1<A<1.
% X = Input. Should be a column vector
% between -1 and 1.
%
%coded by: Steve McGovern, date: 09.29.04
%URL: http://www.steve-m.us
[x,fs] = wavread(x);
k = 2*a/(1-a);
x = (1+k)*(x)./(1+k*abs(x));
wavplay(x,fs);

I also tried another one:

function [output]=fuzzy(sound, amount)
[sound,fs] = wavread(sound);
norms = norm(sound);
output = sound/norms;
amount = (1- amount)/100;
for i = 1:length(output)
if ( output(i) > amount )
output(i) = amount;
end
if ( output(i) < -amount)
output(i) = -amount;
end
end
output = output*norms;
wavplay(output,fs);

Both algorithms are to be found in my folder on ESAT (gdist.m and dist.m)

see above
I tried to catch a book which was referenced from an article I read (sorry, forgot which one). The book from Roads is called 'the computer music tutorial'. It became a real quest to find it. I didn't yet succeed. I'll keep trying.
Order of the day. I'll experiment with pitch extraction: see the rest of this post

NARGIN; this is used as an indicator of number of input arguments: 'Number ARGuments IN'
Different methods:

Pitch determination and voice quality analysis using subharmonic-to-harmonic ratio (http://www.ling.northwestern.edu/~jbp/sun/sun02pitch.pdf )
Zero crossings
Autocorrelation
Maximum likelihood (time or frequency)
Adaptive filter (more than one)
Super resolution pitch determination
Harmonic product spectrum
Cepstrum
Perceptual pitch detector (http://www-ccrma.stanford.edu/%7Epdelac/154/m154paper.htm )

Worked awhile on the PDA described in another post. I don't really understand how to interpret the resulting output.

This is two times the result f0_values (resp of x2.wav and x1.wav from my thesis folder on ESAT). I have no idea how to interpret it... help me?
Some other links on pitch detection:

http://ccrma.stanford.edu/~eberdahl/Projects/Grundfrequenzanalyse/index.html (contains on the first sight a very good tutorial in german )
http://www.dsprelated.com/showmessage/35165/1.php (a discussion on pitch detection for singing voices)
http://cnx.org/content/m11716/latest/ (m-files on autocorrelation and HPS method)
http://www.soundmathtech.com/pitch/download/terez_icassp02.pdf (robust pitch determination using non-linear state-space embedding)

I was hoping to be able to try out some m-files. I even did try the one called fastauto.m, but again I didn't understand the results. More on these tomorrow!

matlab session on beat detection

2007-02-21T18:24:00.000+01:00

vandaag beat detection algorithm

enkele interessante links:
http://www.owlnet.rice.edu/~elec301/Projects01/beat_sync/beatalgo.html
http://homes.esat.kuleuven.be/~tvanwate/alari/projects/beat_detection/scheirer98.pdf

uitleg bij termen:
- wave rectifyers
http://en.wikipedia.org/wiki/Rectifier

filterbank.m:
- bl(i) = floor(bandlimits(i)/maxfreq*n/2)+1;
hierin is bandlimit(i)/maxfreq schaalfactor voor uw samples van uw FT
- nog eens nagaan wat juist nen DFT doet
zet n samples om naar n samples (op welke manier stellen die laatste
samples dan het frequentie domein voor ??---> avond lectuur)

andere m-files gewoon uitgevoerd:

de 4 stappen die worden toegepast zijn wel duidelijk
echter algoritme moet wat uitgediept worden...

eens uitgetest op hiphop schijf, kwam niet echt iets deftig uit
volgende keer eens uittesten met verschillende house schijven
beat zou dan gemakkelijker moeten kunnen gedetecteerd worden

conclusie: blijkt wel te werken
algoritme nog niet volledig door --> nog eens verdiepen
volgende stap uittesten om te zien of hij relatieve verschillen goe vastlegt

matlab session on feature extraction

2007-02-20T15:44:00.000+01:00

A short list of several feature extraction methods for audio:

pitch tracking (dafx p.477)
(
http://cnx.org/content/m11714/latest/
http://www-ccrma.stanford.edu/~pdelac/154/m154paper.htm
audio to midi convertion
)
amplitude tracking (dafx p.477)
centroid tracking (dafx p.477)
voice/silence tracking (dafx p.477)
voiced/unvoiced tracking (dafx p.477)
partial tracking (dafx p.477)
rhythm tracking (dafx p.477) / beat / tempo
(
http://www.owlnet.rice.edu/~elec301/Projects01/beat_sync/beatalgo.html
http://homes.esat.kuleuven.be/~tvanwate/alari/projects/beat_detection/scheirer98.pdf
http://www.cs.uvic.ca/~gtzan/work/talks/job/jobtalk.ppt)
timbral texture feature tracking (music genre recognition by tony)

Weekly meeting

2007-02-20T15:20:00.000+01:00

We agreed on many ideas: after experimenting with effects in matlab, we should now start doing matlab on extraction of features in audio. That is what we're going to do first. Then the next step would be to connect extraction features with effects. All this might be done in matlab first.
Of course we really want this to be done in real time (not only to do easy testing, but also for client purposes) and modular.
If we could make standard effect blocks and standard extraction blocks in simulink that would be nice. Simulink might indeed be the best next step after matlab, eg to test in realtime (although this might require some reprogramming of the m-functions).
We also talked about the idea to turn everything in VST plugins or direct X plugins as a final step, but we might lack time for that.
TODO: Our promotor M. Moonen would like us to do an intermediate presentation. Here we might explain some theoretical issues about effects and extraction algorithms we already tried and also we can explain our project status and the next steps towards our goal.

warming up session 2007

2007-02-13T15:35:00.000+01:00

Exams finished, vacation finished. As for me, I'm almost healthy again.
Today Tony and I met to set things straigth and restart our discovery.
Our plan for the nearby future:
This week we will finish the basic literaturestudy (as in, we will read the DAFX book and surf across the internet a lot)
So next week we can start trying out some feature extraction methods (as we already did with effects)

On monday 19 feb we hope to meet with our assistants.

Matlab exchange server examples

2006-12-13T11:11:00.000+01:00

sound_acquisition.m It reads in a number of seconds of audio from your computers soundcard and it is easy to change the duration, samplefrequency and soundcard.
pitch determination algorithm (.zip) This zip-file contains a few files: the algorithm itself and some post-processing things.

The input Y specified here is not a wave file but the vector. I wrote the wavread component inte the m-file.
If you specify the input and the samplefrequency, it's enough.
The algorithm works but I don't really understand the outcome.
% Input parameters (There are 9):
%
% Y: Input data
% Fs: Sampling frequency (e.g., 16000 Hz)
% F0MinMax: 2-d array specifies the F0 range. [minf0 maxf0], default: [50 550]
% Quick solutions:
% For male speech: [50 250]
% For female speech: [120 400]
% frame_length: length of each frame in millisecond (default: 40 ms)
% TimeStep: Interval for updating short-term analysis in millisecond (default: 10 ms)
% SHR_Threshold: Subharmonic-to-harmonic ratio threshold in the range of [0,1] (default: 0.4).
% If the estimated SHR is greater than the threshold, the subharmonic is regarded as F0 candidate,
% Otherwise, the harmonic is favored.
% Ceiling: Upper bound of the frequencies that are used for estimating pitch. (default: 1250 Hz)
% med_smooth: the order of the median smoothing (default: 0 - no smoothing);
% CHECK_VOICING: check voicing. Current voicing determination algorithm is kind of crude.
% 0: no voicing checking (default)
% 1: voicing checking
% Output parameters:
%
% f0_time: an array stores the times for the F0 points
% f0_value: an array stores F0 values
% SHR: an array stores subharmonic-to-harmonic ratio for each frame
% f0_candidates: a matrix stores the f0 candidates for each frames, currently two f0 values generated for each frame.
% Each row (a frame) contains two values in increasing order, i.e., [low_f0 higher_f0].
% For SHR=0, the first f0 is 0. The purpose of this is that when you want to test different SHR
% thresholds, you don't need to re-run the whole algorithm. You can choose to select the lower or higher
% value based on the shr value of this frame.
I think some nice functions in our field are found in the m-file (bottom to top): generate a window function, compute zero-crossing rate, post voicing checking, split signal into frames, determine the energy treshold of silence, determine whether the segment is voiced, unvoiced or silence, compute subharmonic-to-harmonic ratio, do FFT and get log spectrum

AGC.m An automatic gain control script, might be interesting for all the tests it does on the input signal and because it can process both mono and stereo inputs. If we ever need gain, power or energy commands, here's the place to look first for.

unicomb, vibrato, wisperization

2006-12-09T12:39:00.000+01:00

1) unicomb testing for toms_diner.wav

calculations much too long... must be done something about it.
after 20 minutes he did it...
works, but veeerrry slowly

2) trying to make vibrato work

too long in time
bad parameterchoice
now OK

3) testing of wisperization

book is not here, so I'm just fooling around a bit
effect is clearly audible, but somewhat too harsh
experiment a little with the parameters and turn the m-file in one where you can give the wav-file to.

matlabsession ronny and tony

2006-12-06T14:24:00.000+01:00

Today we try to implement some effects. Some notes:

Instead of wavplay() you can also use sound(vector,sampleFrequency), it does fairly the same. Also stereo if the matrix is Nx2. soundsc() even plays the sound at the maximum amplitude without clipping.
With wavrecord() one can record wave audio from the windows wave input device.
When using wavread and wavplay and others, pay attention to play at the same sample frequency as read. Otherwise the sound will mostly play as if it passed through an lowpass filter, because the standard Fs = 11025Hz
Ctrl+C is used to terminate a calculation.
switch/case statement can be used in matlab too
m-files can make use of cells to make the overview clearer

I tried implementing a flanger effect I found on the mathworks exchange server. Herefore I made a function that I can use throughout our quest for cool effects:
function[] = applyEffect(effectFile,soundFile)
Example: applyEffect('flange','x1.wav');

<< soundfile =" 'redwheel.wav';" effectfile =" '';" variation =" .01;" rate =" .3;" outputsignal =" flange(inputSignal," outputsignal =" signal;">>

This works well. I tried it with flange and after experimenting with the parameters variation and rate it worked nice enough.
Now I'm trying to understand every line of flange.m for future use... Some notes:

: Colon.
J:K is the same as [J, J+1, ..., K]
CEIL(X) rounds the elements of X to the nearest integers
towards infinity.
ROUND(X) rounds the elements of X to the nearest integers.
and thus: CEIL(X) <= ROUND(X) CEIL(X) >= ROUND(X) >= FLOOR(X)
>> abs([1 2 3;4 5 6;7 8 -9])
ans = 1 2 3
4 5 6
7 8 9
.* Array multiply.
X.*Y denotes element-by-element multiplication. X and Y
must have the same dimensions unless one is a scalar.

I almost succeeded understanding the whole m-file about flange. Got stuck with this line:
signal = signal(cosineSamples+modulation-lfo);
signal is a column vector and between the brackets there is a row vector, what could this mean then?

I include the flange.m file in the version of today:

function [signal]=flange(signal, sampleFrequency, variation, rate)
%[y] = flange(fs, v, x, r)
%
% This is a basic flanging effect.
%
% fs = Sample rate
% v = Variation.
% x = Input audio signal. This should be a column
% vector.
% r = Rate.
%
% Example:
%
% >>y = flange(fs,0.002,x,0.5);
%
%
% See also WAVREAD and SOUND
%
%Version 1.0
%Coded by: Stephen G. McGovern, date: 08.03.03

modulation = ceil(variation*sampleFrequency);
%a row containing a raising value per sample of the signal
cosineSamples = 1:length(signal)+ modulation;
variation = round(variation*sampleFrequency);
padding = zeros(modulation,1);
%maxSample is the biggest amplitude of the vector or matrix signal
maxSample = max(abs(signal))
signal = [padding;signal;padding];
%period in seconds
period = 2*pi/round(sampleFrequency*rate);
%approximates a cosine which variates about variation/2 as an LFO
lfo = round((variation/2)*(1-cos(period.*cosineSamples)));
%chooses the samples indicated by the vector cosineSamples+modulation-lfo, from signal
signal = signal(cosineSamples+modulation-lfo);
maxSample = maxSample/max(abs(signal));
signal = maxSample*signal;

unicomb / delay / vibrato

2006-12-06T12:25:00.000+01:00

1) unicomb filter 'all pass' mode implemented

calculations too long; why?

spectrogram plot: search/make something decent

watch out with commands wavread and wavwrite --> you have to give the right sampling frequency, otherwise your sound is totally distorted, while it's not the fault of your algorithm

we worked on a vibrato, nut didn't succeed; maybe just because previous point :( --> we should try it again !!

2) to implement delay mode

problems with calculation time... not able to use it because of duration/ why?
a delay with parameters M=20000 so approx. 0.5 seconds results in no sound when using La.wav (it's own duration is also 0.5sec) This is because everything falls outside the window then. But do you have to sum the original signal with this one? Because it seems that this is already done in the figure (DAFX ch3). --> We must sum, it is NOT done in the picture

3) vibrato

same problem: too long ! frustrating !
ok, it was because of the parameterchoice... shorter, but still not short enough.

spectrogram function by Evan Ruzanski

2006-12-05T17:44:00.000+01:00

function spectrogram(x,fs,bw)
% Plot grayscale spectrogram with variable sampling rate and bandwidth
% Evan Ruzanski, ECE259, 2/26/2003

% Set minimum FFT length
fftmin = 256;

% Set 2*fs/bw variable window length for good resolution with long block length
% will provide higher frequency resolution as main-lobe of the window
% function will be narrow and short block length will provide higher time
% resolution as less averaging across samples is performed for each
% STFT value
winlen = floor(2*fs/bw);

% Get FFT length
fftlen = max([winlen fftmin]);

% Create window (Hamming for favorable sidelobe attenuation) and zero pad accordingly
win = [hamming(winlen) ; zeros(fftlen-winlen,1)];
win = win/sum(win); %normalizing?
windel = (0:(length(win)-1)) * win;
??

hmm from hereon I loose it... someone?
% Set overlap (Effects expansion of spectrogram display as no overlap plots
% fftlen data points and maximum overlap gives 1 data point) ?? This I don't understand?
ntime = 500; % Choose based on trial-and-error for best looking plot
overlap = floor(max(fftlen/2, (ntime*fftlen-length(x))/(ntime-1)));
ntime = floor((length(x)-overlap)/(fftlen-overlap));

% Create arrays
c1=(1:fftlen)';
r1=(0:ntime-1)*(fftlen-overlap);

% Take FFT of real data
b = fft(x(c1(:,ones(1,ntime))+r1(ones(fftlen,1),:)).*win(:,ones(1,ntime)));
if size(b,1) == 1
m = length(b1);
b(floor((m+4)/2):m) = [];
else
m=size(b,1);
b(floor((m+4)/2):m,:) = [];
end
b = b.*conj(b);

% Setup pixel locations for plot on frequency axis and time axis
f=(0:fftlen/2)*fs/fftlen; % Point spacing
t = (r1+windel)/fs;

% Set limit for dB scale
lim = max(b(:))*0.0001;

% Set dB scale
b=2.5*log10(max(b,lim));

% Plot
imh = imagesc(t,f/1000,b);

% Set up axis and labels for plot
axis('xy');
title('Spectrogram');
xlabel('Time (s)');
ylabel('Frequency (kHz)');

% Set and apply grayscale levels
colormap(gray);
map2 = colormap;
map2 = 1 - map2;
colormap(map2);
colorbar; % Applies color legend to plot
orient landscape;

matlab session on my own

2006-12-05T16:04:00.000+01:00

To login to ESAT from a remote computer I use Filezilla:
host: login.esat.kuleuven.be
port: 22
servertype: SFTP using SSH2
user: rvandenb
Here I hold the folder 'Thesis', where all the documents are placed.

To give labels to axes:
xlabel('Time');
ylabel('Frequency (Hz)');
I tried understanding the command spectrogram, but...
Even the chirp example given by matlab I don't understand. I now try searching google on it.

Found an at first sight, very exciting URL: http://www.nd.edu/~nkottens/
Some other m-files about audio on the mathworks file-exchange also with some effects!

There is a spectrogram function on this site with better visual effect.
I'll study this one in another post

weekly update and some matlab 5nov2006

2006-12-05T15:51:00.000+01:00

Just before the first weekly update since like a month, I worked a little bit on matlab.
I tried making a simple m-file which does all kinds of standard stuff for audio processing. Load a file, play a file, view the plot and spectrogram.
Some points of attention:

subplot(numberOfRows, numberOfColumns, indexOfPlot)
after that, comma-separated, you specify the plot (plot(), stem(),...)
and then several things you like, like: title(''), ...
indexOfPlot counts from left-to-right and top-to-bottom.
wavread:
[soundFile,sampleFrequency] = wavread('La.wav');
soundFile is a vector with all the samples (values between -1 and 1). If you want to see only the first n bits, you add n after the filename at the RHS of the equation. To see the vector you omit the point comma.
sampleFrequency in Hertz
You also can specify the number of bits used to encode every sample (Quantisation), therefore you add 'NBits' after 'sampleFrequency'
Supports multi-channel data, with up to 32 bits per sample.
This file reader only supports Microsoft PCM data format.
Also auread is possible
To control the axes of the plot a website previously specified uses: axis('tight')
This sets the limit of the axes to the range of data. I think that the y-axis is too tight then, so I put it on auto (still quite tight though). What I search for is a command to set only the y-axis (to -1, 1)
I think it's best to always add 'clear all;' in the beginning of an m-file.
The website still used specgram, but matlab prefers spectrogram now.
I will study this soon.

Then the update with toon and tony.
We really should start implementing some effects in matlab. When encountering problems we don't have to bother disturbing toon.
We talked a bit about simulink/matlab/... According to toon best is to implement in matlab. He thinks there's no way to implement realtime functions in matlab (although he heard that in a newer version of matlab there'd be a possibility to do exactly so). The normal procedure would be to translate the matlab code to C or JAVA.
More wasn't really said I think. Before the exams we must really try to implement some effects and get used to the matlab environment.
We also can start searching for articles on audio feature extraction.
That's all.

Problems and questions encountered when working with matlab

2006-12-01T13:12:00.000+01:00

I'm doing a matlab session today and it clearly needs a warm-up.
In almost every m-file I try from the book DAFX, I either don't fully understand it, either it gives errors, either I don't see how it can be used with an audio fragment.
A short google search on "matlab commands functions for digital audio" gives me a few interesting looking URL's:
http://ccrma.stanford.edu/~jos/filters/Matlab_Utilities.html
and further on "audio in matlab":
matlab audio processing examples
http://vise.www.ecn.purdue.edu/VISE/ee438L/matlab/help/pdf/audio.pdf
audio processing

Some questions I have:

How do you allow figures to be in one window? And with a title...
What must you do when a function used in an example m-file of the book DAFX isn't recognized by matlab? (like CreateNewTrack)

weekly meeting thursday 9nov2006

2006-11-09T10:26:00.000+01:00

I met Sam, the other two were not present.
I read the chapter about nonlinear processing techniques and finally succeeded installing matlab (after download in hetdepot and problems with the image processing toolbar which I then didn't install)
Next (this afternoon) we will start implementing a delay effect (or so) in matlab. Sam and me talked about simulink and matlab and we'll begin implementing in both languages. To see which one will fit best. For realtime simulink is easier, but maybe this must be a problem for later on.
Next meeting on next thursday two o'clock PM.

weekly meeting tuesday 31oct2006

2006-11-02T11:36:00.000+01:00

I read through chapters 1 to 5 in DAFX. I was also making a list of possible effects and their possible parameters. This was an extensive job leading to the understanding that you can keep on inventing parameters for effects.
We assume that an effect algorithm starts with the basic effect and its parameters and after that you can make it more complex and add parameters. We thought that it might be better to start implementing effects right away starting with the basics and then experimenting with its parameters and adding parameters on the fly. The same with extraction algorithms. This instead of reading about them and then looking after the possibilities, because this might leave some possibilities unsought.
Also we will start searching search engines looking for articles about our subject.

http://scholar.google.com/
Google Scholar (niet specifiek voor audio, maar
wel zowat de beste zoekmachine die er is voor wetenschappelijke
publicaties)

http://www.aes.org/search/
Journal of the Audio Engineering Society + AES Conventions

http://scitation.aip.org/vsearch/servlet/VerityServlet?KEY=JASMAN&ONLINE=YES
The Journal of the Acoustical Society of America

digital effect parameters

2006-10-18T19:04:00.000+02:00

Hier volgt een lijst met een hele hoop effectparameters. Zo gedetailleerd mogelijk, maar daarom zeker niet alles zal hier worden neergepend. Ik noteer ook de webpages die ik gelezen heb.

+ Owner's manual Roland synthesizer SH-32

+ Data list Yamaha music production synthesizer MO8

(+) Owner's manual Korg synthesizer/vocoder microKorg

+ User's manual t.c.electronic dual effects processor M ONE

+ User guide Lexicon digital effects processor PCM80 en quick reference guide

+ Manual t.c.electronic multitap rhythm delay D TWO

+ Authorized product manual Yamaha professional multi-effect processor SPX1000

REVERB

http://www.harmony-central.com/Effects/Articles/Reverb/
http://www.earlevel.com/Digital%20Audio/Reverb.html
http://en.wikipedia.org/wiki/Reverb
http://www.digit-life.com/articles2/theoryofreverb/index.html

Onder reverb vallen oa:
hall, room, stage, plate reverbs, stereo reverb, gated reverb (zie gate), reverse reverb

Eerst heb je enkele early reflections die de eerste weerkaatsingen voorstellen (bv van de muren), daarna de late reflections of diffuse reflections (diffusion), die veel meer random zijn. Deze reflections beginnen na de respectievelijke predelays [ms][initial delay]. Deze predelay kan verschillend zijn links en rechts voor stereo reverbs en stelt eigenlijk kamerafmetingen voor rekening houdend met de plaats van de luisteraar. De reverberation time [ms] stelt de tijdspanne voor waarop het geluid met 60dB van zijn originele amplitude verlaagd is.
Nog parameters zijn:
reverb decay time [ms] stelt de tijdsduur voor dat de reverb te horen is na het stoppen van de input. Hoogstwaarschijnlijk bestaan analoog ook reverb attack time [ms], reverb sustain level [], reverb release time [ms]
dry/wet balance [D>W - D=W - DOf dat natte signaal enkel de reverb is of de reverb + inputsignaal weet ik niet, hangt mss af van de fabrikant.
high/low frequency damp gain [] vertelt hoe sterk het reverbsignaal is boven/onder de hieronder aangegeven frequentie. Als de gain gelijk is aan nul, krijg je de HPF/LPF cutoff frequency [Hz]. Maakt de reverb softer. Simuleert in zekere zin het materiaal dat geluiden absorbeert.
high/low frequency damp frequency [Hz]
diffusion [] ??? randomness van de late reflecties in amplitude en/of tijd ??? als deze parameter laag is, wordt het signaal discreet en lijkt het dus meer op een echo
reverb delay [ms] ???
density []
ER/reverb balance [E>R - E=R - E< R] is de verhouding tussen de early reflectionsen de reverb
ER level []
feedback level [] geeft aan hoeveel van het effectsignaal terug in de reverb gestuurd wordt
feedback high damp [] is waarschijnlijk weer op te splitsen in een gain parameter [] en een frequentie treshold parameter [Hz] waarboven het feedbacksignaal verzwakt tot gecut wordt.