PureData remarks 2

• As long as I don't find a better solution, I'll put the parameters for display and for adjusting both apartly in the display. If I just connect a number box and a slider together, stack overflow errors occur, sometimes slowing down the system.
• I connected my korg microkorg midicontroller to my laptop to use it in PureData. This works fine. I keep a written chart with all the control change numbers of the controls at hand. Because MIDI data ranges from 0 to 127, I adjusted the inputs of some effects I already had. From now on this will be the unified way for ranges of control inlets.
• Apparently when one reads into an array, an array must be added to the patch too. You can make it small by unchecking the graph on parent button, but it is still wondering around there.
• It's important that when you will be using a block several times in the same patch (something we must do), every parameter name that can be sent without a line is named with a $-sign. send, receive, throw, catch, delread, delwrite and several others must have a name like $1-name. When you use the block, you give a unique number as argument. This number is filled where $1 is placed. This works only when the dollarsign is placed in front of the name.
• The dollarsign has another meaning when used in messages. There exists no $0. The dollarsign is replaced by the n-th value of the inlet.
  

monotraditionaldelaybpm~ bad try

monotraditionaldelaybpm~

bad try - discontinued

This block delays an incoming signal and outputs the wet signal on one channels. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop, actually one per channel. Users can change all parameters with the visible controls, or they can vary them by using the three other inputs.
When varying the beats pet minute (BPM) parameter, one can choose what the division is of the time between two beats. Only the grey buttons will keep the delayed signal synchronized with the original signal.

version

v1.0: built 28/03/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the beats per minute parameter
Input 3: the delay time parameter
Input 4: the feedback parameter

Output 1: the delayed, wet monosignal

Controls:
top number box and horizontal slider: adjust the beats per minute (BPM) parameter. It specifies the tempo that synchronizes the delayed signal. It ranges from 20bpm to 250bpm.
middle number box and horizontal slider: adjust the delay time (DLY) parameter. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec tot 3000msec (3 seconds). This is not only the maximum value of the slider and number box, but also the length of the delay line used in the implementation. When these are adjusted, the DLY controls are adjusted accordingly.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands. When these are adjusted, the value of the BPM controls are not adjusted accordingly (read the implementation to see why).
switch button: is used to reset the feedback parameter to 0 and thus turning of the delayed signal. When a cross is visible, the effect is turned on. The previous value is remembered and displayed in the number box above the FBK slider as long as one doesn't touch the number box. When turning the effect back on, the slider and the number box jump to the last value remembered.
16 measure buttons: choose the fraction of time between two beats. It resembles the measure of a musical piece, but it is not. It's more like a measure between two beats, as if between two beats a total measure takes places. The grey buttons will stay synchronized between two beats, but also the other ones can stay synchronized over different beats:

implementation

The implementation is the same as the monotraditionaldelay~ block. Only the new details will be described here.

For smoothing the signal when parameters are changed, a line block is implemented for the value.
Adjusting the BPM slider is the same as adjusting the delay time DLY parameter. The factor is: DLY = 60000*BPM^-1. This is then multiplied by a factor that expresses the measure as a fraction of time between two beats. The mechanism used for this is a message that is banged the moment you press the button on the display. This updates the factor (that has a default value of 1). To make sure it updates directly, a bang block is inserted to the hot input of the multiplication block. Inside the message the measure fractions (DIV for division) are set rounded to 5 numbers after the comma.

TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline.

TO DO: When adjusting DLY, the other one is not updated in the display. In the next version this is taken care of. For this update it would require an inverse recalculation. This consumes CPU time, so the effect of it must be researched first.

The switch toggle switches the effect off by setting the feedback level FBK to 0. The value is remembered and restored when turning the effect back on. In the number display you still see the remembered value. How this system functions can be seen in fig. 1: switch implementation above.

problems

There are some problems, although everything works.

When the output of the bpm slider is connected with the input of the number box above it, an error: stack overflow occurs and slows down the computation. Strange, because exactly the same thing is done with the delay_time controls. The same error shows up when it's adjusted although not connected to each other, but without really slowing things down. The latter happens also with the delay controls. It stops when the r dly block above the controls are uncoupled, for both controls: DLY and BPM.

When the feedback_level controls are adjusted an error: inlet: no method for 'list' occurs if the switch is turned of and an error: stack overflow occurs when the switch is turned on, without really slowing things down.

To make it possible to adjust the DLY control displays when changing the BPM controls the r dly block must stay attached to the delay_time number box. As a result these stack overflow errors keep appearing...

stereotraditionaldelay~

This block delays an incoming signal and outputs the wet signal on two channels. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop, actually one per channel. Users can change all parameters with the visible controls, or they can vary them by using the two other inputs, except for the phase reverse and switch-off. The difference with the mono delay, except for the stereo part, is the variable delay time per channel and the choice for in-phase or reversed-phase delayed signals.

version

v1.0: built 28/03/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the delay time parameter
Input 3: the feedback parameter
Inputs 4 and 5: the left and right channel delay time offset

Output 1 and 2: the delayed, wet stereosignal

Controls:
top number box and horizontal slider: adjust the delay time (DLY) parameter. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec to 1500msec (1,5 seconds). This is not only the maximum value of the slider and number box, but also the length of the delay line used in the implementation.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands.
switch-off button: is used to reset all the parameters to 0 and thus turning of the delayed signal.
TO DO: It might be better to call the switch-off button a reset button, implying that after pressing it, the values will not be remembered.
phase-reverse buttons: toggle on and off to create slight difference due to cancelling out of parts of delayed signals.
TO CHECK: It might be possible that this function doesn't work properly because the difference with both switches off is really negligent.
L-offset and R-offset: adjust the delay time of each channel relative to the delay time DLY. It ranges from -200msec to 200msec.

implementation

The basic delay is implemented the same way as the mono traditional delay, but with two copies then. Both loops have the same FBK parameter in common, that is calculated once.
For smoothing the signal when parameters are changed, line blocks are implemented for the dlyLeft, dlyRight and fbk values. Also a one pole high pass filter is added at the outputs, with a very low cutoff frequency.
The dlyLeft and dlyRight values are calculated by adding the delay time parameter DLY to respectively the L-offset and R-offset parameter. Both values are being input in the left slot of the +, because that's the hot one: everytime a value at the left is changed, the block will execute an addition.
Also here a throw~ and catch~ block are left behind. If you want to add the dry signal to the wet one inside the block, it's done by connecting the unconnected blocks to the nearby slots.
The phase reverse logic works like this: when the button is toggled it transmits a 1 or a 0. The moses block sends a 0 to the left output and a 1 to the right, because x<1 is sent to the left, etc. The following messages contain the factor by which the signal must be multiplied. -1 for a phase reverse (as easily seen when you draw a signal on an x-axis with a phase circle next to it). The multiplication that follows (and begins with a 1) is banged by the message and the factor thus changes inside the *~ block.
TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline.

monotraditionaldelay~

This block delays an incoming signal and outputs the wet signal on one channel. The original signal is not throughput. It works like the traditional delays, this means with a feedback loop. Users can change the two parameters with the visible controls, or they can vary them by using the two other inputs.

version

v1.0: built 28/03/2007 with Pd v0.40-2 [pd] [pdf]

interface

Input 1: the mono signal you want to delay
Input 2: the delay time parameter
Input 3: the feedback parameter

Output 1: the delayed, wet mono signal

Controls:
top number box and horizontal slider: adjust the delay time (DLY) parameter. It specifies how long it takes in milliseconds before you hear the following repetition. It ranges from 0msec tot 3000msec (3 seconds). This is not only the maximum value of the slider and number box, but also the length of the delay line used in the implementation.
bottom number box and horizontal slider: adjust the feedback (FBK) level parameter. The signal is attenuated with this value divided by 1000. In other words, you can vary the level from 0 (no delay audible) to 0.999 (maximum amplitude of the delayed signals, keeps on repeating as if it never ends) with steps of thousands.
switch-off button: is used to reset all the parameters to 0 and thus turning of the delayed signal.
TO DO: It might be better to call the switch-off button a reset button, implying that after pressing it, the values will not be remembered.

implementation

The delay itself is pretty simple: the incoming signal is read by a delread~ object that writes the signal into a delay line called delay1 and with length 3000msec. Although there's no visible loop, this is how the loop is formed: after a delay time DLY the associated delread~ object reads what's in the delay line, outputs it and sends it back into the delwrite~ object. Before outputting the delayed signals they are attenuated every time by a factor FBK ranging between 0 and 0.999 (with 1 clipping is to easy), in steps of thousands. That's why the control parameter lies between 0.001 and 999.
A possibility is left in the implementation to also mix the wet output with the original, dry signal, by connecting the unconnected throw~ and catch~ objects.
TO DO: It's possible to also implement an HF DAMP function which simulates a retro analog delay effect. It's nothing more than adding a lowpass filter with fairly high cutoff frequency (or even adjustable) in the delayline.

beat detection in PureData

A block exists that tracks beats. I didn't yet read the details, but it's standard in Pd and is called bonk~. Obiwannabe wrote an article on it here. He also wrote other interesting stuff about music synthesis and all using Pd.
I also found a library that collects a bunch of feature extraction scripts written in C. I want to download it, but I don't know how I should compile it or at least make it work anyway. It's named aubio.

weekly update

This week we both read through the tutorials of PureData as indicated last week. Also the book of Puckette M. is not virgin anymore.
For the next week we'll implement some concrete effects. Concrete means that they really work with a given input, from input to output. If there's time we might also try an extraction algorithm already. But we have to keep it simple so as to achieve something concretely this week.
Things we talked about during our meeting:

• I tried already measuring the latency by just connecting the input of my external soundcard (EDIROL FA101) with the output through PureData. With a sample rate of 44,1kHz and quantizing at 24bit I reach a minimal latency of 37msec, these is too audible and 1msec less results in a big mess. This is not the case when using other profeesional audio software on my computer (adobe audition 2.0, Live 6.0.1) I didn't really search further, I ony tried it out once and it works at least. I know there exists a latency test patch for Pd on the internet. I'll search it the next time I do real input tests.
• Anyway, Sam says that an audible latency is OK as long as we have a realtime working effect. The latency of the program would be rather normal because of the complexity of the code.
  

monopanner~

monopanner~

discontinued

This block places a mono signal in a stereo panorama. You can choose the panning curve that is applied. An input is also provided to vary the pan parameter externally.
This block is discontinued but it's implementation is used in the series of channel blocks.

version

goginLeft.txt - goginRight.txt

linearLeft.txt - linearRight.txt

rootLeft.txt - rootRight.txt

sineLeft.txt - sineRight.txt

pancurves.m

v1.0: built 27/03/2007 with Pd v0.40-2 [pd]
v1.1: adjustment 28/03/2007 with Pd v0.40-2
the output of the horizontal slider is now connected to the number box above it. This way the value in the number box is updated also when you slide the slider.
v2.0: built 29/03/2007 with Pd v0.40-2 [pd]
extra input for reset
PAN parameter input rescaled for MIDI data: 0-127
separate controls and displays for PAN parameter
rescaling calculated once instead of twice

v2.1: adjustment 1/04/2007 with Pd v0.40-2 [pd][pdf]

Other connections made it able to leave some away, for the initialisation logic

The reset knob only resets the PAN to the center. The table is left untouched.

Initialisation sets the table too, to the root function, because it's a more normal pan control curve than the gogin curve that was the default before.

interface

Input 1: the mono signal you want to pan
Input 2: the pan parameter, that is also included in the front of the block. It's range goes from 0 to 127 in steps of 1, where 0 stands for LEFT and 127 stands for RIGHT. This results in a PAN parameter that changes with a step size of 2, zero is not reachable with this input.
TO DO: re-size the tables, so that the step size of the input as well as the slider is equal.
Input 3: reset the PAN parameter to CENTER. That is the value read on sample 128 of the curves

Output 1 & 2: the stereo output signal.

Controls:
number box and small slider PAN: This displays the current value of the PAN parameter on a scale of -127 for LEFT to 127 for RIGHT with 0 as CENTER (when controlling from the second inlet, the value 0 is not reachable. One can do it with the reset button. This display is included to view the changes while the effect is controlled externally, as you see on the top figure: the current PAN value is 127, apparently controlled from the outside, because the controls themselves are still centered.
reset: resets the pan parameter to 0 or CENTER
number box and horizontal slider: adjust the pan parameter
gogin, linear, root and sine: choose a panning curve. root is the default value after loading the patch.

implementation

For the pan control curves a table is used. Per curve type there are two tables: one for the left and one for the right channel. The vectors are calculated in matlab with the following m-file:

xMin = -128;
xMax = 127;
x = [0:1/(xMax-xMin):1];

% gogin
arg = x*pi-pi/2;
goginLeft = sqrt(2)/2*(cos(arg)-sin(arg));
goginRight = sqrt(2)/2*(cos(arg)+sin(arg));
figure;plot(goginLeft,'b');hold on;plot(goginRight,'r');

% square root
rootLeft = sqrt(1-x);
rootRight = sqrt(x);
figure;plot(rootLeft,'b');hold on;plot(rootRight,'r');

% sine
sineLeft = cos(x*pi/2);
sineRight = sin(x*pi/2);
figure;plot(sineLeft,'b');hold on;plot(sineRight,'r');

% linear
linearLeft = 1-x;
linearRight = x;
figure;plot(linearLeft,'b');hold on;plot(linearRight,'r');

As you might have already remarked a choice is made here about how many values the control curve has. This can be changed afterwards in this m-file. xMin en xMax are defined here the same way as they are defined in the block itself: ranging from -128 to 127. A larger range would result in a smoother transition when panning, but with these values there's already no audible step.
The plotted figures can be seen in the section interface.
After calculating the values, this vector is copy pasted into a .txt-file. This space separated list is directly readable by PureData through the command read placed inside a message box. The values are thus read and written away in an array called left Channel and right Channel.

The inlet accepts values between 0 and 127, the controls range from -127 to +127 and the control curves are written in tables with a width that ranges from 0 to 256. So the range of the inlet must be multiplied with two and a one has to be added. The controls just have to be shifted over 128 samples. Then the tabread block looks up the corresponding value, depending on the chosen control function. This value is used to attenuate the incoming audio signal.

The center values per type are:

• gogin: L=0.693; R=0.720
• linear: L=0.494; R=0.505 (-1 is a better center)
• root: L=0.702; R=0.711 (-1 is a better center)
• sine: L=0.700; R=0.713 (-1 is a better center)
  

The table that draws the array as a graph in the patch itself as an y-index that ranges from -1 to 1. The negative values are necessary for representing all the values of the gogin function.
TO DO: maybe I can change the range of the control curves to an odd number of indices, so as to provide a real centered center, with equal loudness for both the left and right channel.

PureData remarks

• Apparently Pd organizes the inlets and outlets of a subpatch according to their horizontal position. So no matter if they're placed at the top or at the bottom of the subpatch, you will find the leftmost inlet also as the left inlet of the blok in the parent patch.
• The delete knob of the computer does NOT work...
• I had problems moving a canvas around in a subpatch, but apparently this is done with the canvas properties box (right-click on an empty spot in the window). The margins stand for the distance form the left and top margins of the window.
• Also the placement of the title of a box (knobs, etc.) is moveable by adjusting their x_off and y_off parameters.
• Also something strange happens when I insert my monopanner~ block in the main file (just an input and an output). The array drawn in the subpatch appear at the far bottom of my parent patch main.pd... Any idea why?
  

And now for the monopanner~ object I implemented today. You can find the documentation in the folder monopanner~ or in the post with the same name. The latest document I keep as a google document on my google account.

weekly update and information resources to PureData

Short update today. I showed some basic features of PureData I discovered last two days. Conclusion: try to work with PureData as fast as possible (as in next week).
That's what we are going to do. Firstly by running through all the tutorial patches.
I also discouvered the accompanying book (which might turn into a reference work) written by the author of PureData. It gives mathematical background to all the exercices and more: The techniques and theory of electronic music. I printed the book and now I'm looking in it and reading interesting parts.
Another essential work, but I'm putting it here more for private reasons than for this thesis is the book of Benson: Music, a mathematical offering.

PureData session on beat controlled delay (pt1)

Today I start already a practical example as an extra means to try to learn PureData. I'm still very pleased by the program. I can run it from my cheap memorystick and it's so small I can even save all the soundfiles and pd files with it.
So I began with an abstraction for the output like the one given in the tutorial (only changing the control to a slider).
Then I added the delay loop. It has the control parameters DELAY TIME (DLY) and FEEDBACK LEVEL (FBK) which are now controlled by a slider. This slider can receive a number as input, so we can attach the result of the extraction algorithm or mapping to the slider itself without having to change anything else.
I now want to implement a multitap delay (where you can specify the number of delayed samples you hear) but using the delay with feedback loop (not with the easier but totally other way).

First I tried to calculate when the level of the nth delayed signal would be smaller than the fraction of the n-1th version. But after a while I realized that the only easy way to do this was by stopping the input of the delay loop after n * DLY.
But also this doesn't work because if you stop input for the loop, also the ongoing delays are not written anymore and only one delay tap rests.
Therefore I tried starting from a multitapdelay to join the two types of delay. But I don't find a way to implement it without having a number of delay reads equal to the max number you want to have. So I don't find a way to do it in a loop...

Then I also want to try to attenuate every feedback copy while it's playing, because now it is played with a constant attenuated level. This can be done sith an envelope maker in the loop or the same effect might be reached by giving an envelope to the overall delayed signal. wow I'm going too fast. This is a feature that migh be useful when using really long delays, otherwise it's really not needed. But in this case it might give some very surprising and creative results (eg by not attenuating but making the feedbacked signal louder as it is played). It would require a lot of control signalling at first sight, so I put it on a long trail. Maybe one evening, who knows...

When I draw it on a paper, it seems that it is equivalent to, or that the only possible thing is to apply a control curve to the feedback level parameter. Let's try that sometime!

I also implemented a panorama block with which you can choose different connections between the left and right channels. One problem I encounter here is that when I try to mix both signals in both outputs, it doesn't give the expected result. Any ideas why? See the image.

Installing and learning PureData

For future reference:
Installing PureData in Windows XP is as simple as downloading a zipfile and saving the extracted files somewhere in a folder where you like. You find the program itself in the subfolder /bin/pd.exe .
From there you can run it, or you can create a shortcut to somewhere else.
I also checked if we could indeed add our own blocks in C and that's OK, it's even possible in C++ or FORTRAN (whatever).
Now I'm running over a bunch of examples to learn working with the platform.

• a question that rises now is that for example a delay (even very pratical and complex ones) can be made in PureData without 'descending' to level of matlab language. Everything can be done at the higher level, they even got analysis blocks (but their complexity and functionality I must first research), so why bother with the matlab level?
  This is a question for tomorrows weekly meeting.
  

matlab session on the signal routing for testing my delay and other effects

I worked on the file effectSystem.m and all it needs still is the output part: playing and plotting the results of the effect. And also the part where the different frames are glued together again to form a soundfile and the mixer part.
After that I can start implementing and testing effect files I already have, or new ones.
I did not yet pay attention in effectSystem.m to the feature extraction part.

EFFECTS

heb nog paar linkjes gevonden met implementaties voor effecten
moesten we er toch nog willen bijnemen:

REVERB

http://cnx.org/content/m14222/latest/

DISTORTION

http://cnx.org/content/m14221/latest/

http://www.cs.stir.ac.uk/~lss/lsspapers.html
http://www.cs.stir.ac.uk/~lss/recentpapers/IEEETransNNSept2004.pdf

using pitch detection
http://ismir2005.ismir.net/proceedings/1008.pdf

information:
http://www.cs.uvic.ca/~gtzan/work/publications.html

Onset detection

veel links, veel lectuur, veel informatie beetje verwarrend...

http://www.music-ir.org/evaluation/mirex-results/articles/all/short/west.pdf
http://www.music-ir.org/evaluation/mirex-results/articles/onset/collins.pdf
http://www.speech.kth.se/prod/publications/files/875.pdf
http://ieeexplore.ieee.org/iel5/10376/33958/01621213.pdf?tp=&isnumber=33958&arnumber=1621213

Er bestaat zelfs zoiets als de onset detection Contest
Hadden we dit geweten....

http://www.lam.jussieu.fr/src/Membres/Leveau/SOL/SOL.htm
http://www.lam.jussieu.fr/src/Membres/Leveau/ressources/Leveau_ISMIR04.pdf

nog iet gevonden over Piano note detection
te vinden op http://cnx.org/content/m14196/latest/
dit zijn een aantal m-files, waarvan de onset detection
eigenlijk vrij goed algemeen werkt... momemteel
gaan we hiervan uit, indien achteraf blijkt dat dit niet voldoende is,
zullen we toch teurg op die papers moeten hervallen...

beat detection: the sequel

Als vervolg op vorige topics over beat detection/extraction, zie hier:
http://idafx.blogspot.com/2007/02/some-links-for-beat-extraction.html
http://idafx.blogspot.com/2007/02/vandaag-beat-detection-algorithm-enkele.html

Na overleg met Toon besloten da de subbandinformatie moet gewogen worden.
Bij het herbekijken van de code beseft dat ik de subbandverdeling met parameter
Bandlimits had aangepast aan de sample frequentie wat feitelijk niet de bedoeling was,
omdat daardoor de hogeren frequentiebanden een hoger gewicht krijgen. Het is de
bedoeling om de lage frequentiebanden het meest te laten doorwegen.
Bandlimits = [200 400 800 1600 3200 samplefreq] zoals in de oorspronkelijk code
stond dus.

test results ---> geprobeerd met die wegingsfactoren
---> wegingsfactoren ??---> bandbreedte subbanden aanpassen
---> oorspronkelijke hersteld --> juiste results voor loopkes

AC_100A_kit1.wav ---> 100
AC_75_kit2.wav 76 acc,bpm:
AC_90_kit1.wav 90
BB_135_Lp10.wav 135
BB_95_K03_Lp03.wav 95
FHP_130_DRMLP_26.wav 130
FHP_130_DRMLP_48.wav 130
Jw_65_kit2.wav 65
Jw_70_full1.wav 70
Jw_80_full2.wav 80
Jw_95_perc3.wav 95

Sh_120 Rhodes Loop 7C.wav 120
Sh_120 Synth Loop 3C.wav acc,bpm 1,118 .5,120.5 .3 119.6 .2,119.6 .1,119.6
Sh_125 C Live Bass 2.wav 125
Sh_128 Drum Loop 1.wav 128

Toms_diner.wav 86, kan ook kloppen want heb ongeveer schatting gedaan op basis aantal tellen in die 4 seconden ---> 8/9 tellen

Conclusie: juiste resultaten, algoritme detecteert ook wanneer er geen beat aanwezig is
het gaat hier dus om RITME extractie.

Op zoek naar niet iteratieve methodes !!?
interessante links:
-->selfsimilarity matrix: http://www.fxpal.com/?p=similaritymatrixFAQ
-->http://www.cs.cmu.edu/~lebanon/IR-lab.htm


heel interessante site !!
http://mue.music.miami.edu/thesis/KurtJacobson/KJacobson_msmue_final.pdf
Hier staat op het einde van de pdf een heel uitgebreide matlab code voor het berekenen
van allerlei features... mss moeten we daar eens in gaan snuffelen...

Nog nen site met allerlei tools uitgewerkt in matlab code:
http://labrosa.ee.columbia.edu/matlab/

Brainstorm session on our goal

After the presentation of last friday we decided to choose a few effects and a few features and to implement these in real-time. The idea is to write working code with matlab at first and we can use simulink for the inputs/outputs and signal routing. Then we'll translate it to c code, using PureData probably for the routing part.
For feature extraction we would surely like to try onset detection and the other two we already worked on (rhythm/beat and pitch extraction).
As for the effects: delay and vibrato for sure, and then maybe a wah effect, panning, (,...)
The basic ideas we want to try with these effects and features are:

• the delay time parameter of the delay effect controlled by the tempo of the song (auto-adaptive as well as adaptive (master channel or drums channel, with a mic or with insert cables))
• the rate and depth of the vibrato will be faster with high pitched notes and/or loudness
• the wah effect and the panner can be controlled with onset and beat detection, but this already works this way
• when a singer sings a high and loud note, the delay effect can be more distinct and with a nice long delay (to make it stronger. this is the same as the sound engineers do by hand now)
• ...

There might be added some other idea later on, but this are the starting blocks.
So the beat extraction will extract the tempo of the music andwe will use this to sync the other effects. This is one example of cross reference for our list of effects and features.

delay in simulink with adjustable parameters and weekly update

Problems:

• How do I make in simulink a vector with N zeros and thereafter a one, while the N is supplied by an input? I didn't find blocks for creating matrices, so I wanted to do it with a matlab code snippet. But this gave a lot of strange problems. delayVector = [zeros(N,1);1]; This was my function, but the debugger said that this N gives problems and that the matrix must be constant or non-empty. Has it got something to do with iterations? I don't know what to do with this, so temporally I will use a fixed matrix.
  

  Non-constant expression or empty matrix.
  This expression must be constant because its value determines the size or class of some expression.
  

  This is still a problem after applying toons proposal (see comment). The identity matrix seems not to output a matrix. Here you see the error code simulink gives us:
  

  Element 1 of port 2 in 'effectSystem/Delay1/Selector' has a value of 257. This value is not within the range of permissible values (1 through 1) that allows selection of a valid input data element.
  

  
  I tried narrowing down all possible causes so far that I ended with another little document:
  And even this won't work. So I ain't got no clue for what's happening.
  

• Now I got another problem with our FIR filter: there would be a singularity problem.
  

  Trouble solving algebraic loop containing 'delaySubSystem/Subsystem/ -N z' at time 0. Stopping simulation. There may be a singularity in the solution. If the model is correct, try reducing the step size (either by reducing the fixed step size or by tightening the error tolerances)
  

  I don't understand what they mean, and even so I can't find a way to adjust the step size like they suggest.
  
  

weekly update

First we talked about the things we did and didn't understand during the last days.
And of course our discovery of extensive literature on the topic and what would be our place now. Is it changed or not?
For the presentation on friday:
Tell something general, about the effects, about the feature extractions, about the proposed mapping method of Verfaille. And ending with our future vision.
As we think now there are two ways to continue from here:

1. Or we try the general framework idea and after that we try all sorts of combinations
2. Or we try some specific new effects out, but then we don't have to generalize

Tomorrow we schedule the presentation making.

important papers

• A-DAFX adaptive digital effects
• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations

These two papers show that there's already been some research on our topic.Ihaven't got much time to conclude them now, but by showing a few images fromthesecond paper, you already see how they made a general framework like we werethinking of.




I'll summarize the papers by summing up the things I marked:

• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p2:
  In the class of loudness effects (perceptive classification) the first adaptive effects already exist: compressor and expander, limiter and noise gate. They're auto-adaptive effects.
• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p4:
  Chorus and flanging are specific cases of phase modifications known as phase shifting or phasing.
• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p5:
  Here they advise lowpass filtering the pitch extraction model (based on autocorrelation). I marked this because it could be useful for my pitch detection algorithm, they wrote this to say that it's might be enough to use not so good algorithms depending on what you're going to use the feature for.
• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p5:
  Here they introduce the three layer mapping strategy using a perceptive layer. There are sound feature extraction, sound feature combination and signal conditioning.
  The sound feature combination using warping (after normalizing and before combination). This can be used for different purposes: a truncation of the feature in order to select an interesting part, a low-pass filtering, a scale change (from linear to exponential or logarithmic).
  What is meant by this?
  

  Fourth, and finally, a warping function can also be applied to the feature combination output in order to symetrically provide modifications of features before and after combination.
  

  
  Control signal conditioning is usually used to protect hardware from an input signal.
  Long term features can be used as controls of the mapping (fi(n)), while short term features are useful as inputs.
• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p7:
  Real-time implementations are performed in Max/MSP and non-real time in Matlab.
  I've seen this programming environment Max/MSP being named a couple of times in papers about our subject. It looks a bit like simulink for musical purposes.
  
• Adaptive digital audio effects (A-DAFX) : a new class of sound transformations p3:
  A beautiful classification scheme of audio extraction features is given:
  
  

>> still two other papers to summarize!

simulink and some reading

I worked in simulink and things I encountered to remember are:

• delay blocks are calculated this way:
  I put the delay in frames. 1 frame has a default value of 256 samples. Because the sampling period (Ts = 1/Fs) equals 2,27E-5, one frame takes 5,8 milliseconds to end. So if the delay value is 10 frames, we have a delay time of 58 msec.
  
• so the general formula could be stated as: delay time = Nsamples * Nframes / Fs with Nsamples the number of samples per frame and Nframes the number of frames.
  Normal delay times are:
• ROLAND SH32
• stereo delay: 0-500 ms
• long delay: 0-1000 ms
• stereo and long delay sync type: (Thirty-second note), (Sixteenth-note triplet), (Dotted thirty-second note), (Sixty-fourth-note triplet), (Sixty-fourth note), (Thirty-second-note triplet), (Eighth note), (Quarter-note triplet), (Dotted eighth note), (Quarter note), (Sixteenth note), (Eighth-note triplet), (Dotted sixteenth note)
  13 possibilities!
• feedback : -98% - +98% (no mapping given)
• YAMAHA MO8
• 0.1 - 1638.3 (with value range: 1-16383)
• feedback level: -63 - +63 (0-127)
  they also use a feedback time, which could mean
• they use delay time parameter for the first delay and the feedback time for a new delay in the feedback loop, so for the rest of the delays. If so then it's better to rename the delay time parameter to initial delay time.
• Another possibility is that it is used to specify a kind of gate time, like the number of copies you allow.
  
• T.C.ELECTRONIC D2
• feedback time in stereo mode is maximally 5 sec.
  
• KORG MICROKORG
• Here they don't really give a time but the values: 0-127
• For the syncing they give as parameters: 1.32 - 1.1 (see the diagram) 15 possibilities!
• feedback (delay depth): 0-127
• So it seems that for our purpose, our delay must have a range which is the same as the range our bpm algorithm will work with. And there might have to be an ability to choose the sync type too, with a knob or something (=gesture control)
• I designed two delay implementations in simulink. The only one I'm going to speak of is the one with the feedback line. The other one has a more flexibel, but too complex way of assigning each delay or feedback level a gain value.
  For the feedback looped delay, I have set a feedback level of 50%. It thus has identically the same effect as the other implementation.
  
  To see what a negative percentage for the feedback level could mean, I tried one out and also calculated the gain after every round through the loop: it gives the same values as for the positive counterpart, but alternating negative/positive. Is it true that this is just a phase reverse after every loop? And is that the reason why I don't hear a difference at first sight? And why do professional keyboards give this choice than? Or is there another meaning connected to this negative percentage? I just tried adding two delays: one with a positive feedback level of .5 or 50% and one with a negative one of -.5 or -50%. This gives a more beautiful sound, and on the image you can see that some delaypeaks filter each other out due to a the sum of opposite magnitudes. (in my thesis folder on ESAT, you can find an image an the model too). This is an image where the original signal is not shown!
  
• Now I am trying to include a HF damp filter in the loop (because it reappears in many synths). For info on filters for audio applications, this website seems interesting: http://ccrma.stanford.edu/~jos/filters/.
  The ranges are:
• YAMAHA MO8: 0.1 - 1.0 kHz (1-10)
• ROLAND SH32: 200Hz - 8kHz or bypass
  
I read that this HF DAMP function tries to simulate analog delays. Because trying to simulate other effects will focus too much on effect design, and it's something extra that can be added later, I stop implementing it any further than this. So I'll use a lowpass filter modeled with a chebyshev II filter for which Fstop can be controlled with gestures (like a knob or slider). The A-stop could also be controlled, and many other things typical to a filter can be, but no effect device that I know of provides more control than the Fstop.

• Oh yes, sometimes you can choose how many times a sound is repeated, this is possible with a multitap delay. eg the T.C.ELECTRONIC D2 has a max delay time of 5000msec stereo or a total of 10000msec mono. When you want 5 taps you have a max delay of 2000msec on every tap, when you want 10 taps, you still have 1000msec available. I don't know if the implementation differs rather than the parameter calculation. Therefore I'll just remember the possibility but I won't implement it now.
  

-6/3/2007-

• Another thing with simulink: a single line represents a scalar value per iteration. A double line represents a vector. In this case there are double lines because of the fact that the audio input is given in frames. There is a way to see how the signals look on every line: Format -> Port/Signal Displays -> Signal Dimensions
• The simplest way to use and process a stereo signal is to split the two channels in two mono channels right after the input block and to merge them again right before you enter the output block. This is done with the following commands:
• select columns (stereo > mono)
  Simulink -> Signal Routing -> Selector
• merge columns (mono > stereo)
  Simulink -> Math Operations -> Matrix Concatenation

some more reading on adaptive effects

Today is a reading day, because I found a lot of interesting articles about our subject.

1. First I read the article on 'an interdisciplinary appproach to audio effect classification'. The best things to remember: they explained some vocabulary (effect, sound/audio effect, sound tranformation) that make it somewhat easier to classify them in a first way. Then they state that the most general way to classify audio effects is by their perception, what is heard by the listener. They also made layered model of levels on which you can speak about effects: (low-to-high level) digital implementation technique, processing domain, applied processing, control type, perceptual attribute, semantic descriptors. What we can remember here is not how to exactly classify effects, but more a clear view on them.
2. Then I discovered the thesis of our friend V.Verfaille: Effets audionumeriques adaptatifs: theorie, mise en oeuvre et usage en creation musicale numerique. Over 300 pages about our subject. I won't read that now, because it's too much. But we must try to read as much of it as possible!

Also I wanted to try simulink again. (it's a long time ago). Look at the last weekly meeting post to see why I want to do this. I'll try some effects or so out. Probably enough problems to solve before I succeed in that.

• signal processing blockset contains wave in and out blocks. You must be in the right working directory for the audio files to be found. When starting simulink the working directory is changed!
• I also have a problem when trying to link more than one signal. I think I'm just using the wrong blocks. (mux or merge) Idea of sam is to use an adder block and maybe scaling blocks per signal. I'll try this out.