AudioNyq | Previous Section | Next Section | Table of Contents | Index | Title Page

Interactive Nyquist

Nyquist is not intended for real-time performance, but it has some features that allow you to adjust parameters interactively. The basic idea is that there is an array of 1000 floating point values, called sliders, that can be accessed while synthesizing sounds in Nyquist. The slider unit generator returns a signal that copies the current value of a slider value. You can change the slider value while playing sounds using either Open Sound Control or the NyquistIDE.

Sounds are normally computed on demand. So the result returned by slider does not immediately compute any samples. Samples are only computed when something tries to use this signal. At that time, the slider value is read. Normally, if the slider is used to control a sound, you will hear changes in the sound pretty soon after the slider value changes. However, one thing that can interfere with this is that SOUND samples are computed in blocks of about 1000 samples. When the slider value is read, the same value is used to fill a block of 1000 samples, so even if the sample rate is 44,100 Hz, the effective slider sample rate is 44,100/1000, or 44.1 Hz. If you give the slider a very low sample rate, say 1000, then slider value changes will only be noticed by Nyquist approximately once per second. For this reason, you should normally use the audio sample rate (typically 44,100 Hz) for the rate of the snd-slider output SOUND. (Yes, this is terribly wasteful to represent each slider value with 1000 samples, but Nyquist was not designed for low-latency computation, and this is an expedient work-around.)

When you load sliders.lsp, which defines a number of slider functions, two important settings may be changed. First autonorm-off is called. The problem with auto-normalization is that it works by computing 1 million samples ahead of real time to determine a normalization factor. If Nyquist computes ahead, it will be unable to respond to control changes until the million samples (about 20 seconds) have been played. Secondly, snd-set-latency is used to set the audio latency to 0.02s (20 milliseconds). Normally, Nyquist uses a generous 0.3s latency which allows Nyquist to stop computing audio and run garbage collection without breaks in the audio output. At 20ms, interactivity is greatly enhanced because changes do not sit in audio buffers for 300ms, but you may notices some break-up in the audio, especially when garbage collection takes place. The latency can be changed to any value you like after you load sliders.lsp.

In addition to reading sliders as continually changing SOUNDs, you can get the slider value as a Lisp FLONUM (a floating point number) using get-slider-value. This might be useful if you are computing a sequence of many notes (or other sound events) and want to apply the current slider value to the whole note or sound event.

Other unit generators exist to instantiate behaviors and to stop sounds according to slider values. These will be described below.

Interactive Control with the NyquistIDE

To control sounds interactively using the NyquistIDE, you first create a control panel, then populate the panel with sliders and buttons. These will send values into the sliders array in the Nyquist process. To control a sound, you use built-in functions to retrieve real-time values from the sliders array as sound is being played. Further discussion and examples can be found in demos/slider-demos.sal.

Creating a Control Panel

A control panel is created with make-slider-panel, which takes a panel name and color as parameters. Control panels can only be created by executing code in Nyquist. There is no way to configure control panels directly using the NyquistIDE. Control panels can be deleted interactively using the close button on the panel or through code by calling close-slider-panel.

make-slider-panel(name, color) [SAL]
(make-slider-panel name color) [LISP]
Create a control panel in the IDE. The title of the control panel window is given by name, a STRING. The color of the panel is given by color, a FIXNUM from 0 through 12. The color 0 is gray. Other colors are implementation-dependent, but different numbers give distinguishable colors. You must load sliders.lsp to access this function.

close-slider-panel(name) [SAL]
(close-slider-panel name) [LISP]
Close a control panel in the NyquistIDE named by name, a STRING. Any embedded controls (sliders or buttons) are also destroyed. You must load sliders.lsp to access this function.

Creating Controls

You can create slider and button controls. A slider control adjusts a floating point value in the sliders array and accessible as a time-varying signal (a SOUND) or as a FLONUM. A button control sets a floating point value in the sliders array to zero (0.0) or one (1.0). The value changes when the left mouse button is pressed over the button control and changes back when the mouse button is released.

make-slider(name, init, low, high) [SAL]
(make-slider name init low high) [LISP]
Create a slider in the most recently created control panel. (Thus, you should populate a control panel with sliders and buttons before creating another control panel.) Sliders have a label specified by name, a STRING, an initial value specified by init, a FLONUM, a minimum value specified by low, a FLONUM, and a maximum value specified by high, a FLONUM. Sliders are added to the current panel in order from top to bottom. You must load sliders.lsp to access this function.

make-button(name [, normal]) [SAL]
(make-button name [normal]) [LISP]
Create a button in the most recently created control panel. Buttons have a label specified by name, a STRING, and a "normal value" specified by 0 or 1 (a FIXNUM). If 0 is specified, the value controlled by the button is 0 when the button is released and 1 when the button is pressed. If 1 is specified, the normal value is 1 changing to 0 when the button is pressed. Buttons are added to the control panel in order from top to bottom. You must load sliders.lsp to access this function.

Accessing Control Values

Each control created by make-slider or make-button is assigned a slider index from 10 to 999. Control changes are passed via hidden text input from the NyquistIDE to the nyquist process, where the values are converted to floats and stored in the slider array. You can then access these values with either slider, lpslider, or get-slider-value.

slider(number [, dur]) [SAL]
slider(name [, dur]) [SAL]
slider(panel, name [, dur]) [SAL]
(slider number [dur]) [LISP]
(slider name [dur]) [LISP]
(slider panel name [dur]) [LISP]
Create a SOUND that reads signal values from the slider array. In the first form, the first parameter is the index (a FIXNUM) of the value in the slider array. In the second form, the slider value will be controlled by the NyquistIDE control created by make-slider or make-button using the same name, a STRING. The control must be in the most recently created panel. In the third form, the panel named panel is searched for the control named name to determine the value. In all cases, the optional dur, a FLONUM, is used to determine the duration of the sound. This duration is scaled by the environment in the usual way. You must load sliders.lsp to access this function.

lpslider(number [, dur]) [SAL]
lpslider(name [, dur]) [SAL]
lpslider(panel, name [, dur]) [SAL]
(lpslider number [dur]) [LISP]
(lpslider name [dur]) [LISP]
(lpslider panel name [dur]) [LISP]
Create a SOUND based on the value of an interactive control. This function is exactly like slider, except the sound is low-pass filtered to avoid sudden jumps when the control value is adjusted. The low-pass filter cutoff is determined by *lpslider-cutoff*, which is initialized to 20Hz when slider.lsp is loaded. You must load sliders.lsp to access this function.

get-slider-value(number) [SAL]
get-slider-value(name) [SAL]
get-slider-value(panel, name) [SAL]
(get-slider-value number) [LISP]
(get-slider-value name) [LISP]
(get-slider-value panel name) [LISP]
Get a value (a FLONUM) stored in the slider array. The array is accessed directly if the parameter is number, a FIXNUM. The index can be determined by searching the most recently created control panel by name, a STRING, if only name is given. If both panel and name are given (STRINGS), the named panel is searched for the named control. The result is a FLONUM. You must load sliders.lsp to access this function.

snd-slider(index, t0, srate, duration) [SAL]
(snd-slider index t0 srate duration) [LISP]
Create a sound controlled by the slider named by index (an integer index into the array of sliders). The function returns a sound. Since Nyquist sounds are computed in blocks of samples, and each block is computed at once, each block will contain copies of the current slider value. Normally, you would call slider (see above) rather than this low-level function.

Starting and Stopping Sounds

All Nyquist sounds have a duration. Even the slider unit generator has a duration and terminates at the end of that duration. In most cases, you will instead want interactive functions to run until you interactively ask them to stop. The stop-on-zero function terminates when an input signal (typically a slider) goes to zero. This can be used to terminate a complex interatively controlled sound. For example, here is a tone that terminates when the Stop button is pressed:

exec make-button("Stop", 1)

function can-stop()
  play (hzosc(1000) * stop-on-zero(slider("Stop"))) ~ 100

The stretch factor of 100 will cause this tone to play for 100 seconds. However, if the button named "Stop" is pressed, it will change value from the "normal" value 1 to 0. When the signal goes to zero, stop-on-zero will terminate. The multiplication then immediately terminates (because anything multiplied by zero will be zero; termination is just an efficient way to return zeros from now on). Since the multiplication is the top-level sound being played, the play stops.

Another thing you might want to do with interactive control is start some sound. The trigger function computes an instance of a behavior each time an input SOUND goes from zero to greater-than-zero. This can be used, for example, to create a sequence of sound events interactively. For example:

exec make-button("Trigger", 0)

function trigger-me()
  play trigger(slider("Trigger", 100), pluck(c3))

Here, a button control changes value from 0 to 1 when the button is pressed. This value is retrieved by the slider function, which runs for 100 seconds. When the button and hence the slider goes from 0 to 1, the behavior, pluck(c3) is instantiated. Many instances can be triggered by the button.

stop-on-zero(s) [SAL]
(stop-on-zero s) [LISP]
Return a SOUND that is identical to s, a SOUND, except the returned sound terminates when s first goes to zero. When a sound terminates, it remains at zero. A SOUND multiplication terminates when either parameter terminates, so multiplying by stop-on-zero is a way to terminate a sound interactively. (See the example above.) You must load sliders.lsp to access this function.

trigger(s, beh) [SAL]
(trigger s beh) [LISP]
Returns a sound which is the sum of zero or more possibly overlapping instances of the behavior beh. One instance is created each time SOUND s makes a transition from less than or equal to zero to greater than zero. (If the first sample of s is greater than zero, an instance is created immediately.) The sample rate of s and all behaviors must be the same, and the behaviors must be (monophonic) SOUNDs. The sample rate of the result is also the sample rate of s. The stop time of the result is the maximum stop time of s and all sounds returned by instances of the behavior. This function is particularly designed to allow behaviors to be invoked in real time. See the trigger-me function definition shown above. An implementation note: There is no way to have trigger return a multichannel sound. An alternative implementation would be a built-in function to scan ahead in a sound to find the time of the next zero crossing. This could be combined with some LISP code similar to seq to sum up instances of the closure. However, this would force arbitrary look-ahead and therefore would not work with real-time inputs, which was the motivation for trigger in the first place.

snd-stoponzero(s) [SAL]
(snd-stoponzero s) [LISP]
This function is identical to stop-on-zero. You should use stop-on-zero instead.

snd-trigger(s, closure) [SAL]
(snd-trigger s closure) [LISP]
This is a low-level support function for trigger. The closure takes a starting time and returns a SOUND. See trigger above for more details. Use trigger as described above and do not call this function directly.

Using Open Sound Control

Open Sound Control (OSC) is a simple protocol for communicating music control parameters between software applications and across networks. For more information, see http://www.cnmat.berkeley.edu/OpenSoundControl/. The Nyquist implementation of Open Sound Control is simple: an array of floats can be set by OSC messages and read by Nyquist functions. That is about all there is to it.

The slider and get-slider-value functions, described above, can be used to access these values within Nyquist. Each of these functions can take a slider array index to specify which value to use. Since make-slider allocates slider indices starting at 10, it is recommended that you control sliders 0 through 9 via OSC. If you change a slider array value via OSC that is already controlled by a graphical slider in the NyquistIDE, the graphical slider will not be updated or synchronized to the OSC value. (And there is no current way to send OSC command to the NyquistIDE.)

Note: Open Sound Control must be enabled by calling osc-enable(t). If this fails under Windows, see the installation instructions in sys/win/README.txt regarding SystemRoot.

osc-enable(flag) [SAL]
(osc-enable flag) [LISP]Enable or disable Open Sound Control. (See Section Using Open Sound Control.) Enabling creates a socket and a service that listens for UDP packets on port 7770. Currently, only two messages are accepted by Nyquist. The first is of the form /slider with an integer index and a floating point value. These set internal slider values accessed by the slider and get-slider-value functions. The second is of the form /wii/orientation with two floating point values. This message is a special case to support the DarwiinRemoteOsc program which can relay data from a Nintendo WiiMote device to Nyquist via OSC. The two orientation values control sliders 0 and 1. Disabling terminates the service (polling for messages) and closes the socket. The previous state of enablement is returned, e.g. if OSC is enabled and flag is nil, OSC is disabled and T (true) is returned because OSC was enabled at the time of the call. This function only exists if Nyquist is compiled with the compiler flag OSC. Otherwise, the function exists but always returns the symbol DISABLED. Warning: there is the potential for network-based attacks using OSC. It is tempting to add the ability to evaluate XLISP expressions sent via OSC, but this would create unlimited and unprotected access to OSC clients. For now, it is unlikely that an attacker could do more than manipulate slider values.

Sending Open Sound Control Messages

A variety of programs support OSC. The only OSC message interpreted by Nyquist has an address of /slider, and two parameters: an integer slider number and a float value, nominally from 0.0 to 1.0.

Two small programs are included in the Nyquist distribution for sending OSC messages. (Both can be found in the same directory as the nyquist executable.) The first one, osc-test-client sends a sequence of messages that just cause slider 0 to ramp slowly up and down. If you run this on a command line, you can use "?" or "h" to get help information. There is an interactive mode that lets you send each OSC message by typing RETURN.

The ser-to-osc Program

The second program is ser-to-osc, a program that reads serial input (for example from a PIC-based microcontroller) and sends OSC messages. Run this command-line program from a shell (a terminal window under OS X or Linux; use the CMD program under Windows). You must name the serial input device on the command line, e.g. under OS X, you might run:

./ser-to-osc /dev/tty.usbserial-0000103D

(Note that the program name is preceded by “./". This tells the shell exactly where to find the executable program in case the current directory is not on the search path for executable programs.) Under Windows, you might run:

ser-to-osc com4

(Note that you do not type “./” in front of a windows program.)

To use ser-to-osc, you will have to find the serial device. On the Macintosh and Linux, try the following:

ls /dev/*usb*

This will list all serial devices with “usb” in their names. Probably, one will be a name similar to /dev/tty.usbserial-0000103D. The ser-to-osc program will echo data that it receives, so you should know if things are working correctly.

Under Windows, open Control Panel from the Start menu, and open the System control panel. Select the Hardware tab and click the Device Manager button. Look in the device list under Ports (COM & LPT). When you plug in your serial or USB device, you should see a new entry appear, e.g. COM4. This is the device name you need.

The format for the serial input is: any non-whitespace character(s), a slider number, a slider value, and a newline (control-j or ASCII 0x0A). These fields need to be separated by tabs or spaces. An optional carriage return (control-m or ASCII 0x0D) preceding the ASCII 0x0A is ignored. The slider number should be in decimal, and theh slider value is a decimal number from 0 to 255. This is scaled to the range 0.0 to 1.0 (so an input of 255 translates to 1.0).

There is a simple test program in demos/osc-test.lsp you can run to try out control with Open Sound Control. There are two examples in that file. One uses snd-slider to control the frequency of an oscillator. The other uses get-slider-value to control the pitch of grains in a granular synthesis process.


Previous Section | Next Section | Table of Contents | Index | Title Page