PETZOLD BOOK BLOG

Charles Petzold on writing books, reading books, and exercising the internal UTM


Recent Entries
< PreviousBrowse the ArchivesNext >
Subscribe to the RSS Feed

A Simple Silverlight 3 Synthesizer with Keyboard (of Sorts)

July 24, 2009
Roscoe, N.Y.

Many, many years ago when multimedia was new to Windows, I wrote a bunch of articles for PC Magazine about sound and music programming. Many of those sample programs ended up in Chapter 22 of Programming Windows, 5th edition, including KBMidi ("keyboard MIDI"), a program that let you play the sound board's MIDI synthesizer through your PC's keyboard.

KBMidi featured a diagram of your PC's keyboard labeled with the notes of the octave, which are highlighted when the key is pressed. Here's me playing a C-minor chord:

I decided to try something similar for Silverlight 3 but with a completely reconceived screen image. The SimpleKeyboardSynthesizer program displays a music keyboard labeled with the letters and numbers of your PC keyboard, which you can run from here:


SimpleKeyboardSynthesizer.html

Keys are colored gray when they're pressed. If the Shift key is down when you press a key, the pitch is raised an octave; the Ctrl key drops an octave. The ScrollBar is a volume control.

SimpleKeyboardSynthesizer has several problems and issues, some of which are my fault (because I've only recently started messing around with Silverlight's real-time sound capabilities and obviously I haven't gotten very far), and some of which are intrinsic to Silverlight:

Here's the source code. Compared with my first blog entry on Silverlight Sound Synthesis, you'll notice that the Petzold.SoundSynthesis library now contains a Mixer module that combines outputs from multiple oscillators. Here's the Mixer class in its entirety:

public class Mixer : ISampleProvider
{
    public IList<ISampleProvider> Inputs { get; protected set; }

    public Mixer()
    {
        Inputs = new List<ISampleProvider>();
    }

    public short GetNextSample()
    {
        int sample = 0;

        if (Inputs.Count > 0)
        {
            int multiplier = 65536 / Inputs.Count;

            foreach (ISampleProvider input in Inputs)
                sample += multiplier * input.GetNextSample();
        }

        return (short)(sample >> 16);
    }
}

The Mixer class implements the ISampleProvider interface, which means that it includes a method named GetNextSample that returns a short. This method is called asynchronously (that is, not in the UI thread) 44,100 times per second. The class also includes a public property named Inputs of type IList<ISampleProvider>. In this program, these inputs are Oscillator objects.

It is tempting to write code for a music synthesizer using mostly floating point and then convert to 16-bit samples at the very tail of the process. However, if you consider that 16-bit samples need to be produced at the rate of 44,100 per second (for so-called CD Quality), it's best to minimize the use of floating point throughout the sample pipeline. (I still have floating point in the generation of the sine curve, but I'll be switching to a lookup table at some point.)

Because samples are only 16 bits wide, you can use use a 32-bit int to represent a 16-bit whole number with a 16-bit fractional part. This is basically what the Mixer module is doing: It's effectively multiplying each incoming 16-bit sample by 1/N, where N is the number of inputs, and then summing the results. But 1/N is really the integer 65536/N.

It seems reasonable that a program of this sort could dynamically create Oscillator objects and add these inputs to the Mixer as the user is playing the music. But this won't work right: Suppose the user presses a key for C. The program responds by creating a new Oscillator object, setting the correct frequency, and adding it to the Inputs collection of the Mixer. Then, without releasing the C, the user presses a key for an E. Now the program responds by creating a second Oscillator object, setting the correct frequency, and adding that the Inputs collection. But now there are two inputs to the Mixer, and it's multiplying the samples from each input by 1/2, and the sound from the first Oscillator suddenly drops in volume!

The correct approach is to create N Oscillator objects at the outset, set all the frequencies to zero, and add all N to the Inputs collection of the Mixer. Then as the user is playing the keyboard, just pick an available oscillator when needed. (That's also why I didn't implement any thread synchronization in the access of the List object, but I really should have.) In SimpleKeyboardSynthesizer, N is hard-coded as 8.


Comments:

Hey, I think I figured out how to get rid of that latency. Set the AudioBufferLength = 15 on your PcmMediaStreamSource (inherited from MediaStreamSource). This is new I guess in Silverlight 3 official release.

It almost makes the lag acceptable.

Toby Teel, Sun, 2 Aug 2009 17:48:05 -0400 (EDT)

Gosh, how did I miss that? Thanks a lot! — Charles

this is great! Thanks Charles.. and toby ;)

— Tony Merante, Tue, 20 Oct 2009 16:59:54 -0400


Recent Entries
< PreviousBrowse the ArchivesNext >
Subscribe to the RSS Feed

(c) Copyright Charles Petzold
www.charlespetzold.com