PETZOLD BOOK BLOG

Charles Petzold on writing books, reading books, and exercising the internal UTM


Recent Entries
< PreviousBrowse the ArchivesNext >
Subscribe to the RSS Feed

A Simple Electronic-Music Sequencer for Silverlight 3

July 12, 2009
Roscoe, N.Y.

The audio support in the Silverlight 2 MediaElement is limited to WMA and MP3 files. I always thought the omission of WAV file support to be a bit peculiar. After all, at some point during playback, the WMA and MP3 files are decompressed to the pure audio samples that comprise WAV files, so why not accept WAV files directly?

It's not that I wanted to play actual WAV files in a Silverlight app. No, I had something a little more ambitious in mind. I thought it might be fun to generate audio data dynamically and feed it into a MemoryStream attached to a MediaElement through the SetSource method. This would make possible an interactive electronic music synthesizer right on a web page. But the omission of WAV file support from Silverlight 2 was a big problem: I'd have to generate the audio data, and then convert it to either WMA or MP3, which involves lots of messy code and license issues.

In Silverlight 3, however, we have a solution. You still can't attach a WAV file or stream directly to a MediaElement, but you can derive from MediaStreamSource and attach it to the MediaElement and funnel either a WAV file or WAV data into that.

I don't think I'd want to figure out how to derive from MediaStreamSource on my own. Fortunately there are a couple of pioneers who have posted solutions. Gilles Khouzam demonstrated how to play WAV files in a blog entry entitled Playing back Wave files in Silverlight and Pete Brown demonstrated real-time sound generation in Creating Sound using MediaStreamSource in Silverlight 3 Beta.

With those preliminaries done for me, I was able to slap together a primitive sequencer for Silverlight 3. (A sequencer is a electronic music instrument that plays a sequence of notes repetitively.) You can run it from here:


www.charlespetzold.com/silverlight/SimpleSequencer/SimpleSequencer.html

The button at the lower-left plays and pauses. Use the mouse to drag notes up and down the two staves, from two octaves below and two octaves above middle C. Press Shift and click a note to toggle between sharp, flat, and natural. Radio buttons give you a choice of simple waveforms. The top scrollbar controls the tempo; the bottom one controls the volume.

Very little testing has been done with this program. I'm not even sure the pitches are correct, and I'd be shocked if I got the sharps and flats logic right! The sounds I'm generating in this thing are just naked waveforms without any filtering or processing or envelopes. Obviously getting better-sounding sounds is high on my agenda of enhancements.

Although I am overjoyed I can now generate music in real time in Silverlight, there are some problems, in particular a very noticable latency issue. Internally, somebody (MediaElement?) wants a very comfortable buffer, so it won't start playing anything until it has that buffer, and if you change something dynamically, it won't be reflected in playback immediately. I suspect this problem will eliminate any type of interactive playing of music via the computer keyboard, but I'm curious to see how far I can get otherwise.

The source code is here. When you derive from MediaStreamSource and attach it to a MediaElement, it's going to want data at the rate of 176,400 bytes per second (for a 44,100 sample rate, 16-bits per sample, and stereo — half that for my mono sequencer), and one way to do this is to set up a pipeline that delivers this data on demand. My MediaStreamSource derivitive requests samples from a class called Attenuator, which controls the volume, which then requests samples from the Sequencer class, which controls the note-to-note sequencing, which then requests samples from the Oscillator class, which actually generates the waveform data. All timing in the program is based on those original requests.

As I get deeper into this stuff (and if people seem to be interested), I'll discuss some of the principles of digital electronic-music synthesis in upcoming blog entries.

This looks like great fun, and I'm only getting started.


Comments:

Nice project

I'm curious to know how the interactive playing of music via computer keyboard will work out... I see that latency and jitter are an issue with flash (As3.0).

At worst the only multiplatform winner might be java...

bdk, Mon, 13 Jul 2009 14:53:04 -0400 (EDT)

Charles, you can see from the URL that I have been working on a piano chord finder in Silverlight and am looking to get it playing the chords/scales - so your sample code looks interesting.

I haven't looked through your code yet, but I was wondering how easy it would be to get it working polyphonically? Obviously essential for playing chords...

Joan Miro (aka Gordon Mackie), Wed, 15 Jul 2009 07:40:06 -0400 (EDT)

Multiple Oscillator objects are needed plus a new Mixer component. I'll be posting some sample code shortly — Charles

My browser (IE 8 under 64-bit Vista) does not want to install Silverlight 3, so I can't play it.

Sheryl Canter, Sat, 18 Jul 2009 11:08:08 -0400 (EDT)

Never mind - mystery solved. It won't install because I need the developer's installer (because I was doing development).

Sheryl Canter, Sat, 18 Jul 2009 11:12:24 -0400 (EDT)

Is it possible to to write a silverlight application which allow playing virtual pairy , we have an windows based application which do the same but want online presentable solution.Can we also write virtual guitar. Is this some thing out of boundary for Silverlight.

— Jack, Thu, 23 Jul 2009 14:12:24 -0400 (EDT)

Pretty cool example. I gotta take a look at the source code.

One question about the red dot indicating the current note. With the latency of the MediaElement, how do you keep that dot accurate?

Tobyteel, Sun, 2 Aug 2009 16:20:07 -0400 (EDT)

I was waiting for someone to ask me that: Look at the use of the timedPositions collection in MainPage.xaml.cs. I record the time the note begins, then wait for that time from the MediaElement. — Charles

Nice work...as usual Charles. I was browsing through your site...you are a true renaissance man.

Do you think Silverlight would be capable of making an audio mixer? Something that has...say five channels of audio with volume control for each channel, then a "master" play/pause/stop function?

I am new to Silverlight and want to try to make one...but not sure if Silverlight could do it.

— Dav, Sat, 28 Nov 2009 20:20:14 -0500

Hi Charles

This is great. I have been working on a 100% SilverLight music teaching system myself and I can now get it to play tunes synched to notation (mainly Guitar tab but I am also offering standard music notation). I can also synch it to video of the teacher playing the tune. Have a look at http://strumdiddle.com/Lessons.html

The secret for me was to include the music sound samples in the XAP package and that way I don't have any wierd timing issues. It works pretty well.

I have always loved your books and I've been reading them since your first Programming Windows way back.

Keep up the good work. Silverlight is superb!

Ian Gray, Mon, 30 Nov 2009 13:26:50 -0500

Charles,

This is great stuff! I've been planning to do an online music composer app that allows users to create compositions and play them back on the browser. This will point me in the right direction.

Johan, Thu, 1 Apr 2010 20:26:00 -0400

very nice example - but missing flat "b" and sharp # notes ;)

And You Can not make accords.

Please go on..

— Harald, Mon, 12 Apr 2010 07:44:43 -0400

You apparently missed the part of the blog entry that says "Press Shift and click a note to toggle between sharp, flat, and natural." — Charles


Recent Entries
< PreviousBrowse the ArchivesNext >
Subscribe to the RSS Feed

(c) Copyright Charles Petzold
www.charlespetzold.com