Harmony 4 Development Blog

Mission complete - Blog closed

2007-03-05T07:29:00.000-08:00

This Blog was used by TC-Helicon development and product specialists to communicate with the real-world during the long development of Harmony4 for Powercore and Pro Tools HD.

That development was completed on all counts on February 28th, 2007. For more information on Harmony4, visit:
www.tc-helicon.com/harmony4

Driving on the other side of the road

2006-11-23T02:50:00.000-08:00

Although we took a lot longer than expected to complete Harmony4 Powercore, we do have a few excuses. Unfortunately, at the moment, I’ve forgotten what they are. Moving right along…

In the business of writing software, it’s generally “reuse or perish.” Writing software takes a lot of time, and testing sometimes longer. If you’ve got software that’s been field-proven to work well, and it’s possible to reuse it in whole or in part, why reinvent the wheel?

Speaking of wheels, top automobile manufactures don’t generally build their own cd-players, nor do they make their own wheels and tires. They reuse off-the-shelf components. I’m sure they also reuse their own designs all the time. Perhaps the same steering wheel or shift knob can be found across an automaker’s entire product line.

Harmony4 is also made up of many components. We call them objects. For example, each harmony head you see in the display is actually a virtual object. They’re like the wheels on a car. We bolt them on, spin them around, and hope they don’t fall off.

Now we have Harmony4 working under Powercore. Our current task is to make it work under Protools. Happily, and as planned, we get to reuse a lot of our original work. It’s a little like switching the steering wheel from left to right, because Protools users drive on the other side of the road.

This philosophy isn’t new to TC-Helicon. TC Electronic has spent countless hours building up a wonderful virtual world of plug-and-play components, since the very first plug-in they shipped. We’ve leveraged their work a great deal here, and evolved it a little ourselves.

So, yes we took a long time for Powercore, but hopefully Protools will be done much quicker (not to mention whatever future plugins we embark on for both platforms).

Stephen Evans
Developer

Hear Harmony4!

2006-10-02T01:15:00.000-07:00

Have a listen to two songs on the TC-Helicon brand website that were created with the help of Harmony4. The only section in which an actual human vocal was used for harmony is in the bridge of "Together". The rest of the harmony and doubling vocals were created with the Powercore plugin.

Click here to get there!

This Plug-in is Hot

2006-09-27T01:14:00.000-07:00

We’ve enjoyed a very sunny summer this year. Mid July our air conditioner decided this was unfair and decided to make its displeasure know by going on strike. It didn’t all out quit, but to make sure it wasn’t ignored it started dripping water into our studio. Soon ceiling tiles were mush and trashcans filled with water. In the picture you can see our intrepid CEO taking matters into his own hands and bashing out one of the soggy tiles. Apologies for the poor picture quality, but I could only get my hands on a cell phone to take the picture, and once the urge to ‘CEO-go-smash-now’ got in his head there was no delaying him for a better photo-op. The air conditioner really didn’t appreciate being prodded and started leaking more water. Our only recourse was to shut down the whole system. I suspect this was the AC’s intent from the start. The office soon turned into a muggy jungle environment – which persisted for a few uncomfortable days. We measured the air temperature in the office at 32C (90F) one afternoon. Harmony4 plug-in development continued.

Eugene Pretorius
Developer

DSP 101

2006-09-13T02:45:00.000-07:00

Ever wonder what DSP code looks like? Well, the answer is more than likely “No”; but I’m going to pretend you said “Yes”. Harmony4 is rather DSP intensive. In fact, we require all the resources (i.e. memory and processing power) from an entire DSP (PowerCore or TDM) to produce harmony. The following is a snippet from one of the mixers in Harmony4. It takes two stereo signals and mixes it down into a single stereo signal. All the gains you see allow one to fool with the balance of the final mix (everything after a semi-colon is a comment – we need some hints as to what we did).

The example in the graphics (Click here for a larger version) is a rather simple example of signal manipulation, but it provides a brief glimpse under the hood of Harmony4. Maybe next time I’ll post an uncommented snippet and have you can let me know what you think it does…

Eugene Pretorius
Developer

After testing

2006-09-11T02:52:00.000-07:00

Developers at Helicon don’t always get a lot of time to experiment with products once they’re shipping. The necessities of good business mean we almost immediately find ourselves working on updates, new products, old products; whatever requires our attention. Sometimes, however, I say “screw it” to good business. This has been one of those times.

I recently spent several days “testing” Harmony4, which has by the way, already received the golden stamp. Testing is an interesting word. For example, what I’ve been doing could be compared to “test” driving a new car, or “testing” a new pair of skis. Both of these activities are very different than say, “testing” software for bugs, or “testing” the potency of rat poison. So, when Kevin (TC-Helicon CEO) asks me what I’ve been doing lately, I simply say “testing”.

Yesterday I went out and bought a “Silent Brass” system from Yamaha, which is basically a trumpet mute with a pickup. This weekend I plan to route it through Harmony4. Should be interesting!

This will be my last Harmony4 Powercore related post, but I’ll be appearing in a dramatic new series this fall called “PSI” (Pro-Tools Software Implementation).

Thanks for reading!

Stephen Evans
Developer

Alternative sources

2006-08-28T00:25:00.000-07:00

It’s been a busy several weeks finishing off Harmony4 for Powercore. We’re at the end of beta testing and things are looking great. One thing we discovered during this long testing phase is that Harmony4 can be very useful applied to all sorts of different audio sources – not just vocals. Because we had so much fun with this, we decided to spend just a little time making the necessary optimizations to our algorithms, for even better processing of all types of monophonic instruments. Of course, our first priority is still vocal harmonization, and nothing has been compromised here! I’m looking forward to hearing how customers find new and interesting ways to use our harmonies once this ships.
Click here for a larger version…

Stephen Evans
Developer

Q&A: Status August 2006

2006-08-16T06:24:00.000-07:00

First of all, thank you for the massive interest in the Harmony4 plug-in, and especially all the questions and thoughts you have send us.

Within these questions, comments and tangents we have received from the blog, we have extracted some popular questions about the plug-in. As Harmony4 is now in betatest, we will also be posting Q&A’s on www.tcsupport.tc

Does Harmony4 work with ProTools ?
There are two versions of Harmony4 – one for Powercore, the other for ProTools HD systems. If you are using a ProTools HD system you can run the Harmony4 TDM version. The TDM version will ship in the fall of 2006

If you are running ProTools without an HD system, there are wrappers available that can sometimes allow use of VST plug-ins. Harmony4 for Powercore supports VST and may work through such a wrapper.

Does Harmony4 work as an RTAS plug-in ?
Harmony4 for Powercore runs as a VST plug-in. There are wrappers available that can sometimes allow use of VST plug-ins in a RTAS environment. Harmony4 for Powercore supports VST and may work through such a wrapper.

Which Powercore unit(s) will Harmony4 run on ?
Harmony4 for Powercore will run on any Powercore device created. However, the trial version of the plug-in will not function on the original Powercore PCI card.

How much DSP processsing power does Harmony4 require ?
Harmony4 for Powercore uses one DSP. Therefore, up to four instances of the plug-in can be used at one time.

Harmony4 for ProTools HD (TDM) uses one DSP. Depending on the DSP card you have, the number of instances of the plug-in you can run will vary.

What is the quality difference between Harmony4, VoicePro and VoiceWorks ?
Harmony4 uses the same algorithms and feature set as the Harmony function in VoicePro. The harmony algorithms in Harmony4/VoicePro use newer versions of our pitch detection, shifting and humanization than do VoiceWorks, VoiceLive and previous TC-Helicon products.

Will Harmony4 operate at 96KHz sampling rate ?
Yes, Harmony4 for Powercore and ProTools HD (TDM) support 86 KHz sampling rates.

Can Harmony4 be used in real-time ?
The latency of the Harmony4 algorithms are similar to our hardware products which are used live. However, the latency of your audio I/O devices and host will add latency. If the latency they add is small, then real-time use would be similar to our hardware products.

Presets

2006-08-02T01:43:00.000-07:00

I spent a day helping Tom Lang out with presets. It gets a bit frustrating at this point in the project when it comes to presets. Sometimes you have 10 or 15 build (just as Tom had), then an updated version makes those presets incompatible. When the plug-in is finished, preset compatibility is always easier to handle. During development we have internal tools to convert presets. But at this point, it's better not to tie up Stephen or Eugene's time on running tools - rather than testing and fixing the last bugs. I'll stop complaining now. I built new presets using VoicePro harmony block presets as a reference. A lot of time went into creating these presets. Stephen and I talked about how cool it is to put other plug-ins in line, parallel, etc. with Harmony4 to create new sounds. Some of them you'd never want to hear again - others you'd want in a mix.
Kevin Alexander

Testing

2006-07-13T06:51:00.000-07:00

As part of my testing, I’ve put together a sample of Harmony 4’s output for you. It’s attached in the bottom of this post.

Basically, I was testing Harmony4 in what we call Notes 4 channel mode. This is a cool way of controlling the plug in that allows you to use a MIDI keyboard to create up to 4 unique harmony parts on 4 MIDI channels/tracks, each with their own pitch bend articulation, gender and pan position. I prefer using this way of using Harmony4 because when you assign a gender setting and pan position to say, voice 1, it stays at those settings regardless of how many notes are being played on the keyboard. In the other Notes mode, which is used for performing the harmony parts live on one MIDI channel, the note to gender/pan assignment can switch depending on how many notes you have held and which ones came in first. When recording, Notes 4CH (as it is known in our GUI) is the stuff!

In this example I arranged 4 harmony parts to sound roughly like a SATB vocal quartet. The voices have gender assignments ranging from deep bass to female and the voices are panned medium-wide. To show how a pitch bend on only one channel can be cool, I did a little bender wiggle on the word “… free” on the high voice for fun.

For Humanization features I used different vibratos and intensities, and a tad of all the pitch and time randomization features without going overboard. Actually the recording does sound a little overboard, but hey, it’s part of the testing process and I don’t have a record producer/A&R guy looking over my shoulder! You can see the settings I used in the graphics.

Lastly, I added a quiet guitar part to give it some rhythm. We’re racing to get the PowerCore Harmony4 beta version out to our testers. Stay tuned for more developments.

Listen to the Audio sample (mp3)

Tom Lang
Product Specialist

Q&A: Plug-in vs Hardware, DSP and Harmonies

2006-07-07T04:01:00.000-07:00

Hi, I’m Tom Lang, the Product Specialist at TC-Helicon. First of all, thanks for all of your questions and interest! Right now I’m testing the plugin versions delivered from our developers. It’s exciting to finally see (and hear) a version of our technology running in real time in a DAW program. Yeah!

I’ll take a moment to address some of the more common concerns.

Sound Quality Compared to VoiceWorks and VoiceLive
The pitch detection and harmony generation comes directly from our flagship vocal processor, VoicePro. Which helps answer the question of its necessarily heavy DSP consumption… VoicePro has extra humanization controls that help with making the virtual harmonies sound more real. Also, because the plug in doesn’t have to take an extra trip through ADA converters and an analog mixing console, it’s just that much cleaner.

DSP Consumption
Harmony4 Powercore will require the full resources of one of the DSP’s on your Powercore card. Most Powercore cards have 4 DSPs, leaving plenty of room for other effects.

How real do the harmonies sound?
If you’ve got four professional singers in your studio, use them. Otherwise, with very little effort, Harmony4 can produce realistic results. Stay tuned for some audio clips generated from the plugin.

That’s it for now, back to testing…

Tom Lang
Product Specialist

Quality Control - Part 2

2006-06-26T05:29:00.000-07:00

The trick to quality software development is knowing when to use each method as described in my last entry titled Quality Control Part 1.

The “excessive traffic light” method should be used in places where performance is not an issue. For example, when the user tries to store a preset, we want to communicate well with the operating system, making sure the preset was properly saved. If it wasn’t, we want to notify the user in a meaning manner. Accomplishing this in 10 milliseconds is not our goal!

The “black box method” is great during development, but is not much use after the software has been released to users (unless they are friendly enough to send the error reports back to the developers). During development, this method is sometimes the best way to understand complex interactions between separate yet coordinated operating system processes, because you can “observe” the system without really affecting too much. Though Heisenberg’s principal definitely comes into play (see today’s link). The more data you try to observe the worse it gets, and searching for meaning in a report with sometimes tens of thousands of entries can be very difficult.

One of my favorite development methods is the “driver training” method. Whenever I identify an especially complicated or error prone part of a system, I employ what are sometimes called “asserts” in programming lingo. An “assert” is like a resident driving instructor, who if observing something unexpected, will immediately yell at me during execution. The instructor stays around during development cycles, but once the system has been adequately trained, the instructor vanishes, unburdening the system for ultimate performance.

Today’s link: http://en.wikipedia.org/wiki/Uncertainty_principle

Stephen Evans
Developer

Quality Control - Part 1

2006-06-23T03:54:00.000-07:00

When implementing any large and complicated system, error management is crucial. Where do errors come from? For a plugin like Harmony4, here are just an example of possibilities:

User’s Computer: Runs out of memory, disk fills up or has an error, a conflicting or malice 3rd party software process, CPU runs out of gas, O/S error.

Software Bug: Error introduced during development or design.

User Error: User does something bad like deleting important files (you could argue that the O/S should prevent this). The user might try to save a preset with an invalid file name, or they may try to overwrite a read-only file.

There are many possibilities beyond these. The trick is knowing how to deal with each error that may arise, properly informing the user, and if required, offering a decipherable choice of action.

During development, we are imminently concerned with our own errors. Trapping errors properly at this stage can mean the difference between chasing bugs for a day, or a month. We generally use three methods which I’ll describe metaphorically. I’ll call these methods “excessive traffic lights”, “black box recorder”, and “driver training”.

Excessive traffic lights
If you wanted very safe streets, you’d throw up lots of traffic lights, stop and yield signs, and impose all sorts of traffic bylaws everywhere you thought an accident could happen. While this might work, you’ll likely infuriate people with extremely slow traffic. You may also create grid lock, and perhaps a violent coup would spell the end of your governing term.

Black box recorder
Another good way to determine what went wrong, is to install a black box recorder in all vehicles. After an accident, read the log of exactly what happened. Then, improve the intersection as required.

Driver training
Improve the quality of your drivers! At the beginning, force a driving instructor to sit in the car, asserting complaints and warnings whenever the new driver does something unexpected. Sure, the car may have to be pulled over, and you may have to drag the lousy driver back to the class room, but eventually they’ll learn, and fewer traffic accidents will hopefully be the result.

Part 2 of “Quality Control” will discuss how we decide when to use each of these three methods in development. (To be posted June 26th)

Stephen Evans
Developer

Q&A: Real-time usage

2006-06-21T07:56:00.000-07:00

- Question
Dear Harmony4 developers, I am currently planning to pre-order Harmony4, but I have one question that has to be clarified before: Will Harmony4 work in realtime, that is, can it be used in the same way as your hardware harmony processors (that is, I play midi chords, record a vocal audio track in Cubase, and Harmony4 generates additional voices using the Midi chords as input)? What is known about latencies? I work with an up-to-date Notebook (2 Ghz Pentium, 1 Gb Ram, Powercore Compact, Cubase SX3 under WIndowsXP).

Thank you for your support - I'm looking forward to the release of the software.

Best regards,
Thomas Goschke, Germany

- Answer
Hello Thomas,

Thank you for the question. In a word, yes, it will work in real-time.

The Harmony4 plug-in will create harmony voices just as our other hardware products do. It will support all of our regular harmony modes, including chord mode which you were referring to. Of course, the question of real-time processing and latency is, on the surface, as innocent looking as an iceberg. Latency is something we are always battling against (for our hardware and software products). The type of signal processing we do requires that we first analyze the input audio before we can actually produce any output. For example, one of the key requirements for creating harmony is knowing what the singer’s pitch is. There is no way to do this without waiting for a “small” amount of audio to show up and then crunching through it to figure out what the pitch is. Once this process has started we’re running in real-time, but we’re behind just a little due to that initial pause we needed to straighten our bearings.

Now with a DAW the concept of real-time is a bit more pliable than with our hardware products. If you have your audio (and MIDI) pre-recorded and use Harmony4 as an insert or aux effect, the DAW will be able to compensate for the processing latency automatically. So when you hit the play button everything should be nicely aligned. In fact, if you’re using it this way latency is hopefully something you don’t need to worry about. On the other hand, if you want to monitor everything in real-time (more or less treat your DAW rig as a hardware effect processor) you will most likely notice the latency. And here enters another, more variable, aspect of latency – your specific DAW setup. The DAW software, OS version, audio card, ASIO driver, etc. will all impact the latency. But, Thomas I’m sure you and the other audiophiles out there are all too familiar with this.

As far as specific numbers go, I’m still trying to optimize the DSP processing latency; but at the moment the Harmony4 plug-in is introducing about 30 ms of latency.

Eugene Pretorius
Developer

From Venus to Mars

2006-06-20T03:53:00.000-07:00

It's not how long I work, it's what I'm working on that determines if I'll be able to even see straight, let alone function at the end of the day. Lately I've been quite pleased with progress. I've dug myself out of several holes that were easily big enough to burry a man my size. Today unfortunately, I discovered a whole network of underground caverns to explore.

Cut to the chase.. I'm dealing with Windows XP and Mac OSX. One comes from Venus, the other from Mars. Then I've got all your favourite VST hosts to deal with. Unfortunately, they all do things differently when it comes to hosting plugins. I'm not going to lay any blame here; I've simply learned to accept the fact that this has happened. (For your info, the whole point of VST and having a nice SDK is so that all hosts behave the same, and all plugins behave the same, so they can all happily work together). Finally, I've got various flavours of PowerCore to deal with, and various versions of PowerCore drivers (pre 2.0 and post 2.0). Oh yeah, did I mention Intel Mac? To call having to develop for Intel Mac a "wrench in the system" would be like saying a comet hitting earth 65 million years ago and killing all the dinosaurs was a “wrench in the system”. By the way, I just read that at the Permo-Triassic boundary (some 250 million years ago), 90% of all species died. Even insects took a huge hit! (This info comes from a book called The Ancestor’s Tale by Richard Dawkins).

Ok, really cutting to the chase this time… It’s all about MIDI, and accuracy of triggering harmony voices from your sequences. Sounds easy, and normally would be. The above paragraph is my only defence.

Favourite unrelated link today: http://www.tolweb.org/tree

Stephen Evans
Developer

Three levels of interaction

2006-06-19T05:20:00.000-07:00

We were reminded today of how some time consuming up-front work on this project is saving us in the end.

In any plug-in or effects processor you could say that there are three levels of interaction for the product to work. The person using the product interacts with the plug-in. This is often called the GUI, or Graphic User Interface. The GUI is largely supported by the operating system. Then, the GUI interacts with a plug-in’s proprietary UI (User Interface). Finally, the UI layer must communicate with the DSP layer which actually performs the audio processing. When there is a bug or a feature change we need to make sure all layers of the plug-in software are updated and in sync. In many plug-ins this updating is easy, as there are only a few parameters to manage. In Harmony4 there are hundreds of parameters to manage.

In order to deal with the challenge of integrating these three layers, we use advanced in-house tools, originally developed for our flagship vocal processor, VoicePro. The tools can be used from the product specification stage, right through to the smallest change in beta, and synchronizing between the layers is automatically taken care of!

The Platforms

2006-06-16T06:11:00.000-07:00

This morning we had a quick get-together with everyone working on the project. These sessions are informal. We usually end up on a tangent discussion that involves solving the world’s problems, or what trouble we got into the night before. But we do get chance to make sure we’re working on the right issues in the right order.

One complication with this plug-in is that it’s being developed for PowerCore and ProTools HD systems. Powercore will ship first. However, to maintain and improve the plug-in over time, it’s best that as much of the final software is shared between the platforms. When you factor in the variants of Powercore and ProTools hardware, there are over 11 different combinations of DSP and memory configurations that have to be implemented and tested. If you’re just into using the plug-in, all this will be transparent to you. If you’re interested in why we often stay late at work, sorting out the memory allocation for all these different platforms is like playing two games of high-speed Tetris at the same time – one in your left hand, one in your right;-)

Ask The Development Team

2006-06-15T06:19:00.000-07:00

Our main thoughts when launching this Blog was to give you some insight in all the aspects when developing a Plug-in like the Harmony4. Not only technical aspects, but also just thoughts about the entire process from idea to final product. We hope, now that we are this close to the goal, this will be an exiting journey until the day we launch the final Harmony4 Plug-in. Within this process we would like to share your own thoughts, and try to answer your questions about the Plug-in and its development. We are a small and busy team so we cant promise you to answer all questions, but urge you to send your thoughts to us. Apart from our daily postings, at least once a week we will try to pick up questions and post the answers in this column.

Contact the Development team at: harmony4@tc-helicon.com

The Challenge

2006-06-14T01:07:00.000-07:00

Creating Harmony4 is truly a challenge. I’ve been working heavily on all TC-Helicon’s harmony projects ever since VoicePrismPlus, including Quintet, VoiceWorks, VoiceLive, and most recently VoicePro. VoiceDoubler isn’t a harmony product, but I simply have to plug it here because it was one of my favourites to work on (next to VoicePro). Anyway, Harmony4 is a massive challenge to TC-Helicon and myself, because it’s the first software plugin we’ve done all by ourselves. Our previous plugins were worked on in conjunction with TC-Works. We’ve now taken complete control over our plugin destiny! It’s a massive challenge because the differences between software land and hardware land are immense. Let me explain.

Hardware:

We build the operating system ourselves (sorry Apple and Microsoft, we won’t be upgrading to the newest version). We do this to be as efficient as possible. We also have complete control. As a user, you can’t install a bunch of lousy software (not your fault, trust me, I know!) on your machine that brings the rest of the system to its knees.
We build the hardware ourselves. 3rd parties can’t sell you hardware devices with poorly written drivers that seem to enjoy the colour blue (yes, a little Windows jab, but OSX isn’t innocent either).
We select the memory and processor speed. We don’t have to worry about how recently you as a user had to shovel money over to Intel or AMD or whoever, to keep up with a recent operating system upgrade.

Software
Software is just the opposite. Sure, I can abuse high-level programming languages, play with gobs of RAM and hard-drive space, make easy upgrades in the field with Internet downloads, but all the other benefits are gone. And everyone thinks plugins should be free. Wonderful!

That about ends my rant for the day.

Stephen Evans
Developer

The Final Interface

2006-06-13T00:26:00.000-07:00

We thought we’d post a shot of what the final interface looks like. It is slightly different than what you might have seen in the advertisement. One nice feature we implemented is that the four parameters stacked next to the input/output meters are dynamic, based on the harmony mode selected. This is great as you get top level access to parameters associated with that mode.
Click here for a larger version…

The Harmony4 Almost Shipping Blog

2006-06-12T03:03:00.000-07:00

Welcome to the Harmony4 Almost Shipping Blog. We set this up because we have the same feeling as you, “when is Harmony4 going to ship?”. Don’t get us wrong, we’re confident we’re almost finished. But it’s interesting that what we as developers and testers experience at the end of a project is similar to what customers experience; We both have a date when we want the product to be ready – and when we miss that date we both start wondering, when will it be ready ? We hope this blog can help communicate our passion for putting the finishing touches on the Harmony4 plug-in. It will all be worth it.