Pianoteq is an effort to model, rather than sample, acoustic pianos and other instruments on the computer. Now in its third major release, its interface and sound generation have each matured. Using mathematical models in place of recorded sounds, an entire grand piano fits in just a few megs of space, rather than requiring several DVDs, and the software maker claims the results can be more natural and playable.

Pianoteq, which runs as do its rivals on Mac and Windows, is also unusual in providing support for the Linux operating system – something some developers have claimed isn’t practical with commercial music software. And a new “Player” addition, announced this month, makes it more affordable. In addition to software development, the team has even launched an extensive piano restoration effort:
http://www.pianoteq.com/kivir

I’ve been impressed with my time playing with Pianoteq’s software. That’s especially meaningful to me, as my background in music has been on acoustic pianos, back to when I was literally old enough to reach the keys for the first time. But I also wanted to know more about how this software developed. Its rigorous approach to modeling has attracted a lot of attention in the virtual instrument world, and the fact that it targets Linux alongside Mac and Windows challenges notions that commercial software can’t make it on the free operating system.

Pianoteq sent along some extensive answers, which I’m pleased to be able to share. Naturally, they’re proud of their software, so there is a bit of expected boasting here. (I’ll discuss more of the experience of using the tool, and the new Player version, shortly). But they also have some fascinating commentary on sound design, modeling, and the development process. In the “geeky as we want to be” spirit of this site, here’s the full scoop.

CDM: Can you talk about the background of the company? How does one make a shift from dealing with the physical instruments and tuning to thinking about mathematical models, let alone translate that into actual software?

Philippe: the Pianoteq history is strongly connected to my first job as a piano tuner. Then at the age of 31, I started a new life with basic studies in mathematics at the University of Toulouse, France. After what I prepared my PhD thesis on the parametrization of vibrating phenomena, without imagining that it would be the basis of my third life with Pianoteq. Thanks to these two skills and to an exceptional scientific environment in Toulouse, I succeeded in identifying important phenomena responsible for the generation of the piano sound and proposed a model which describes the whole interaction of the soundboard, strings, bridge and air.

Julien: I was working as an engineer at Institute of Mathematics of Toulouse, with Philippe (who was also my teacher when I was a student). My focus was on an open-source finite element package ( http://home.gna.org/getfem ) when Philippe told me about his project of piano sound synthesis. I took charge of the development of the real-time engine, and we quickly decided that we wanted to turn this research project into a commercial product. Thanks to the French law on innovation and research (1999), the support of INSA Toulouse, and the Institute of Mathematics, the start-up MODARTT was created in 2006 to sell Pianoteq, which was at that time the first fully modeled piano instrument.

Niclas: I represented many of Pianoteq’s customers of today, being a part time piano composer with a love for the piano instrument. I was updating an extensive article on digital piano technology in 2004 when someone advised me to have a look at Philippe’s research. I immediately understood its potential, which is why I suggested Philippe to assist in marketing and product development. Since then, I have participated in the product development and testing and am also in charge of sales and customer service.

Modeling the Piano with Math

I remember trying previous attempts at piano modeling and finding them interesting but ultimately unsatisfying. I think people who pick up Pianoteq have, immediately, a different experience. What’s different about this modeling approach than those that have come before – and, for that matter, why did it take until now?

Philippe: The idea of modeling musical instruments is very old and has always faced great difficulties: the complexity of physical phenomena, the sensitivity of the human ear to the slightest imperfection, and the difficulty of running a complex model in real-time. The latency needs to be so small that it gives to the musician the impression of playing a real acoustic instrument. Until now, attempts have only confirmed that the task was not easy. The state of the art of digital pianos is based on sampling technology. Each note is a recording of how it sounded at a specific moment, without taking into account the complexity of the instrument. The huge data generated by sampling can reach 40 Gbytes for a single piano. The flow rate of data transmitted from the hard drive to the sound device is too high for the current hardware capacity and it can happen that one hears crackles. [Ed.: I would say fast hard drives, optimized software streaming, and other intelligent configuration can certainly avoid crackles, but the fact that, say, a low-end hard drive might choke means that Philippe's point about data intensity here is nonetheless well taken. -PK]

Moreover, the reproduced sound lacks vividness. Hence, creating a piano model which takes into account the interaction between hammer and strings, the interaction between strings and soundboard via the bridge and the interaction of the soundboard with the air is of great interest. Based on mathematical models, Pianoteq allows parameters to be stretched as long as the model permits, resulting not only in new performance styles but also in new piano sounds. Pianoteq is thus also an innovating tool for music creation and can be useful not only to musicians but also to piano manufacturers and piano tuners for simulation and training purposes. Pianoteq makes excellence in piano available to all. Among Pianoteq users, composers and professionals in music creation are certainly the most excited with our innovation. Pianoteq offers what acoustic and sampled pianos cannot offer: new opportunities for music creation and a pure piano sound that is not altered by its environment (reverberation) or by recording devices.

Can you talk more about the model itself? I can see the components that are modeled, but – realizing we have fairly technical people among our audience – what are the basic modeling techniques?

Philippe: the modeling technique is based on various standard techniques issued from mechanics theory including modal analysis (calculation of vibration frequencies and the corresponding modes) and parametrization techniques that we developed at the university, as well as from a precise knowledge issued from my previous job as piano tuner/restorer of what is going on in a piano and what is important.

How did that approach to modeling evolve? Obviously, there’s this strong mathematical research background. But what’s the process like of translating that theory into something that’s usable? Were there mistakes or adjustments along the way?

Philippe: I don’t think there were mistakes along our model evolution, but more a constant improvement in the details taken into account by the model, looking closer to the physics of the piano and finding suddenly some simplification in the algorithms that allow to take include more details for the same computational cost or being more precise in the simulation.

Developing for Linux

How did you make the decision to support Linux in addition to Windows and Mac?

Julien: The initial prototypes for pianoteq were developed on Linux, using [audio system] JACK, with no GUI. Later, when we added a graphical interface and turned it into a VST plugin, we used VSTGUI for its interface, which is not available on Linux. However we had quite a few requests from Linux users, and we did make sure that pianoteq was running fine in WINE [an open-source implementation of Windows' APIs, allowing Windows programs to run in Linux]. During the development of Pianoteq 3, we switched to the JUCE toolkit, which is a great piece of cross-platform software. Thanks to JUCE, the Linux port was really easy to do, so we decided to give it a try and see what happens.

One complaint I hear from developers about Linux is that it’s “impossible” to do commercial development, because you “can’t” distribute binaries. Obviously, that didn’t stop you. I’ve tried Pianoteq on Fedora and Ubuntu, though, under both the real-time and default kernels, and had immediate success. Now, I imagine there’s a good bit of work that goes into making that happen. What was your experience like as a developer? Do you feel that the result is successful, that it is a usable solution for users?

Julien: This was in fact something that we also feared, that the Linux port would turn into a support nightmare. However a good example of a successful application that is distributed in binary form is Renoise. That showed us that it was possible to do. In fact Renoise also uses JUCE, but I was not aware of that fact at that time. What helps here for binary portability is that we have very few dependencies. JUCE is statically linked, so pianoteq depends on very few dynamic libraries: ALSA, X11, libc (even old versions), and basically that’s all. We had to hack some sort of weak linking for JACK in order to allow pianoteq to run even when libjack.so is not available. Of course, if you want better integration in the desktop, things get much more complicated.

What’s your own Linux testing setup like? (distro? kernel?)

Julien: Pianoteq is built on a Debian Sarge box, otherwise we generally use Ubuntu for the desktop, with the default kernel. [Ed.: The distribution Ubuntu is itself built on packages from Debian; 10.04 LTS uses Debian Testing.]

I’ve likewise been impressed with the vanilla kernel (as opposed to the “real-time” branch), which can save some setup time and configuration work. (My audio interface is a Native Instruments Audio Kontrol 1). Any thoughts on what setups may be most advisable? (You document some of this in the readme.)

Julien: I’ve never been lucky with the kernels labelled “rt” , and I really hate when the computer randomly hangs so I prefer to stick with default kernels. We don’t have issues with them, as long as your user account has been granted real-time priviledges. I believe that for now, the most overlooked setting for realtime audio is the CPU frequency throttling, which is a real audio performance killer, especially on the less powerful machines such as netbooks. You really need to have your cpu running at full speed 100% of the time, especially with a software like Pianoteq which needs quite a bit of CPU power.

Ed.: Before we give the realtime kernel a bad name, some of those “random hangs” were not necessarily the kernel’s fault – a bug in Ubuntu’s implementation caused the system to crash when combining the RT kernel with proprietary NVIDIA drivers, for instance. But if this sort of thing scares you, the vanilla kernel remains a strong option – it’s the default for a reason. The larger discussion is best saved for another article, but suffic,e to say, if latency-sensitive piano instrument developers are okay with the vanilla kernel, you shouldn’t feel you have to install a realtime kernel just to make music. If you want to test it, projects like Fedora’s Planet CCRMA can make it easier to use.

Are you finding that there is some positive response to the Linux version?

Julien: Yes, very positive response. In fact, a bit more than what we expected at the beginning. Approximately 4% of our customers are using the Linux version.

I could even imagine it working on netbooks. Based on load, it appears perfectly workable, which means a really cheap ultra-portable piano you can take anywhere.

We spent some time to make sure that the latest versions could run on netbooks, altough with very high CPU load (80% or more). However I’d recommend to use a more powerful laptop in order to have more room for the cpu. Ed.: Given the range of Atom netbooks out there now, I may have to test some of the newer models on this.]

There’s not currently a Linux plug-in version, correct? I’ve been just as happy using JACK [a standard for routing audio between applications], but what went into that decision?

Julien: Right, no plug-in version on Linux for now. The problem is that “plug-in” may mean any combination of VST, DSSI, and LV2. VST would be the easiest for us, but very few hosts support it ( basically only proprietary hosts such as renoise and energyxt, and also jost). DSSI is said to be obsolete, while being not to hard to support (except the GUI has to run in a separate process..). LV2 is said to be the future, but
it seems to be quite complicated to fit a “vst-like” plugin into an lv2 plug-in. We have not yet taken a decision. It is already enough of a pain to support the numerous plug-in formats on Mac and Windows. We will probably add support for JACK sessions quite soon.

In Use

Initially, having so much power over sound could be overwhelming – looking at the number of parameters you can adjust in the real-time mathematical model. Aside from the presets (which themselves sound pretty terrific), how would you suggest someone go about beginning to explore the options? Is there a workflow that makes sense for approaching adjusting the sound?

Answer from Pianoteq:
http://www.pianoteq.com/faq?category=pianoteq_working

Ed.: So I should have read the *** manual! Here’s their advice:

If you need to adapt the piano sound you could for example try adjusting the hammer hardness (2) to achieve a different brightness of the hammer strokes. Increasing unison width (3) makes it a bit out of tune (resembling certain acoustic pianos). The new powerful sound recording feature (4) lets you place up to 5 virtual microphones anywhere around the piano to achieve ultimate ambience and tone colour. The dynamics and velocity curve (5) will most likely need to be adjusted to the keyboard used in a MIDI file performance.

Indeed, this commentary makes sense. Hammer hardness is something that could be adjusted in the maintenance of an actual piano. Since you listen to a software piano model as though it is amplified, adjusting mic placement (as on a number of piano software emulations) is a no-brainer. And dynamics and velocity curve are essential not only for MIDI files, but if your keyboard controller lacks these controls onboard.

As my friend Jim Aikin noted in his review of Pianoteq – why would you adjust the speed of sound? (I suppose you could account for different altitudes; I could replicate the Aspen Music Festival!)

Julien: Well, why not ! Being in a virtual world gives you access to some parameters that cannot be easily modified in the real world, if they give interesting variations of the sound, then they are worth being adjusted!

One small note – it seems the metronome is not connected to the playback and recording, which means that MIDI sequences won’t export to SMF correctly? (Or is this a Linux bug?)

Julien: Yes, it’s not a bug, the MIDI recording and playback abilities of the standalone application are very minimalistic. It is best to use a real sequencer for serious work.

Where can people read more about the featured historical instruments?

http://www.pianoteq.com/cimbalom
http://www.pianoteq.com/grimaldi
http://www.pianoteq.com/blanchet
http://www.pianoteq.com/schmidt
http://www.pianoteq.com/graf
http://www.pianoteq.com/cp-80

There are some fascinating add-ons that aren’t pianos. Is it possible at some point that other sound designers might be able to use the sound engine to design their own instruments?

Julien: We believe that Pianoteq PRO is already a first step in that direction, with its ability to edit each parameter note by note.

How are users using this onstage and in the studio? What sorts of users have you found using the product?

Niclas: There are many composers and keyboardists that use Pianoteq, from amateurs to professionals, on stage as well as in the recording studios. We have presented a few reference users here: http://www.pianoteq.com/references

Thanks to the Pianoteq guys for being thorough in the answers. I know not everyone responds in the same way in regards to the perceived quality of the model, but my own feeling is that the effort makes the instrument terrifically playable and responsive. The best way to see for yourself is to give the demo a go, and listen to the results. I’ll follow up more on this instrument, and how it’s become a central part of my Linux music workstation, soon. Let us know if you have questions for the developers I missed.

Here is the original post:
Makers of Pianoteq Talk Piano Modeling, Developing for Linux