Conducting shootouts-part 2 mic shootouts

In my previous post I talked about the overriding philosophies I employ when doing gear shootouts. In this post I'd like to address microphone shootouts specifically.

Not all mics are the same and not all mic shootouts should be the same, but what I'm going to do here is outline some specific guidelines that I'll employ in a future shootout and that others can use to do their own shootout and listening tests.

------

I'd like to lead off by offering what is probably going to be the most controversial recommendation I'll make here - to prefer the exact same performance over the exact same signal path.

What I mean by this is that if you have multiple channels on a mixer that are all rated within tight tolerances of one another (as in a sound devices 442 or 552) then the difference between cables and preamps will not add up to a greater difference than if you run multiple performances in front of the mics through the exact same cable and pre. This is especially true with tests that involve the human voice, and less true with mechanical or speaker produced sounds.

So given that I'm going to recommend (and use) a methodology that uses multiple cables and preamps simultaneously in order to capture the exact same performance in front of the mics.

The other possibly controversial thing I'm going to advocate is calibration of the mics based on output level and not input level. This would have the effect of not measuring mic sensitivity against one another for the purposes of better evaluation of tone. As most audio people know, the way we hear tonality varies pretty dramatically depending on the spl we're listening to, so the question is this: is it more important to hear the differences in tone or sensitivity between mics? My answer is tone, and that's how I'm setting up my tests.

My evaluations will look to pit mics against one another in the following areas:

- price
- features
- self noise
- transient response
- off axis respnose
- on axis human voice
- on axis sfx

----------

Lets begin with setup:

Whether you're testing one mic or several, you'll want to keep the signal path as short and stable as possible. This will be my setup for some upcoming tests:

- bypass all switches in the mics. Highpasses and pads specifically should be disengaged
- place the mics as close to each other as possible, and at the point of an equilateral triangle relating to the speakers in your room.
- run the shortest reasonable cables to consecutive channels on a sound devices mixer
- bypass all signal changes on the mixer. Highpasses, pads,limiters, stereo links, etc all disengaged
- set faders to unity
- run direct out prefader to the recorder (a 744t or 788t)
- bypass all signal changers on the recorder. Again, highpasses, pads, eqs, limiters, all disengaged.
- set recorder to 24 bits 96kHz

In some cases I may skip the mixer entirely and just run straight into the recorder. The 788t has 4 xlr inputs, so if I'm testing 4 or fewer mics and have that recorder available to me at the time I'll run the tests directly to it. In all other cases I'll be using the mixer almost purely as a preamp and nothing else.

This is pretty similar to most industry standard setups for field recording and production recording, so I think the end results should translate very cleanly into the real world.

-----------

On to calibration of the mics:

The most simplistic way to calibrate a mic input is to simply mark a level on a preamp and roll with that. I prefer to to with a slightly more complex route - pink noise and dbspl.

When choosing level to calibrate to, one must consider what's going to be tested, and in the case of many microphones you're talking about the human voice, musical instruments, and sfx recordings. The level chosen will dictate both headroom and noise floor, so doing this too wrong can mess up the tests pretty badly.

For my tests I'm going to calibrate to 90db SPL = -12dbfs.

The primary reason for this is to give myself ample headroom for transient response tests (I should be able to get to 102dbspl before clipping the A/D) and to better utilize the full 24bits of headroom.

Steps to calibration should go as follows:

- send pink noise through the speakers
- measure the volume that measures 90dbspl C weighted at the microphone position with an spl meter
- adjust the preamps until the recorder's digital meters show consistent peaks at -12db spl

---------

setting it all up and calibrating it can often take as long or longer than actually running the tests. double checking to make sure it's all perfect is key to not running tests that will be at best misleading and and worst completely useless.

---------

With the mics calibrated and positioned, it's time to run some tests.

First I'll just roll on the calibrated pink noise.

next I'll turn everything off and run very quiet room tone.

third I'll speak into the mics. I'll tend to just describe what I'm doing and what the setup is as a test. This leaves me at a natural speaking voice and cadence. All voice tests will be done about 2 feet from the mics.

The fourth test is to move to 45 degrees off axis and speak again. here I'll just say that I'm speaking 45 degrees off axis.

Next is perpendicular, using the same language and distance.

with voice done I'll move on to transient and harmonic response. I have a set of small tingsha bells that I'll use, but generally you can use any type of ringy metal to do a similar test. First, from about 2 feet away I'll strike the bell on axis and let it ring out.

next I"ll do a rotation where I strike the bell 90 degrees off axis, and while its ringing I'll slowly move it to on axis. Once the bell is on axis I'll strike it again, then slowly move it to 90 degrees off axis on the other side of the mics. Once in position I'll strike it one last time and let it ring out in that position.

Next is a guitar chord (I like G) Guitar chords show off how the mics handle midrage frequencies amongst other things. This one is just done on axis.

last test is to grab a bag and record some foley. Bags are fun because they have a fair amount of complex cloth and clasp sounds that can cover a lot of frequency and dynamic range in a pleasing and complex way.

--------

With the recordings done, I prep the presentation. Presentation consists putting a chart together comparing all of the pertinent info: price, switches, other features. Mics are pretty straightforward, so this chart isn't usually too much of a burden.

I'll then edit all of the audio at native resolution in a way that best pits the mics against one another. 2 to 5 seconds clips of each mic test in a consistent order placed back to back to back in a single file - one file per test. This will yield about 9 files. Spectrograms of the pink noise can also be useful.

That's a pretty thorough test. Even more can be done though, (like testing handling noise and proximity effect) but really too many tests can cause analysis paralysis. This is probably plenty.

Conducting shootouts-part 1 Philosophy

I’m going to do a couple of posts here about conducting gear shootouts. In this first installment I’ll discuss the purpose and philosophy of shootouts that I personally subscribe to, and in the second I’ll go over doing mic shootouts specifically.

So generally, why do a shootout?

I do these to clearly define the strengths and weaknesses of one piece of gear vs another in order to make buying and session decisions. I personally took a lot of inspiration from the zacuto great dslr shootouts conducted last year, and you’ll see the principles that they applied in those tests echoed in my own.

With that in mind, what are the main things that need to be dealt with when conducting a shootout?

Here’s my list:

• The gear needs to be tested fairly against one another.

• The gear needs to be tested in situations that approximate common and specific real world conditions.

• The gear needs to be stress tested.

So fundamentally here’s how I go about covering those three main points:

Testing fairly:

Testing apples to apples isn’t always possible in the purest sense of the word because manufacturers will do different things with the gear in order to differentiate themselves, but the objective should be to get as close to that reality as you can.

Generally testing fairly means:

• Setting up all devices in a manner that is as close to equal as possible
• this means setting all filters, sample rates, bit depths, and other things to the same settings and otherwise bypassing anything proprietary to a single device if possible.
• calibrating all devices to the same baseline
• I’ll go over calibration in more detail later, but generally you want to lock down a baseline testing level for any signal that’s going to pass through the devices being tested in advance of the tests. Pink noise and db spl meters are your friends here.
• removing or reducing all outside influences on the test
• Outside influences are the things that are required to be in the test in order for the test to happen, but that aren’t being tested themselves. This can be things like the sounds that are being recorded in order to test, the mics, cables and recording devices and preamps and rooms etc that are part of a recording signal chain that are both part of the recrording chain and not part of the equipment being tested.

Testing situations that approximate specific real world situations

Tests are not useful unless they can give information that will illustrate how gear works in the situations you’re most likely to use them. This means that testing an akg D112 as a distant mic on a woodwind section (while potentially interesting) is not particularly useful.

A whole lot of recording gear is tuned up to record the human voice, so generally voice recording is a good baseline for a wide variety of gear. Shootouts can either be wide and shallow or narrow and deep, but not often both wide and deep. Find the appropriate balance there and choose your subject matter accordingly.

As an example, in my handheld recorder shootout I focused on a few handheld recorders that I had available to me. But beyond that I consciously tested their performances using the internal mics only, and I tested them in the context of sound effects recording. I could have tested the preamps separately with external mics, and I could have tested them in other contexts like interviews or musical performance recording, but I wanted my tests to be focused on how these devices perform as impromptu sfx capture devices specifically so I tailored my tests to that specific end.

Think about your scope before doing the tests, then work that scope thoroughly.

Stress Testing

With a scope defined, the next step is to really test the boundaries of what the gear does within that scope. That means if you’re testing a recording device you’ll do things like peg its limiters to see how they react, roll quiet room tone to see what happens down on the bottom end, and otherwise put the gear in a variety of tough recording situations. Microphones can be stress tested similarly, but can also have things like off-axis response, transient response, and proximity effect really run through the ringer.

Stress testing gear is important because a fair amount of what differentiates one piece of gear from another is how it handles extreme situations. The better a piece of gear is designed, the more gracefully it will deal with what you can throw at it. Its often a combination of price and extreme end performance that determines one’s buying and usage decisions.

A note on calibration:

Calibration is a little more complex than just setting everything at the same place. It requires thought into what you’re about to record.

Calibration is usually done by sending tone or pink noise through the devices and setting inputs to a level that is consistent across all devices. In the case of my HH shootout I sent pink noise through at 70db spl and adjusted the pre on each to -12dbfps. This meant that at about 92db I was going to clip, which was a pretty hot setting. That setting was done on purpose because my intent was to both test the limiters on the devices and to test the room tone.

The level at which you calibrate for the test has a big influence on how those tests come out, so choose your calibration (and headroom) level carefully.

Presentation:

One of the things that I really liked about the Zacuto presentation that I tried to incorporate into my own shootout was the presentation.

Specifically, I loved the quick edits from one camera to the next of the exact same footage. I really felt that gave me the best opportunity to make a blink assessment of what I was seeing. In my tests I modeled my edits the same way, and really put the devices right next to each other in the presentation. If I had put each test in its own soundcloud player, the reader (listener) would not really get to make a split-second evaluation, and prejudice with regards to brands and price points would have an opportunity to creep in.

I also did my best to document and describe every step of the test so that the reader could listen to them with a clear sense of what they’re actually hearing. This included descriptions and photos of the setups, tables describing features and functions of the devices, and wraps ups with my own personal impressions and thoughts.

So that’s the philosophy and general theory behind how I run my shootouts. In the future I’ll do a mic shootout and get into some details about how to do mic shootouts specifically.

On art

On Art:

A while back I stopped thinking of myself as a technician and started thinking of myself as an artist. This came around the point at which I had gotten enough of a handle on my techniques as to feel the overwhelming need to do something personal and creative with them.

As time wore on and I had conversations with my incredibly artistic wife I started to turn over in my mind the question of what exactly art is, and how I can get better at it.

What I came up with was this:



This post will discuss why I define art this way and why it matters to me. All of the content here is my own, and was not copied or borrowed from anyone else. Feel free to read this as my opinion on the matter, but don’t read much more into it than that.

So put simply:

Art is creativity expressed through technical means.

The reason I love this definition so much is because of both what it allows in and what it excludes.

I’ll start at the beginning.
-----------------

Art requires creativity.

Creativity is as it sounds. It is the creation of something new. If there is no creativity then there is only repetition of things before, and rote repetition is not art. A mass manufactured bottle churned out by a machine is as much not art as the original design for the first one is. Plagiarism is also not art unless it required creativity beyond what the original creator did in order to replicate it exactly.

Art requires a medium.

Art requires translation from one individual to others in order to exist. It needs a medium. Without a medium you don’t have art, you only have a dream. Dreams and thoughts are important, but alone they don’t constitute art.

A medium can consist of anything from words to paint to physical movement to sounds to ones and zeroes. The only concrete requirement for a medium is that it be observable by others. Some art exists in multiple mediums simultaneously, but all art requires at least one medium in order to exist as an actual work.

Manipulation of mediums requires technique.

There is a craft that comes before the art can arise. Whether in sounds or images or piles of dirt or pencil and paper, the prospective artist begins the artistic journey as a technician. The more refined the artists technique in manipulating the medium, the more fully and truly the creative ideas can be expressed through that medium to others. The less refined the technique, the more frustrated the artist.

-------------------

So what?

Well, if I now consider myself an artist I need to be able to evaluate the art of those around me as well as to do the things that will make me a better artist tomorrow than I am today.

Now that I have in my head what I believe to be the building blocks of art, I can begin to pursue some specific things that will make the art that I do better and at a faster rate than those who don’t think about these things.

I naturally gravitate towards the technique side of the artistic equation, and by recognizing that in myself I can do things like force some creative endeavors here and there, and also measure myself objectively against the technical achievements of my peers.

I also just enjoy art generally, so now that I have it defined more clearly in my head I feel like I’m able to look at things like computer programs and paintings and guitar performances and every part of a film (acting, makeup, dialogue, set dressing, lighting, cinematography, foley, sound design, mix, location, coloring, etc) and say “Man, that’s really great art” and know what I really mean by it.

So there it is, my definition of art. I like it and I’m sticking by it. In the future I may discuss the way I measure high art vs low art.