Conducting shootouts-part 2 mic shootouts

In my previous post I talked about the overriding philosophies I employ when doing gear shootouts. In this post I'd like to address microphone shootouts specifically.

Not all mics are the same and not all mic shootouts should be the same, but what I'm going to do here is outline some specific guidelines that I'll employ in a future shootout and that others can use to do their own shootout and listening tests.

------

I'd like to lead off by offering what is probably going to be the most controversial recommendation I'll make here - to prefer the exact same performance over the exact same signal path.

What I mean by this is that if you have multiple channels on a mixer that are all rated within tight tolerances of one another (as in a sound devices 442 or 552) then the difference between cables and preamps will not add up to a greater difference than if you run multiple performances in front of the mics through the exact same cable and pre. This is especially true with tests that involve the human voice, and less true with mechanical or speaker produced sounds.

So given that I'm going to recommend (and use) a methodology that uses multiple cables and preamps simultaneously in order to capture the exact same performance in front of the mics.

The other possibly controversial thing I'm going to advocate is calibration of the mics based on output level and not input level. This would have the effect of not measuring mic sensitivity against one another for the purposes of better evaluation of tone. As most audio people know, the way we hear tonality varies pretty dramatically depending on the spl we're listening to, so the question is this: is it more important to hear the differences in tone or sensitivity between mics? My answer is tone, and that's how I'm setting up my tests.

My evaluations will look to pit mics against one another in the following areas:

- price
- features
- self noise
- transient response
- off axis respnose
- on axis human voice
- on axis sfx

----------

Lets begin with setup:

Whether you're testing one mic or several, you'll want to keep the signal path as short and stable as possible. This will be my setup for some upcoming tests:

- bypass all switches in the mics. Highpasses and pads specifically should be disengaged
- place the mics as close to each other as possible, and at the point of an equilateral triangle relating to the speakers in your room.
- run the shortest reasonable cables to consecutive channels on a sound devices mixer
- bypass all signal changes on the mixer. Highpasses, pads,limiters, stereo links, etc all disengaged
- set faders to unity
- run direct out prefader to the recorder (a 744t or 788t)
- bypass all signal changers on the recorder. Again, highpasses, pads, eqs, limiters, all disengaged.
- set recorder to 24 bits 96kHz

In some cases I may skip the mixer entirely and just run straight into the recorder. The 788t has 4 xlr inputs, so if I'm testing 4 or fewer mics and have that recorder available to me at the time I'll run the tests directly to it. In all other cases I'll be using the mixer almost purely as a preamp and nothing else.

This is pretty similar to most industry standard setups for field recording and production recording, so I think the end results should translate very cleanly into the real world.

-----------

On to calibration of the mics:

The most simplistic way to calibrate a mic input is to simply mark a level on a preamp and roll with that. I prefer to to with a slightly more complex route - pink noise and dbspl.

When choosing level to calibrate to, one must consider what's going to be tested, and in the case of many microphones you're talking about the human voice, musical instruments, and sfx recordings. The level chosen will dictate both headroom and noise floor, so doing this too wrong can mess up the tests pretty badly.

For my tests I'm going to calibrate to 90db SPL = -12dbfs.

The primary reason for this is to give myself ample headroom for transient response tests (I should be able to get to 102dbspl before clipping the A/D) and to better utilize the full 24bits of headroom.

Steps to calibration should go as follows:

- send pink noise through the speakers
- measure the volume that measures 90dbspl C weighted at the microphone position with an spl meter
- adjust the preamps until the recorder's digital meters show consistent peaks at -12db spl

---------

setting it all up and calibrating it can often take as long or longer than actually running the tests. double checking to make sure it's all perfect is key to not running tests that will be at best misleading and and worst completely useless.

---------

With the mics calibrated and positioned, it's time to run some tests.

First I'll just roll on the calibrated pink noise.

next I'll turn everything off and run very quiet room tone.

third I'll speak into the mics. I'll tend to just describe what I'm doing and what the setup is as a test. This leaves me at a natural speaking voice and cadence. All voice tests will be done about 2 feet from the mics.

The fourth test is to move to 45 degrees off axis and speak again. here I'll just say that I'm speaking 45 degrees off axis.

Next is perpendicular, using the same language and distance.

with voice done I'll move on to transient and harmonic response. I have a set of small tingsha bells that I'll use, but generally you can use any type of ringy metal to do a similar test. First, from about 2 feet away I'll strike the bell on axis and let it ring out.

next I"ll do a rotation where I strike the bell 90 degrees off axis, and while its ringing I'll slowly move it to on axis. Once the bell is on axis I'll strike it again, then slowly move it to 90 degrees off axis on the other side of the mics. Once in position I'll strike it one last time and let it ring out in that position.

Next is a guitar chord (I like G) Guitar chords show off how the mics handle midrage frequencies amongst other things. This one is just done on axis.

last test is to grab a bag and record some foley. Bags are fun because they have a fair amount of complex cloth and clasp sounds that can cover a lot of frequency and dynamic range in a pleasing and complex way.

--------

With the recordings done, I prep the presentation. Presentation consists putting a chart together comparing all of the pertinent info: price, switches, other features. Mics are pretty straightforward, so this chart isn't usually too much of a burden.

I'll then edit all of the audio at native resolution in a way that best pits the mics against one another. 2 to 5 seconds clips of each mic test in a consistent order placed back to back to back in a single file - one file per test. This will yield about 9 files. Spectrograms of the pink noise can also be useful.

That's a pretty thorough test. Even more can be done though, (like testing handling noise and proximity effect) but really too many tests can cause analysis paralysis. This is probably plenty.

Conducting shootouts-part 1 Philosophy

I’m going to do a couple of posts here about conducting gear shootouts. In this first installment I’ll discuss the purpose and philosophy of shootouts that I personally subscribe to, and in the second I’ll go over doing mic shootouts specifically.

So generally, why do a shootout?

I do these to clearly define the strengths and weaknesses of one piece of gear vs another in order to make buying and session decisions. I personally took a lot of inspiration from the zacuto great dslr shootouts conducted last year, and you’ll see the principles that they applied in those tests echoed in my own.

With that in mind, what are the main things that need to be dealt with when conducting a shootout?

Here’s my list:

• The gear needs to be tested fairly against one another.

• The gear needs to be tested in situations that approximate common and specific real world conditions.

• The gear needs to be stress tested.

So fundamentally here’s how I go about covering those three main points:

Testing fairly:

Testing apples to apples isn’t always possible in the purest sense of the word because manufacturers will do different things with the gear in order to differentiate themselves, but the objective should be to get as close to that reality as you can.

Generally testing fairly means:

• Setting up all devices in a manner that is as close to equal as possible
• this means setting all filters, sample rates, bit depths, and other things to the same settings and otherwise bypassing anything proprietary to a single device if possible.
• calibrating all devices to the same baseline
• I’ll go over calibration in more detail later, but generally you want to lock down a baseline testing level for any signal that’s going to pass through the devices being tested in advance of the tests. Pink noise and db spl meters are your friends here.
• removing or reducing all outside influences on the test
• Outside influences are the things that are required to be in the test in order for the test to happen, but that aren’t being tested themselves. This can be things like the sounds that are being recorded in order to test, the mics, cables and recording devices and preamps and rooms etc that are part of a recording signal chain that are both part of the recrording chain and not part of the equipment being tested.

Testing situations that approximate specific real world situations

Tests are not useful unless they can give information that will illustrate how gear works in the situations you’re most likely to use them. This means that testing an akg D112 as a distant mic on a woodwind section (while potentially interesting) is not particularly useful.

A whole lot of recording gear is tuned up to record the human voice, so generally voice recording is a good baseline for a wide variety of gear. Shootouts can either be wide and shallow or narrow and deep, but not often both wide and deep. Find the appropriate balance there and choose your subject matter accordingly.

As an example, in my handheld recorder shootout I focused on a few handheld recorders that I had available to me. But beyond that I consciously tested their performances using the internal mics only, and I tested them in the context of sound effects recording. I could have tested the preamps separately with external mics, and I could have tested them in other contexts like interviews or musical performance recording, but I wanted my tests to be focused on how these devices perform as impromptu sfx capture devices specifically so I tailored my tests to that specific end.

Think about your scope before doing the tests, then work that scope thoroughly.

Stress Testing

With a scope defined, the next step is to really test the boundaries of what the gear does within that scope. That means if you’re testing a recording device you’ll do things like peg its limiters to see how they react, roll quiet room tone to see what happens down on the bottom end, and otherwise put the gear in a variety of tough recording situations. Microphones can be stress tested similarly, but can also have things like off-axis response, transient response, and proximity effect really run through the ringer.

Stress testing gear is important because a fair amount of what differentiates one piece of gear from another is how it handles extreme situations. The better a piece of gear is designed, the more gracefully it will deal with what you can throw at it. Its often a combination of price and extreme end performance that determines one’s buying and usage decisions.

A note on calibration:

Calibration is a little more complex than just setting everything at the same place. It requires thought into what you’re about to record.

Calibration is usually done by sending tone or pink noise through the devices and setting inputs to a level that is consistent across all devices. In the case of my HH shootout I sent pink noise through at 70db spl and adjusted the pre on each to -12dbfps. This meant that at about 92db I was going to clip, which was a pretty hot setting. That setting was done on purpose because my intent was to both test the limiters on the devices and to test the room tone.

The level at which you calibrate for the test has a big influence on how those tests come out, so choose your calibration (and headroom) level carefully.

Presentation:

One of the things that I really liked about the Zacuto presentation that I tried to incorporate into my own shootout was the presentation.

Specifically, I loved the quick edits from one camera to the next of the exact same footage. I really felt that gave me the best opportunity to make a blink assessment of what I was seeing. In my tests I modeled my edits the same way, and really put the devices right next to each other in the presentation. If I had put each test in its own soundcloud player, the reader (listener) would not really get to make a split-second evaluation, and prejudice with regards to brands and price points would have an opportunity to creep in.

I also did my best to document and describe every step of the test so that the reader could listen to them with a clear sense of what they’re actually hearing. This included descriptions and photos of the setups, tables describing features and functions of the devices, and wraps ups with my own personal impressions and thoughts.

So that’s the philosophy and general theory behind how I run my shootouts. In the future I’ll do a mic shootout and get into some details about how to do mic shootouts specifically.