The musical ‘observer effect’

In scientific audio circles, it is believed that if you are aware (or think you are aware) of what hardware you are listening to, then you are incapable of any sort of objective assessment of its quality. This leads to the blind listening test being held up as the Gold Standard for audio science.

But here’s an irony: almost everything of value that man creates comes into being through a process of ‘sighted’ creation and refinement – and it seems to work. Bridges are designed by architects who refine CAD models on a screen, but the finished products don’t fall down, and are admired by ordinary people for their appearance. Car bodies are designed by engineers and stylists in full sight, yet the holes line up with the rest of the car, and they achieve great measurements for aerodynamics and the cars look good as well. Pianos are tuned by people who know which way they are turning the lever as they listen.

So if ‘sighted-ness’ leads to a completely fictitious, imaginary perception, then presumably our pianos are not really in tune, but we imagine they are? Maybe everyone but the piano tuner would hear an out-of-tune cacophony when the piano is played? But no, it turns out that everyone, including the piano tuner, can tell consistently when a piano is in tune without resorting to blind tests, and this can be confirmed with measurements.

So how come ‘sightedness’ is so problematic for the creation or assessment of audio equipment? I think that the question is “not even wrong”. The faulty logic lies in the erroneous idea that audio equipment is being listened to, as opposed to through, and that the human brain when listening to music is similar to a microphone. There is no reason to believe this at all; to me, it is just as likely that the brain is acting as an acquirer and interpreter of symbols. The quality of the sound is part of the symbol’s meaning, but cannot be examined in isolation.

As a result, it may just be that there is no way for a human listener to reliably discern anything but the most obvious audio differences in A/B/X listening tests. Using real music, the listener may be perceiving sound quality differences as changes in the perceived meaning of the symbols, but repeated listenings (like reading a phrase over and over), or listening to extracts out of context, kills all meaning and therefore kills any discernment of sound quality. Consciously listening for differences as opposed to listening to the music, pressing buttons while listening, breaking the flow of the music in any way, all have a similar effect. Alternatively, using electronic bleeps, or randomised snippets as the ‘test signal’, the listener is effectively hearing a stream of noise without any context or meaning, so the brain has nothing to attach the sound quality to at all.

In effect, the act of listening for sound quality in scientific trials may kill our ability to discern sound quality. Can this be proved either way? No.

I don’t see this as a problem to be ‘solved’; it is simply the kind of paradox that pops up when you start thinking about consciousness. Music has no evolutionary survival value, but we enjoy listening to it anyway – so we are in Weirdsville already. The extreme ‘objectivists’ who hold up ABX testing as science are extremely unimaginative if they think their naïve experiments and dull statistical formulae are a match for human consciousness.

Within the limitations of their chosen technology, most hi-fi systems are created with the aim of being ‘transparent’ to levels that exceed the known limitations of the physiology of the ear, and people seem keen to buy them. Without referring to scientific listening test data, the customers know that, in normal use, proper hi-fi does sound better than an iPod dock with 2″ speaker. But, as their own preference for the sound can’t be proved scientifically because of ‘the observer effect’, and because a human is bound to be influenced by factors other than the sound, then at some level they have to buy their hi-fi equipment ‘on faith’; maybe being influenced by the look of it, or because they believe the meme that vinyl is superior to digital. So be it. But they may find that, later, the system fails to meet their expectations and they are on a ruinous treadmill of “tweaks” and “upgrades”.

On a strictly rational basis, bypassing all that anguish, the new generation of DSP-based speakers gets even closer to the ideal of transparency by virtue of superior design – no listening tests required. I am confident they will sound great when being used for their intended purpose.

[Last edited 06/08/16]


8 thoughts on “The musical ‘observer effect’

  1. I suspect that the piano tuner is rather neutral compared to most audiophiles. She’s paid to tune somebody else’s piano, and doesn’t care so much about creating the best possible piano, and has never taken side in bike-shed debates about pianos. Most audiophiles seem to end up in one of various factions and subscribe to a belief system, including many people one would hope are more independent. Perhaps it would work better if hi-fi reviewers were not people with significant personal investment or experience and had fewer preconceptions. Even then, it may be necessary to make some attempt to stop conflation of ‘sounds good’ and ‘looks good’, although buying processes certainly do conflate that for most people.

    Liked by 2 people

  2. This is interesting because not long ago there was a debate on a well known blog about the late 1970’s Sansui amplifiers vs the older pre 1978 ones. I have a Sansui A-80 that I love but is built with plastic sides but has a aluminum face with nice vu meters definitely not built like the big black AU series and earlier ones. One member said he preferred the sound of his A-80 over a comparable AU unit and then the salvo’s ensued. The automatic thought was the older units are built more solid so ergo they must sound better. I think this is used in some higher end modern amplifiers. If you put a good mid fi amplifier into a chassis of steel with 1/4″ aluminum front and sides and compare the same exact amp in a more traditional case, thin aluminum front, sheet metal top, plastic knobs and of course black paint. I would bet that almost every person listening would say the High End looking unit sounds miles better than the BPC version even though they would both have identical sound. Which brings perception of sight back into the equation if it looks better so it must sound better.


  3. I’ve spent almost a year preparing myself for the purchase of a decent ‘hifi’. I’ve been reading avidly – both the audiophile press and also some of the more ‘sceptical’ commentators. And I’ve found it mostly completely baffling. When amplifiers are reviewed and seemingly night-and-day distinctions are made about the ‘air between the instruments’ and the ‘timing of the music’ I struggle somewhat. And don’t get me started on cables…

    Your post crystallised my thinking – I believe you are absolutely right – this isn’t a speaker/mic relationship – and once that’s realised then the language of the reviewers makes more sense *to them* – but may be off little use to me as I’m me… and they are them.

    So… having taken that all on board – and realising that neither reviewers, or indeed me at an audition, can tell me much about whether one amp/dac/cable is ‘better’ than another I’m thinking I’ll skirt the issue completely and go with something that makes sense to me… possibly a pair of Kii Threes…


    1. Yes, I think that after a few false starts, the ‘neutral’ system is finally here, and it is strikingly better than what went before. Peter Walker of Quad espoused the idea that if the design and measurements were right, a listening test was just confirmation that a product was working i.e. that the customer could confidently buy it and listen to it without the psychological overhead of wondering whether it really sounded any good or not. This wasn’t really applicable to speakers at the time, but may be now.


      1. Hurray for Peter Walker, a man possibly before his time! I wonder what he’d make of DSP-engineered coherence/response etc.? Given the lengths he went to make the ESL63 behave as transparently as possible with the technology available, I rather think he’d be right on the case. In fact… maybe I should convert my ESL63s from “delay lines via lengths of wire” to “delay lines via DSP”! That would be a challenging project…


        1. It has crossed my mind that the ESL63s *could* be improved by using genuine delay lines. I think the existing system is a phase shifting network. Transients would be reproduced even more accurately if this was replaced with a multi-tap delay line using, of course, DSP. Please correct me if I am wrong…


          1. Looking at the service manual, it’s an inductor/capacitor ladder, with 8 steps, each connected to a separate electrode set, starting from the middle and moving out. I can’t find any values for the inductors which are listed as a specific part number but the caps are mostly 22pF with one set being 10pF. Basically, as I’m sure you’re aware, it changes the signal to simulate a point source some 30cm behind the speaker, which I guess is essentially a phase change that is equivalent to the required delay. I’m not sure how it deals with different frequencies, I’d have thought that the phase shift would have varied by frequency, in which case the effect would also vary. Any thoughts on that?

            Modifying it to use DSP-implemented delay lines would be pretty challenging, I’d guess you’d need 16 amps, and of course there’s the minor problem of 5.3kV and the cunning spark detection antenna etc… They don’t have to be huge amps, max of 40V swing required before the protection cuts in anyway. But hey, we’ve got time, right?! I have a pair, but I think I’d have to have a reasonably firm plan for a possible rebuild before I started dismantling them 😮 . What a great project though!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s