Inherent emotional quality of human speech sounds

In Plato’s work Cratylus, Socrates and his two interlocutors discuss the relationship between names and their referents.  One interlocutor, Hermogenes, argues that the relationship between a name and its referent is simply arbitrary and based on social convention.  The other interlocutor, Cratylus, rejects this view and argues that a name has a natural, nonconventional connection with its referent (e.g., perhaps due to certain qualities that the name shares with its referent).

This discussion continued throughout the Greco-Roman world, with many philosophers developing accounts that incorporate features of both Hermogenes and Cratylus (see Frede and Inwood, 2005).  In the seventeenth century, John Locke, by contrast, took a more exclusive approach, arguing for a thoroughly Hermogenean view.  In An Essay Concerning Human Understanding, Locke writes, “[W]ords being sounds, can produce in us no other simple ideas than of those very sounds; nor excite any in us, but by that voluntary connexion which is known to be between them and those simple ideas which common use has made them the signs of” (III.4.11).

Locke’s exclusively Hermogenean view is also shared by many philosophers today.  Michael Rescorla, in his (2011) Stanford Encyclopedia article on convention, writes, “Nowadays, virtually all philosophers side with Hermogenes.  Barring a few possible exceptions such as onomatopoeia, the association between a word and its referent is not grounded in the intrinsic nature of either the word or the referent.  Rather, the association is arbitrary.”

In opposition to the exclusively Hermogenean view, a new paper by Maia Pujara, Richard Wolf, Michael Koenigs, and myself provides evidence that phonemes (the human speech sounds that constitute words) have an inherent, non-arbitrary emotional quality.  Moreover, our data suggest that the perceived emotional valence of certain phoneme combinations depends on a specific acoustic feature—namely, the dynamic shifts within the phonemes’ first two frequency components.

Our study is predicated on a number of previous observations.  First, during the production of human phonemes, air within the vocal tract vibrates at several different frequencies simultaneously; each of these frequencies is known as a “formant frequency.”  Second, humans’ ability to auditorily differentiate between phonemes is largely mediated by the relative positions and transition patterns of the first two formant frequencies (F1 and F2).  Third, numerous nonhuman animal species lower their vocal tracts (thereby lowering the frequencies of their vocalizations) in order to appear larger and more threatening to antagonists or competitors.

Thus, we predicted that strings of phonemes characterized by downward shifts in F1/F2 formants (perhaps evolutionarily rooted in antagonistic/competitive behavior) would be associated with negative emotion, whereas strings of human phonemes characterized by upward shifts in F1/F2 formants (perhaps evolutionarily rooted in conciliative/submissive behavior) would be associated with positive emotion.

To test this prediction, we adapted a two-alternative forced-choice test in which subjects were instructed to match strings of phonemes (comprising nonsense words) with pictures.  The non-words were constructed so as to exhibit either an overall upward or downward shift in F1/F2 frequencies.  The pictures were selected on the basis of eliciting positive or negative emotion.  During the test, consisting of 20 experimental trials, subjects saw a pair of non-words (one with upward F1/F2 shifts, the other with downward F1/F2 shifts), and a pair of pictures (one positive, one negative).  Subjects were instructed to mentally sound out each word, and then match each word to one of the two pictures.  As predicted, subjects reliably paired the downward F1/F2 shift non-words with the negative images and the upward F1/F2 shift non-words with the positive images (see Figure 1 below).  A detailed description of the data and methods can be found in the paper.

These results seem to suggest that (contrary to the exclusively Hermogenean view popular amongst many philosophers) certain strings of phonemes have a non-arbitrary emotional quality, and, moreover, that the emotional quality can be predicted on the basis of specific acoustic features.

 

 

Figure 1. A, B: Spectrograms illustrating the first four formants (F1-F4) of the nonsense words “bupaba” and “dugada” as obtained with Praat software (version 5.1.20).  When distinguishing between these words, the most salient formant transitions are the F2 transitions from consonants to vowels (outlined in red), which move slightly upward for “bupaba” and downward for “dugada”.   C: Example of a trial from the visual task.  Subjects pressed a vertical arrow button to match the non-words/pictures vertically, or a horizontal arrow button to match the non-words/pictures horizontally.  D: Visual task data (n=32 adult subjects, 15 males, mean age 35.5±15.2).  The proportion of individuals selecting a majority of predicted responses (i.e., on more than 10 out of the 20 trials) was significantly greater than expected by chance (Yates’ χ²= 23.8; p=0.000001).

 

 

Comments

[John Turri]

This is a super-interesting study, Blake!

Looking quickly at your methods section, I couldn't tell how you recruited participants or what their demographics were (I just picked up "two different sets of healthy adult participants). I expect that your results will not be limited to a specific culture. But it would make your case stronger if participants were from different cultures. Otherwise you're open to the worry that any observed differences are an artifact of cultural convention. Like I said, I'm not inclined to worry about this much, but it automatically occurred to me as something a friendly-but-skeptical adversary might bring up. Or am I overlooking something?

If you're looking to launch this research program into a more traditional philosophically-oriented venue, you might be interested to know that Thomas Reid disagreed with Locke and took a more inclusive, hybrid Hermogenean/Cratylan view. Reid included "modulations of the voice" in what he called the "natural language" of humanity. Elements of the natural language "have a meaning which every man understands by the principles of his nature" and "prior to all compact or agreement" (See Reid's _Inquiry_, ch. 4; also http://john.turri.org/research/Reid_natural_language.pdf .) Grice's "natural meaning" points in the Cratylan direction too. More generally, I bet a number of people have some sympathy for hybrid views, and this work of yours could help reinvigorate the discussion.

[Blake Myers]

Hey John, thanks so much for the helpful comments!

Regarding your first point, participants were recruited from the University of Wisconsin-Madison campus area, as well as the Madison community at large. Thus, we had a mixed sample of UW-Madison undergraduate and graduate students along with some adults from the Madison area. In order to qualify for the study, all participants had to be native English speakers. That said, I think you’re exactly right that our case would be stronger if it were shown that the effect also holds across non-English speaking cultures. That seems to be one of the next key things to explore.

I wasn’t aware of Reid’s work on this general topic. That’s great to know. I just downloaded your paper. It looks fascinating! I look forward to reading it. I’ve also come across some literature indicating that Leibniz was critical of Locke, as well, and that he argued for something like a hybrid Hermogenean/Cratylan view. Timothy Baxter (The Cratylus: Plato's Critique of Naming, 1992, p. 67), for instance, quotes the following passages from Leibniz (the first passage is from Leibniz’s Aarsleff, and the second is from Leibniz’s New Essays on Human Understanding):

“[Languages] have a certain natural origin, from the agreement of sounds with the disposition of the mind [or ‘affects’], which the appearance of things excite in the mind. And this origin I believe occurs not merely in the primal language, but also in languages that have grown up later in part from the primal language and in part from the new usage of men dispersed over the globe.”

“[I could] mention any number of similar terms which prove that there is something natural in the origin of words—something that reveals a relationship between things and the sounds and motions of the vocal organs.”

[Dustin Locke]

Interesting study. As I understand it, the Lockean/exclusively Hermogenean view is that the phonetic qualities of a word play no constitutive role in determining its referent. This would seem to be compatible with the thesis that some sounds have natural emotional associations and thus that some words might even have been assigned (by convention) the referent they have because of their phonetic qualities. In other words, a convention of using t to refer to x might have arisen because t has certain phonetic qualities. The Lockean thesis is simply that it is this convention (regardless of its origins) that constitutively determines reference. And so the Lockean thesis seems quite compatible with the results of your study. Or am I missing something?

[Blake Myers]

Hey Dustin, thanks! That’s an interesting suggestion. After reading your comments, I had to go back and reconsider some passages in Locke. It still seems to me that Locke held to view that’s a bit stronger and more exclusive than the one you propose. In An Essay Concerning Human Understanding, he argues that the signification of words is “perfectly arbitrary, not the consequence of a natural connexion” (III.2.8). This seems to preclude the possibility that words might be assigned a referent “because of their phonetic qualities.”

Also consider the following two passages in Locke's Essay:

“[W]ords…came to be made use of by men as the signs of their ideas; not by any natural connexion that there is between particular articulate sounds and certain ideas…but by a voluntary imposition, whereby such a word is made arbitrarily the mark of such an idea” (III.2.1).

“[S]ounds have no natural connexion with our ideas, but have all their signification from the arbitrary imposition of men” (III.9.4).

Locke’s continued persistence here that a word’s sound is *arbitrarily* related to its referent, and his contention that there is *no* natural connection between a word’s sound and its referent, seem to suggest that he would have resisted the claim that “a convention of using t to refer to x might have arisen because t has certain phonetic qualities.”

Now, even if we suppose that Locke did in fact hold to a weaker Hermogenean view as you suggest, the results of our study still seem to provide him with a challenge. In particular, if I understand your proposal, this weaker Hermogenean view maintains that while phonetic qualities may play some type of auxiliary role in connecting a word to its referent, they are never sufficient to do so; for it is convention that plays the constitutive (and thus necessary) role in connecting a word to its referent. In contrast to this, however, the results from our study suggest that certain phonetic qualities—apart from any social conventions—are indeed sufficient for connecting a word to a referent (since the words in our study were all artificially constructed and thus had no conventional associations, and each word pair was such that the two words differed only with respect to phonetic qualities).

[Dustin Locke]

Thanks for the reply. I don't see how your results show that "certain phonetic qualities... are indeed sufficient for connecting a word to its referent." The non-words in your study didn't have referents, did they? Subjects in your study "matched" non-words with pictures. You don't take this to imply that the non-words have certain referents, do you?

[Dustin Locke]

I also think it would be helpful to distinguish the view prevalent among contemporary philosophers---which I was referring to as the "Lockean/exclusively Hermogenean view"---from whatever view the historical Locke might have ascribed to. But those are some surprising passages you've found in Locke. I wonder if charity dictates we should read them in a weaker way than they might at first appear. It's strange to think that Locke might have been unaware that certain sounds had natural emotional associations. But maybe he was. In any case, my purpose above was to defend the view prevalent amongst contemporary philosophers---not the view of the historical Locke.

[Anibal]

In contemporary research, coming from neuroscience, there are many authors arguing for a Cratylian view of names sharing the essence with its referents.

I think for example in Michael Arbib and G. Rizzolati.

They believe in words as phonological actions where semantics is provided by the linkage to neural systems supporting perceptual and motor shemas.

Reference:
"Language within our grasp": http://www.ncbi.nlm.nih.gov/pubmed/9610880

[Brent Strickland]

Hi Blake. Thanks for an interesting study. I think this is an excellent example of evolutionary psychology at because instead of using evolutionary theory as post-hoc justification for some effect, it was a necessary component in creating your experimental hypotheses in the first place. I don't think there is any way you would have run this study without evolutionary theory as a motivating factor. So it's a good example of how evolutionary psychology should work.

As for Dustin's comments on your paper's relevance to the "Hermogenean view" of the connection between words and referents, I think his comments are spot on. Showing that people have slight preferences for associating certain speech sounds of non-words with certain emotional qualities isn't decisive one way or the other on the ultimate connection between actual words and their meanings. However, I think there are ways you could address this issue. One compelling piece of evidence would be to do a huge cross linguistic analysis in which you would look at actual words from many different languages, and show that in virtually all of them, there is a tendency for word meanings to follow the pattern you paper suggests they would (e.g. with F1/F2 formants being more likely to have a negative valence than F3/F4 formants).

[Dustin Locke]

Thanks for the follow-up comment, Brent. But I don't see how the results you imagine of the cross linguistic study would be any problem for the Hermogenean view. Those results would fit nicely with the hypothesis that there is a causal link between the phonetic quality of words and the conventions that arise concerning their usage. And as I've indicated, that hypothesis is entirely consistent with the Hermogenean view. The Hermogenean view Is a view about what constitutes the reference relation. It is not a view about what Is involved in the causal history of why words refer to what they refer to.

[Blake Myers]

Hi Dustin, thanks for the helpful follow-up remarks. Yes, I agree that those statements by Locke are surprisingly strong. One way I’ve thought about interpreting him is as follows. Perhaps he would say that the essential phonetic components of a word’s sound (e.g., the F1/F2 frequency positions used to differentiate “tale” from “kale”) are never naturally connected with a meaning. And perhaps he would still acknowledge that there are other acoustic features—i.e., ones that are irrelevant to the phonetic qualities that compose a word (prosody, for instance)—that are in fact naturally connected to a meaning.

There seems to be a significant amount of evidence in *support* of this latter claim; i.e., the claim that phoneme-irrelevant features, such as prosody, are naturally connected to certain meanings.

And I take the results of our study as providing (preliminary) evidence *against* the former claim; i.e., against the claim that the phonetic components of a word’s sound are never naturally connected with a meaning. In particular, given that subjects were presented with several pairs of images and consistently matched the words characterized by upward F1/F2 transitions with positive images and those characterized by downward F1/F2 transitions with negative images, I take this as (preliminary) evidence that certain phoneme-relevant features are naturally connected (even if only weakly) to certain emotional qualities.

[Brent Strickland]

Hey Dustin. I probably wasn't as clear as I could have been. The principle question I was concerned with was whether or not the connection between phonology and meaning is arbitrary or if there is a "natural" causal link between the two. I was loosely using the term "Hermogenean view" to refer to a potential association between phonology and meaning which is arbitrarily assigned. If that's not the right term, then maybe it's better to call it the "arbitrary" view? (I actually don't care much one way or the other about the terminology since that seems to be a definitional issue)

The main point was that I don't think Blake's data directly address whether the "arbitrary" view is correct. Hopefully my suggested study will be helpful.

[Blake Myers]

Hey Brent! Yes, you’re exactly right that evolutionary theory is what shaped the overall hypothesis and design of the study from the start.

Regarding your comment that our study “isn't decisive one way or the other on the ultimate connection between actual words and their meanings,” I certainly agree. I also readily acknowledge that the connection between *actual words* and meaning is probably driven largely (if not mostly) by convention.

My main point regarding our data is that it seems there are cases in which reference can get off the ground without the help of convention. This interests me largely because of how it might relate to the early evolution of language. That said, I wholly agree with you that the main limitation of our study is that all participants were English-speaking adults. And I think a large-scale study, like the one you suggest, is a great way to address this issue.

[Blake Myers]

Anibal, thanks for the helpful pointer! I look forward to looking at that paper. Yes, it does seem that a number of people from psychology and neuroscience are sympathetic to a Cratylan-like view (or perhaps a hybrid Hermogenean/Cratylan view). In our paper, we cite several previous studies suggesting a psychological association between sounds and meanings.

[Dustin Locke]

Let's simply grant that "certain phoneme-relevant features are naturally connected (even if only weakly) to certain emotional qualities." Are you saying that this entails that *reference* can occur without the relevant convention?

"x is naturally associated with y" is one thing. "x refers to y" is quite another.

[Blake Myers]

Hi Dustin, thanks.

I think my statement that you quote suggests something a bit more than "x is naturally associated with y." Or rather, I think it’s important to specify the nature of x and y and say something like:

“Phonetic quality x is naturally associated with meaningful content y.”

This statement seems (to me at least) to suggest that x refers to y (though perhaps only on a loose sense of reference). I’ll grant that on a more restricted view of reference, it may still be an open question as to whether x refers to y.

[Dustin Locke]

Subjects matched non-words with certain phonetic qualities to pictures that evoked certain emotions. Does it follow from this that those emotions are the "meaningful content" of those non-words (or their phonetic qualities)? I doubt it. If you ask subjects to match uniforms with jobs, you'll find certain correlations. This certainly does not mean that those uniforms have those jobs as their "meaningful contents".

(In other words, you've shifted the discussion from "reference" to "meaningful content", but my objection is still the same.)

[Blake Myers]

Hey Dustin, it seems that the debate here could go one of two ways:

(1) First, one could start with the claim that linguistic reference is an association—perhaps natural, perhaps conventional—between a string of phonemes and some entity/quality. Then, the real focus of the debate is whether, and to what extent, there are natural (or alternatively: conventional) associations between strings of phonemes and particular entities/qualities.

As I understand it, this was a key part of the debate among many Greco-Roman philosophers (see Frede and Inwood 2005, linked in my post above), as well as between Locke and Leibniz.

(2) Secondly, one could start with the claim that linguistic reference is a socially governed relation between a string of phonemes and an entity/property. Then, the Hermogenean view is, of course, immune to any empirical scrutiny.

I’m more interested in the Greco-Roman, Lockean-Leibnizian debate, along the lines of (1). And it seems to be a debate worth reviving; if not for its use in present philosophical discussions on reference, then for its potential relevance to questions concerning the early evolution of language (questions such as, what was the early adaptive value of categorical phonetic perception, and why is categorical phonetic perception found among certain nonhuman species that lack any clear complex system of speech; and why did certain acoustic features, rather than others, take on the psychological associations needed for speech).

[Dustin Locke]

That seems mostly* right to me. I think your study and others like it are important for understanding how it is that phonemes come to be associated with entities/qualities.

My only concern was that you were taking this study to show something "contrary to the exclusively Hermogenean view popular amongst many philosophers".

*I say 'mostly' because you mistakenly suggest that the only alternative conception of the debate is one on which the Hermogenean view is true by definition.

[Blake Myers]

Dustin, thanks. That makes sense. However, it makes me wonder what an alternative conception of the debate would look like that didn’t make the Hermogenean view true by definition. Specifically, what types of facts would count as evidence for or against the Hermogenean view?

[Dustin Locke]

Thanks, Blake. I fear we are teetering on the edge of a much larger philosophical debate. You seem to be suggesting that if we cannot "specify facts that would count as evidence for/against a view", then that view must be true/false by definition (or else nonsense). This suggestion is commonly known as the "verification principle", and it is the core of logical positivism (my apologies if you know all of this already).

As the verification principle is now widely rejected, I'll give you the chance to clarify whether you meant to be presupposing it in your question before I respond.