Does language affect my survey responses?

Over 60% of the global population lives in more than one language. How do you accommodate for that in your surveys?

A few weeks ago, I agreed to help at a Careers Fair about the benefits of studying languages for use in the workplace. The format was group speed dating. A group of teenagers rotated around the hall where the fair was taking place, asking questions of a number of different people who use languages at work. 

One group came to me to ask why they should bother learning a language when so many people speak English. 

“About 1.1 billion people of the world’s 7.7 billion people speak English. That’s not many. But sure, at least you can speak to them,” I answered. “Me, I also speak French (280 million), German (132 million) and Welsh (about 1 million). So, I can speak and do business with around half a billion more people than you can.” 

They got it instantly.

We’re a Welsh company. Both me and my co-founder Steve live at least half of our lives in the Welsh language. We speak five languages between us and doopoll is available in four languages (English, Welsh, French and Spanish)

And we’re not alone.  

In fact, 60-75 percent of the world’s population speaks at least two languages. 

If you only speak one, you’re in a relatively tiny minority of people. (But it’s never too late to start)

But living bilingually is weird too. It has all kinds of crazy effects on your day to day. I’m 100% grateful for the weirdness, but still, it’s definitely worth considering how you should go about accommodating the two thirds of global population who live in more than one language.

Let’s take a look at how this applies to surveys (because, you know, we’re a company that makes an amazing survey tool)

How does language affect decision making? 

A study which is outlined by Albert Costa in his marvellous book The Bilingual Brain, demonstrates just how important the language a respondent uses can be in the choices they make. 

The study offered a couple of situations to different groups of participants. They were told that in a shop there were two items they wanted to buy. One for £125 and the other for £15. But then, there’s a twist:

  1. A salesman tells you that the £15 item you want to buy is available for £10 at their other shop which is a 20 minute drive away
  2. A salesman tells you that the £125 item you want to buy is available for £120 at their other shop, which is a 20 minute drive away

Participants are asked what they would do. 

Despite both these options being the same in cost and in discount, would you make the trip in one situation or the other? 

What Costa and his colleagues discovered was that when the scenario was presented in the participants’ first language, 40% of them said they would make a 20 minute trip to get a discount on the lower value item. 

Only 10% of them would do the same for the higher value item when asked in their first language. 

This difference was reduced by half when the problem was presented in their foreign language (English). 

An example of a bilingual survey on doopoll

When presented the scenario in a foreign language, it seems as though participants made a much more thoughtful evaluation of the options when they had to consider the problem with the added difficulty of a second language added in. 

This attitude to risk is reflected by empirical evidence from many other studies. It’s also commonplace to hear people who have considered their bi- or mutlilingualism talk anecdotally about their attitudes to risk in different languages. 

Expert advice

How do you do a bilingual survey on doopoll?

There’s a few ways that you can do surveys in two languages on doopoll. Here are the two most common: 

1. Duplicated surveys

Using this approach, you’ll create the survey in one language and then duplicate it in another language. 

Pros

  • Easy to do
  • Simple to get it translated
  • Separate data for each language

Cons 

  • You have two separate survey links to share
  • The results are separated out

I don’t recommend this approach but it’s sometimes the one that people like to take if they’re producing something that absolutely needs to differentiate between languages. Most of the time, it’s better to appreciate that bi- or multilinguals switch between languages many times a day and opt for the following instead: 

2. Parallel translation

In this method, you add all questions and options like this: “Language 1 / Language 2”. You keep the same survey link. 

Pros

  • Keep all the survey data on the same survey
  • Super easy to do
  • Display the results on a single screen - perfect for event usage 
  • Shows ‘parity of esteem’ (you value both languages equally)

Cons

  • Question labels can become a bit long for mobile devices – but this can be avoided with careful thought into what you need to ask
  • Our product currently supports one language at a time for buttons like ‘Next question’

Overall, this is the method that most people on doopoll use to do a bilingual survey. In countries where there are two official languages (like in Wales), people are extremely au fait with this method. 

Tip: You can use a qualifying question to find out which language is preferable to the respondent

For example, Boaz Keysar a psychology professor at the University of Chicago said on this episode of the Freakonomics podcast:

“I realise that I am different when I’m using almost any language other than Hebrew. Things like making investments. I’m much more likely to invest in the stock market when I do it in English. If I think about it in Hebrew I feel that I become more conservative.”

Personally, I’m a lot funnier in French. I’m more considered in German. In Welsh, I’m friendlier. 

But what could the impact be on surveys? 

“Be careful when comparing surveys that are carried out in languages other than the participants’ first language,” says Costa. “Think about, for example, job satisfaction surveys among migrant workers.”

This makes sense from a number of perspectives. 

As well as the computational power necessary for even a ‘fluent’ bilingual to make a choice in a foreign language – usually, you’ll still need to question whether you’ve got the full sense of the question being asked, for example – there’s also an impact caused by personality differences in each language. 

Things to think about when you’re creating a survey in more than one language

Here’s a list of things we think are important about setting up your survey in more than one language

  1. Do the translations you’ve had done accurately reflect the questions that you want to ask? – Words, emotions, cultural significance of ideas. These things all change language to language. Check with a native speaker before you send it out.
  2. Have you already planned how you might look at the data? – Will you factor in the language a person uses into the results. Remember Costa’s advice about employee engagement surveys for migrant workers above. 
  3. Will you present the responses bilingually too? – You should have a policy for presenting the results of your survey in any case, but considering how you will do this for people who speak different languages is useful.