Research Methodologies

TechCrunch's flawed reportage about Wikipedia and Tumblr

2011-10-01T11:28:00.000-07:00

Recently, TechCrunch did a blog post that Tumblr has surpassed Wikipedia in page views per month with 6.5 billion page views compared to Wikimedia foundation sites's (Wikipedia and its sister sites) only 5.6 billion page views per month. Their reportage is based on data from ComScore, if TechCrunch had double-checked their source, they would have found that Wikimedia family sites, actually have about 14.6 billion page views. Unlike most other big websites that like to keep their analytic data secret, WMF (my employer) makes all the analytic data that we collect public. If you are a data nerd, you might want to spend few minutes checking stats.wikimedia.org. A side note we are committed to the privacy of our readers, and do minimum tracking. For example, we are one of the few websites that doesn't deploy cookies to track our users. We also use ComScore data to track our unique visitors since we don't deploy cookies on our website, and therefore have no way of figuring out our unique visitors.

Now, going back to the discrepancy between page views. There is obviously a big difference between 14.6 and 5.6 billion. But 14.6 billion is a more reliable stat since it is based on actuals, whereas ComScore data is based on a ComScore panel that has opted in to have their Internet surfing behavior tracked.

As for TechCrunch they should provide a clarification to their story. My colleague Amit Kapoor did point towards the discrepancy in data in the comments section.

Editor Survey Report

2011-08-29T19:40:00.000-07:00

Since the beginning of this year, I have been working on the survey of Wikipedia editors. This is the first time WMF (my employer) has conducted a survey of Wikipedia editors internally. We plan to continue doing surveys of Wikipedia editors regularly to keep a tab on the health of the community. Finally today, we released the final report from the editor survey. The report can be downloaded as a PDF or read in a Wiki format.http://commons.wikimedia.org/wiki/File:Editor_Survey_Report_-_April_2011.pdf

BBM, London riots, teenagers in India

2011-08-10T18:25:00.000-07:00

Much has been written about the popularity of BBM (Blackberry Messaging Service) and its role in helping flash mobs organize themselves during the London riots. Blackberry has been primarily geared towards the enterprise market in the US. But clearly Blackberry and its messaging services has resonated with an entire different set of users that Blackberry engineers and designers didn't have in mind. The use of BBM also speaks to improvisational use of emerging technologies for unintended uses. Messaging is great not only for enterprise clients, but also youths organizing themselves in a revolution or a riot.

When I conducted research earlier in the year in India to understand how mobile users in India use the mobile web, I was surprised to find that Blackberry had captivated teenage girl market in India. I interviewed several girls who had upgraded to a Blackberry only to use BBM. As one of them put clearly, "all my friends are chatting on BBM, I have to be there." She told me that all the girls in her high school class had Blackberrys as they wanted to be part of the gossip. Blackberry has also seen a growth outside of the enterprise market in India because an unlocked Blackberry is available for about Rs 6000 or $150. It does make a good birthday or anniversary gift. When I visited a mobile phone shop at Khan Market in Delhi, I spotted a guy buying a Blackberry as a gift for his girlfriend.

But the future still looks bleak for Blackberry as it continues to lose its market share rapidly. Both Facebook and Apple have launched or plan to launch BBM-styled messaging service. This would make Blackberry less attractive certainly for the teenage girls I met in India: they would rather use Facebook for mass messaging since all their friends are on Facebook already.

New Mobile Readers Survey

2011-07-22T11:44:00.000-07:00

At Wikipedia, we have made it our strategic priority to increase mobile page views to 2 billion in the next one year from about 760 million currently. In order to achieve our objective, we are redesigning our mobile site to provide enhanced reading experience and introduce some editing functionalities on the mobile. Me and my wonderful colleague Parul Vora have conducted user experience research in India and Brazil to understand needs of our current and potential mobile users. We are also in the process of launching our mobile survey that we are planning to conduct in 9 languages.
At Wikipedia we have an inclusive culture and encourage participation so I have shared the draft of the mobile survey on our website to elicit feedback on making the questionnaire better. If you are interested, you can take a look at the questionnaire and provide us feedback.

Brazil mobile phone market: Two markets in one country

2011-07-02T22:14:00.000-07:00

Pelorhino_ historial center in Salvador

Favela in Salvador

Brazil provides an interesting case study of the mobile phone industry. Like most other countries both in the global south and north, the Brazilian market is dominated by a handful of big players: Oie, Vivo, Claro and Tim. But when it comes to cost of data usage, Brazil almost has two markets: the affluent South and poor North. And the biggest difference between the two markets is cost. While Brazilians living in Sao Paulo or Rio pay almost $80 - $100 for data for the mobile, north Brazilians pay $20 or less in a month for mobile phone data. Why such a remarkable difference in the cost? Simple, people in the North are much poorer than those in the South, and cell phone operators have adopted different strategies for the two regions. The operators have inflated costs for data in Sao Paulo, Rio and other cities in South because they know people can afford to pay high prices. But in Salvador they offer cheap plans that provide unlimited data access for a day for about R$ .50 or 30 cents. The plans are pay as you go plans so people can access data if they want and not use it when they don't need it. Everyone I interviewed in Salvador was using these plans and most operators offered similar plans. Most people did not use data pans everyday, they used it only if they needed it.

When I was recently in Salvador (which is capital of the Northern state of Bahia) everyone I met took it upon themselves to educate me about the economic difference between South and North Brazil. Having visited Sao Paulo last year, I could clearly see the difference between the two places. As a visitor, you can see that Salvador and its population is much poorer compared to Sao Paulo. The roads are less nicer, the hotels although less expensive appear to be dated (my hotel seemed to be a set of a movie from the 1970s), restaurants don't look as upscale as what you would find in Sao Paulo or Rio, and you can't spot helicopter taxis in the sky that are a fixture in Sao Paulo used by the rich to beat the crazy traffic.

I will do another post about other interesting things that I saw/researched in Brazil.

Indian & Chinese phone manufacturers are here

2011-05-30T22:34:00.000-07:00

Chinese phone "iBall" with extra battery pack

On my recent trip to India, I bought mobile testing devices. The Wikimedia engineering team (Tomasz & Patrick) is going to simulate the Wikipedia experience in India on these phones. I found the Indian phone marketplace flooded with new choices. There are obviously usual suspects like Nokia, Sony Ericsson and Samsung. Although, Nokia is losing market share in India rapidly, it continues to be popular among most mobile users. But the devices that I found really interesting were Chinese and Indian knock-off phones like iball (see the picture). Chinese phones started becoming popular and ubiquitous in India about 3-4 years back. The Chinese phones took off because they were cheap and many of phones came with 4-5 external speakers enabling them to double as boom boxes. But Chinese phones have ever increasing competition for the bottom tier of the market from Indian manufacturers. This year I saw several folks (especially all my taxi drivers) with phones from Indian manufacturers like Micromax and Karbonn.

These mobiles have several additional features apart from low price point that make them attractive in the Indian market (a) Super loud: Indians like their phones to be loud: loud ringtones, loud music -- boombox style. These phones do exceptionally well in the loudness functionality. Many phones like Nokia Express Music are branded and sold as music phones since people like to listen to their favorite Bollywood songs on their phones. On my last visit, I remember speaking with a villager who had migrated from Bihar to Delhi who had spent almost his entire month salary on a phone that provided him the functionality to listen to music on external speakers. (b) Dual sim card: Many Indians have 2-3 phone numbers with different service providers to avail of free phone-to-phone calling within the network. Although India has one of the lowest prices for voice calls, people prefer to have more than one number so they don’t have to pay for outgoing phone calls. Having a phone with a dual sim card means that you are free of the hassle of carrying two phones. (3) Extra battery power: Many phones come with extra batteries, and extra battery in a place like India is always useful since it is standard to have power cuts. Who knows when you might need the extra juice for the phone since there is always a chance that you might not be able to charge your phone.

I believe that these low-end and knock off phones will squeeze the market for phone manufacturers like Nokia that have made profits on low cost, but high volume sales, and they will also take a slice from the second hand mobile that has been booming in India, Africa and elsewhere.

Welcome to India, Madam

2011-05-08T19:25:00.000-07:00

Auto-rickshaw ride

When I boarded the plane for New Delhi in Munich, a copy of The Times of India, complete with a copy of Delhi Times, beckoned me home. I was a member of the original team that launched Delhi Times when I worked in TOI in the 1990s right after college. I leafed through the paper to see familiar bylines from some of my old friends and colleagues. It was deja vu & for a brief second I missed seeing my byline in the newspaper. I instantly felt home as memories tumbled, although I have not lived in Delhi for almost 13 years.

Later, the flight attendant walked up to me and said aloud: “Ms Pande, you ordered a special meal.” This caught the attention of the passenger who was sitting in front of me in the airplane. He turned around and inquired, “ Are you a Pande, I am a Tiwari.” Code word for: we are both Brahmins and hail from the Hindi heartland. When he realized that I was traveling alone, he even offered to help me through customs and immigration since he was some sort of VVIP (although everyone likes to believe they are one) and was being picked up at the plane (no queue to go through immigration), and he was not lying, I did see someone carrying a placard with his name in the restricted zone. I think he made the offer due to our caste allegiance. My thought: Welcome to India!! A trip to India is not complete without some red tapeism and caste allegiance. I did not take him up on the offer and decided to go through customs like a normal and weary traveller.

I will do some more blog posts later on the research that I conducted in India on how users access Wikipedia on their mobile.

Wikipedia and Internet switch-off in Egypt

2011-02-05T09:30:00.000-08:00

My colleague at Wikipedia, Erik Zachte who is in-charge of web analytics on Wikipedia has this great graphic on his personal blog that shows what happened to Arabic and English Wikipedia page views from Egypt after the government switched off Internet in the country. As you can see in the graph page views from Egypt almost disappeared after Internet was switched off on Jan 28, 1AM, Egypt time, but there were some folks still accessing Wikipedia even though Internet did not come back on till Feb 2.

The Case for Facebook Zero and similar applications

2011-02-01T10:47:00.000-08:00

Early last year Facebook launched Facebook Zero, a stripped down version of mobile website at no data cost primarily to attract new users in the Global South. Facebook Zero unlike the standard version of Facebook mobile does not serve multi-media content like images and videos (which can clog data pipes) making it possible for carriers to make it available for free. Multi-media are stored at the backend, and if a user decides to access images then standard charges apply. Facebook Zero was launched in about 45 countries in the Global South with the help of strategic partnerships with 50 carriers allowing users to access Facebook Zero at no data costs. Facebook has been working at extending partnerships with more carriers even in developed countries like UK and Austria.

Although smart phones and 3G networks are becoming ubiquitous across the globe, I believe that there is a strong business case for services like Facebook Zero because a sizeable segment of mobile users will not have access to faster networks (3G and beyond) or afford smart phones (despite the prices dropping significantly). For further details, please see below:

Despite the rapid deployment of 3G networks across the world, which provide faster connectivity, 2G networks will still dominate in the global South in the coming years. For example, about 3 billion people in Asia Pacific will be still using 2G networks in 2014. In comparison, 3G networks will cover only over 1 billion people in Asia Pacific. We see a similar story in MENA and S&C America. There will be more people in these regions who will be still using 2G networks v/s 3G networks.

2G will dominate in 2014

Another barrier to the adoption of 3G is the cost of 3G devices. 3G devices continue to be extremely expensive in the global South. With GDP/capita as a base we find that 3G devices are the most expensive in Africa. A 3G device in Africa is about 6.9 percent of GDP/capita. Unlike the US where carriers subsidize mobile phones, most users in the global South have to buy the mobile phone at full price themselves. This is the number one reason why iPhone, which was available at $500 did not take off in India despite India being one of the hottest mobile phone markets. Another interesting point to note here is that the cost of a 3G device as percent of GDP/capita is inversely correlated with 3G penetration.

3G devices are unaffordable in many developing countries

What this is telling us is that in the coming 2-3 years, the mobile phone market will not uniform, but will continue to be fragmented into two. There will be two kinds of users: those who have 3G devices and primarily use 3G networks. The other users will be those who will still be using older, 2G devices that run on 2G networks. These numbers make a strong business case for investing in lightweight versions of mobile websites, and Facebook is leading by example.

In addition, the mobile web is forecast to overtake the desktop web in 2014 making it imperative for companies to have a strong mobile presence.

What's wrong with NPS?

2011-01-31T11:17:00.000-08:00

Net Promoter Score (NPS) is a popular measure of customer loyalty that many companies like EBay, PayPal and Amazon swear by to measure customer loyalty. Bonuses in many companies for top executives are also tied to this single metric, making it even more important come bonus season. Here is an explanation of Net Promoter score by Fred Reichheld from Bain & company who made a business case for instituting NPS for measuring customer loyalty in his book, The Ultimate Question.

Net Promoter Score (NPS) is based on the fundamental perspective that every company’s customers can be divided into three categories. Promoters are loyal enthusiasts who keep buying from a company and urge their friends to do the same. Passives are satisfied but unenthusiastic customers who can be easily wooed by the competition. And detractors are unhappy customers trapped in a bad relationship.

NPS is measured on a scale of 0-10, based on likelihood of recommendation: “How likely is that you would recommend company X to a friend or colleague?” Customers who provide a score of 9-10 are categorized as promoters, those who give a score of 8-7 are dubbed passive and those who give a score of six and below (6-0) are classified as detractors. Any company’s NPS score is equal to the percentage of customers who are promoters subtracted by percent of customers who are detractors. NPS=P-D

Here are three problems that I have with NPS:

Statistically speaking, it is highly biased towards detractors since anyone who scores between 0 and 6 is a detractor. Among the three categories, detractors have the widest range (0-6), while both promoters and passive have a range of two with promoters ranging from 9-10 and passive 8-7. In any introductory statistics or survey research class, one of the first things that they teach you is to have equal ranges while designing choices to a survey question, and to make room for a neutral response also. This ensures that respondents to the survey have an equal chance of picking each point of view (very positive, positive, neutral, negative, very negative). In other words, the probability of someone choosing a neutral rating is equal to someone choosing a very negative rating. This is not rocket science; it is just survey design 101. The problem with NPS is that it increases the probability of someone being classified as a passive compared to a promoter & even a detractor. I am sure proponents of NPS might argue that the bias is by design, but it compounds the second issue that I have with NPS discussed below.
Different societies have different ways of evaluating quantitative metrics. When I did a cross-national studying looking at youth in India, China and the US, I found that young people in India and China are more stingy with their scoring: they are less likely to give higher ratings, and this extends to their own self-assessment. So someone in China is less likely to give a score of 10/10 then someone in US. Modesty is considered a great virtue in these societies and it extends to answering questions on surveys. There is the possibility that if someone in India gives a score of 8, according to him/her the service is great, but NPS will categorize them as passives.
This brings me to my third point. It is extremely important to label scales. I have seen most quantitative market researchers ask respondents to choose one option on 0-10 scale, and when they want to look at top responses, they just look at top two boxes and classify that as extremely/very important. Honestly, in my mind 9 classifies as extremely good (yes, I did grow up in India and am less generous while providing a rating), whereas for someone else 10 will be extremely good. I strongly believe that it is important to label scales to eliminate response bias and ensure that respondents are interpreting the question in the same way.

What is my advice to companies?

If you want to continue using NPS in your organization, rescale the metric to ensure that it is not negatively biased, provide labels to ensure that all the respondents are on the same footing and lastly segment your customers to account for regional and other disparities like customer engagement.

Street ethnography from Brazil

2010-09-01T14:16:00.000-07:00

This blog post has been in the works for over six months. Earlier this year, I went to Brazil for work, it was a very short visit. Brazil has been on the list of places that I had always wanted to visit. So I was really happy to visit the country even if it was for a very brief visit. I spent only 4 nights in Sao Paulo, but I was able to find sometime to discover the city. It was my first time visiting a country where I did not speak the language (Europe does not count). I can speak a few things in Portuguese now, I only knew how to say thank you in Portuguese: O brigada. As a qualitative researcher who has traveled globally to conduct research, it was a very interesting trip. Here are some quick glimpses of Sao Paulo seen through the eyes of an Indian who has lived in the US most of her adult life.

Moto-boys

On the streets of Sao Paulo you can see men on motorcycles whizzing past. It is the fastest way to beat the insane traffic in Sao Paulo. These drivers often drive between lanes squeezing past cars. They are Brazilian version of errand boys, but on motorcycle. Thus popularly called moto-boys. Brazilians pay them money to take packages from one place to another. They are the wheels that keep small businesses running in Brazil. So imagine instead of sending a package to FedEx for shipping you can call a moto-boy to take your package from one corner of a city of 20 million people to another.

Helicopters

Another thing that struck me in Brazil was the number of helicopters that you can spot in the sky at one time. While some of them are privately owned, others are helicopter taxis. Obviously, they are a privilege that only the rich can afford. They are the preferred mode of travel of very rich Brazilians (and there are many) to beat the insane rush hour Sao Paulo traffic. When we were there we heard the story that CEO of one of the big Brazilian companies on earth day decided to take the helicopter to work instead of driving to make Sao Paulo cleaner and less polluted. And, yes the irony was not lost on anyone.

Pao de Queijo

In my jet-lag mode in Brazil, I did not feel hungry. But there was one thing on the menu that I always had an appetite for. .. Bread stuffed with cheese. They are as yummy as they sound. Soft, warm, fresh from the oven, they melt in the mouth. I wasted a lot of food when I was in Brazil because the helpings at restaurants are really big. Each helping would be 3 meals for me, not one. It is considered impolite to take the left-overs home. I thought that it was odd, in a country where there is a still a large poor population, why would you waste so much of food.

The Urban Jungle

Sao Paulo is a city of skyscrapers. Skyscrapers stretch into the horizon almost as far as the eye can see. Almost nestled between the skyscrapers are dots of favelas. While the buildings are white and pastel sheds, the roof-tops on favelas are fiery red making it easy to spot them among the sea of concrete. I did not go on a favela tour, I don't like slum tourism. But in the taxi ride from the airport to the hotel, the taxi driver drove through a favela to avoid the traffic, and I have to add the Brazilian favela on the face looks better than Dharavi or other slums in India.

Making my dissertation public

2010-04-22T20:18:00.000-07:00

I finished my PhD degree six years back, and like most PhDs I had envisioned a career in academia. As a grad student I taught several courses in introductory sociology and some higher level courses like Research Methods (I often joked that I had made a date with the devil for agreeing to do it, Sociology undergrads hate anything to do with Math) and Sociology of Women. I was a reluctant teacher. I did not do any teaching in the first three years of school, instead I was a research assistant.

But once I started teaching like many people I really enjoyed it. My best teaching experience was teaching research methods (it did not turn out to be a date with the devil). It was very fulfilling to share my ideas, learnings, wisdom and understanding of statistics with students. I also had a selfish reason I realized that teaching statistics made me a better statistician myself. I had to clarify all issues in my head before I could explain them succinctly.

But as it happened I never took a job in academia. After my graduation I moved to California to be with my husband who started working as a software engineer in the bay area. Six months later I got a job with a research company in the bay area. I did get some calls for job interviews, but they were in places like Fresno and Grand Rapids. After having lived in the bay area for a while, I did not move to Fresno. No one can blame me for that.

I still thought I would publish more chapters of my dissertation as scholarly articles. And one of the conditions of publishing in a scholarly journal is that the research should not have been published anywhere else. It was this single reason that kept me from making my dissertation public for so long.

Well, now I am never going back to academia and I have no plans to publish any scholarly articles. So I decided to make my dissertation public. And in one day it went from having been read or seen by two people (me and my advisor) to having been read by 70 people. In my mind that is one good reason for making it public.

If you would like to read it, please click on this link: http://www.scribd.com/doc/30373090/dissert

Simulations as a tool for research

2009-07-23T12:35:00.000-07:00

It often amazes me that things we research and talk about at IFTF remain relevant for so many years after we have written about them. Just to give one example, when we worked looked at the future of science and technology about five years, I was tasked with looking at the future of sociology as a discipline. One of the subjects that I wrote about was: the advent of simulations in sociological research. As I stated earlier, I wrote about it almost five years back, it is still a relevant topic. Here it is, reproduced from the original:

Quantitative analysis methods in social sciences that use statistical and numerical rather than verbal data, have traditionally been limited by the amount of information and data that can be processed. Researchers have typically relied on smaller sets of variable data and as a result, have tended to use the same information in many different circumstances.
Simulations that take advantage of vastly increased computing power could be used more heavily in the social sciences, eventually becoming the more dominant means of analysis as a method of predicting human behavior.

Computer simulations that borrow methods from applied physics and business decision make it much easier to use a greater range of information and throughly evaluate each alternative. Increased computing power and storage capacity means that there are potentially no limits to the size of datasets and the number of variables that can be analysed and the variables can also be measured across time repeatedly. Social scientists are expected to increasingly use simulation, as they learn to rely on computers and an aleatory (random) approach to knowledge that takes into account the vastness of available data. The big challenge however will be to develop statistical techniques for finding patterns in these huge datasets.

Twitter Rankings

2009-05-20T13:42:00.000-07:00

There are so many apps out there to figure out one's ranking on Twitter. There is Twitter Grader (http://twitter.grader.com/), Twitterank (http://twitterank.com/) just to name the ones that I am familar with. As a person who works with numbers and statistics everyday, I have been wondering what is the algorithm they have to rank people.
If I were to create some form of ranking metrics for twitter, I would factor in ratio of followers versus followees. But I would give more weight to the number of updates. I have seen there are several people who follow many people, and many people follow them, but have a handful of updates. They are, obviously, not engaged with the platform or they have nothing interesting to say, probably. This formula might be effective in weeding out a lot of noise in the ranking system. For all you know, the current apps use similar formulas for coming up with their rankings.
If we would like to make it more accurate, we can ask other twitter users to give a thumbs up or a thumbs down to other users. This would help us in qualitatively ranking updates.

Cyberethnography

2009-05-19T22:03:00.000-07:00

I have always wanted to work on a project that would let me do cyberethnography, and as luck would have it, I have never come across one. But on the upside when IFTF did a science and technology scan for the UK government, I wrote about cyberethnography and how it is done, what are the tools that are available. It is a little dated, I wrote this is in 2005 just after I started working full time for IFTF, so bear with me on this.

Anthropologists have traditionally conducted fieldwork by living in distant cultures, conducting interviews, and observing participants. As people conduct more and more activities online and leave digital tracks (pictures, blogs, emails, and such), anthropologists have begun to study human behaviour in cyberspace. Cyber-ethnographers participate in and observe blogs, Web sites, and chat rooms. They analyse how people form social networks or groups online and establish cultural identity. Visual anthropologists, media and cultural studies scholars, ludologists (who study video games from social science/humanities perspective), and science and technology scholars are among those who are building cyberethnography. See http://www.cas.usf.edu/anthropology/cma/CMAnthropologists.htm for an example of the wide range of emerging cyberanthropologists.

Anthropologists have traditionally conducted fieldwork by living in distant cultures, conducting interviews, and observing participants. As people conduct more and more activities online and leave digital tracks (pictures, blogs, emails, and such), anthropologists have begun to study human behaviour in cyberspace. Cyber-ethnographers participate in and observe blogs, Web sites, and chat rooms. They analyse how people form social networks or groups online and establish cultural identity. Visual anthropologists, media and cultural studies scholars, ludologists (who study video games from social science/humanities perspective), and science and technology scholars are among those who are building cyberethnography. See http://www.cas.usf.edu/anthropology/cma/CMAnthropologists.htm for an example of the wide range of emerging cyberanthropologists.

Cyber-ethnography is part of the move to reconceptualize the traditional notion of 'the field'. In cyberspace, the boundaries of the observed field are both virtual and embedded in place, discursive and geographical. New methods for understanding the nature of virtual experiences and environments will explode as the variety and frequency of cyber-experiences grow.

What is the implication?

Increasing need for anthropologists to learn new skills to explore a new domain -- online personal media -- by transforming traditional methods such as participant observation and traditional tools such as firsthand contextual information about place.

The entire original post is available at: http://humanitieslab.stanford.edu/2/280

Writing survey questions

2009-05-13T09:47:00.000-07:00

My colleague Alex Pang yesterday tweeted this:

"RT @pomeranian99: I... wish social scientists would tackle... questions, like, "what percentage of people in the US are assholes?"

I would love to know the answer to it too, but I think it is hard to design a question which tries to get a correct response to this question: "Are you an asshole or not?" No one is going to answer yes to this question, especially, the assholes are not going to be falling over to say yes, yes I am an asshole. So, how do we tackle this question. One of the first steps in designing a survey is not to use words that evoke a strong emotional response or even not to use negatives of any kind in the question. You could may be phrase a question like this: On a scale of 1 to 10, how would you rate yourself as a human being? This is not foolproof still, most people have a better self-perception of themselves, but the responses will still be more accurate than asking the asshole question.

Just to eliminate response bias further, you could ask, how their friends would rate them and how their family would rate them? You would be able to see if there are any gaps between how they feel they perceive themselves and how people they know perceive them.

Efficacy of focus groups

2008-07-11T12:02:00.000-07:00

I was in India last month conducting focus groups with young people. I have conducted focus groups in the past, and every time I finish a focus group, I wonder about their effectiveness as a tool for collecting data. It is a good way to be able to talk to a large number of people in a short span of time. For example, I had spoken to 90 people after I finished conducting 9 focus groups. But it is hard to get deep insights, you can at best just scratch the surface of information. Often people who might be very interesting in a small group setting don't open up in a large group setting. You get to hear more from people who are outgoing, not necessarily those who have best ideas. An ethnographic interview involving face-to-face talk is far more insightful and more fulfilling to do as a researcher.

Typical day versus yesterday

2008-05-02T15:08:00.000-07:00

I was talking to someone the other day. They told me that people when asked the question in a survey "How much time you spend working on your desk in a typical day?" most people almost doubled the amount of time they spend on their desk. People have a lot of trouble recollecting events, and if you ask them a typical day then without malice or bad intention, they don't answer accurately. It is therefore better to rephrase the question and ask them what they did yesterday or last week. It is easier to recollect in this fashion. You are still more likely to get more accurate estimates even if yesterday was atypical for some people.

Scalar options

2007-11-21T15:56:00.000-08:00

I have never been a fan of providing "open ended" scalar options like 0-10 when asking respondents whether they agree or disagree with a question with 0 being completely disagree to 10 being completely agree, I feel options like strongly agree, somewhat agree, somewhat disagree and strongly disagree work much better. The former can be pretty vague and different respondents will have a different interpretation of the scale leading to a lot of measurement error. Interpretation of results on the scale is not fun, if I tell you that respondents on average scored 4.2 /10, it is a little difficult to understand as against saying that 20% of people strongly agreed.

How truthful are people when answering surveys?

2007-11-15T11:06:00.000-08:00

In the past I have conducted surveys on issues around sustainability and using technology to extend the capabilities of the body, whenever I presented the result people questioned the validity of the data because most respondents had very positive attitudes on these issues. "They just want to look good," everyone says. Interestingly, I came across this study that found that most respondents are more likely to give honest answers to a self-administered questionnaire compared to an interview questionnaire. More so, young people people are more likely to be honest when responding to a computer based questionnaire (a tool that I use often) (Turner, Ku and Rogers, 1998)

Hello

2007-11-08T12:26:00.000-08:00

Hello everyone,
Welcome to this blog on research methodologies for social sciences. I am a social scientist, sociologist to be precise, with specialization in research methodologies. I conduct both statistical and qualitative research, and I will regularly post my learnings conducting research on the blog.