I stand before you today to confess perhaps my greatest clusterfuck of the year: the Skeptoid Twitter Experiment, which rendered your Twitter account nearly useless on September 14 and 15, if you follow me or anyone else who follows me.
I have an upcoming Skeptoid podcast episode for which I want to include some informal survey data (it was episode 173 on Astrology). I’ve also been thinking a lot about Twitter for its potential to virally spread information. So I thought it would be a clever idea to combine my survey with Twitter, which (I thought) would be a lot of fun for everyone and would accomplish two goals:
- Virally spread awareness of my podcast, Skeptoid
- Get a huge number of respondents to my survey
Well, it worked. The good parts worked better than I hoped, and unfortunately, undesired side effects were just as potent. Now, before I describe what happened, let me state outright that it was shockingly naive of me not to foresee what would happen. It was dumb, it annoyed a lot of people, and I have no excuse other than failure to think it through very well. So, my apologies, and I offer no defense of what turned out to be a giant mess.
Here’s how it worked. I posted questions to Twitter in this form:
Skeptoid Twitter Experiment, Question 7, is now live. Get it now at http://skeptoid.com/twitter
If you followed the link, it would take you to a page where you would answer some questions, and it would give you a code to post your response back onto Twitter, like this:
I’m doing the Skeptoid Twitter Experiment (http://skeptoid.com/twitter) – My answer is #skep_4_4 – Follow @BrianDunning to join
The effect was twofold. First, people unknown to me who follow my followers saw these posts, were intrigued, and followed me, boosting the numbers of my Twitter followers, and reaching a larger and larger number of people with the question announcements. Second, among groups of people who follow each other (like many of my followers who know me from Skeptoid and know each other from JREF, TAM, etc.), great colliding volumes of redundant answer messages from everyone they follow clogged their Twitter inboxes and made it virtually impossible to see normal messages. I heard from one guy who shares 80 Twitter friends with me, and if they each answered the 8 survey questions, that made 640 tweets.
Many called my attention to other less intrusive ways the survey could have been given, including a number of free web survey tools. Having a web development background, I’m aware of these, and even built a languishing one myself – http://GroovySurveys.com (a day’s work over Christmas break a few years ago) – but there’s a reason I didn’t do it that way. It wouldn’t have been a Twitter survey. And the whole point of my experiment was to see how well a survey worked where the entire thing – questions and answers – were sent via Twitter; and particularly to evaluate its viral effect.
There were some interesting observations to be made. Three times, before the survey started, I made one daily post to Twitter like this:
Please RT: Be a part of the Skeptoid Twitter Experiment: http://skeptoid.com/twitter
And, predictably, this enticing message began to spread. I started seeing it being retweeted (Twitterese for “repeated”) among people who did not follow me. My follower count began to increase. By Monday morning, when I posted the first question, the rise was on a very solid trend.
The first half of the questions were posted throughout Monday, and if things grew before, they exploded now. I use Tweetdeck to view Twitter, and I have columns to view mentions of my name (@BrianDunning) and the podcast (#Skeptoid). Both were absolutely clogged. I thought this was wonderful; but since nobody else is likely to follow either of those search words, it never occurred to me that anyone else was having the same problem.
Tuesday morning I awoke to a barrage of emails, direct messages, and public replies. Obviously, all was not well. Accusations of spam were flying everywhere – which hurt, I’ll admit, because I hate spam as much as the next person and it’s the last thing I’d ever want to be associated with. The growth in my follower count had stopped.
What it reminded me of was junior high school, when a few Kool Kidz wore Izod Lacoste shirts. Soon everyone had to have one. And then, once they became popular, the Kool Kidz wouldn’t be caught dead in them. My survey grew quickly when it sounded like fun, swamped the market, and everyone wanted out.
I posted some hasty apologies and did what little I could to reduce any perceived spamminess. I took my name and the site name out of all the messages, killed the invitation to follow me, and put instructions on the landing page for economizing posts. But then I made things worse: I publicly offered to terminate the experiment if 10 people asked me to do so, but I suggested that they do so by Twitter direct message. I was unaware that direct messages can only be exchanged between people who follow each other, and the handful of personal acquaintances whom I follow did not happen to be the same people as those out in the real world that I’d annoyed. Oops. At least I can boast to having received no such direct message requests, but it’s a pretty thin boast. ;-)
My stuffing of fingers into the dike seemed to help a lot. Adoption rose again, and through the end of Tuesday, my follower count resumed its growth (though I can’t figure out why, since there was no mention of my account anymore), and answers continued flooding in.
So, as far as providing the survey data I wanted, it was a massive success. As far as being user friendly, it was a massive failure, and would probably get my account banned if I did it again. So take what lessons you can from this – I certainly have.