For those who do not know him, Beppe Grillo is an Italian former comedian and the leader of “5 Star Movement” (Movimento 5 Stelle), a political movement with some populist nuances, which is about to run for the first time in Italy’s parliamentary elections, and -according to the latest polls- to gather about 20 percent of total votes. Grillo’s grass-roots movement started on the internet, where he has a very popular blog, by international standards as well. Beppe Grillo is also on Twitter, where -as of today- about 623,000 users follow him.
What happened yesterday? Marco Camisani Calzolari, a web entrepeneur who teaches PR methods at Milan-based IULM University, issued a press release about a forthcoming empirical study, according to which 54 percent of Grillo’s followers are presumably BOTs, i.e. they are not humans.
Needless to say, Grillo did not take it well and wrote on Twitter that he might sue Camisani Calzolari for slander. In fact, there are serious concerns about the method employed in this study, some of which have been highlighted by Paolo Bottazzini on Linkiesta (the piece is in Italian).
I add some comments here. In a nutshell, before embarking into the type of research done by Camisani Calzolari, one should bear Alan Turing in mind: a robot -or an automatic algorithm- cannot reliably tell robots from humans, even in the case of Twitter followers. The Turing Test is the standard method to ascertain whether a machine “thinks”, i.e. it is endowed with artificial intelligence. A given machine is intelligent if a human being that is engaged in a dialogue with it cannot tell whether it is a machine or a human being. Of course, the format of the machine’s replies should not be directly informative about its nature. How so? Turing originally proposed the idea of typed replies, while now it is straightforward to imagine a text-only, internet-based dialogue.
What is the procedure used by Camisani Calzolari? He picked up a large and random sample of Grillo’s followers on Twitter, and built an automatic algorithm, which attributes to each sampled follower a “robot score” and a “human score“. For example, a follower that is accessing Twitter from multiple platforms (e.g. both a PC and a smart phone) obtains three additional points to its human score. On the other hand the procedure gives an additional point to the robot score for each feature for which he/she/it has not received a human point (note: not all human features are relevant here, see page 4 in the press release).
If the overall human score for a given user is higher than the overall robot score, then the algorithm would code him/her as a human. Instead, if the robot score if four points larger than the human score, then the user is classified as a BOT. Finally, for intermediate values of this difference the procedure would classify the user as “uncertain“.
Let me be clear on this: a priori, I am very sympathetic to replicable, automatic methods that allow to classify very large datasets. Moreover, those procedures can be applied to several other Twitter users and/or to the same user across time, with the purpose of making comparisons and of running more sophisticated statistical analyses.
Still, one should be very aware of the Turing test and abstain from overambitious empirical claims: only a human being can tell whether a machine “thinks”, while the opposite does not hold. A machine cannot provide any definitive answer to the question of whether a given subject (in this case: a Twitter follower) is a human being or a non-thinking machine.
I am not commenting here on the entrepeneurial skills of Camisani Calzolari. Still, it would be a little far-fetched to define him as a very influential scholar. However -as a lecturer in PR methods- he is definitely knowledgeable about how to (over)sell his empirical findings. Marketing rules are important, but scientific rules matter even more.