Tweetlevel – and a PR top 100 in ‘tweetlevel’ order.

The latest in our efforts to quantify the unquantifiable and reduce you to a mere number was launched yesterday. Yet again we brutalise complex human concepts like Trust, Popularity, Influence and Engagement with our over-simplistic (but pretty bloody clever) algorithms. With the Social Media Index (SMI) we did it to try measure ‘influence’ across a variety of social media platforms and now we have had a bit of a think about Twitter. Well, I say “we” but really I mean the fiendishly smart Jonny Bentwood.

Twitter is, as you know, the platform de-jour, but how do you begin to sort through it to find out who is important to you or your client? How can you use it better, particularly if it’s having an increasing role in how you manage relationships? To help us do this across all our Edelman teams and at scale (and dynamically as the twitterverse is developing and changing so rapidly) we have built a tool which we have been using for a while and which from today is open for use by anyone.

It’s called Tweetlevel. It looks at Twitter users four ways:

1. Popularity (how many people follow you – the easy one)
2. Influence (is what you say interesting and how many people listen to it)
3. Engaged (are you actively participating within your community – vs just broadcasting)
4. Trusted (do people believe what you say)

It does all this based on this formula (there’s always a formula or an algorithm or something):

The full methodology is at the bottom of this post. Crudely, though, this is how we come to those different weightings:

Weighted for Popularity

The key variable is the number of people someone has following them. There are many online tools that show this such as Twitterholic.

Weighted for Engagement

The key variables are an individual’s participation with the Twitter community (as measured by the Involvement Index), with additional emphasis on the frequency of people name pointing an individual (via @username), the numbers of followers and the signal to noise ratio. Other attributes were included in the final score but were given a lower weighting.

Weighted for Influence

The key variables in this instance is a combination of the number and authority of someone’s followers together with the frequency of people name pointing an individual (via @username) and the how many times and individuals posts are re-tweeted. Other attributes were included in the final score but were given a lower weighting.

Weighted for Trust

The best measure of trust is whether an in individual is will to ‘trust’ what someone else has said sufficiently that they are also prepared to have what they tweeted associated with them. The key metric in this instance looks at combination of retweets and references (shown through ‘via’. Other attributes were included in the final score but were given a lower weighting.

As ever, these are big value judgements and I am sure our labelling of them will raise the usual concerns. It is, we stress, a fairly blunt tool, but it does begin to help you think about twitterers in a way that we have found useful.

The proof of the pudding is always in the eating they say and so here I have taken Valeria Maltoni’s list of 100 PR people worth following on Twitter and put it through Tweetlevel. You can of course put any group or universe of people you follow through it and come up with your own rankings. We routinely do this now across all sorts of sectors and industries for our clients. Valeria listed her 100 in alphabetical order, but I have listed them below in Tweetlevel ranking order. Yes, yes it is list bait, but it is interesting to see where people fall and how they score so differently across the four criteria.

Have a go yourself and as ever let us know what you think.

Name Influence Popularity Engagement Trust


1


ginidietrich


80.6

59.2

87.2

72.8


2


steverubel


74.9

67.6

60.6

69.1


3


arikhanson


71.3

55

77.3

57.5


4


shonali


70.8

55.3

79.5

57.8


5


zoeyjordan


70.7

52.6

72.7

58.7


6


BethHarte


70.1

61.5

76.3

64.6


7


kamichat


68.4

55.3

72.1

56.6


8


briansolis


68

69.6

52.5

75.8


9


prblog


67.3

61.7

58.7

56.6


10


shel


66.3

57.2

70.8

55.4


11


laermer


66

59.4

44.9

61.9


12


CubanaLAF


64.7

53.3

73.2

56.9


13


PRsarahevans


64.4

68.6

57.6

66.1


14


rachelakay


64.1

57.2

71.2

50.6


15


skydiver


63.5

72.2

40.8

68.3


16


ikepigott


63.4

54.5

67

51.9


17


TDefren


63.3

61.9

58.6

57.5


18


jasonkintzler


63.3

59.6

63.4

52.4


19


DougH


62.8

64.7

62.5

54.5


20


trevoryoung


62.6

54.4

58.4

49


21


mikeschaffer


62.5

48.8

61.9

50.4


22


dbreakenridge


62.2

55.3

64.4

50.2


23


elizabethsosnow


61.5

53.8

56.7

52.1


24


kmatthews


61.5

52.3

67.3

49.2


25


GeoffLiving


61.3

58

64.9

53.3


26


Steveology


61

67.9

50.3

62.5


27


dmscott


60.7

68.1

57

61.3


28


dmullen


60.7

54.8

70

45.5


29


davefleet


60

57.8

66.6

57.8


30


BarbaraNixon


59.4

57

61.2

57.2


31


jspepper


58.9

59.4

72.6

47.4


32


jpostman


58.5

58.9

53.3

45.3


33


martinwaxman


58.4

50.6

58.5

43.7


34


jangles


58.1

56.4

55.3

56.5


35


leehopkins


58

52.9

61.2

37.1


36


alanweinkrantz


57.7

54.1

56.2

42.4


37


DoctorJones


57.5

52.4

69

46.9


38


MikeLizun


57.5

55

49.3

50.1


39


davidparmet


56.9

53.5

72.1

39.9


40


charshaff


56.8

49.8

60.1

41.9


41


ryananderson


56.7

51.4

61.5

43.9


42


paullyoung


56.5

53.6

54.2

42.9


43


wiredprworks


56.5

56.8

58

45.3


44


cbasturea


56.4

50.3

60.1

40.5


45


kdpaine


55.6

57.5

68.4

52.5


46


DannyBrown


55.6

63.3

73.3

53.2


47


RichBecker


55.2

50

63.6

45.7


48


brittanymohr


54.8

49.3

57.1

40.2


49


drewb


54.7

54.8

51.2

54.8


50


princess_misia


54

50.4

53.1

44.5


51


PRwise


53.6

57.1

39.7

55.8


52


KarenRussell


53.2

51

67.9

47.1


53


DebInDenver


52.6

48.6

64.7

39.7


54


johncass


52.2

55.5

61.5

45.8


55


thornley


52.2

55.8

49.6

48.9


56


BillSledzik


51.7

47.9

64.4

36.2


57


benrmatthews


51.7

51.4

51.9

47.9


58


stevemullen


51.6

50.7

53.5

38.1


59


PeterHimler


51.5

50.3

49.1

42.4


60


CathyBrowne


51.2

53.8

72.4

28.9


61


jbell99


51

53.2

47.9

37


62


DavidBrain


50.9

51.2

44.2

47.3


63


cherissef


50.4

45.7

48.3

37.2


64


khartline


50.1

55.1

57.8

33.8


65


leeodden


49.5

64.4

55.7

31.2


66


wadds


49.4

50

57.8

42.1


67


perfectporridge


49.4

53.4

43.2

36.2


68


AdamSinger


49.3

53.7

42.4

44


69


CatrionaPollard


49.1

50

44.9

43.1


70


jackmonson


49

48.1

51.1

33


71


ThePRDoc


48.5

46.2

43.5

43.8


72


LindsayLebresco


48.2

49.6

60

31.6


73


domw


48.1

49.4

56.4

39.7


74


jedhallam


47.6

50.3

54.7

38


75


simoncollister


46.6

50

58.3

37.7


76


sherrilynne


46.1

49.1

46.9

37


77


missusP


45.6

63.9

53.9

43.4


78


KellyeCrane


45.5

55.9

51.3

46.7


79


LuAnnGlowacz


45.3

44.2

50.4

33.8


80


LukeArmour


44.8

49.4

49.5

28.4


81


tpemurphy


44.1

47.1

47.6

36.4


82


ShaneKinkennon


43.9

45.1

46.2

32.8


83


andismit


43.9

47.8

53.5

32.8


84


RTorossian5wpr


43.7

47.4

27.8

45.7


85


stuartbruce


42.7

52

46.1

34.7


86


Paul_Stallard


42.6

48

44.8

37.1


87


GRIPCOMMPR


42.4

46.6

50.6

25.9


88


KyleFlaherty


42.2

51.5

49.8

40


89


SarahWurrey


41.6

50.6

47.1

24.5


90


hyku


41.4

54

32.7

23.1


91


bmcmichael


40.2

48.4

49.5

23.6


92


ealbrycht


39.6

49.7

45.5

20.2


93


mmanuel


38.6

49.7

44.6

19.4


94


csalomonlee


38.4

47

51.2

26.2


95


gdugardier


37.2

44.2

32.1

24.8


96


mpwatson


35.7

45.9

45

21.4


97


robskinner


29.4

42.5

32.7

18.6


98


davidreich


27.3

42.2

43.6

16.8


99


samoakley


23.9

45.5

39.3

17

Full Methodology (or, ‘the science bit’)

TL TwitterLevel Rg Range assigned to score
Fo Number of followers Fg Number users following
Up Number of updates @U Number of name pointing
Rt Number of retweets Ta Twitalyzer score
TaN:S Twitalyzer noise to signal ratio Ti Twinfluence score
Tg Twittergrader score Ii Involvement index score
Vi Velocity index score w Weight assigned to each attribute
Z Standardised score p Popularity
e Engagement i Influence
t Trust

Following – Twitter lists the number of people each user follows. The tendency for most celebrities is to only follow a few individuals – the more people that someone follows, there is an increased likelihood of them actively participating in conversations with the community instead of simply broadcasting to it. Following ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm.

Followers – Twitter lists the number of followers each user has. Like subscribing to a feed, this is a clear indication of ‘popularity’ as it requires someone to actively request participation. Follower ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 50) that was used as part of the algorithm.

Updates – How often does someone update what they are doing. This number is purely objective as it scores someone highly no matter what the content of their post (i.e. how relevant is it). Nevertheless it is assumed that if someone posts frequently but has poor content then their ‘followers’ will decrease. Update ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 30) that was used as part of the algorithm.

Name Pointing – e.g. @name – How many people engage in conversation with a celebrity or point to their name. The clearest way to establish this is to run a search on the number of people who reference @username in a message. This calculation is based upon a one month period combined with a 24 hour period. The number of times this happens is calculated with each range was assigned a number (0 to 30) – again this was then used as part of the algorithm.

Retweets – Has a tweet caused sufficient interest that it is worth re-submitting by others? Despite a great deal of ‘noise’ (i.e. posts that are not relevant or interesting), when someone sees something that is of high interest, their post can be re-tweeted. The clearest way to establish this is to run a search on the number of people who reference RT @username in a message. This calculation is based upon a one month period combined with a 24 hour period. The number of times this happens is calculated with each range was assigned a number (0 to 50) – again this was then used as part of the algorithm.

Twitalyzer – “This is a unique (and online) tool to evaluate the activity of any Twitter user and report on relative influence, signal-to-noise ratio, generosity, velocity, clout, and other useful measures of success in social media.” This 3rd party tool is a useful method to combine automated metrics dependent upon criteria within posts and publicly available numbers. Where tools such as this are available, we incorporate them into the algorithm to achieve a more confident score. Twitalyzer gives users scores from 0 to 100. Ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Twitalyzer noise to signal ratio – Signal-to-noise ratio is a measure of the tendency for people to pass information, as opposed to anecdote. Signal can be references to other people (defined by the use of “@” followed by text), links to URLs you can visit (defined by the use of “http://” followed by text), hashtags you can explore and participate with (defined by the use of “#” followed by text), retweets of other people, passing along information (defined by the use of “rt”, “r/t/”, “retweet” or “via”). If you take the sum of these four elements and divide that by the number of updates published, you get the “signal to noise” ratio. Twitalyzer gives users scores from 0 to 100. Ranges were determined (i.e. more than 20, more than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Twinfluence Rank – Twinfluence is an automated 3rd party tool that uses APIs to measure influence. For example: “Imagine Twitterer1, who has 10,000 followers – most of which are bots and inactives with no followers of their own. Now imagine Twitterer2, who only has 10 followers – but each of them has 5,000 followers. Who has the most real “influence?” Twitterer2, of course.” As with Twitalyzer, this index uses 3rd party tools to add greater confidence in the overall Twitter score. Similar to the other criteria, ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Twitter Grader – Twitter Grader is the final automated tool to add greater confidence to the final index. This site creates a score by evaluating a twitter profile. Similar to the other criteria, ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Involvement Index – As the only personal subjective measure in the algorithm, opinion points were assigned to each celebrity. People who scored highest in this category had frequent, relevant, high-quality content that actively involved the twitter community (asking questions, posting links or commenting on discussions) and did not purely consist of broadcasting. Ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Velocity Index – As more people engage on Twitter, it may become harder to keep activity going. The velocity index measures changes on a regular basis and assigns a score based on increased or decreased participation. Ranges were determined (i.e. less than 20, less than 30, etc.) and each range was assigned a number (0 to 20) that was used as part of the algorithm.

Weighting – Each specific variable listed above was given a standard score out of 10. Using a weighting scale I varied the importance of the each metric to establish an individual’s total score.

Weighted for Popularity – the key variable is the number of people someone has following them. There are many online tools that show this such as Twitterholic.

Weighted for Engagement – the key variables are an individual’s participation with the Twitter community (as measured by the Involvement Index), with additional emphasis on the frequency of people name pointing an individual (via @username), the numbers of followers and the signal to noise ratio. Other attributes were included in the final score but were given a lower weighting.

Weighted for Influence – the key variables in this instance is a combination of the number and authority of someone’s followers together with the frequency of people name pointing an individual (via @username) and the how many times and individuals posts are re-tweeted. Other attributes were included in the final score but were given a lower weighting.

Weighted for trust – the best measure of trust is whether an in individual is will to ‘trust’ what someone else has said sufficiently that they are also prepared to have what they tweeted associated with them. The key metric in this instance are a combination of retweets and number of followers. Other attributes were included in the final score but were given a lower weighting.

Criteria for inclusion – There are many lists of top celebrities on Twitter – every one of these use ‘popularity’ as its main criteria. Edelman have used all these lists (such as The Times, Celebrity Tweet and Mashable together with other ‘interesting’ names and used its algorithm to establish their importance.

See, told you it was clever!

David Brain

30 Comments

  1. Excellent breakdown on helping everyone understand the new Edelman formula…problem is that it doesn’t measure influence at all…not even close.

    It’s weakness is in the design of the forumla. By including multiple layers of twitter functionality, such as following, involvement, engagement, updates, etc. actually degrade the potential for measuring influence. Did I miss “Twitter Lists?” Is that accounted for in this formula? As a digital sociologist, I can assure you that the more you engage, the lower the velocity of influence. That’s just the beginning…

    Influence is a very deep discussion and unfortunately, this metric, doesn’t measure authority as much as it does stature.

  2. Thanks for your comment Brian. One of the problems when explaining the algorithm is to put in sufficient detail so that people understand the flavour of the way it works without obfuscating this matter with complex maths. I have a strong feeling that if you looked under the bonnet, your argument may largely come down to semantics.

    Please allow me to address each of the points you raised.

    It’s weakness is in the design of the forumla. By including multiple layers of twitter functionality, such as following, involvement, engagement, updates, etc. actually degrade the potential for measuring influence.

    I disagree, it is by having so many metrics that we are able to measure influence with a higher degree of confidence. For example with the involvement index we are able to understand and prioritise tweeters by the way an individual engages with their micro-community. For example look at @momsofamerica – she is someone who is incredibly influential and with less than 9,000 followers she is rated as one of the most engaged.
    What you cannot see is the individual weighting behind each metric. For example, for influence, we put a higher degree of importance on the authority of the people who retweet or comment on someones conversation as oppose to the number of followers someone has.

    Did I miss “Twitter Lists?” Is that accounted for in this formula? As a digital sociologist, I can assure you that the more you engage, the lower the velocity of influence

    Twitter Lists is something that is critical. As I reference in the methodology this will be included as soon as twitter releases this part of its api. With regard to influence, there is a fundamental difference between high engagement which leads to mass amplification of thoughts, and little engagement but with great ideas that can spread through other nodes. Again this is addressed through the algorithm.

    Influence is a very deep discussion and unfortunately, this metric, doesn’t measure authority as much as it does stature.

    Again I disagree as authority of the person speaking is at the heart of what it does. What Edelman are not saying is that they have solved the ‘influence’ formula. We have open sourced and beta tested this over several months with many key people in this space and the feedback is excellent. We are always trying to improve the way this works and would welcome a more detailed discussion with you about how this works.

  3. Jonny, you’re more than welcome to call me…but I have to be honest, I study influence. The results, while very interesting and useful, do not measure influence nor authority…they measure community stature. That is not semantics when you compare it to how I define influence in the era of socialized media, the ability to inspire action and measure it.

    I suppose part of the problem is in the positioning of the formula and the list itself, ” Twitter Users by Influence”

    If you’d like insight in how to measure influence, I’m more than happy to help…

  4. Jonny, in fact, I encourage you to reach out to discuss this as I would like to write about it in a way that doesn’t mislead or confuse people. It’s important to clarify the differences between influence, authority, and community stature when exploring and planning engagement and communications programs.

  5. I read the comments and I don’t totally understand the science in this. But what I do know is I must spend WAY too much time on Twitter! It’s an addiction. Thanks for the compliment…now, how do I monetize this?? :)

  6. If “studying influence” makes you an egotistical blow-hard, then I’m glad I don’t listen much to Brian Solis. Calling someone out on their list just to promote the way you’d measure something? Low, Solis, low.

  7. Brian – i think we are going to have to agree to disagree. Not for one second do we believe that we have found the holy grail and have solved the influence formula. This is our understanding of how we can easily identify which people use twitter well. Semantics aside whether you call it influence, stature etc, it still is a great tool to identify via the many metric available who is important on twitter.

    Ari – i completely agree that there are fundamental problems with some twitter measurement tools. The example you cite on your blog is a great case study. This is why tweetlevel takes all these things into consideration and gives a far more representative score IMHO. I’d be interested to know if any of the results you found havce a value relative to other people that you disagree with.

    Obviously one of the main advantages of this tool is not to try and look at the whole world but to understand who are the important people within micro-communities.

  8. When Valeria published this list, I was honored to be among the listed. I appreciate what you are trying to do here, but I am grateful to be among this “micro-community” no matter how we are ranked. Thanks to social media I consider the people on this list mentors. The numbers do not matter to me because at the end of the day I interact with these colleagues on a daily basis and know they are professionals who have “influence”.

  9. Craig, just have to ask, what are you talking about? I have nothing to promote here. I study influence as part of social studies and I don’t sell or market that service nor the research results. I give it away for free so that we all learn together. I’m also offering to help here.

    The truth is that when you rank unsuspecting people based on influence and publicize it, there are aspects that must be considered in order to not lead or cause further confusion or dismay.

    Jonny, you’ve done great work here and it’s definitely a helpful and meaningful tool. It serves a purpose indeed. I guess we will agree to disagree as the use of the word “influence” in the product is misleading and unfortunately confusing people who are struggling with the greater meaning and mission of social media in general.

    I agree with this point and the application of the service on this premise however, “This is our understanding of how we can easily identify which people use twitter well.”

    In that sense, well done…

  10. Brian, it does seem that even though we are agreed in principle with the benefit of the tool – ie identifying who uses twitter well, where we have a disagreement is over the term ‘influence’ especially when it comes to online conversations.

    This is certainly a moving goal post – when I first looked at this, I confused popularity with influence. The work that you have done looking at influence seems very interesting and I’d be delighted to take you up on your offer and discuss this with you.

    As an aside when we were discussing which of the four metrics (influence, popularity, engagement and trust), we should lead with we opted for ‘influence’ as we thought most people would be able to understand that whereas if we went with engagement, it may be to esoteric. However, in my opinion within the micro-communities I often talk about it is engagement and trust that is far more important. Not surprisingly these two terms are something that Edelman champions too.

    You can reach me via jonny.bentwood at edelman.com and I look forward to continuing this conversation. After all isn’t conversations what this is all about?

  11. Influence is great but it all depends on who, if anyone, is listening (or in Twitter’s case – following AND reading).

    Right now Twitter is still a huge novelty that could easily become swamped with spam (you can only block a spammer once you know their ID – i.e. after they have spammed you). It could easily fall out of fasion and go the same way as Beebo, Friends Reunited, Second Life….

    There are millions of users who will lose interest in building lists of followers simply to compete with their friends (by becoming a follower of others). Those accounts will become idle and then how can you influence them?

    There is a very good article in this month’s Wired (I can’t find it on the Wired website): “How will Twitter grow up?”. To which the answer seems to be – we dunno!

    Many years ago a guru said to me that the Internet’s real power would come when it became an intrinsic part of every day life and when people no longer noticed it or saw it as a novelty. It could be that, as Twitter “grows up”, it’s longevity as a communications tool will depend on it becoming an intrinsic element within other platforms. It could then become a vital channel within reach of the majority.

    And that would be influence worth measuring.

  12. Chaps (Brian and Jonny) – it’s great to watch this conversation playing out and for what I can recall, is one of the deepest (thus far anyway) conversations about what influence really is.

    On the surface, agreed, influence can appear to be simply the share of voice. Conversely, this same term can be perceived as popularity.

    The re-tweetability (as worked on by Dan Zarella) of content, I believe, comes closer to influence, but this too, stops short of actually measuring influence – because this is simply on the Twitter channel.

    An “influencer” on Twitter I would argue, is typically more re-tweeted because they have something unique, new or different to say that others find of value. The weakness of this as a measure of influence is that simply the act of re-tweeting is not tantamount to “being influenced”.

    Herein lies an interesting point. In the same way that with analytics, early ecommerce stores fogured that their goals for conversion were simply sales, in the social web, are things like re-tweet and other social gestures such as friending, liking, commenting etc. equally important in determining the influence of an individual?

    Over times, ecommerce sites have come to realise that conversions can take many forms, each with perhaps a different, longer sales funnel – each perhaps with its own strategy.

    Inevitably, only when the social graph becomes truly open will we be able to determine the true influence rather than community stature of an individual but I honestly think that the this is a wonderful start.

    I talked at a Figaro Digital event a few months back looking at ways in which we can try and understand the ways in which we can best (as an agency on behalf of brands), spend our time more effectively on Twitter by finding and following those people most likely to re-tweet our work, engage with us (based on their previous tweets and interests) which can, done properly be a prohibitively laborious process.

    Whilst there are things that I would like to understand further about this formula (and its weightings), this is a great start guys to making my approach much more srtreamlined.

    Jonny, if you haven’t already spoken to Marshall Manson about the tools he is using for understanding influencers in networks (DM @paulfabretti for more info), then I would recommend it heartily! The Influencer Analysis tool is a really, really interesting piece of kit!

Leave a Reply

Your email address will not be published. Required fields are marked *