RSSAthens Startup Weekend I
Put together on November 24, 2008 7:42 pm by Dimitris
What do you think?
When the Athens Startup Weekend was first announced I wasn’t really sure what to expect. It was a totally foreign idea to me and I could never imagine a group of strangers forming a team and managing to go from conception to prototype in 2.5 days. The event was to take place in a 3-story building which was only recently put to use by Microsoft but which proved very handy for the job. I got there at 16.30 and met with George Tziralis, Efthimios Mpothos of askmarkets and George Kasselakis hoping that we could all use the opportunity of the SUW to make something useful. By 17.15 when the event started the ground floor seats were pretty much all taken – a pretty impressive turnout.
After an intro by Patrick Malone, the Microsoft host, Alexandros Pagidas, the organiser, and Andrew Hyde, who conceived SUW, the idea pitches started. Hesitantly, at first but quite confidently eventually, more than a dozen ideas were laid in front of us on a slide. George K took on pitching the nugget of an idea I had suggested earlier to them: ‘a metric of how popular or social or influential one is by measuring the impact they have not just on one platform like in Twitter (where such services do exist – albeit being very simplistic) but in all major social networks: both individually but also combined in a single index’. I pitched another one (hopefully to be written up at a future post) and George T yet another.
And once the pitches had ended the chaotic and difficult to follow forming of teams started. After a bit of talking around eventually we decided to go with the social impact thing. Somehow apart from George K, George T and Efthimios, another two people joined us around the white-board where we had started scribbling some first notes about the idea. That was Vicky Kolovou of Netwire and Alexandros Georgiadis of wiredpot. The team seemed suited for the project and its scope: we had two data mining/analyst people, two developers, a web designer and a technical/project manager, respectively.
Our first task for the day which was quickly drawing to an end was to basically think through the idea and identify what we would focus on qualitatively and draw some specs or paper prototype the project so that in the next morning we would be ready to start working on it. Part arbitrarily and part due to some already known technical characteristics of those services we initially selected Twitter, FriendFeed, Facebook and blogs as the platforms we would calculate the combined social impact of users for.
The rest of the evening up until 9 or so was spent scouring the web mainly for technical information as to what could be done and could not be done using the API in these platforms as well what scripts have been written by the developer community that would be useful to us. Eventually, we ended up with a list of a few parameters per platform that would be available to us to use in our algorithm that would calculate a user’s impact factor. Having settled on them we called it a day and agreed to reconvene the next day at 9.
The following morning we got down to work right away. Us ‘analysts’ started looking for previous efforts in quantifying the impact people have in their communities online while the developers looked in detail into the APIs starting with Twitter and Friendfeed and started the coding of the crawlers based on the variables we had agreed we wanted. We needed basically two elements for our project. Primarily, it was necessary to have the social graph of users for each platform we were to include and its associated data. Secondarily, we wanted an algorithm to process the data on the graph. It was important get crawling as soon as possible so that when the algorithm was ready there would be enough data to work on.
We were fortunate to confirm both our guesses. Indeed, we had two very competent coders in our team who before noon had written a first version of a crawler for Twitter and FriendFeed and set them running. Moreover, from what we could find in a cursory search at least, most attempts to monitor what the social impact of people online were rather oversimplifying things. We had to – and it seemed straightforward – to have a more sophisticated go at the problem.
A major problem we had was that different online platforms handled people and their interaction differently and in order to be able to combine the impact a person had online you had to have a common method that could be applied/adopted on most platforms out there. The following day I wrote up a detailed description of what we do to calculate the social impact someone has online but the main concept we ended up with George Tziralis in the first day can be summarised in the following principles:
1. For technical purposes (mainly computational power, time limitations and API restrictions) we need to process only a few and cheap variables. That excludes text processing (e.g. what a tweet says exactly) and going moving way back in time.
2. Taking a few parameters into account – just the most important ones – helps in ensuring, to some extent at least, that all platforms will have an equivalent across them. All platforms have the concept of friending in one way or another, for example.
3. It is very important to understand about a certain user whether they are surrounded at the 1st degree mostly by ‘black hole’ connections (sucking in information but not retransmitting it) or ‘megaphone’ connections (retransmitting a large fraction of what comes their way thus amplifying the message)
4. A measure of how effective one is in attracting attention is what comes back to them as an answer once they have said something. And although it’s too expensive computationally and usually of dubious quality to do a text analysis of responses or to cover the entire time after a post has been made looking for responses, you can still get some information by looking at, say, the last 20 responses (20 being a semi-arbitrary maximum) to an action and over how long a time they are spread out. For instance, when Robert Scoble twits something he will probably get some replies and they will be spread over a couple of minutes. When I say something, I may not get any replies due to my smaller following and if I do get any they are likely to be spread over a longer time – again just because my possible responders are fewer.
5. The same way of thinking can be applied for ‘likes’ or the votes of confidence people send to each other (diggs, stars in Twitter, etc).
6. Factoring everything in has to be somehow weighted of course. For one thing, these parameters are not equal in monitoring impact and for another, people are not equally active in all the platforms they have a profile. Of course, not all platforms are equally influential overall (i.e. regardless of a particular user) – for example FriendFeed has too few users to have the same overall effect as Facebook.
It took us the better part of the day to sort out these details of the algorithm, the formula to find the impact factor per service, to estimate the weights and to combine everything in a single impact factor in a way that makes sense. Considerable back and forth took place between us analysts and the developers on what exactly was computationally feasible and somewhere along the road we discovered that Facebook unfortunately had to be excluded due to their API not allowing us access to crucial information.
However, the first steps in developing our (yet unnamed) howsocial.ru had been taken…
(Photos by Andrew Hyde, Vicky Kolovou and Robert Scarth)
tags: idea, project, report |
Digg | |  |
Del.icio.us | |  |
Ma.gnolia | |  |
Newsvine | |  |
Technorati |
Comments
Leave a Reply - in English Please
Good Tip... - last month