Don’t Use the “I” Word

I’ve been told for the past year not to use the “I” word, “intelligence”. Oh f’gods sake. IMHO NGOs haven’t, are not, and are certainly not intending to, collect any form of intelligence about people, nations etc. But they are starting to build assisted intelligence systems, as in computer systems that help human beings make decisions by improving their access to information. And some of the techniques used in intelligence systems also turn out to be very useful for working out what’s happening in a crisis, i.e. who needs help where and when. So let’s put on the big-girl pants and look at what these are, and how they might be used to help people in crises (as opposed to, say, make them nervous enough to stick tinfoil to their ceilings). Development Intelligence I noticed the term “development intelligence” being used more often this year (and no, not by us)….

Data Science

Big data – wozzat?

So what is this big data thingy? Big data has become a hot topic lately. The people who deal with it (“data scientists”) have become much in demand by companies wanting to find important business insight in amongst their sales data, twitter mentions and blogposts. Which confuses three different concepts. Big Data is defined as the processing of data that’s larger than your computer system can store and process at once. It doesn’t matter so much where the data is from – it’s more important that it’s too huge to handle with ‘normal’ processing methods. Social media mining looks for patterns in the posts, tweets, feeds, questions, comments et al that people leave all over the Internet. It’s a logical consequence of Web 2.0, that idea that we could not only read what people put on their websites, but contribute our thoughts etc to it too. Data analysis looks for…

Data Science

Creating humanitarian big data units

Global Pulse has done a fine job of making humanitarian big data visible both within and outside the UN. But it’s a big job, and they won’t be able to do it on their own. So. What, IMHO, would another humanitarian big data team need to be and do? What’s the landscape they’re moving into? Why should we care about humanitarian big data? First, there’s a growing body of evidence that data science can change the way that international organisations work, the speed that they can respond to issues and degree of insight that they can bring to bear on them. And NGOs are changing. NGOs have to change. We are no longer organizations working in isolation in places that the world only sees through press releases. The Internet has changed that. We’re now in a connected world, where I work daily with people in Ghana, Columbia, England and Kazakhstan….

Code for Good

What is a hackathon?

I accidentally ended up organising a hackathon recently. RHOK NYC could have been a tragedy. I was too overloaded to help organize it, the local Crisiscommons lead was too busy, and the young man who stepped in to lead was inexperienced with hackathons and unsupported but managed to pull things together well until the fortnight before, when RHOK NYC lost its venue. Which is a big deal in New York – – they’re not easy to come by for a weekend event with a sleep-over (or rather a crash on the floor for a couple of hours in-between coding -over). Oh, and the young man was unexpectedly out of the country at another event. So we cancelled, got talked back into trying again, and put out the call to the local volunteer technical community. The community answered us, in spades. Within a week, Phil from Open Plans found us a venue…


Where next for Hunchworks?

Hunchworks is just one of the technologies that we designed this year.  It’s time to talk about why, and what some of the others are. Why build Hunchworks? Imagine the scene today. I’m an analyst at the UN, and most mornings I get to my desk in the morning and think “what’s going on in my area” or “something’s not right here”. What do I do about that today, and what would be useful to me next year if I want to do it better? Today, I rely mostly on personal connections, colleagues, skype chats, newsgroups and websites. If I want to know what’s going on, I log into accounts in several different systems (my UN and personal emails, a group of Skype chats, some irc channels and a set of newsfeed websites) and scan the entries in them for things (specific event types, geographical areas etc) that are urgent,…


Lessons from mapping Sahel

We needed an example problem set for our current version of Hunchworks (note that this is a very early, i.e. pre-alpha version of the code and a lot of the cool Hunchworks features aren’t in it yet). The UN’s main use for Hunchworks is to gather up the weak signals that people put out about emerging development crises – those small hints that something isn’t right that appear all over the world before they coalesce into ‘obvious’. Awareness of development crises can happen very quickly. One minute there are whispers of a potential problem – a chat here, an email or text asking for a bit of data there. And then a tipping point appears and there’s suddenly data everywhere. And we have a great example of this happening just at the time that we’re demonstrating Hunchworks to the UN General Assembly. We had one of these serendipitous test sets…

Augmented Intelligence

Strata talk on hunchworks technology

I try not to put too much dayjob stuff here, but sometimes I need to leave less-tidy breadcrumbs for myself.  Here’s the 10-minute (ish) talk I gave at Strata New York this year. Intro I’m Sara Farmer, and I’m responsible for technology at Global Pulse. This brings its own special issues.  One thing we’re passionate about is making systems available that help the world. And one thing we’ve learnt is that we can’t do that without cooperation both within and outside the UN.  We’re here to make sure analysts and field staff get the tools that they need, and that’s a) a lot of system that’s needed, and b) something that organisations across the whole humanitarian, development and open data space need too. <Slide 1: picture of codejammers> We’re talking to those other organisations about what’s needed, and how we can best pool our resources to build the things that…


What’s my real job again?

Where does Global Pulse start and end technically? It’s a question I often have to answer, mostly because of my very ambiguous role leading work both inside (at Global Pulse), outside (with the crisismappers, open data peeps, volunteer hackers etc) and across (UNGIWG, UNSDI) the UN. The UN, both traditionally and increasingly, has the role of coordinator between both its internal agencies and external NGOs, government agencies and affected communities in the geographical areas in which development and humanitarian efforts take place. On top of that, it’s still doing the political and physical (food aid, heritage sites etc) legwork that helps to keep the world safer and more stable. That’s a lot to ask of any organization, and it’s a huge amount to ask of one whose systems until very recently were based on a slower-moving, less-connected social world. There’s a lot to be done, data-wise, to support all of…


Lessons from the (latest) internet revolution

Once more with publishing some notes that I’d left lying around.  I was thinking about the new Internet – the one that’s emerged over the last year and is becoming more real-time, reactive and uses more data analysis, maps and geography.  And I was wondering what lessons we could learn from this shift. 1. Stop observing your users and start interacting with them Huge just published a book on the shift from customers to users, i.e. from people who buy one-off purchases from a firm, and people who are engaged in a conversation with that firm, whether they’re purchasers, potential purchasers, bloggers, employees or fans. That resonates.  It’s a shift already happening in the commercial world, and it’s a shift going on in the development world too.  I’ve blogged before about the shift in organisational focus from vulnerability to resilence and the work of people like CDAC and the World…