Tanzania day 5: Internet! (ish)

Download PDF

Woken at 4am by mosquitoes – I left the bathroom door open, and the insect screen only works so far.  None of the hotels here have mosquito nets, so I spend an hour listening and swatting before going back to sleep.  I breakfast on scrambled egg and bananas: my substitute for all the bread products I keep being offered.  The team is here: I watch them do a soil type analysis (using a flowchart: seeing if they can form muddy balls from the soil samples, seeing how long a ribbon of mud they can make (1cm? 2? more?) then finally rubbing in their hands to see if it’s “gritty” or not.  Most of the soil we collected is clay or sandy loam.  It rains.  Msofi and I swap British and Kiswahili words for different types of rain (we both have many of these).

I watch the team do rapid roadside assessments.  In rural areas (red dirt roads), these are done every 2km; on main roads every 20km, and in LOs (regions with a large concentration of surveys) every 4km.  A roadside assessment is just that: the team stands at the roadside, looks at the 50m by 50m area of land in front of them, assesses plant cover, plant types and use, and takes a panoramic photo of the area.  We stop and do 4 of these assessments on a dirt road, and mark the sheet for this with the from and to village names of the road.  I chat with Gervase, who is a seriously good systems thinker – he talks about how he persuaded rural people to adopt more efficient cooking stoves not because of environmental concerns: people living out here don’t understand the concept of save the forest, since the forest is all around them, but do understand the concept of “your eyes won’t get so red from cooking”.  In some areas, red eyes are seen as a sign of witchcraft, and people’s grandmothers have been killed for this.  I have serious respect for this guy and how he gets things done.

We go back to the hotel and go through the data entry and upload process, where data from the paper forms are entered into a tablet, then sent up to the head office servers.  The team are using Samsung tablets which have slots for simcards – available from Amazon, but I’m warned that the Dubai versions have a smaller simcard slot which means cutting down larger cards.  The team uses ODK Collect for its forms – these are available from the main server, but have changed many times already (mostly bug fixes, e.g. not being able to enter lat/longs ending in 0). We start with the soils form, but Joseph has the old version of the form on the tablet he’s holding.  He finds the right form on another tablet (internet here is still terrible), and walks me through the data entry.   There are many forms (and many versions of those forms): Eplot, soils, rapid roadside assessments, water rapid assessments, water lab reports, household surveys, agriculture surveys, farm field and crop surveys, the contents all of which we will have to make comprehensible (along with satellite data, open data etc) to decision makers.

We say our goodbyes and thank yous to the field teams (all Tanzanian, all experts who play games like “name that tree” with each other) who are starting their Easter break after 40 days in the field.  They’ve done roughly 50 of 350 Tanzania sites so far, and have much still to do.

We set out for the bright lights of Iringa, the big town in this area and a 2 hour drive away.  We talk about water sampling methods and the issues of vandalized equipment and data not getting from the basin offices to the central ministry of water.   There are potatoes roadside now, and big boulders that look like glacial moraine, which confuses me – were there glaciers here?  We talk about the timber lorries we pass – there’s a huge need for timber across Africa; there’s much construction, and the people who own woods will become rich on this.  I think about the transitions between old African and new Western-style systems that I’m seeing, and think about the things that get lost in that transition.  Some of these are tragic, e.g. there are many blue babies (brain damaged from lack of oxygen at birth) born here now because the traditional practice of midwives sucking on babies’ noses to clear gunk has been lost in the new Western-style hospitals, but hasn’t been replaced by the Western-style use of suction bulbs to do the same thing.

We reach Iringa, and start hotel-hunting: Avery knows two places in town with wifi. We try the first one: a craft shop, café and guesthouse run by disabled people.  The rooms are beautiful, but their wifi is out.  We hear about a guesthouse by the university, and try that – no wifi, but there’s a strong signal from the nearby internet café.  The five-story glass building opposite seems surreal after a week of one-story houses.  Life now is all about getting wifi, and getting back to ‘normal’ work.  It’s a catholic guesthouse – we’re staying amongst nuns and looking at pictures of the pope, but there’s also a bar in the evening.  We pick up wifi vouchers from the café (5000 ksh for 5Gb) and head off to lunch.  We’re in the tourist zone, and the first café we try is full of earnest young Americans, English menus and high prices.  We go round the corner to a local place and eat lentils, beans, rice and salads off tables with tablecloths and matching cruets then head back to the hostel to get online.  Nicky goes off to get the car fixed – the long fast drives over rough roads have damaged a pipeline and bearing. Which is painful… the bandwidth is so slow that OpenStreetMap goes to the low-bandwidth version, and I can see the titles of my emails (eventually) but not the contents.  I manage a brief Skype conversation with Nairobi before giving up.  Avery goes off to buy her bus ticket (she’s going up-country from here for Easter), and I haggle for local fabrics (blue chickens!).  Then we switch to plan b: the other hotel with wifi has a restaurant, so we head up into the hills above Iringa to a place that even my clean jeans feel a bit underdressed for, and eat Indian food with our laptops in front of us.  I finally get an OpenStreetMap editor open on the area that we were lost in, and show the team how well the red dirt roads and waterways stand out against the vegetation.  When I have good internet again, tracing will happen, so the field team have a small-scale map to start from next time.

We head back to bed – the hostel rooms have mosquito nets, so I sleep (in the trying-not-to-stick-anything-outside-the-net position, waking once to the sound of a frustrated mosquito in the room.

Tanzania day 4: Field Trips

Download PDF

We saw a lot of schoolchildren in uniforms yesterday – in the afternoon, they were walking past carrying hoes.  We go shopping for a cable to charge my phone and  camera: I’ve lost one cable and broken the other, so it’s off to the local shops, each of which points us to another one: general store, electronics shop, phone shops, camera/video shops, some in buildings, others in plywood shacks. Finally have two colourful cables with smiley faces on the ends (the powerstrips in the shape of hearts are tempting too). Avery buys fruit from a lady with a huge basket on her head: it takes 2 of us to the lift the basket back up.  We check the cable: it’s the car adapter that’s broken, not the cable: the team lends me an adapter for the ride.  Add to field shopping list: car chargers, and lots of them.  The car is filthy: I just miss a shot of it parked next to the same colour and model: left car is red; right car is brown.

Gervase and I talked about sample areas last night. One set of sites is in a wildlife area, with twin dangers – from the wildlife, and from the poachers chasing the wildlife.   The team can look “official” in their khakis – Joseph tried to ask a local man for directions yesterday, but the man ran away – a common thing here.  I have huge bruises up my arms, splinters everywhere, torn jeans – but no bugbites. We saw trees, plants, flowers yesterday but almost no wildlife: some insects and butterflies (not many), birds (ditto): nothing larger, although we did find wild pig poo and some elephant damage on a tree.  We also found an animal trap on the path (Joseph got caught in it), but the jungle was eerily silent.

We arrive and walk to the first site (300 metres through open woodland – yay!), but the site’s start point is in a swamp.  This happens occasionally, and there’s a protocol for it: the team moves the corner to dry ground with similar vegetation to the swamp, and marks on the papers that they’ve done this.

Joseph explains the sampling protocol in detail and I take many many photos of soil, buckets and poles.  First tree-counting: they use the ranging pole (long pole with tape markers every 50cm) to find all the trees within 1.5m of the plot start point (“corner”), then they measure every tree over 5cm diameter and 50cm height (knee height).  They do this so the analysis team can estimate carbon sequestration in this area.  Each tree’s diameter is measured with a tape; the small tree heights (5m or less: the ranging pole is 2.5m long) are measured with the ranging pole; the larger tree heights with a clinometer; the tree’s canopy width is measured with the ranging pole (in dense woodland, one person shakes the tree so the others can see where the canopy is).  Once the team starts on a plot, the same team member does the same measurement jobs at every “subplot” point: because people’s estimates vary, this gives some consistency of measurement across the plot.   The team takes soil samples by putting down the metal plate and pushing a corer (also marked up with tape) through the hole in it; putting soil into buckets marked 0-20cm (topsoil), 20-50 (subsoil), 50-80 and 80-100cm (the 80-100 sample is only taken in the first corner): these depths are based on the Afsis (African Soil Information System) sampling protocol.  The team also throws down a quadrant (50cm metal square) and counts the species in it, how much of each it contains, how much bare earth, dung etc; and uses a densiometer (mirror marked into squares) to estimate the canopy cover north, east, south and west of the start point. They also hang a scale bag in a tree, put 500-900g of each soil in a marked plastic bag, weigh the bags, and also weigh marked metal pots containing samples of each layer of soil (the pots are added to a plastic bag attached to the scale).

We hear a shotgun – the team thinks it’s just a car backfiring or tyre blowing out – there are very few guns in this region.  We move on to the subplots.  What we’re doing is called the “E” plot – we’re sampling a 100m by 100m square area, with “subplot” points every 20m in that grid (e.g. there are 36 subplots to a plot).  The shortest path with the smallest errors through that grid is in the shape of a large “E” (eeeeessssswwwwwneeeenwwwwneeeenwwwwn), hence the name.   At each subplot, the team counts and measures trees (trunk diameter, height, canopy width, species) and estimates the biomass only in the quadrant (from the average plant cover and heights).  The team used to do the full works (everything done at the first corner) at each subplot, but the process was long and has now been streamlined.

There are lots of trees here – I learn Kiswahili for “please shake the tree”.  It rains a little, and we have 33 subplots to go.  There’s more noise here than in the jungle: birds are singing, insects are buzzing.  I ask about wildlife protocols – there aren’t any animal, insect or bird protocols yet.  The team were going to set camera traps for animals, but they were too expensive; I wonder if there’s something small and cheap we can hack together with a camera board, motion detector component and microprocessor.

We move over the road to a Pinus Patula (misheard as “Spatula Pine”, which causes much amusement) plantation – well-spaced small trees with soft spines and wild mint growing underneath.  We talk about Apopo’s work on mine-sniffing and TB-sniffing rats, and how this works in Africa but not so much in SE Asia.  At every corner of 100m grid, we take soil samples and do detailed quadrant analysis; we also do this at the centre of the plot, where we take a panoramic photo (the camera has built-in GPS).  I’m still picking thorns out of my head; at the centre, I sit on a fallen log, and Avery rushes to brush the ants off me (I now have ants in my pants).  It rains heavily. The team gets wet, and the car goes red again.

The team has spilt into two, to get two sites done for the day.  We drive over to the other site; it’s raining heavily so Avery and I sit in the car and work on our laptops.  She gets a good data signal for the first time in days (the signal at the hotel is non-existent) and I manage to post an “I’m safe” home.  The flowers here are beautiful, making the site look like an English cottage garden: huge purple mallows and something that looks like clematis overlaying delicate yellow flowers – I wonder how many common British garden plants have come from here.  It stops raining for a while (this rain is monsoon-grade), so I walk out to the site: across a swampy valley, through woods, across grassland to a firebreak between woods.  I find the team’s start point (an umbrella over the sample pots), and track them to the edge of the woods – which look impenetrable: someone has chainsawed the plantation but, puzzlingly, not taken away the fallen wood.  Trees have grown up through the fallen wood, and the whole effect is one of a giant woven basket.  I hear the team’s voices, find a not-so-bad patch, and push through, under, over, across trees to reach them.  They seem surprised: they’ve been pangaing through these woods for hours now, and weren’t expecting anyone to just go through them.   They’re still on the outside legs of the “E” plot – with 22 subplots to go.  These are very different woods than the morning, so I tag along to see how they sample in dense woodland.  There’s a lot of scrambling but not so much wait-a-minute here: today is branch scratches rather than thorns.  Moses the biologist tells me that there are two species of wait-a-minute, and that the one by my head is related to the orange tree – I crush a leaf, and yes, it smells of oranges.  In the firebreaks, I see animal tracks going into the wood – later, whilst crouched under a fallen tree, I hear what sounds like a boar grunting annoyance.  It gets late: 5:30pm in a place where the 6:30pm sunset is a sudden from light to dark.  The team has 10 plots left to do, and push on, quickly measuring trees and assessing the ground cover.  Just before sunset, we finish and rush back to the cars before dark.  I’m wet, cold, muddy, sunburnt (regretting not bagging that aloe) and happy that I understand a lot more about what it takes to collect this data, what it means and what we could do to help.

Tanzania day 3: Welcome to the Jungle

Download PDF

Today we go into the field.  Woken by laptop charger fizzling – electricity is available but a bit variable here.  Breakfast with sweet milky tea – the tea in it is grown and picked here in Mafinga.  We drive past tree plantations – pines, for their wood. I ask about the rice; Tanzania is a major rice grower, and much of it comes from Morogoro.  I meet the rest of the team, and explain Ushahidi and my own skills to them, as asked, armed with a notebook, pen and much arm-gesturing.   We drive off to the site; on red mud roads, fast.  The team truck has a snorkel and I worry they might be using it.  Finding routes to the sites is an issue, and the gps units fill up if the team trys to track roads: we talk about roadmapping using their GPS-connected tablets and Funf, and about OpenStreetMap traces from Bing’s satellite images.

This area (Mafindi)’s economy is based on trees, tea and maize; I see eucalyptus (grown for its wood) and other trees grown for paper.  I see some cows – we talk about the perceived difference between pastoralists (many cows, with perhaps a small piece of land for vegetables) and farmers (mostly crops, with perhaps few cows), the conflicts between them, and how farmers often don’t count the cows as part of their farming.  We see the first tea plantations of many – I’m surprised that the tops of the plantations are flat rather than tea bushes.

We drive fast for 2 hours on dirt roads – we’re remote but trade is visible; motorbike repair shops and women wearing cloth made in Nigeria.  We see cabbage fields – European missionaries settled here because the climate was familiar to them; there are some churches but also many schools in this area.  It rains again – the roads are getting muddy now.

We stop on the road.  We’re 7km from the site, so we go back and try another road.  The logging trucks are out now (11am) – Mafindi Paper company, and little trucks with offcut bark.  If only we had a roadmap.  We talk about how to improve tea production efficiency, and wonder about mechanical harvesters – just before we see mechanical tea harvesters: giant lawnmowers pulled over the tops of the tea plants.  We reach a dead end: 6km away now.  Really really need better roadmaps for this project; I can feel an OpenStreetMap session coming on.   And here’s how to help:  there’s Internet here in the sticks, but it’s slow – too slow to sensibly edit maps from here.  But internet is good in New York, London etc: so if you help OpenStreetMap make better maps of this region (southern Tanzania), local people can get on with the things that they need to do, like plant monitoring, instead of getting lost.  I’m told it takes 4-5 hours to process each plot, and that each plot will be revisited every 3 years.  If you add getting lost onto that, it becomes a very very slow process.

We stop at a dead end 5km away from the sample site. It’s getting late, so Joseph, the field team lead, asks me if I’m okay with walking 5km (3 miles) – he explains that 5km straight is probably going to be a lot further on the trails, and that it might be a bit up and down.  I look at the tea plantations around us, and think “hey, this is just like walking in Dorset”.   We set out… across the team plantation and down into woodland – walk downwards for a while, then retrace our route because two of the team are yelling that there’s a river in our way.  We set out again… down through woodland, across a small brook (which I’m hoping is the river), up past a small shack with a fire going, and small garden with mint and vegetables (Joseph explains that sometimes the farmer stays with his fields), and up through a maize field.  The field is closely planted – I follow the voices ahead of me.  The field is underplanted with courgettes, which I take care to step around; and then we’re out into another field – some type of wheat?  at the top of a hill.  We go down through woodland again – a slippery muddy path that looks well-used. Someone mentions that we’re going down to the bridge the plantation workers told us about.   I think “ah – a roadway; great’. They don’t mention that the ‘bridge’ is a pair of tree branches across the river.  The guys walk across one of the trees then jump onto the far bank; Avery and one of the guys wade waist-deep across instead.  I take the tree – the guys build me steps down out of their field plates.   This is the first time I hear them say MamaSita; I hear that a lot soon.  We walk along a small trail going up through the trees – and then the trail stops.  It’s panga time: Joseph starts hacking a trail through the jungle, and the walk becomes a long repeat of ducking under vines, picking “wait a moment” (bramble-like plants) off our arms and heads and waiting for enough of a trail to be cleared.  There are holes in the jungle; I step over most of them, but it’s muddy, and sometimes I slide thigh-deep into them.  The guys talk all the time – laughing, teasing each other, talking about politics.  We stop every so often, and call out the distance to the sites.  First stop, it’s still 5km.  Then 4.8 km; we have a
picnic” of samosas (or in my case samosa innards: it’s tough being gluten-free in the jungle).  After 3 hours ducking through the wait-a-moment (jungle: I thought snakes and big cats would be an issue: turns out it’s falling into holes and picking big thorns out of your head), it’s 3:30pm, we’re 3.6km away from the sites, and have 3 hours of light left for the day.  We turn back, planning to return the next day.  The route back takes 1.5 hours: when we return, Nicky has collected local pears for us all.  We drive back past towns with repair shops and chickens, past recently-logged areas and more logging trucks.  I’m exhausted and covered in scratches and bruises – I crawl off to sleep for a while.

Tanzania day 2: the Safari Commute

Download PDF

Today I meet the team: we breakfast, talk about the plan for the week (travel, measure, measure, rest, travel) and set off on the road to Iringa.  I’ve now been in 2 of the “big 5” wildlife countries and so far have seen: 1 dog.  I’m hoping we might see something else in the parks.  The road is very quiet – most of the traffic has stopped because of the traffic jams around the flooded bridge, which is great in terms of having the road to ourselves, but no so great in terms of being the only car around for the traffic police to stop.  They stop us and show the radar gun (the most common sensor that I see around here) – speeding.  We stop next to one of the communities making woven baskets – I’m tempted to go shopping but know that would just increase our chance of a fine.  I see another dog.  We pass the waterfall where our driver once took a hippy who tried to teach him about transcendental meditation.  All the police want to talk to us, to see the car’s papers – today they must be bored.

And then we enter the Mikumi National Park.  Right away there are baboons – mothers with frisking children, big proud males with bright bulbous bottoms.   Then giraffes, posing tall under shady trees. Impala peeking through the bushes.  Elephants ranging in the distance. Wildebeast and zebra sharing a watering hole.  And more giraffes.   Someone jokes that this is the “safari commute”.  We eat good African food just outside the park (last night the hotel staff made me a late meal: of chicken and chips, then eggs and frankfurters for breakfast), then continue through the plains and along a river.  As promised, there are Masai herding cows whilst on their mobile phones (“they carry two at least”), and small boys with sticks and goats.  I look at an NGO crew in their big white 4×4 and wonder how many of them actually know what it’s like to be poor – not poor as in student, but poor as in having to make the difficult choices you make to survive.  We see onion stands near the river – you can map your location here by what’s on the vegetable stands: onions, mango, tomato, peppers, and finally, in the high plains past Ikaya, potatoes.   We climb, past a crashed lorry, up a mountain road that Nicky tells us once had a phone at each end because it was too narrow for vehicles to pass).   We’re in high meadowlands now, and there are many sunflower fields.

We stop in Mafinga and choose a hotel – the cheaper one that just opened today.  I sign in as their first guest, and put “Webb” in the “tribe” column.   We eat local chicken, plantains, rice and big sweet avocados and talk about maps.

Tanzania Day 1: how was your day at the office?

Download PDF

Day 1 – not going well. 5am start in Nairobi – check.  Flight over Kilimanjaro – check. Car waiting at airport – check. Takes credit card as promised – nope. Hack Tanzanian cashpoint to get enough cash to pay.  Buy sim card: simple, but not instant (forms!). And off.. to mall to get cheap camera and supplies.  Cheapest camera = 220000 ksh ($130).  Asked $80000ksh for bugspray and sunblock. Decide to buy supplies in tourist area upcountry.  But first, driver1 has forgotten the car’s id docs – an issue on a road with policement every mile or two.

And so we get into the first traffic jam (10 mins): the president’s mother lives nearby, and visits his mother every weekend (even presidents aren’t immune to this).  Everything stops as his cavalcade drives past – except today it’s the vice-president visiting.  Second traffic jam: 10 minutes for a busy traffic junction in Dar – the traffic policeman routes us onto a lane full of motorbikes coming towards us, and we test the 4×4 going offroad to get round a car stuck in front of us.  And onto the main road out of Dar, and from Tanzania to Zambia, Malawi and Congo.  Third traffic jam: 10 minutes as everything stops, and the president’s cavalcade dashes past.  The rainy season has started, and we hear that the road is damaged ahead.  Fourth traffic jam:  we drive past a long line of stopped trucks, to where the president is inspecting a subsidence (kind of a deep wrinkle across the road) – everyone is walking forward out of their cars to see and hear him.

And then the fifth traffic jam.  Everything stops. For miles.  We drive past lines of trucks, then cars, then buses (there are people everywhere now) then cars.  We stop on a bridge – below us, a group of Masai are stood on the riverbank, watching the trafficjam like tourists.  We’re told that the road ahead is flooded, and a channel’s being cut to divert the floodwater: perhaps a 1-2 hour wait for this to be done.  Motorbikes and big 4x4s with snorkels swarm past us: I make a mental note to hire something with a snorkel next time.  As we drive forward past the stopped cars, a passerby shouts that the problem is “too much crocodiles”. Everyone laughs; we’re all stuck, and this is rapidly becoming a social occasion.  A vanload of traffic police drive past in the opposite direction – perhaps they’ve given up?  It rains again, and we sit and wait for news.  And the president returns – slowly this time because there’s no room left on the road to go fast.  Merchants appear, selling fresh-roasted cashewnuts, which I eat looking at the trees they came from (and smelling the next batch roasting).  I read the boring textbook about javascript that I’ve been avoiding, chat with Nicky (the driver that Esther insisted I had for Tanzania) and people-watch. I learn that Masai carry a stick and an umbrella (and a gust of wind reveals a silky pair of pale-blue boxershorts.  I learn how useful traditional dress is when a girl has to go off into the bush. I learn Kiswahili for “white” and “cashew” and “bugger off, we’re not paying that”.

It’s now 4 hours since we parked on the bridge (we’re now at two lanes of parked traffic, with the 4x4s and motorbikes still swarming past), and we start to move., hopeful. People drive past the jam, taking video of it on their mobile phones.  My own phone dies (add to shopping list: car charger for phone).

We hear that the bridge is clear now, but a line of cars has overtaken the waiting buses, and another line of cars has overtaken that one, on both sides of the bridge. The main route from Tanzania to Kenya now has 3 lanes of static traffic (4 or 5 if you count the bikes; 5 if you count the 4x4s creating a new cross-country trail) with nowhere to go, and no traffic police in sight.  It’s going to be a long time getting over this bridge.

7.5 hours after entering the trafficjam, we see what it’s all about.  First, we drive down onto the river’s floodplain: it’s dark now, and there’s water either side, with crocodile eyes reflecting in knobbly heads just enough distance away to be menacing but hard to see.  The the water – first a static flood across hald the road; not moving, nothing to worry about. Then it’s moving, and quickly becomes a torrent, sideways across the road, getting deeper and deeper as we cross until we reach two trucks stranded on their sides, motorbikes piled across their chassis.  This isn’t a good sign; I start regretting my earlier “just line up the crocs” jokes, and wonder who the god of angry rivers is hereabouts.

Then we’re through, breathless, stunned that we just tried that (but after 7 hours’ wait, what the heck) – and stop again.  We’re stalled on the other side of the river by a mirror of the lines of cars that we just left.  We’re hungry and thirsty – and by luck, stranded in the middle of the night market that many people had come to in the morning before the river broke over the road.  I learn how to jostle for drinks with Masai tribesmen (British hard stare doesn’t work, waving money at barkeep does), and Nicky and I settle into trafficjam number 6 (a van has broken down on the road ahead of us).   We move at last, and drive past the jam on the other side of the bridge. This goes on for miles. People are sleeping in front of their trucks, and in the roadside near buses.  And I’m quietly struck by the thought that this is what a crisis evacuation looks like- the jams, the queues, the rumours, the hope, the decisions: do I stay here with the bus or walk forward in the jams? What do I eat? What should I save just in case? And how can I help other people?  The answer to the last one is “with information” – as we drive, Nicky shouts to stranded drivers about the situation, the bridge, and the length of the queue ahead of them.

Writing an Ignite Talk

Download PDF

Ignite talks are the standard format of events like ICCM and other GIS-focussed events. They look great on stage, and might seem impossible to do if you’re not used to speaking. But it’s not that bad really. You too can write and present and ignite talks!

So what *is* an Ignite Talk?

An ignite is a 5-minute talk where you supply 20 slides. Each of those slides is shown for 15 seconds before automatically moving to the next one.

5 minutes. That’s not too bad.

How do you start planning an Ignite talk?

Here’s how I do it. This isn’t the “right” way – it’s just one of many – but it’s a place for you to start.

First, know what you want to talk about. For instance, I want to give an ignite talk about giving ignite talks. You know the general area of the event (e.g. “crisismapping”) – what about or around that area excites or worries you? What have you been talking a lot about this year? Tell the stories you’re already telling… for example, this might be Leesa talking about virtual PTSD, Om about organisation, Rose about some VOST work she loved. Or tell a new one you want to explore – “if we could do this, then…”. Write a first sentence about each story.

Then start thinking about what’s important to you about that story. Where are you going for information about it? Who’s done really useful things about it? What would you do if you had unlimited resources? Berkun suggests picking 4 important points to make in your story, but you might have 2, or 5 or 6. Start listing your points for each story.

Then try talking for 5 minutes about each story. It took me years to figure out that a talk isn’t about you standing up and being judged by the audience – it’s a conversation between you and them, a way of getting people to talk about and act on things that you care about. Think about telling your mother or grandmother or best friend about this theme… what would you tell them? Write it down – or if you’re not great at writing things down, either record yourself talking and write it up later, or talk to someone else and get them to write down what you say. And draw pictures (they’ll be useful later).

Outline your script. I usually start a googledoc that looks like this:

 Title of talk
 Slide 1: introduction
 Slide 2: point 1
 Slide 20: thank you and goodbye

– I usually have the first slide for an introduction (and getting on the stage), the last slide for thankyous and reiterating those major points (and getting off the stage), and give each of points an equal number of the remaining 18 slides. At least, that’s where I start – I often realise that some points are bigger than others, and adjust the slides accordingly. Sometimes it makes sense to devote some of the earlier slides to background – that’s fine too. The important thing is that you start writing, and that you know that at this stage it’ll be a long way from the perfect performances you see up on the stage.

Start writing your script. You should by now have 1) an outline document, and 2) the text from talking to your friends, grandmother etc. Start putting them together: put your words into your outline, and adjust both of them to fit. Remember that it’s okay to “cheat”: for example, if you want to talk longer about one slide, then repeat it; and go watch some videos of ignite talks (www.crisismappers.net has lots of these) to see how other people do it. At this point, you don’t need to write essays – 15 seconds of talking isn’t much more than one paragraph of text, so a sentence of two per slide is fine.

Find images. You’re going to need something on your slides. At this point, your talk isn’t polished, and that’s a good thing – because when you start looking for images, you’ll probably want to adjust it again. We’re lucky – we do a lot of work that’s visual (e.g. maps and documents) and can be either used directly (jpgs) or captured using a screen grabber (see below). There are also a lot of free images and clipart (cartoon images: try googling “free clipart”) on the internet too. Avoid bulletpoints and lots of words if you can – your audience will be reading those rather than listening to you (which isn’t a good thing,no matter how shy you are); using a single word or sentence can be very powerful though, so consider this as an option too.

Tidy up your script. By now, you have 20 images and a script. You remember that 15 seconds per slide? Time to practice it. Pick a random piece of text, find a stopwatch, breathe slowly, talk slowly and read out the text for 15 seconds, leaving a short gap between each sentence. For me, that’s a small paragraph – about 3 sentences. Go back over your script, and first tidy up by eye (editing and moving text so you get your points across in the time that you have available), then time reading out the script for each slide, and adjust until you’re somewhere near 15 seconds, speaking slowly.

Record your talk. Now you have 20 slides and a 5-minute script that matches them. Time to record yourself. Powerpoint allows you to auto-advance slides and include an audio track (see below for details); it also allows you to re-record the audio for each slide, so you can record each slide separately and overwrite anything you’re not sure about. Go do this. And now you have an ignite talk!

Write an abstract. Nearly done. A lot of conferences ask you for an “abstract”, or summary of what the talk is about. You have your story above- write a paragraph that describes it, and send it on in!

Where can you find more advice?

Here’s some advice from people who’ve given ignite talks before:

Useful Tools

Future cities

Download PDF

Cities are apparently the future. All the predictions I’ve seen for the next few decades show the world\’s population concentrating in cities, but our development indicators and policies are still listed by nation state. Perhaps they should be wider, for instance by including developing cities on the lists.

I said “developing” there – which begs the question “how are these cities developing?”.  This isn’t just a Las Vegas-style spreading of suburbia across the desert: many of the cities I’ve visited in the past year have shanty towns, and these appear, at least from outside, to be where a lot of the city development is happening (btw, I wanted to use a less emotive word than ‘slum’ here: although it’s what Slum Dwellers International uses, there’s still a lot of negative feeling about it).  From Lagos to Guatemala to Haiti, I’ve seen dozens of homes and businesses under tin roofs looking across at smaller numbers of tower blocks, and wondered “how do these economies fit together”, and “where does it go from here”.

Good old BBC gave me a few more answers… and a few more questions (like how does the nation-based world fit with people this adaptable and informal), and case studies, Medellin and London, of both positive and negative ways that the shanty and non-shanty worlds can start to fit together.

Perhaps it’s the way you look at it.  If you look at the Wikipedia links above, you’ll see shanty towns and slums described in very negative terms… impoverished, illegal, lack of services.   If you hang out with people who live or work in shanty towns, they’re communities and neighbours and businesses and services – and quite possibly the adaptable, informal, majority economic future of the cities they\’re part of.  Whichever way you look at it, there are a lot of people in shanty towns, and how (and what) they develop is important.

Creating humanitarian big data units

Download PDF

Global Pulse has done a fine job of making humanitarian big data visible both within and outside the UN. But it’s a big job, and they won’t be able to do it on their own. So. What, IMHO, would another humanitarian big data team need to be and do? What’s the landscape they’re moving into?

Why should we care about humanitarian big data?

First, there’s a growing body of evidence that data science can change the way that international organisations work, the speed that they can respond to issues and degree of insight that they can bring to bear on them.

And NGOs are changing. NGOs have to change. We are no longer organizations working in isolation in places that the world only sees through press releases. The Internet has changed that. We’re now in a connected world, where I work daily with people in Ghana, Columbia, England and Kazakhstan. Where a citizen in Liberia can apply community and data techniques from around the world, to improve the environment and infrastructure in their own cities and country.

We have to work with people who used to be outsiders: the people who used to receive aid (but are now working with NGOs to make their communities more resilient to crisis), and we have to work with data that used to be outside too: the tweets, blogposts, websites, news articles and privately-held data like mobile money logs and phone top-up rates that can help us to understand what is happening, when, where and to whom.

UN Global Pulse was formed to work out how to do that. Specifically, it was set up to help developing-world governments use new data sources to provide earlier warnings of growing development crises. And when we say earlier, we mean that in 2008 the world had a problem. Three crises (food, fuel and finance) happened at once and interacted with each other. And the first indicator that the G20 had was food riots. The G20 went to the UN looking for up-to-date information on who needed help, where and how. And the UN’s monitoring data was roughly 2 years out of date.

What have we done so far?

So what are the NGOs and IAs doing so far? The UN has started down the route to fix this with a bunch of data programs including Global Pulse and FEWSnet. Oxfam connected up to hackathons last month; the Red Cross has been there for a while. The World Economic Forum has open data people, as does the World Bank. And other groups as different as the Fed and IARPA are investigating risk reduction (which is the real bottom line here) through big data techniques.

What should we be doing?

But what do the NGOs need to do as a group? What will it take to make big data, social data, private data, open data and data-driven communities useful to risk-manage for crises?

1. First, ask the right questions.

When you design technology, the first question should be “what is the problem we’re trying to solve here?” Understand and ask the questions that NGOs do and could ask, and how new data could help with them. There is data exhaust, the data that people leave behind as they go about their lives: focus on the weak signals that occur in it as crises develop. Reach out to people across NGOs to work out what those questions could be.

2. Find data sources.

We cannot use new data if we don’t have new data.

Data Philanthropy was an idea from GFDI to create partnerships between NGOs, private data owners like the GSMA mobile phone authority and other data-owning organisations like the World Economic Forum. Data Commons was a similar idea to make data (or the results of searches on data – we want to map trends, not individuals) available via trusted third parties like the UN. It’s gone a long way politically but still has a lot of work to be done on access agreements, privacy frameworks and data licensing.

Keep encouraging the crisismapping and open data communities to improve the person-generated data available to crisis responders, to improve the access of people in cities and countries to data about their local infrastructure and services, and to voice their everyday concerns to decision makers (e.g. via Open311). Encourage the open data and hacker movements to continue creating user-input datasets like Pachube, Buzz and CKAN. All this is useful if you want to understand what is going wrong.

3. Find partners who understand data.

Link NGOs to private organisations, universities and communities who both collect and process new types of data. Five of these recently demonstrated Global Pulse led projects to the General Assembly:

    • Jana’s mobile phone coverage allowed us to send a global survey to their population of 2.1 billion users in over 70 countries. There are issues with moving from household surveys that need to be discussed, but it allowed us to collect a statistically significant sample of wellbeing and opinion faster and more often than current NGO systems (an authoritative survey I read recently had 3500 data points from a 5,000,000 person population. Statistical significance: discuss).
    • Pricestats used data from markets across Latin America to track the price of bread daily rather than monthly. Not so exciting in ‘normal’ mode or in countries where prices are regularly tracked. Incredibly useful during recovery or for places where there is no other price data gathered.
    • The Complex Systems Institute from Paris tracked topics emerging in food security related news since 2004. This showed topic shifts from humanitarian issues to food price volatility (with children’s vulnerability always being somewhere in the news). More of a strategic/ opinion indicator, but potentially incredibly useful when applied to social media.
    • SAS found new indicators related to unemployment from mood changes in online conversations – several of which spiked months before and after the unemployment rate (in Ireland and the USA) changed. This gave new indicators of both upcoming events and country-specific coping strategies.
    • Crimson Hexagon looked at the correlation between Indonesian tweets about food and real food-related events. The correlations exist, and mirrored official food inflation statistics. Again, useful if gathered data isn’t there.

And reach out to the communities that are forming around the world to process generated data, from the volunteer data scientists at Data Without Borders to the interns at Code for America and the GIS analysis experts connected to the Crisismappers Network.

4. Collect new data techniques and teach NGOs about them.

There is a whole science emerging around the vast ocean of data that we now find ourselves swimming in. It has many names, Big Data and Data Science being just two of them, but it’s basically statistical analysis of unstructured data from new sources including the Internet, where that data is often very large. Learn about them, play with them (yes, play!), and teach people in NGOs about how to use them. The list of things you probably need to know include data harvesting, data cleaning (80% of the work), text analysis, learning algorithms, network analysis, Bayesian statistics, argumentation and visualization.

And build a managed toolkit of open-source tools that NGOs and analysts in developing country can use. For free. With support. Which doesn’t mean “don’t use proprietary tools” – these have a major part to play too. It just means that we should make sure that everyone can help protect people, whatever the funds they have available are.

5. Design and build the technologies that are missing.

Like Hunchworks. Hunchworks is a social network-based hypothesis management system that is designed to connect together experts who each have part of the evidence needed to spot a developing crisis, but don’t individually have enough to raise it publically. It’s a safe space to share related evidence, and give access to the data and tools (including intelligent agents automatically searching data for related evidence) needed to collect more. It’s still in alpha, but it could potentially help break one of the largest problems in development analysis: namely, the silos that form between people working on the same issues and the people that need to see their results.

6. Localize.

Build labs in developing countries. Build analysis capacity amongst communities in developing countries. People respond differently to economic stress, and environments, data sources and language needs are different in different countries. The labs are there to localize tools, techniques and analysis, and to act as hubs, collectors and sharing environments for the types of minds needed to make this work a reality. No one NGO can afford to do this in all countries, so connections between differently-labelled labs will become vital to sharing best practice around the world.

7. Publicise and listen.

Be there at meetups and technology sessions, at hackathons and in Internet groups, listening and learning to do things better. And never ever forget that this isn’t just an exercise. It’s about working better, not building cool toys – if the answer to a problem is simple and low-tech, then swallow your pride and do it – if the answer is to share effort with others to get this thing worker faster to protect people around the world, then do that too. We do not have the luxury of excessive time or meeting-fuelled inaction before the next big crisis strikes.

Lessons from mapping Sahel

Download PDF

We needed an example problem set for our current version of Hunchworks (note that this is a very early, i.e. pre-alpha version of the code and a lot of the cool Hunchworks features aren’t in it yet). The UN’s main use for Hunchworks is to gather up the weak signals that people put out about emerging development crises – those small hints that something isn’t right that appear all over the world before they coalesce into ‘obvious’.

Awareness of development crises can happen very quickly. One minute there are whispers of a potential problem – a chat here, an email or text asking for a bit of data there. And then a tipping point appears and there’s suddenly data everywhere. And we have a great example of this happening just at the time that we’re demonstrating Hunchworks to the UN General Assembly.

We had one of these serendipitous test sets before: we tracked the Horn of Africa crisis emerging across newsgroups as one of our early will-this-work paper exercises (this btw is also why we’re suddenly interested in data mining googlegroups).  But the Horn of Africa crisis is well established in the public eye now, and there is both too much online data on it to pick out the early weak signals, and many of the early traces (e.g. anecdotes and messages) have been lost in both human and machine memories (yes folks, not everything on the Internet is logged).  But there’s a new thing starting to happen (which is potentially very very bad for the world and certainly for the people caught up in it) – over the last month or so there have been mysterious messages here and there about something starting to happen in Chad and Niger, across the African region known as Sahel.

So this week we started collecting information about Sahel and turning it into hunches and evidence in Hunchworks. First we had an email from a trusted colleague containing potential places to look. Then an Internet search for news and background information, followed by more digging across the Internet (including recent reports from the UN and other NGOs), a colleague searching in the food-related UN agencies and a Twitter search for hashtags and interested parties.

We could have got a lot more information faster and with more insightful comments into it if we’d crowdsourced its collection, but the EOSG couldn’t be seen getting involved prematurely (i.e. before the dedicated HLWG team) in this crisis. We could have also involved more people in a non-public search if we’d had Hunchworks at its Beta testing stage, but doing the first search by hand is a sane early stage test that exposes bugs early to a small number of people before a larger group (e.g. the Alpha and Beta testers) get annoyed by them.

So what did we learn about ourselves, our information and Hunchworks from this exercise?

We ran the exercise using a spreadsheet (no, we can’t just do this for hunches because it will quickly become overwhelmed – see below). Its first worksheet was a list of hunches: this was quickly populated with a mix of hunches and evidence for hunches that took some time to separate out. We also discovered that the evidence that we gathered often contained new places to look for evidence, suggested new problems that should be proposed as hunches and spawned a whole pile of other evidence-gathering activities.

  • Lesson 1: people confuse hunches and evidence. Sometimes evidence is posted on the hunches list; other times hunches are posited as evidence to another hunch.
  • Lesson 2: evidence generates hunches. For example, we realized that a hunch about a famine in Sahel also contained hunches about famines in Mali and Chad that the country teams there needed to investigate.
  • Lesson 3: evidence generates evidence-gathering activities. We ended up with a to-do list linked to each hunch.

Some things were confirmed for us. We suspected that we could map the connections between hunches as though the hunches were propositions in a reasoning system. We also suspected that there were a set of basic search actions that we would do at the start of most hunches, that some of them would be ongoing (i.e. to catch new information being added to the Internet) and that we could automate many of these. Yes.

  • Lesson 4: when we draw graphs using our hunches as nodes, the links between nodes look suspiciously like the links in semantic networks. This should come as no surprise to anyone working on linked data.
  • Lesson 5: Google searches, news searches, twitter searches, UN report searches and emailing around likely suspects are obvious first things to do on any new hunch.
  • Lesson 6: We can automate some of the above searches, especially if we have search terms (e.g. Sahel) and tags to start from. Our hitlist for Sahel was: twitter stream, food price, migration from/to Sahel and news monitoring agents.

We’ve tried to build a system that doesn’t need much management or moderation. We might need to revise that: in the initial excitement of chasing up leads and links from the original hunch, it was difficult to maintain momentum (e.g. amount of evidence added) and completeness at the same time. I had to do a lot of reading and editing – both to disambiguate hunches and evidence as discussed above, but also in generating tags, thinking about the links between hunches and managing the list of actions that happened most times we added any evidence. Some more lessons from this:

  • Lesson 7: Hunches have information-gathering actions attached to them.
  • Lesson 8: Once we get textual evidence, it’s pretty easy to create tags from it.

And then we’ve got some very specific lessons about the system.

  • Lesson 9: if two hunches are related, they probably need the same people involved in them. Can we start the “involve x” list of one from the other?
  • Lesson 10: Some places had 2 locations, e.g. migration from Libya into Chad.
  • Lesson 11: We have the same problem as crisismappers with location accuracy, e.g. sometimes we want to mark a region rather than a single point on the map.
  • Lesson 12: Using tags brings a set of questions about how we find related things again. This is the same issue we’ve seen in crisismapping and Twitter feeds, and we have tools that can help with this.

There are more lessons learnt, but we’re somewhat busy today. More soon.

Strata talk on hunchworks technology

Download PDF

I try not to put too much dayjob stuff here, but sometimes I need to leave less-tidy breadcrumbs for myself.  Here’s the 10-minute (ish) talk I gave at Strata New York this year.


I’m Sara Farmer, and I’m responsible for technology at Global Pulse. This brings its own special issues.  One thing we’re passionate about is making systems available that help the world. And one thing we’ve learnt is that we can’t do that without cooperation both within and outside the UN.  We’re here to make sure analysts and field staff get the tools that they need, and that’s a) a lot of system that’s needed, and b) something that organisations across the whole humanitarian, development and open data space need too.

<Slide 1: picture of codejammers>

We’re talking to those other organisations about what’s needed, and how we can best pool our resources to build the things that we all need.  And the easiest way for us to do that is to release all our work as open-source software, and act as facilitators and connectors for communities rather than as big-system customers.

<Slide 2: open-source issues>

Open source isn’t a new idea in the UN – UNDP and UNICEF, amongst others, have trailblazed for us, and we’re grateful to have learnt from their experience in codejams, hackathons and running git repositories and communities.  And it’s a community of people like Adaptive Path and open source coders who make HunchWorks happen, and I’d like to publicly thank their dedication and the dedication of our small tech team.  We have more work to do, not least in building a proper open innovations culture across the UN (we’ve codenamed this “Blue Hacks”) and selecting licensing models that allow us to share data and code whilst meeting the UN’s core mandate, but in this pilot project it’s working well.

<slide 3: bubble diagram with other UN systems we’re connecting to>

We’re already building out the human parts of the system (trust, groups etc) but Hunchworks doesn’t work in isolation: it needs to be integrated into a wider ecosystem of both existing and new processes, users and technologies.  The humanitarian technology world is changing rapidly, and we’ve spent a lot of time thinking about what it’s likely to look like both soon and in the very near future.

So. For hunchworks to succeed, it must connect to four other system types.

We need users, and we need to build up our user population quickly. We’re talking about lab managers, development specialists, mappers, policy coordinators, local media and analysts. Those users will need also specialist help from other users in areas like maps, crowdsourcing, data and tools curation.

  • UN Teamworks – this is a Drupal-based content management system that a UNDP New York team has created to connect groups of UN people with each other, and with experts beyond the UN.
  • Professional networking systems like LinkedIn that many crisis professionals are using to create networks.
  • Note that some of our users will be bots – more on this in a minute.

Users will need access to data, or (in the case of data philanthropy), to search results.

  • UN SDI – a project creating standards and gazetteers for geospatial data across the UN.
  • CKAN  – data repository nodes, both for us and from open data initiatives.
  • Geonode, JEarth et al – because a lot of our data is geospatial.

They’ll need tools to help them make sense of that data. And bots to do some of that automatically.

  • We need toolboxes – ways to search through tools in the same way that we do already with data.  We’re talking to people like Civic Commons about the best ways to build these.
  • We’re building apps and plugins where we have to, but we’re talking about organisations putting in nodes around the world, so we’re hunting down open source and openly available tools wherever we can. We’re waiting for our first research projects to finish before we finalise our initial list, but we’re going to at least need data preparation, pattern recognition, text analysis, signal processing, graphs, stats, modelling and visualisation tools.
  • Because we want to send hunchworks instances to the back of beyond, we’re also including tools that could be useful in a disaster – like Ushahidi, Sahana, OpenStreetMap and Google tools.
  • And there are commercial tools and systems that we’re going to need to interface with too. We’re talking about systems like Hunch and a bunch of other suppliers that we’ll be talking to once we get the panic of our first code sprints out of the way.

And they need a ‘next’, a way to spur action, to go with the knowledge that Hunchworks creates.

  • We’re adding tools for this too. And also connecting to UN project mapping systems:
  • UN CRMAT – risk mapping and project coordination during regional development
  • UN CIMS – project coordination during humanitarian crises, an extension of the 3W (who, what, where) idea.

Which is a big vision to have and a lot to do after our first releases next spring. And yet another reason why we’re going to need to do all the partnering and facilitation that we can.

<slide 4: algorithms list>

So. You’ve seen how we’ve designed Hunchworks to help its users work together on hunches. But Hunchworks is more than just a social system, and there are a lot of algorithms needed to make that difference.  We have to design and implement the algorithms that make Hunchworks smart enough to show its users the information that is relevant to them when they need it (also known as looking for all the boxes marked “and then a miracle happens”).

And the first algorithms needs are here:

  • Similarity and complementarity metrics.  We need to work on both of these.  Now there’s a lot of work out there on how things are similar, but there’s not so much around about how people and their skills can complement each other.  We’ve been looking at things like robot team theories, autonomy and human-generated team templates as baselines for this.
  • Relevance. And for that, read “need some interesting search algorithms”. We’re looking into search, but we’re also looking at user profiling and focus of attention theories, including how to direct users’ peripheral attention onto things that are related to a hunch that they’re viewing.
  • Credibility. We’d like to combine all the information we have about hunches and evidence (including user support) into estimates of belief for each hunch, that we can use as ratings for hunches, people and evidence sources. There’s work in uncertain reasoning, knowledge fusion and gamification that could be helpful here, and there are some excellent examples already out there on the internet. As part of this, we’re also looking at how Hunchworks can be mapped onto a reasoning system, with hunches as propositions in that system. Under “everything old is new again”, we’re interested in how that correlates to 1980s reasoning systems too.
  • Hunch splitting, merging and clustering. We need to know when hunches are similar enough to suggest merging or clustering them.  We also would like to highlight when a hunch description and the evidence attached to it deviates far enough from its original description to consider splitting it into a group of related hunches. Luckily, one of our research projects has addressed exactly this problem – this is an example of how our internal algorithm needs are often the same as the users’ tool needs – and we’re looking into how to adapt it.

Mixing human insight with big data results. One of the things that makes Hunchworks more than just a social system is the way that we want to handle big data feeds. We don’t think it’s enough to give analysts RSS feeds or access to tools, and we’re often working in environments where time is our most valuable commodity.   The big question is how we can best combine human knowledge and insight with automated searches and analysis.

Let’s go back to Global Pulse’s mission.  We need to detect crises as they unfold in real time, and alert people who can investigate them further and take action to mitigate their effects.  It’s smart for us to use big data tools to detect ‘data exhaust’ from crises.  It’s smart for us to add human expertise to hunches that something might be happening.  But it’s smarter still for us to combine these two information and knowledge sources into something much more powerful.

We’ve argued a lot about how to do this, but the arguments all seem to boil down to one question: “do we treat big data tools as evidence or users in Hunchworks”?  If we treat big data tools as evidence, we have a relatively easy life – we can rely on users to use the tools to generate data that they attach to hunches, or can set up tools to add evidence to hunches based on hunch keywords etc.  But the more we talked about what we wanted to do with the tools, from being able to create hunches automatically from search results to rating each tool’s effectiveness on a given type of hunch, the more they started sounding like users.

So we’ve decided to use bots. Agents. Intelligent agents. Whatever you personally call a piece of code that’s wrapped in something that observes its environment and acts based on those observations, we’re treating them as a special type of Hunchworks user.  And by doing that, we’ve given bots the right to post hunches when they spot interesting patterns; the ability to be rated on their results, and the ability to be useful members of human teams.

<Slide 5: System issues>

And now I’ll start talking about the things that are difficult for us.  You’ve already seen that trust is incredibly important in the Hunchworks design. Whilst we have to build the system to enhance trust between users and users, users and bot, users and the system etc, we also have to build to deal with what happens when that trust is broken. Yes, we need security.

We need security to ensure that hidden hunches are properly distanced from the rest of the system.  I’ve worked responses where people died because they were blogging information, and we need to minimise that risk where we can.

We also need ways to spot sock puppet attacks and trace their effects through the system when they happen. This is on our roadmap for next year.

And then we have localisation. The UN has 6 official languages (arabic, chinese, english, french, russian and spanish), but we’re going to be putting labs into countries all over the world, each with their own languages, data sources and cultural styles.  We can’t afford to lose people because we didn’t listen to what they needed, and this is a lot of what the techs embedded in Pulse Labs will be doing. We’ll need help with that too.

<Slide 6: federation diagram>

Also on the ‘later’ roadmap is dealing with federation.  We’re starting with a single instance of Hunchworks so we can get the user management, integration and algorithm connections right.  But we’re part of an organisation that works across the world, including places where bandwidth is very limited and sometimes non-existent, and mobile phones are more ubiquitous than computers.  We’re also working out ways, as part of Data Philanthropy, to connect to appliances embedded within organisations that can’t, or shouldn’t, share raw data with us. Which means a federated system of platforms, with all the sychronisation, timing and interface issues that entails.

We aren’t addressing federation yet, but we are learning a lot in the meantime about its issues and potential solutions from systems like Git and from our interactions with the crisismapper communities and other organisations with similar operating structures and connectivity problems. Crisismapping, for example, is teaching us a lot about how people handle information across cultures and damaged connections whilst under time and resource stress.

Okay, I’ve geeked out enough on you. Back to Chris for the last part of this talk.


Focussed on the technical challenges of building Hunchworks… slide titles are:

  • Open source and the UN – pic of codejam – licensing models, working models for UN technology creation (h/t unicef and undp, big h/t adaptive path and the codejammers)
  • Integration – bubble diagram with the other UN systems that we’re connecting to (CRMA and CIMS for actions, TeamWorks for the user base, non-UN systems for tools – CKAN for data, CivicCommons for toolbag etc)
  • Algorithms – list of algorithms we’re designing, discussion of uncertainty handling, risk management, handling mixed human-bot teams, similarity and complementarity metrics (including using hand-built and learnt team templates) etc
  • Security – how to handle hidden hunches, incursions and infiltrations. Diagram of spreading infiltration tracked across users, hunches and evidences.
  • Localisation – wordcloud of world languages; discuss languages and the use of pulse labs
  • Federation – clean version of my federation diagram – describe first builds as non-federated, but final builds being targeted at a federated system-of-systems, with some nodes having little or no bandwidth and the sychronisation and interface needs that creates