Month: October 2011

(Apologies – I’m tidying up my backlog of draft posts, so some of the following posts will be a little late and/or sketchy.)

Most objects have a physical geography.  Some of the ones that might not (the argument continues) are part of the Internet, but most objects and the data they generate can be tied to a specific place or places (e.g. a crisismapper in London mapping Haiti on a server in Luxembourg has three).

So is what we’re doing here an extension of human geography, the study of the interactions between people and place?

Maps contain objects that are important to people. So a ‘conventional’, bought-over-the-counter paper map will contain things like roads, trees, buildings, post offices, hospitals, trig points.  I’m wondering today if we’re adding vulnerability overlays or complete layers to our maps.

For instance, if a conventional map contains health facilities and we’re tracking an epidemic, then are we creating the ‘shadows’ of those facilities.  Is deforestation the shadow of trees? Food crisis the shadow of markets, fields and roads?

It’s also interesting to think about the touch points between visualization and gis.  If a visualization has place, does that become a map?  Or a vizualisation informed by a map?

Hmm… time to wander over to the W3C Points of Interest working group, and see where they’ve got to with this.

I’ve spent almost two years now thinking about and doing humanitarian crisismapping – about the sources, analysis and communication of information available to organisations like the UN before, during and after a natural disaster (fire, earthquake, tsunami, floods, snows etc), but that’s too diverse and uncertain for a small number of people in these organisations to distill into usable, searchable knowledge in the timeframes involved in any disaster (seconds, minutes, hours, days before it’s all over and all responders can do is help people to recover from its effects).

There have been some amazing things done in those years. The UN cluster system, although imperfect and not mandatory to join, does have a structure that field organisations can join to improve their combined effort in an area (nutrition, health, water/sanitation, emergency shelter, camp coordination, protection, early recovery, logistics, telecomms). Teams like UN OCHA, the standby task force, crisiscommons, sahana, humanitarian open street map, humanity road and mapaction are building maps and situation reports from data across the internet and reported from disaster zones, have helped open up new data sources ranging from near-real-time satellite images to gazetteers and citizen journalists, and built new tools, processes and techniques to do this.

But quietly, behind the big stories like Haiti and Libya, there’s also been a lot of work on how to map and use new sources of development data. The same sources, tools and techniques can also be used to map water availability, health outbreaks, famines and migrations.

But there’s more. The crisismappers have been using people (lots of people, all over the world) to hunt through open data streams like twitter, facebook, website updates, news streams and satellite images, and creating tools to help with searching and tagging information in these (this is exactly the type of search through data that happens as part of big data analysis). But also out there are slower forms of datastream on the internet, like website contents, updates to blogs and open data from governments, companies and institutions. And data with a lower psychological distance but more uncertainty as to intent, like trends in the searches made on google. And if you’re part of an organization, you also have access to your institutional data (and incidentally should be asking yourself what the barriers are to making this data, a cleaned version of it, or results from analyzing it, openly available), and to data from other institutions and companies. You might also have access to sensor outputs and people’s knowledge and opinions.

And more. A lot of the big data systems out there are designed to infer people’s behaviours and status from the text (twitter, facebook etc) and data (phone positions etc) they output. Which is great, but a bit one-way and unfocussed. Which is why crowdsourcing systems like ushahidi and open311 become important. And just one of the reasons why the hunchworks system is being built.

Most search applications are about similarity – about finding items or people who are similar in some way, and ranking their appearance by that similarity (amongst other things –there are also things like connection and advertising cash to take into consideration). I have a slightly different problem to solve: that of suggesting connections and forming teams between people who are complementary.

Which is a great problem to have: new turf is always exciting. But before I get too carried away, I need to look for places where this problem might already have been solved. The obvious places to look are management theory, robot team formation and autonomous agents. Each of these is focused around a goal or task that requires a set of skills, time availability, location availability etc., and includes the study of communication, coordination and cooperation between team elements. In robotics, this is autonomy theory. Multi-agent theory is also a rich ground, as is team forming (from team development models).

A lot of human team theory is about personalities and how they complement each other – the completer-finishers and plants of the Belbin Team Inventory and group behaviour. We need to be aware of these (and there is much work in community management on to manage personality conflicts), but they’re not the main focus of this problem.

A lot of search now includes the networks connected to an item – for instance, the pages cited by and citing other pages, the people linked by social networks. That gives us similarity again, which isn’t what we’re looking for, but the network idea is powerful and still potentially useful to us. As our system develops, we should see teams forming around ideas, and groups forming around subject areas. Perhaps we could mine those groups and the people profiles in them, to template the types of team that form around different problem types and topics. Combining that with a ‘skills needed’, location, tags etc template for each hunch might produce a very powerful team suggestion tool.

Which brings us to robotics. There is a lot of work (e.g. here) on team formation in multi-agent theory and autonomy. Mostly these are goal-based, an extension of things like belief-desire-intention models from single agents to multiple cooperating actors.

There are overlap areas too – like the I-MINDS system for creating student teams.

And then there’s the whole area of autonomy that’s dedicated to mixed human-robot teams. Which is exactly what we’re planning to do by creating agents (wrappers for data feeds and sniffers) as users on a par with the human users in our system.

There’s a lot of work here, something worthy of at least one thesis methinks. But we have to make it work, and quickly, so the trick for now is to think about the above, and create something simple and intuitive that ‘just works’. Onwards!

I need to understand what human development is, how it can be measured, what interruptions (‘shocks’) and damage (‘reversals’) to it look like, and the methods that people are using to a) reduce the risk of development reversals or b) reduce the impact of shocks when they can’t be avoided. And then look at who is doing what and think about how we might do more to help this work.

First, human development. Wikipedia is surprisingly insightful (ed: not really – it’s been lifted from the UNDP website) about this – “Development is … about expanding the choices people have, to lead lives that they value and improving the human condition so that people will get the chance to lead full lives. And it is thus about much more than economic growth, which is only a means – if a very important one – of enlarging people’s choices”… “The most basic capabilities for human development are to lead long and healthy lives, to be knowledgeable, to have access to the resources and social services needed for a decent standard of living and to be able to participate in the life of the community. Without these, many choices are simply not available, and many opportunities in life remain inaccessible”.

So it’s not just about money, drugs and food then (and no, I didn’t think that it was). There’s a lot about the value of people there, and the right of all of us to more than just basic subsistence survival. Which means thinking less in terms of individual vulnerability and starving kids and more about resilience and healthy communities that have the resources to make good decisions and survive shocks with help rather than imperial-style handouts.

The UN has two (at least) main measurements for development: the Human Development Index (HDI : life expectancy, literacy, education, standard of living and GDP) and Human Poverty Index (HPI: gaps in ‘long and healthy life’, ‘knowledge’ and ‘decent standard of living’). These all got rolled up into the Millennium Development Goals (MDGs), a commitment by the UN to reach 8 major development goals by 2015.

So a development reversal would theoretically be something that reverses progress in one or more of these areas. A quick Google for “development reversal” shows the main source of information on these to be the annual UNDP Human Development Reports. And from a quick skim, the 2010 report has a lot of useful background in it that I really need to spend some time reading in depth.

I was at the first Data Without Borders Data Dive this weekend, and needed a tool to view and play with some GIS layers. I remembered that this was possible in Open Street Map, and dived into the OSM wiki to find out how. The answer (amongst several other options) was an open-source tool called Quantum GIS (QGIS). Which has been impressing me greatly since I first started playing with it.

Notes to self on this:

• The best guide to starting with QGIS that I’ve found is MapAction’s Field Guide to Humanitarian Mapping.

• Loading an OSM map into QGIS needs the OSM plugin (QGIS top menu bar > plugins > manage plugins > Open Street Map plugin) and an OSM map file (there’s a QGIS tutorial on this here – I downloaded the Uganda .osm file from cloudmade).

• Labels on a map is as easy as double-clicking the layer (QGIS top menu bar> ) and clicking “show labels” in the top left corner of the window.

Loading data from an excel spreadsheet is easy, but you need to enable the plugin for this first (QGIS top menu bar > plugins > manage plugins > Add Delimited Text Layer), then click on the little blue box with commas in it icon that appears in the top icon bar. As far as I know, only csv files can be imported this way, but that’s an easy “save as” from most big excel files.

Loading in a WMS stream (e.g. this http://cernunosat05.cern.ch/arcgis/services/THA/FL-2011-000135-THA/MapServer/WMSServer) is also easy, but you have to remember to click “connect”, then click on the stream names in the window below it, and “” below that.

Back to blogging. I write mainly to keep sane, but also to think through subjects that are complex or I worry that I need to but don’t completely understand yet.

Today it’s development crises. My day job deals with helping to detect large-scale human crises that aren’t natural disasters, but apart from the obvious categories (famine, conflict, poverty limiting access to basic resources like food, medicine, education), it’s not very easy to pin down a) what a non-disaster crisis is, and what is typically done to detect and act to mitigate it. Once I’ve thought about what large-scale human crises are, I also need to think about how the other work that I believe is relevant to them is starting to form a connected space, and what help that space needs from us to keep developing well.

So first. Nomenclature. A quick Google search on “development crisis famine” shows phrases like “social crisis”, “humanitarian crisis”, “famine crisis” and “food crisis”. A wander into Google trends with “development crisis” shows “human development crisis”. Wandering around some on the pages (e.g. a Guardian piece of the impact of global crises on development) shows global drivers like the food, fuel and financial crises, climate change and confidence. Hang on? Confidence? I know that’s a big driver in financial markets, but it’s interesting to think about its role in development.

And examples. There’s a crisis across the Horn of Africa at the moment. The rains failed, crops have failed, livestock is dead and people are starving. But it’s more complex than that, as well explained in this allAfrica.com piece. The guardian piece has a good summary of the IDS Re-imagining Development report  which looked at the complex effects of the 2008 food, fuel and finance crisis interactions. And on a smaller scale, there are the knock-on effects of the natural disasters (that we don’t deal with) on development, for instance the heavy effect that the Eisfjallsjokull eruption had on industries like Kenyan flower-growing  (which relies on air traffic between Africa and Europe).  And then we have this quote in sustainable development: “[A]n environmental crisis, a development crisis, an energy crisis. They are all one.… Ecology and economy are becoming ever more interwoven—locally, regionally, nationally, and globally—into a seamless net of causes and effects.”

So. We have drivers, which can interact with each other and trigger development crises either directly or through causal chains that we may or may not know about yet. And we have crises, and the effects that those crises have on people, environments and social systems (e.g. education and conflict). This sounds very much like a system to me – and one amenable to some serious analysis, sensitivity modeling and risk management. Which must be going on somewhere in the world.