Boosting the data startup

[cross-post from Medium] Some thoughts from working in data-driven startups. Data scientists, like consultants, aren’t needed all the time. You’re very useful as the data starts to flow and the business questions start to gel, but before that there are other skills more needed (like coding); and afterwards there’s a lull before your specialist skills are a value-add again. There is no data fiefdom. Everyone in the organization handles data: your job is to help them do that more efficiently and effectively, supplementing with your specialist skills when needed, and getting out of the way once people have the skills, experience and knowledge to fly on their own. That knowledge should include knowing when to call in expert help, but often the call will come in the form of happening to walk past their desk at the right time. Knowing when to get involved is a delicate dance. Right at…


What could help fix belief problems?

[cross-post from Medium] Last part (of 4) from a talk I gave about hacking belief systems. Who can help? My favourite quote is “ignore everything they say, watch everything they do”. It was originally advice to girls about men, but it works equally well with both humanity and machines. If you want to understand what is happening behind a belief-based system, you’re going to need a contact point with reality. Be aware of what people are saying, but also watch their actions; follow the money, and follow the data, because everything leaves a trace somewhere if you know how to look for it (looking may be best done as a group, not just to split the load, but also because diversity of viewpoint and opinion will be helpful). Verification and validation (V&V) become important. Verification means going there. For most of us, verification is something we might do up front, but…


What goes wrong in belief-based models?

[cross-post from Medium] This is part 3 of a talk I gave recently on hacking belief systems using old AI techniques. Parts 1 and 2 looked at human and computational belief, and the internet and computer systems development as belief-based systems. Now we look at the things that can (and do) go wrong with belief-based systems. Machine Belief Machines have a belief system either because we hard-wire that system to give a known output for a known input (all those if-then statements), or because they learn it from data. We touched on machine learning in part 1 of this series (remember the racist chatbot?), and on how it’s missing the things like culture, error-correction and questioning that we install in small humans. But before we get to how we might build those in, let’s look at the basic mechanics of machine learning. At the heart of machine learning are algorithms:…


Why should technologists care about belief?

[cross-posted from Medium] This is part of a talk on hacking belief systems using old AI techniques. The first part covered human beliefs, machine beliefs, and deep learning and the internet as belief-based systems. Lean and Agile are based on belief   The build-measure-learn cycle. Image from Most system development I see now uses agile principles: short, iterative cycles of development, and an emphasis on learning, preferably alongside the system’s users, and adapting their systems to increase some measure of customer or user value. Lean is a similar idea at the company level, based around learning quickly about what works through prototyping and measurement (aka the “build-measure-learn” cycle). Lean Startup is the small-scale version of this; Lean Enterprise is the large-organisation one. I’m going to focus on Lean Enterprise because it’s more complex. In an established organization, Lean’s usually a combination of learning and adapting existing business, and exploring new…


Mixing human and computational belief

[cross-post from Medium] This is part 1 of a talk I gave recently on hacking belief systems using old AI techniques. I have a thing about belief systems. Not religious belief, but what it means to believe something, when accuracy does and doesn’t matter, and why “strongly held belief” is a better concept to aim at than “true”. It’s something I’ve been thinking about for a while, and I believe (!) it’s a conversation we need to have both as technologists who base many of our work decisions on beliefs, and often work on a belief-based system (the internet), and in the current climate of “fake news”, uncertainty and deliberate disinformation. I care about belief because I care a lot about autonomy: the ways that humans and machines can work together, sharing control, responsibility and as teams, and we don’t talk enough yet about the human-machine sharing of belief systems…


Hacking elections with data

[cross-post from Medium] Cambridge Analytica basically used customer segmentation and targeting: standard advertising stuff (and some cynicism about that: iirc one of the other campaigns ditched them) that will probably become standard for campaigns if it hasn’t already (full disclosure: am helping out on a campaign). Cool (if unethical) use of surveys as probes though. Get the feeling they didn’t do as much as they could have done, but that was enough. Not sure how I feel about gaming elections right now: part of me says bad, another part that it’s politics as normal, just scaled and personalised. Meanwhile, on the Democratic side, big data seems to be a problem. We need to fix this. So repeat after me “big data is not data science”. Get the data, study the data, but understand that exploring data is just part of the arc between questions and storytelling, that humans are complex…


Bursting the right bubbles

[cross-post from Medium] First, understand the bubble It’s hard to argue with people if you don’t know where they’re coming from. One way is to ask: engage with people who are vehemently disagreeing with you, find out more about them as people, about their environments and motives. Which definitely should be done, but it also helps to do some background reading… The Guardian’s started in on this: a round-up of 5 non-liberal articles every week, complete with backgrounder on each author and why the article is important. It doesn’t hurt that some of these authors are friends of friends and therefore maybe approachable with some questions. It’s also worth checking out things like BlueFeed RedFeed. I’ve taken some flack lately for trying to understand Trump supporters. I’m slowly coming round to amending that to trying to understand Trump voters — especially the ones who voted with their noses held. Us vs Them…


Fake News Isn’t About Truth, It’s About Gaming Belief Systems

[cross-post from Medium] Thinking about #fakenews. Starting with “what is it”. * We’re not dealing with truth here: we’re dealing with gaming belief systems. That’s what fake news does (well, one of the things; another thing it does is make money from people reading it), and just correcting fake news is aiming at the wrong thing. Because… * Information leaves traces in our heads, even when we know what’s going on. If I jokingly tell you that I’ve crashed your car, then go ‘ha ha’, you know that I didn’t crash your car, but I’ve left a trace in your head that I’m an unsafe driver. The bigger the surprise of the thing you initially believe, the bigger the trace it leaves (this is why I never make jokes like that). * That’s important because #fakenews isn’t about the thing that’s being said. It’s about the things that are being…


The Internet is made of beliefs

[cross-post from Medium] “Most people don’t have the time or headspace to handle IW: we’re going to need to tool up. Is not much, but I’m talking next month on belief, and how some of the pre-big-data AI tools and verification methods we used in mapping could be useful in this new (for many) IW world… am hoping it sparks a few people to build stuff.” — me, whilst thoroughly lost somewhere in Harlem. Dammit. I’ve started talking about belief and information warfare, and my thoughts looked half-baked and now I’m going to have to follow through. I said we’d need to tool up to deal with the non-truths being presented, but that’s only a small part of the thought. So here are some other thoughts. 1) The internet is also made of beliefs. The internet is made of many things: pages and and comment boxes and ports and protocols and tubes…

Augmented Intelligence

The Ethics of Algorithms

I opened a discussion on the ethics of algorithms recently, with a small thing about what algorithms are, what can be unethical about them and how we might start mitigating that. I kinda sorta promised a blogpost off that, so here it is. Algorithm? Wassat? Al-Khwarizmi (wikipedia image) Let’s start by demystifying this ‘algorithm’ thing. An algorithm (from Al-Khwārizmī, a 9th-century Persian mathematician, above) is a sequence of steps to solve a problem. Like the algorithm to drink coffee is to get a mug, add coffee to the mug, put the mug to your mouth, and repeat. An algorithm doesn’t have to be run a computer: it might be the processes that you use to run a business, or the set of steps used to catch a train. But the algorithms that the discussion organizers were concerned about aren’t the ones used to not spill coffee all over my face….