Collateral Damage: The Ethics of Lean UX

Some techniques that are gaining respect in design and development attempt to decrease the potential for failure (in projects, big companies, or startups) by testing work with real people early and often, in order to validate (or, as is more likely, invalidate) assumptions. A process called the Lean Startup is, as Janice Frasier described at UX London, about "hypothesis recognition", "experiment design", and "smallification." It's about creating low fidelity interactions with people called, in this process, "experiments", and understanding if your assumptions around purchasing behavior are true. Janice encouraged designers engaged in this Lean Startup process to quickly identify a way for testing an assumption about how well a product fits in a market, "smallify" that test by making it even more concise and quick, and then run an "experiment" with real people to understand if you were right. It's only after this process is done that you can move on to the next stage of adding features or functions to a product.

It's a process that appears sensible when you examine it as a broad reaction to two different ways of launching a startup that many of us have encountered: either a process with multiple constituents (both founders, advisors, and directors), all competing to have their opinions and requirements heard, or a process driven by engineering of "rip it and ship it", where engineered ideas are thrown out into the world with little thought for the customer. This lean approach was first created by Steve Blank, a professor at Stanford, and called Customer Development in a book called "Four Steps to The Epiphany" which, according to Janice, is "a terrible read. No one finishes it." If you do read it, you learn some new language to describe your process. By calling your design a hypothesis, you change the nature of the conversation with other members of your organization. By releasing only the smallest necessary unit of design (usually, as Janice described, a landing page that describes what your company is going to do), you can move quickly. And by engaging in these "experiments", you can learn if your assumptions were right, most notably, if people would actually be willing to pay for a particular product or service. This is judging product/market fit.

Something about the technique rubbed me the wrong way, and I couldn't quite put my finger on it until I was discussing it with some folks at dinner. And then, another designer summarized my reaction to it, by describing the way users were treated during these "experiments": as collateral damage.

In this Lean Startup approach, people are treated like guinea pigs; literally, subjects of an experiment. But unlike usability testing, or a true scientific experiment, these people aren't giving informed consent for your experimentation, they aren't learning of risks of engaging with your product or service, and they aren't considered to be on equal footing with the design team. For most product contexts, this probably seems like splitting hairs: if I release a poorly designed version of a restaurant finding app in order to check my assumptions about a particular audience, and it causes discomfort through poor usability, or I abandon my user-base halfway through when I find out it doesn't work quite right, it might be thought of as no big deal: it's just a silly restaurant finding app, and the negative implications on these users is just the necessary cost of learning.

But if I'm working on products and services that are explicitly intended to shift negative behavior, change the fabric of society, and serve an at-risk population, there's a pretty dramatic ethical question related to this collateral damage. It seems irresponsible. The tenor is quite similar to the idea of centralizing our business workflow to make it easier for our accounting department, rather than our users, or optimizing our website for our developers, rather than our users: our focus has shifted from serving people to using people.

And I think this begins to describe my concern and negative reaction with this idea: as Janice described, "this is a reinvention of user-centered design practice, from a different point of view. There's a little skew on it. Instead of asking questions about needs and what people do, [Steve Blank] is more interested in finding out what marketing messages would be best." The whole technique is about marketing, and not about people. And for all the conversation about "get out of the building" in order to talk to users and validate your ideas, the tone of the process seems to force us back to the smarminess of marketing, but under the guise of science. It's co-opted language that seeks to prove that people will pay for things, and while these may be important for all sorts of various reasons, they don't mesh with design.

Although there are lots of different ways of framing and defining design, I've found that the commonality of nearly all definitions is the focus on people, and in making life better. Usability is about reducing our propensity for errors or increasing our efficiency; aesthetics contribute to powerful emotions; even discursive design is about raising awareness and making people think in new ways.

I'm concerned that the methods we're beginning to discuss in our community are misguided. They may work, or they may not; but if they work and produce collateral damage, are they really successful, and are they really in-line with the human-centered qualities of our work?

My raw notes from Janice's talk are below.

*

Janice Frasier

UX Practice for Lean Startups

Lean startup machine is one of the leading educators in startup methods, organized around these ideas and experimentation. They say "Get out of the building!"

I founded LuxR, and was previously a founder of Adaptive Path. On March 2^nd, 2001 we launched a company that helped define what user experience would become. I was the first coo of the company for 6 years. Before that, in the distant past, I was at Netscape, and was there 6 months after the IPO. I realized how much it feels like right now. Right now, there's a sense of permission and obligation to reinvent things. To reinvent the world as we see it. We all have this opportunity, because of the problems our world is facing, economic, or employment, environmental, there's a sense of permission and obligation to change everything. All that's limiting us is our courage and imagination. I would love to see every single one of you quit your shit job and get a better one. Great jobs are ones that you make yourself.

There are startups that are commercially viable to satisfy every personal need. If there isn't one, go make one. Be it social, environmental, technology.

One of the companies in our program reinvented the Texas Instruments graphing calculator. Why does that matter? It turns out that there's a worldwide community of people drawing pictures with a web based graphing calculator. They made valentines. It's putting joy into the act of making math happen; these are high school kids. They come on day one, do their homework, come back to finish what they imagined, not because they have to but because they enjoy it. If we can change the stupid graphing calculator, we can change anything.

All that's missing is you. The startup industry, that segment, is missing the wisdom of the user experience community. I get regularly asked for an introduction to a rock star designer, that can do front end production, and information architecture, and everything, and that will be there design department, and my job is to tell them that they're missing the point, and I help educate them. My purpose is to create fertile ground, so when you partner with them as co-founders, you have a ready environment where you can make the most impact that I know design can have.

At luxr, we're educating entrepreneurs. Netscape led me to design, and I've gotten here through the design world.

I'm an advisor to some startups; foodspotting, diaspora, mingly, taskrabbit.

Today we'll do a bunch of stuff. A talk about Lean Startup, and then, I chose three parts of what a lean startup means; what are the new skills in the toolkit? These new skills change how designers think about their work. We've had 35 companies in our 10 week program, and they have had seasoned people, and they've told me time and time again that they will never go back.

The focus is on hypothesis recognition, experiment design, and "smallification."

Lean Startup.

Eric Reiss; he's the guy who came up with the idea of lean startup. He made it to the cover of Inc magazine; he wasn't always that well presented. I want to talk about a few things that lean startup isn't. it is not cheap startup. There are lean startups spending truckloads of cash, but they are spending it efficiently. It also doesn't mean fast. It's not the easy way to do a startup. The easy way is to get money and build an idea. And then it sucks, and everyone's unhappy. The best way is to run experiments to find out if I deserve any more. You find out that your original idea was wrong, but in interesting ways, and you find your way to the best product market fit.

It's also not shortcut startup. It requires that you get out of the building, and do user research. It's mandating that you do user research, and you do it continually in the cycle. And that you do it over and over and so it becomes a way of life. Isn't that what we always wanted?

It's not low ambition. It doesn't say I'm going to make a little thing, and be OK with that little thing. The little things are in service to a higher vision, and that will create massive global change, with economic value on the other side.

It's not the opposite of fat startup. It's about efficiency, and innovation accounting.

This is Steve Blank. He's the guy that created customer development. Steve says "Get out of the building." It's his mantra, and what's amazing about this is that you shouldn't rely on your own gut idea. Go learn about humans, and develop empathy. It's generative research, it's contextual inquiry. Out there you have acolytes who are following Steve Blank and Eric Reiss, asking if they are doing customer research. When I found out there was a business community that evangelized research with stupid t-shirts? Those guys get it, and they get it in a way I've never seen in 20 years of working in industry. I'm on fire; the opportunity for change and impact is greatest when you have alignment in the belief systems in your companies.

You no longer have to persuade people of the right way to do things. I want to get the people that want to do this to the people that are really good at doing this. Get out of the building.

Steve Blank introduced Customer Development in, uh, 2006. He's a professor at Stanford, and he's a very warm and wonderful man. Epiphany was a big hit; he realized why his companies failed. He called it four steps. Customer discovery, customer validation, customer creation, and company building. The first two phases are about finding product/market fit. This is a reinvention of user-centered design practice, from a different point of view. There's a little skew on it. Instead of asking questions about needs and what people do, their more interesting in finding out what marketing messages would be best. Suddenly, everyone's talking the same language, and you're all valuing that. Let's talk more about what he means.

4 Steps to The Epiphany was a collection of his lecture notes, and so it's a terrible read. No one finishes it. I asked a room of 500 people, and like 7 people read the whole thing. So Patrick Vlaskovitz and Brant Cooper rationalized it and made a shorter, more useful book. They laid out these ideas in a more accessible book. What I want you to note about this is how similar it is to what we do for a living.

Problem solution fit is "understand people and their needs." Proposed MVP is sketching. The proposed funnel is the onboarding process. These are things we've spent decades learning to do well.

The thing to know is that as designers, we can say "I've been doing that all along, none of that is new." And we're right. Everyone tells Eric the same thing. No matter who he's talking to, he says "None of them knew about it..." Now everyone else can come along too. The words customer development, there's nothing I can do to get rid of the phrase. People you work with will call it this, even if you call it UCD. All we have to do is connect the dots so our language matches. Customer development, as if I could develop customers. Steve used this language because he was trying to challenge "product development" being software. You also have to develop your customers; he meant "develop empathy for your customers." He was presuming there would be a department called a customer development department. It hasn't worked out that way.

When I first encountered this, I went out to lunch with a few VC colleagues, coincidentally, they asked me to figure out a design program for their portfolio companies. Design is increasingly realized as being important. I was introduced by a mutual friend, and I had always poo-pooed lean startup. Pivot, whatever, we called is changing strategy. MVP? You mean, a prototype? Shut up. I was really eye rolling about it. Boy, was I wrong. I was wrong, wrong, wrong.

During my exploration, after being prompted by Rob and Mitch, I was introduced to this kid, who was 18, deferred Stanford enrollment, and was thinking about all sorts of other things. We were going to lunch so he could advise me. He laid out what customer development meant; his name is max marmer. From then on, it was like. "Holy crap. This changes my life." It was very humbling. I like to think I'm a little famous? Turns out, not really.

I can't draw, just like other people, but I had to understand the idea, so I had to make a drawing of it.

Customer development demands that, when you have an idea for a startup, ask who is it for? What can they do with it that wasn't possible before? What features do they need? How do those features fit together? I mapped them to things I knew. Personas, design targets, concept drawings, 2x2. This is what we do.

It says fake it, and then make it. This means usability testing. Awesome, still on the same page.

How do you know if you're right? This is one of the things we can work on today. Designers are culturally, subtly, told that they need to be right the first time. If you aren't right the first time, your design sucks. It doesn't suck. It's probably fine for a situation. How does the person casting judgment know? We're all judging these things from our gut, and what we need to do is judge them based on evidence. We change the role of design from "selling in" and "managing stakeholders" to solving the more important customer problem to identifying if the thing you made solves the problem enough.

We're making attempts to solve the most important problems. It's a really different way to think about the problem. I would bring a wireframe to a client and say "This is a hypothesis I'm working with right now."

And then you have a different kind of conversation about it.

You are looking for product/market fit. Once you start collecting evidence through qualitative and quantitative means, you realize you have to go through this a few times before you get it right. Now we have permission and the requirement to do iterative design. Then, you finally get it right and move on. But you also ask other questions. People say "I want to have a seat at the strategic table." Problem solving. You have to decide if you are solving a high value problem. Will someone pay? Who are the market stakeholders, and the other players in the market? How does the money flow? This is all strategic, and it's all design. This may not be about optimizing an interface or making it beautiful, but I think it's core of design.

All of this says, it doesn't matter how beautiful it is, let's solve the right problem first.

So the catch with lean startup, this is what makes it not fast startup; you'll find that the answer to these questions is sometimes no. Maybe there's no way to make money off it. Maybe it's Napster and you are stealing someone's idea. It can be an awesome thing, but if it's not viable, you have to go back to the drawing board. A pivot is where you change your tactics without changing your vision.

A pivot can set you back a year. But that's OK. It's unassailable logic. It's absolutely rational. You prioritize their needs, and solve for their highest value needs, and you make sure money flows in the situation comfortably. I can't think of any flaw in that process. It demands the talents we have as designer, and embeds them in the highest levels of decision making. If you want permission to launch your solutions into the real world, and you want to be a strategic player in the player, become a designer co-founder of a startup. Full stop.

I talked about product/market fit a few times. There's kind of litmus test that is an approximation of this. It means when you've created a product that represents a high value problem and the product you've created is beloved by the people you've made it for. The test is, say you have a small number of users. If half of those people would be very unhappy if your product went away, you can claim to have product/market fit. Relatively small number, really unhappy. We aren't talking about a perfect product. That gives you permission to move into optimizing, making it perfect, delivering on the bigger vision. What you are asking for at each stage is permission to move to the next step. And until you have permission to move to the next step, you keep working on.

There are little stop signs at each stage, and you stop until you have the stage right. And then you move on. And you stop there until you get it right. And if you don't have it right, you may have to go back and start again.

Eric Reis' first blog post was three and a half years ago. He took four steps. Make products customers want, and combine it with agile development. It tells you why agile works and what it's for. It's a very important book for us to read, because we need to develop empathy for our developer colleagues. It's a brilliant book. Reduce the batch side. The Toyota Way, fundamental to the lean manufacturing movement: it's about eliminating waste.

Make something customers want, release in small batches and do experiments, and remove waste. It's unassailable logic.

The lean startup book debuted at #2 on the New York times best seller list. He's an EIR at the Harvard Business School. He costs $30,000 to book for a keynote address at a conference; it's the real deal, it's really happening. And he now has a much better stylist.

Build, Measure, Learn. It's a cycle, and then build the next thing. Optimize for cycle time, not on-budget, on-plan delivery. This is really similar to the UX cycle of think, make, check—which I was taught by Mike Kuniavsky, 12 years ago. But Eric, being a developer, started with build. And Mike, being a designer, started with think. Developers who don't write code feel unproductive. Designers who work so fast feel unfinished. We want to optimize for cycle time. The principle that I emphasize is flow.

I don't care where you start; we want to flow through the cycle again and again, and it's by going through the process of making a hypothesis and testing it, adapting based on our learning, it's this kind of build/measure/learn, or think/make/check.

The work we've done is invalidated effort. Time is across the x axis. We do thinking, and we make things, and then we revise them, and then we release it, and we do usability testing, and we release, and so on. We never pay down the risk, because we never know if it's a successful launch. We make things without validating if they are the right things to make.

The alternative is to pay down the risk of think/make/check; you pay down the risk with a measurement. I'm going to imagine a feature my customer will want. I'll make the smallest thing to validate it, and I'll do the evaluation to know if it works. I never build risk. We're constantly de-risking; we move forward with lots of confidence. On-time, on-budget delivery of a project plan; we're just guessing.

Make the right product. User, leads to Needs, to uses, and to features. Then, we get to user stories and themed releases. I always want whatever code is being written, pixels being made into prettiness, to have a rational story back up to humans in the real world with real needs. I know the ten points in pivotal tracker connect to the most important customer problem to solve, and I know it because we've been talking to customers all the way along.

Victory is measured in learning. If you learn something important about customer needs and who will pay for what, you've made progress. This will change about how you think about your role, your work, your team, and your process.

I get asked for a mythical unicorn who can do UX, visual design, research, and development. I want a rock star designer. The answer is no you don't. You want the beginning of a department, or a person you can build a department around. You want a curious person who will develop new ideas. We're at the beginning of the evolution of our discipline. We're bringing experimentation into the practice. You want a curious person to help create the processes to solve world problems.

"You need to think like a scientist"

7 habits of highly effective startup designers.

1. Ideate with friends. Exactly three friends, not four, because four doubles the talking you do.
2. Go broad. Come up with ten ideas, so you have thirty. Get more, better ideas. It doesn't matter if the idea comes out first or last.
3. Say "Tell me about this one." I'm not rejecting your idea; I'm committed to understanding what you mean.
4. Ask "Who has the decision"?
5. Make informed, albeit arbitrary decisions. Get as much information as you can, but accept that you just have to decide. Inaction will kill your company. Not launching will kill your web company more than anything else. I know this because I did it. It's embarrassing.
6. Prove it. Ask how we know this. Do we need to prove it? How risky is it if we don't?
7. Question perfection. I think it is awesome that when a designer is asked to launch something that is imperfect, it makes them a little nauseous. It makes them physically ill, and it's a good thing. It's wonderful that you care. It's a good thing. But know when it's the right time. There's a humility, and there's a good time for perfection and a bad time. The reason I lost 800,000$ of my investors' money is because it was never good enough. I was too embarrassed. If you aren't embarrassed by your first product release, you waited too long.

Originally posted on Fri, 20 Apr 2012