Session Transcript:
Sure, You’re Making Tools. But Do People Use Them?

Session facilitator(s): Becky Bowers, Tyler Chance

Day & Time: Friday, 2:30-3:45pm

Room: Ski-U-Mah

Becky: Everyone is going to want to move this direction as much as possible and try to make sure you are across from someone as much as you can. We might have odd amounts but it will be good to be in pairs and toward the front.

Fill in any spaces you can. We are looking for matched pairs.

Becky: Hi, everybody. I am excited that you are excited about tools. I am Becky Bowers.

The editor I work with 10-15 reporters working with the visual tools team and doing a lot of work helping make sure our economic coverage is as well supported as it can. This is Tyler and I will let him introduce himself but first I will tell you how I met him. He is a product manager in New York and came to D.C. to listen to us talk about your news scheduling habit and whether they could support us.

He sat and took notes about my day, what I did and my job.

Several weeks later I was in New York and he said come on by, I want to introduce you to a group of people in engineering and a whole range of people from around the newsroom. He sat and they showed me a prototype and he went through and referenced the notes from the conversation we had and everything in the prototype reflected what we talked about. This is my jaw.

[Laughter]

Becky: I will let him introduce himself.

Tyler: Hello, everyone. My name is Taylor chance. I am manager of editor tools at the Washington Journal. Before I had this job, the job didn’t necessarily exist. Before we actually get into what we have going to be talking to we will take a quick poll of the room.

By a show of hands, how many people have made a tool for their newsroom before? This is going to be awesome. If you haven’t made a tool, how many people are in the newsrooms and worked with a developer on making a tool? Oh, yes! If you have done neither of those, how many of you have been given a tool you wish had been made differently. Good deal.

So, it is my life to go through all of the 58 tools that we have at the “Wall Street Journal.”

Becky: Is that all? Just 50?

Tyler: That goes from the things that are super, one-offs, and things that are super legacy. As with most news orgs, we have a lot of tools we make.

Becky: We are going to pause to talk about our objectives for the session. We are going to be doing exercises that allow you to share your experiences with one another. Your failures, wins, and things that fall somewhere in between and want you to use as many specific examples. Then we will do group feedback, help one another, document that, and want someone at each table aggressively using Etherpad.

Tyler: Use the hell out of Etherpad.

Becky: In addition to putting together tools, let’s get inspired to take things back to our newsroom to build.

Tyler: Every time I do research and talk to another newsroom about my project I would say almost every time they are trying to attack the same problem I am and nobody is talking to each other the way we should. We really want to use this opportunity and space for everybody to share their expertise and come together a compilation of stories and advice that we can take back to your newsroom to make the process and experience better.

I will start with the fact that making tools for a newsroom is complicated. They generally fall into a few different categories.

Sometimes we build tools and we have very super entrepreneurial fantastic journalist who go and make one-offs and catch on and they will explode. The one on the left I think Becky can talk to.

Becky: This was called Rogue.

A developer we had working with a product we had called DJ FX trader wanted to make sure reporters and others were empowered to make the small and simple charts so he didn’t have to. He observed everyone has a powerful tool for visualizing data on their machine. It is called Excel. If I build a macro that look sort of like FX trader they would use it and they did.

We used this for years.

Thousands of charts. Hundreds every day were made using Rogue.

Tyler: That meant every time somebody had an idea for the future they had to go tap one dude on the shoulder for several Dow Jones brands and these things.

On the right side what we actually have is something we called image grids. It was a tool made for – it was originally made for what we call the off duty 50. This isn’t even a collection of stories. It is a selection of items and gift guides. Things like that. They are superficial experiences but what happened over time because it was a tool that already existed, we had editors that said I have kind of a problem that might fit with this so they – I use the term bastardized but they will apply problems that were not what the tool was built for and make it a solution for that problem which isn’t always the bes way to go about things. We are currently revamping that tool to fit the new problem they did.

Sometimes we put a lot of work and a lot of resources into tools that just don’t take off. Now there is a lot of reasons why this happens.

Sometimes it doesn’t resonate with their users if the process was bunk. Sometimes there is political reasons why thinks go south but this is not without its place in every portfolio.

There is always something people talk about that was a big thing that didn’t go anywhere.

Sometimes we feel like we get it right. If you want to talk about the one on the left.

Becky: One of the recent big wins has been our live coverage tool. Some of this came out of a process that included the Guardian, right? This was a tool when we do live coverage our reporters and editors can jump right in. It is an example of something that has gotten new features almost monthly since it launched. Every time I go back to configure a job statistic because we are cool this helps us quickly do the work. Or adding visuals or being better archived. This is an example of a back and forth and constant conversation that has made the tool continually better.

Tyler: There are features that come into the live coverage tool that are not legit feature requests but born out of live, organic discussion. They are things we are able to fold back in and it resonates with people and people really want to use the things that they didn’t know they asked for but totally asked for it. On the right, we have a new tool we have been rolling out for image management. This is something so if you are not a photo editor but a reporter and you want to crop and insert a photograph into your story this is something that it has gone through a few iterations over time, a few names and monikers but it is something where it is a constant conversation back and forth between the product team that is building it and every part of the newsroom that actually leverages photography which is every part of the newsroom. It is super fascinating watching this product continue to live, thrive, and change over the short months it has been in production. You can talk about this last one quickly.

Becky: To wrap up, come full circle from Rogue, in 2015 I was here at SRCCON doing a session about reporters making their own charts and we were putting examples on the screen from that Excel macro. Now I can say we have a tool that generates charts on all platforms and they look better and it is one where reporters have been intimately involved giving feedback that it continues to roll out additional features and support and serve wider parts of the newsroom. It has been a victory going from what was an incredibly one off piece. This has been a big win for us.

Tyler: What is great about having internal tools is you can start to pull stats from them. I can tell you 20%, or 12% of our charts are made out of Japan and Hong Kong. The charts made are rendered for 17 million readers over the course of the quarter.

A lot of wins all-around but I think the biggest thing to chase here is when we feel like we get it right what do we mean? It is something we are continually chasing.

I am a huge fan of one good thing/one bad thing retro.

Everybody should have pens and pads in front of them. Think about the time you are actually talking about building tools with people who aren’t building tools or people you are trying to commiserate with who are trying to build tools, what is one good example of things that went really well and one where things – we will politically call it – it an opportunity to improve.

Becky: You will be sharing with those so keep that in mind.

Tyler: And we have being transcribed so be mindful of what you do and don’t on the record.

Becky: You will have two minutes which is a nice amount of time.

Tyler: Does anybody need extra time? All right. Cool.

Before we do anything else, I know everybody is shaking off their post-lunch coma so let’s take 10-15 seconds to get up and stretch ourselves really quickly. Give ourselves a good 10-15 seconds. I know everybody is starting to feel their blood sugar drop. Good deal. Everybody good? Verbal conversation is awesome. Sweet.

You want –

This version is more one-on-one. We are going to do someone along the side and in the back to move your chairs so we are in line just like the pairs facing each other. We will do a little sharing speed dating style. We will do that until you have had a chance to talk with everyone in your row. Hone your ideas this way.

Tyler: Who doesn’t have someone sitting across from them?

All right. Everybody, you will have a minute each? You will have a minute to talk your ideas to each other. All right. Ready, set, go!

Tyler: Okay. That is one minute.

Tyler: Okay. Everybody. Now that is time. What we want everybody to do is –

Becky: Everyone move one to the left. If you are on the end switch to the other side. One left rotation.

Tyler: You are sitting over there. You are sitting over there. Take your Post-it notes with you. Minute and a half. 45 seconds. If you haven’t gotten to talk to a second person now is a good time.

That’s time. Quick question for you. How many of you would prefer more time for each individual conversation even if it meant you talk to fewer point.

Becky: Next round we will go a little longer but go ahead and do the switch. Move to the left.

This time you will have three minutes.

You are about halfway until you switch directions again.

Tyler: All right, everybody.

That was time. Everybody get up and move to your left.

While you are moving silence can be good enough. Is three minutes better?

Yes.

Tyler: Good deal. We have one vote for two. Two? Two it is.

That is time. Get up and move to your left. My transcript is just going to be telling people to get up and move. All right, guys. Get up and move to your left. That’s two minutes. It’s just a jump to your left. And that’s two minutes. Everybody, take another move to your left.

Becky: That is two minutes and we are done. If you could wrap up your conversations that would be great. We would like you all to move two spots with a goal of getting to groups of four that are not the people you were just talking to. And this is time to really record what it is you just learned with one another. In groups of four –

Tyler: This is the part where you use the hell out of the Etherpad.

Becky: One of the four should pull up Etherpad.

Tyler: If nobody has a laptop, you can write it down and week – we can add it later.

This is a time to think about what can we put out in the world.

Think about what were the things that were in common for everybody you talked to. What were the things that stuck with you that you didn’t think anybody was going through that you were absolutely going through on your own? What are the things we can put out in the world that will help processes?

What are things you engaged with in your team? Have you addressed hypothesis-driven design? Are there other design systems or exercises out there that help developers get closer with their newsroom? One thing you have done to change the process is improve the relationship between the maker of the tool and the users of the tool. Think about all the conversations you have had with the lovely people in the room that will help make places better for everybody and compile them into the Etherpad or Post-it notes so we can compile it after. You have about 10 minutes.

You have about five minutes.

You have about to minutes to go.

Becky: Wrap up your conversation. Now for the benefit of everyone who may not dig deeply into every detail of the Etherpad later, although I fully expect you to do that, we are going to have feedback from each group for the room. This will help with live transcription for anybody who may be following along from outside to hear some of your key takeaways. Because there are so many of you I think we will try to go in order and start in the front.

One thing we talked about was deciding your metrics of success as much as you can on the outset of the project. If you are asking yourself why you are truly building the tool in the outset you should be able to set your metrics for success for that why not a different why at the end.

Also shared was the fact there is a difference between the success of a tool and the success of a project and the KPIs of the tool are different than the KPIs of the project.

You cannot tie the success of your tool to a project because your tool may be successful but the project applied to may not and throws are two different things.

Tyler: That resonates with me huge. As we move toward an organization thriving on OKRs I am trying to drive home it may not apply. OKRs – objectives and key results. Previously we had KPI – key performance indicators – it is about the measurement you are looking at.

That could be, let’s say, your key performance indicator for whether or not you think a story went viral is you have looking at the number of pages it has.

It is about the measurement.

They give you strategy and conflict around the KPI.

Objective is desired outcome; I want this to resonate with more readers and the result would be resonating with 10 percent more unique visitors.

I am trying to drive home my objectives are around making my editor’s life better.

I want things to go live faster.

When you see things going live faster, that could have a tingible result for the read R but doesn’t mean every result matters to everybody else. That resonates a lot. Thank you.

Becky: Next group.

Some of the things we talked about that worked pretty well were keeping tools as simple and straightforward as possible.

Anything that doesn’t require training and that people intuitively understand tends to take off. Another way you can reduce the friction for people with tools is if you have anything that people are already comfortable using, you have Slack, or a Google Doc, or any of these interfaces that people already really like to use, you can try and adapt the output and APIs from those tools to power new things. So you don’t actually have to teach people a new interface in order to build a new tool.

What didn’t work was talking to people and then going off and building something but not actually working with them after you deliver the tool. It really has to be iterative in order to be successful. Anything that requires too much training doesn’t work very well. Building something that you think would work really well but not actually talking to the people that are going to use it it also causes problems. If your metric for success is how many people adopt the tool, if you are building it for a handful of people and expecting everybody else is going to want to use it it, that doesn’t often happen.

Becky: All fabulous. All resonate in my world entirely.

For things that worked really true collaboration between the builders and the users are the best outcome. Collaboration through the whole process.

Building for the capacities of the users not what you hope the capacities of the users really are. Designing for easy and obvious use and things that didn’t work, not making dependencies on tools known and having that be something only one person knows about. I love the phrase. It is not field of dreams. If you build it they are not necessarily going to use your tool. And clearly understanding the problem trying to be solved.

Becky: Did you talk about figuring out what their capabiliies might be?

I think that is part of the process of having the discussions so you understand throughout.

We started off by acknowledging we are all mentally drained because this is a very full conference. We mainly just talked about sharing our feelings. One; the adoption problem is tough. Then the group selectively sighed. We acknowledged that maintainability is a problem because people don’t always want to keep up the tools that they have built especially years and years into the future. However, what works is sometimes there are just these tiny hack jobs that do one thing very well.

Somehow those are the ones that persist. That is cool.

[Laughter]

One thing that worked really well is –

I work at a company for web development and we always have a strategic lead in every project which is sometimes project imageer, mimes UX lead, and they will be the glue between the client, customer and development team.

Then we ended by talking about even when you build tooling for yourself there is an adoption problem.

Tyler: That is so nice. I love that.

A few common themes we identified in our discussions were sometimes the failure to plan for the future. You are so focused on getting it out the door you don’t know what the future of the tool is going to be. There was some disagreement on should you build the scale?

Should you not? Build it to work and rebuild it later? We talked more about bridging across the newsroom between developers and journalist and that early outreach being important. What was another big thing we talked about? Building tools on top of things that people already use like using Slack integration, using Google Drive, things like that.

We are also kind of tired.

One thing we talked about was the tension between wanting to innovate and play with new stuff and building tools people care about and can use. Something that was actually there but is gone now was talking about how they would like to have an innovation credit. Even if something didn’t necessarily succeed to the level they hoped it would they would get points or kudos for attempting something new, sorry build something outside of their comfort zone. I kind of felt that that was maybe not that easy because I worked in places with innovation and you feel disheartened if it isn’t something people want. Trying to figure out a way to resolve that tension would be good so you could actually do stuff that, you know, makes you happy and makes you satisfied in your work but is very much focused on what people actually need.

Tyler: I like that. There is the idea that, you know, developers are using things that are cool and cutting edge. There is an empowering piece to that and how do you account for the time it takes to learn those things if you want to build something useful.

One thing we talked about was constraining features and how cutting down the features massively helps people learn how to use the tool and makes it easy to share and get feedback.

MVP and prototypes lead to effective learning. There is a reed Hoffman quote if you are not embarrassed by the first product you have launched too late.

In the newsroom we just want to get our mitts on it and start using it. Absolutely.

Perfect is the enemy of shipped.

I am not sure we have anything to add that hasn’t been said except when you are selfishly developing a tool, because it is your idea, you should probably design for at least one other person.

Becky: Design for at least one other person. Write it down.

Tattoo it to your arm.

Tyler: Going to add that to our design principles. At least one other person.

A loft people said one of their regrets was not getting user feedback earlier enough in the process. They got a little, pushed out a prototype, and found that suddenly people had all sorts of things to say that they had not reached out to get before and now they need to make tons of changes. In general, important thing about the long term and how the tool can be maintained and not how it can just work for the specific use set you had in mind. One person built a great tool but it was posted on his desktop. Important to get buy-in from non-technical members of the team. One person had an interesting experience where people on the team didn’t trust the tool and kept going back to the coder to check the math because they trusted the coder but not the code.

I think a lot of the things we talked about came down to expectations and assumptions and trying to make as few assumptions as possible at the beginning and get expectations clear. A few of those things that came out were one a tool really won’t succeed if you don’t take into account what users are going to need to do before they start using the tool. We talked about aligning expectations early among all the people involved and keep realigning them along the way.

One thing I threw in there is I think it is important to find ways to increase the sense of personal responsibility for tool’s success among all stakeholders involved. There are a lot of projects I am involved in where my team is building the tool and a lot of people show tupe the meetings and weigh in and make criticisms or suggestions and go away, but for everyone intimately involved and in the room, the more they are thinking about what they can bring to the process even if they are not building the tool and what their long-term responsibility would be.

Becky: I am nodding so vigorously because reporters on my team are taking greater responsibility for testing updates and iterations of the tools we use and that has been very important for us. For us to feel like we had a key and important role or responsibility to take time. These are reporters. They are jamming. But to take these 15 minutes and push hard on a new rollout.

We had a little bit of a meta conversation about lamenting how many tools are made that might not need to be made in the first place. We came up with a tool to track whether a tool should be made.

Tyler: I love that.

Whether it is a rubric in a spreadsheet is it worth it to build something? Can we hack together other things? Long-term value? Something anyone will maintain? There might be loosing out on some things that are not about a new tool but a new way of thinking so maybe there is a separate rubric for tools that don’t fit in the first rubric.

We created two tools to minimize the amount of tools made.

Becky: It sounds like you have done this before.

We didn’t really talk about solutions until the end. The first challenge is getting buy in from the stakeholders in the newsroom. Make sure everyone is aligned. You deal with there same people oftentimes into the newsroom and lay get tired of you and you get tired of them.

Having multiple contacts across the organization seemed helpful.

Sometimes it is hard to monetarily justify internal tooling over many cool ad or sponsor consent things you are producing for your website. That would drive revenue. There was a case where a dollar amount was saved on a tool and it was –

In the six figures.

How did you track that?

Well, I built a tool.

Essentially the savings was a reduction in workflow time. I built a metrics collection tool in Trello to track cycle time of work and saw a lovely downward trend once we implemented Google – the stuff about integrating into Google and Slack resonates with me. We built tooling in Google documents that allowed for greater efficiency when moving that content to CMS and that caused a 75% reduction which resulted in a low six figure savings for the company.

Tyler: Other things I do to try to effectively connect is lot is through man hours saved.

I always tell people a tag line for my program is salvation by a thousand cuts. It is even like looking at what is one thing. In the live coverage tool we decided to get rid of one feature and that one feature was able at a save like a couple day’s worth of man hours over the course of a quarter and you could totally add like a dollar amount to that.

Just echoing I think what other people have said making sure not only there is a lot of communication at the beginning of the process of building a tool but there is a really tight feedback throughout it so those things can stay closely coupled.

Awesome.

Becky: Thanks, everybody.

Give yourselves a round of applause.

Tyler: Awesome. You all were awesome. A lot of this you guys already touched on but for a lot of the lessons I have been learning it is about integrating more empathy between had maker and users and that is getting started earlier. How many people have made user personas of their newsroom? Anybody? It is a good way to start.

What is a user persona?

Fancy WJS lingo.

Tyler: I have been in the product side of things so long I don’t even realize I am speaking it. That means you are trying to create a snapshot of what a user of your tool or product is going to be. You can come up with just like one ideal user but generally you are trying to break down the personality types engaging with this thing I am building. I was actually finding it pretty odd, though. When you are building internal tooling you are trying to make these personas that are like generalizations of the people that you know and interact with every day. It is a really odd process but one thing that can help was I started to make my user personas based after cast members in office sitcoms. In one of my tools Becky is a liz lemon.

Becky: I take great pride in this.

Tyler: It is a way where the names are there and there is an intended underlying he – hierarchy that can make it easier.

There is a lot of diffusion fine kicking off a problem. So much is around getting the buy in and everybody realizing they are supposed to be a united front against the problem. Instead of one side of the room sitting down saying I have a solution to this problem and the other side saying they have a solution. It is an about recalibrating the conversations to say let’s describe the problem at hand and we can tackle it together. There are benefits around the iterate development and validate, validate, validate. Last but not least, I come from a design background, and for me, design is all about empathy. It really is about taking care of one another and putting yourselves in the shoes of people around the table. It will always lead to better outcomes. For me, what I am trying to do balance it out is figuring out empathy verse efficiency because efficiency can be a cold son of a bitch.

Those are my takeaways or contributions to the discussion.

Final thoughts, Becky?

Becky: I just want to say thank you to you and everybody in the room. Have a great evening.

Tyler: You guys were fantastic. Thank you so much. A quick show of hands. How many people think we don’t talk about this stuff enough? Thank you. We will figure out how to do it more. Thank you, guys. If anyone has notes that did not take it into the an electronic format please bring them to me.