https://www.linkedin.com/in/jeffreygraham1
This great interview is with Jeffrey Graham. Jeffrey is the VP of Market Insight & Analytics at Twitter. Jeffrey has a great background in analytics. He also received his PhD in sociology from the City University of New York.
We all know Twitter. In this interview we get to learn how Jeffrey puts together the insights and analytics program at Twitter. That includes showing clients how Twitter can help them. And a deeper understanding how and why people use Twitter.
This is a wonderful chance to get under the hood of Twitter and Jeffrey’s mind.
Here are some other things we talk about:
-What experience in your career was the most interesting? This early experience seemed to help shape his career.
-Why did you get a PhD in sociology?
-What are you learning about now?
-For your analytics, how do you know if you’re asking the right questions for your data?
-How would you show a company like Pepsi that they’re getting an ROI on their twitter ads?
-Can you tell us about the study where you measured brain activity around Twitter use? Very interesting.
Transcript
David Kruse: Hey everyone. Welcome to another episode of Flyover Labs and today we are lucky enough to have Jeffery Graham with us. And Jeffery is the VP of Market Insight &
Analytics at Twitter, which is quite interesting, because Twitter is in the middle of huge amount of real time analytics and Jeffery has a great background in analytics which he can talk about and he also received his Ph.D., is Sociology from the City University of New York. So we all know Twitter and I invited Jeff on the show because I’m curious how he puts together his insights and analytics program at Twitter. To me it sounds like a very intimidating project, because they have huge amount of data and they have to show clients the value of Twitter at the same time; so I’m excited to learn more. And so Jeffery, thanks for coming on the show today.
Jeffery Graham: My pleasure, my pleasure.
David Kruse: So yeah, I briefly mentioned your background. Can you tell us a little bit about your background and how you eventually ended up at Twitter?
Jeffery Graham: Sure. So I started out in the business about 20 years ago, after moving to New York from Greece, I started at a company that was building websites and online marketing campaigns for clients. I started out as a Research Manager. From there I worked at a startup called Dynamic Logic, which did research measuring the effectiveness of online advertising using cookie data and survey data; it’s basically doing experiment and seeing which online advertising changed peoples’ minds and kind of moved you know from a branding perspective.
David Kruse: What year was that?
Jeffery Graham: That was in the, at the late 90s, 1999, yeah.
David Kruse: So it was ahead.
Jeffery Graham: Yeah, yeah, then moved on to a company called Starcom MediaVest which is based in Chicago and I was the Head of Research for the Procter & Gamble account and basically helping map measurement strategies for digital media and other innovation, but also was Head of Research for their television buying unit as well. I then moved on to the New York Times. I was Head of Research of the New York Times. This was around 1998 or so and basically consolidated all the different research functions at the time from the business side and helped in marketing to new subscribers and developing new strategic initiatives and figuring out after answering some of the basic strategic question for the Times, for Print and Digital and Mobile. I was Head of Research for Google for the Americas, Head of Ad Research and that was right before I moved to Twitter about four years ago and right now I moved teams across both the B2B side doing research for our advertising clients, as well as for the consumer side answering questions about who users Twitter? Why do they Twitter? How we can better grow that user base through help out marketing and products initiatives?
David Kruse: Yeah, you have quite a deep background in media and analytics, that’s interesting. And so why did you end up getting your PhD in Sociology, what prompted you?
Jeffery Graham: So I studied Communication in College in my Bachelors Degree and when I moved I spent some time living in Europe and I lived in Greece and when I moved to New York City in the mid-90s I did a Masters Degree in Media Studies and so its always been an interest, just understanding the relationship between media and society and all the different ways that advertising works and messaging works in that system. So it’s really driven by kind of just curiosity and the desire to continue to learn and answer those questions. So it’s kind of a hobby that I did during that period.
David Kruse: Were you working while going to school too?
Jeffery Graham: Yes, yeah.
David Kruse: Oh wow! You were busy.
Jeffery Graham: Yeah, a lot of early mornings and late nights, but I find that keeping myself stimulated and keeping myself learning, I’m happiest when I’m doing that. So that certainly helps.
David Kruse: Well, I like that. So how else do you continue to learn? Like do you have certain materials you like to read or keep up-to-date with?
Jeffery Graham: Well, right now I’m getting more and more into different data science techniques, so reading books. I get my hands on doing some quasar [ph], courses on data science. So starting to learn programming, you know with R and Python and getting deeper into the you know the techniques that are just evolving in data science. So always trying to pick new skills and learn more.
David Kruse: That’s good. I mean you sound like you’ve always been kind of a curious person based on the roles that you play and always, yeah.
Jeffery Graham: Yeah.
David Kruse: Were you like that growing up too?
Jeffery Graham: Let’s see, I guess so. I think at least starting in high school you know I was on the debate team and used to read a lot of magazines and you know when I skip school I would end up you know going to libraries or spending time in your know reading books during you know. So I think I’ve always wanted to learn stuff. I think that’s just part of who I am.
David Kruse: And in your career, was there one particular project or a job or this was like pretty amazing. I learned a ton, and you know this is one of the key points I probably learned and I know it’s probably a tough question to pick out one, but you know.
Jeffery Graham: Yeah. Well when I joint Dynamic Logic that was a startup. There was like six or eight people in 1999 or 2000 and Dynamic Logic had a service that, it was a research product, but it was a service where advertisers would take a campaign online, tag the campaign so that people that were exposed to that campaign would be tagged with cookies. A survey would run and people that were exposed to that campaign would be compared to people that weren’t exposed to that campaign and then that difference was ascribed to the impact of that campaign and we look at the increases and brand awareness and purchase attempts and things like that. And when I joined Dynamic Logic, you know they were maybe doing six or eight campaigns at any give time and then very quickly we ended up doing 40 or 50 and we didn’t really have much of a process to get them done. So that was a really interesting moment as a small startup to figure out, ‘okay, well how do we do these things so that they don’t get screwed up,’ because it was really technically. You know we’d have to upload tags and we’d have to write surveys and we have to work with publishers to run those surveys and then download the data and analyze the data there was a lot of writing on it. So that was really interesting to deal with clients, to deal with internal technical teams, client service teams, to make sure that the product actually could get delivered and that we could scale and I think that that’s been – that was certainly a big learning experience and something that I really like to do which is to scale things, to take an idea and then to be able to make them happen in a repeatable way across you know bigger teams.
David Kruse: Interesting, which is valuable and not easy to do. But you guys are definitely ahead of your time doing what – that research you were doing. It seems like…
Jeffery Graham: Yeah, yeah.
David Kruse: All right, well go ahead.
Jeffery Graham: Yeah, no I mean that was kind of the first commercially available brand effect in this research and now you know 16, 17 years later it’s a pretty standard measurement. Dynamic Logic was acquired by WPP and so yeah, it was cool to be able to see basically a metric that was developed there become fairly standard in the industry.
David Kruse: Well, I have a lot of questions about that, but we only have so much time, so we should probably talk about Twitter, too. So that’s where you are right now. But yeah, can you tell us kind of what your prior to days are as VP of Marketing Insight and Analytics.
Jeffery Graham: Yeah. So I’d say there were two major areas. So on the B2B side we help advertisers understand how to get more value out of their advertising on Twitter. So we measure their campaigns, but sometimes with the same techniques I was describing that Dynamic Logic developed. But based on their objectives whether its conversion or brand lift or sales, we work with them to measure those campaigns and then to learn how to get more effectiveness out of their subsequent campaign. And we also do thought leadership to help them learn about the Twitter audience and how Twitter should fit in to their overall media mix. So that’s one kind of area. The second area on the consumer side is to get a better understanding of the Twitter audience. We have more than 300 million monthly active users and it’s diverse across many, many different countries. But as we make decisions about how we communicate to those users, to non-users and how we map kind of the future of Twitter, we need a deeper understanding of that consumer and you know understanding of our brand and how our brand and how our product fits into their lives. So that’s kind of second set of questions about just Twitter consumer and the product itself.
David Kruse: All right. So I got a lot of questions, but so with around the consumer aspect, how do you know – how do you know what questions to ask and how do you know you are asking the right questions. I mean I know you have been doing this a while, but…?
Jeffery Graham: That’s a really good. You just asked I think a really, really good question. When we started to build the consumer research side, that’s where we started, which was to start to surface the most important questions and then prioritize and classify those and we created a process that we called question storming; so like brain storming, but really a process around surfacing question in a way that allowed us to come up with you know even the smallest questions, but then eventually ladder them up to the big questions in a structured way. So then we can start to be very simple in the way that we communicate what we are doing, but then make sure that our work is aligned to stuff that’s actually going to move the business. And sometimes those questions can get very simple like, why do people use Twitter? And then be able to then create questions underneath that, that’s going to you know and then they may get down very tactically to a survey question, but you may not ask why do you use Twitter in the survey question; maybe a different type of technique that’s going to ladder up to that bigger question. So we spend a lot of time making sure we were asking the right questions and I think that that’s – no matter what technique you are ending up, going to use, whether its data science or qualitative you know research, getting that question right, getting those questions right is one of the most important steps.
David Kruse: Has there been a question that you asked that you were pretty surprised by the answer?
Jeffery Graham: I think we are surprised all the time.
David Kruse: Fair enough.
Jeffery Graham: Yeah, I think we are surprised all the time. I’m trying to think…
David Kruse: Yeah, try to pick out one without the…
Jeffery Graham: Yeah, I mean we just did some research around customer service on Twitter and we created this research design where we basically were able to observe who had received timely customer service on Twitter and then people who hadn’t and then we went and surveyed those two groups of people blindly. We didn’t – you know we weren’t communicating to them that they were getting, they were being classified in this particular way. And then we brought them through a high sensitivity analysis where for a variety of different services, but I’ll use airlines as an example, we started to ask them about how much they would be willing to pay for different aspects of the service. So on an airline, how much would they be willing to pay more for on an aisle seat or for an upgrade and basically there was a price sensitivity conjoined analysis. We were able to compare the people that had gotten good customer service on Twitter and people that hadn’t, and what we found is that the people that had received good customer service on Twitter were much less price sensitive, because they had this positive experience with the brand that when we started asking them about how much they were willing to pay, they actually were willing to pay more for that airline service than people who hadn’t received good customer service. So we were able to really hone in to the ROI of customer service on Twitter and we were surprised that – we were hoping that that was true, because we want – we believe that that was true, that was our hypothesis, but to see it play out in such a quantified way I think was really exciting.
David Kruse: Interesting and do you guys have your own analytical tools. I know you mentioned R and Python. Do you…
Jeffery Graham: Yeah, we do, we do. We have you know a data stack that has different layers obviously and we have some of our own ways of warehousing the data and clearing the data, like a lot of the big tech companies.
David Kruse: Got you, and if I was Pepsi, you know and there is – I mentioned Pepsi, because you know they don’t have a lot of – a ton of online purchases. How do you, how can you show Pepsi that Twitter is delivering value? Like do you have a scorning system, you know yeah, there’s probably re-tweets you can measure, there is lots of things you can measure, but how do you come down to a score or something?
Jeffery Graham: Yeah well, it depends on the objective for that particular campaign and certainly for you know, if they are trying to deliver you know entertaining content, we can look at engagement and re-Tweets and things like that. Sometimes a brand like Pepsi is trying to build there association with you know the Olympics for example or the Super Bowl, and so we can measure how well the advertising builds that association, the brand attributes, even awareness of the new product. And in the U.S. we can actually measure sales volume. So we can compare households that have received Pepsi advertising for a particular campaign, compare them to like households that didn’t receive that advertising and basically with royalty card data compare that to sets at household and see if there is an uplift in actually sales. So it…
David Kruse: You can tie together the Twitter handle with whether – with the household and see if they actually purchased?
Jeffery Graham: Yeah, we work with third parties and obviously it’s all privacy protected, but yeah there are techniques to basically do blind matches at the household level and see what the impact is on sales.
David Kruse: That’s helpful. Yeah, that’s good, okay. All right, I know we are almost out of time. I did read about one study that I thought was quite intriguing and went around measuring brain activity and to see peoples reactions when they are using Twitter. Can you just tell us a little bit about that?
Jeffery Graham: Sure. Yeah, I mean we ask people questions all the time and we look at their behavior, but we also like to understand what their implicit reactions are and one of the techniques we’ve used is looking at brain activity while people use Twitter and how they are responding to Twitter relative to other online media, to see kind of how Twitter maybe different.
So we recruit people, centralize people to really do this research and what we found is that based on what are peoples kind of neuro activity, electrical activity is observed while they are using Twitter, you can kind of see differences between Twitter and other online media and have found that Twitter is actually really emotionally medium that people kind of have a really personal relationship with the platform and that tends to make the messages they see there very memorable. So we’ve done that in a few markets around the world. It’s just a really interesting way to get insights without having to ask somebody a direct question about something.
David Kruse: This will all be a lot easy when we have brain implants and you can just tap into our brain activity that would be awesome. I’d sign up for that.
Jeffery Graham: I don’t know about that.
David Kruse: I would sign up for that. But you might have a sample of one or a population of one.
Jeffery Graham: All right, well we got to start somewhere.
David Kruse: Yeah, that’s right. All right, well yeah, I definitely appreciate it, your time. I think that just about does it for this podcast, but what you have done is quite interesting and you’ve had quite a career. So I appreciate you sharing a little bit about what you have done.
Jeffery Graham: My pleasure, my pleasure. It was really nice to talk to you.
David Kruse: Definitely and thanks everyone for listening to another episode of Flyover Labs. As always I greatly appreciate it, and we’ll see you next time. Thanks Jeffery, thanks everyone. Bye.