Join Nathalie’s journey leading teams from Dow Jones (Wall Street Journal) to ESPN to CBS Interactive to the ANA and now in her new role as Chief Research & Analytics Officer at Yahoo. Nathalie shares amazing insights around the challenges of cross-platform measurement, viewability, integrating data science and consumer insights and so much more. An episode you won’t want to miss!

Transcript:

Duane: [00:00:00] Legends of Media Research is a podcast series featuring interviews with the Media Industry’s leading researchers, where we go behind the scenes, sharing stories from their greatest achievements and challenges. Brought to you by MediaScience, the leader in Media and Advertising Innovation Research.

Stay tuned at the end of the podcast for more information about MediaScience, but for now, I’m your host, MediaScience CEO, Dr. Duane Varan.

Welcome to another exciting episode of Legends of Media Research. I’m your host, Dr. Duane Varan, and I’m thrilled to have Nathalie Bordes today. Natalie serves as the new Chief Research Officer at Yahoo. Nathalie, welcome to Legends of Media Research. It’s such a thrill to have you on the show today.

Nathalie: Wonderful. Thank you so much for having me, Duane. I’m really excited to be a little bit with you.

Duane: [00:00:00] Now, Nathalie, how did you get [00:01:00] started in media? I know your story and I think the audience is going to find this really, really amazing because you know, throughout your career, it’s like you stumble into these paths and then end up thriving and excelling in them. It’s such an exciting story. How did you get started in media?

Nathalie: I literally stumbled into my career. I was still working in Europe at the Wall Street Journal Europe and Dow Jones, and I was a marketing assistant. And one day my boss, Sabine Schmidt, called me into the office and said, “we’re going to do this thing here called media and marketing research, and I want you to work on it”.

And I kind of like thought, “really?” Really that seems a little bit odd and I was doing everything to convince her otherwise. I was still studying in the evening. I wanted to get my degree and so I was busy with that. I also kind of like was ready to move into a different company at some point. [00:02:00] And lastly, I also told her, “well, I’m not that great at math”. [Duane chuckles]

And she just looked at me and she ultimately told me “the computer is doing all of the math and I’m going to be honest, as long as you’re here, we’re going to do this”. Lo and behold, this woman recognized the talent in me and she gave me a career. She didn’t give me just a little bit of a task. She gave me a real career that I would have never ventured in.

I started to do first syndicated pan regional European and pan regional Asian research, holding research rankers for media executives and agencies. And I would say I caught the bug for media measurement. It fascinated me, the topic.

Duane: [00:00:00] And this was at Dow Jones in Germany at the time. Is that right?

Nathalie: Correct. Exactly. Yes. I was still working in Germany and that’s how it started.

Duane: How did you end up going from being a media researcher in Germany to [00:03:00] becoming a media researcher in the United States?

Nathalie: Again, the amazing Sabine Schmidt, [laughs] the head of the Dow Jones office in Frankfurt, she realized that I started to outgrow my role.

And she ultimately asked me, “hey, do you want to work for a couple of years abroad?” And I kind of like thought, “oh sure, why not?” I was not married. I was kind of like young and kind of like free spirit and I thought, “I’m going to go for two years to London and that’s it”. A couple of weeks after that conversation, she called me from a business trip from the New York headquarters of the Wall Street Journal and said, “they have an opening here for research analysts, but you have to apply today”.

And crazy enough, I did apply today and lo and behold, I got the job again with a lot of her help and support there. And three or four months later, I moved from Frankfurt to New York. And I had planned to stay in New York for two years. [00:04:00] That’s now many, many years later, almost 20 years later.

Duane: [laughs] We all need a Sabine in our life.

Nathalie: To be honest, I think that she not only gave me a career, she also gave me a lesson and that is when you find talented employees, support them, grow them, encourage them in their career path or even push them into a different career path when you see certain talent and just be a good supporter for younger and upcoming talent.

Duane: So you came across to down the U.S. with the Wall Street Journal. What did you do? So you arrived here, you know, fresh from Germany and you came into this U.S. operation in New York. What happened once you arrived here? What did you do in that role?

Nathalie: [chuckles] I did first syndicated research and I was in charge for classified research.

So back then the Wall Street Journal was still in print, or it is [00:05:00] still in print, but it had a big print section. And one of the big businesses were the classifieds. But also I was in charge of also the starch program, which was advertising recall of printed ads. That was what I did really. Right in the beginning, the beauty for my time at Dow Jones and the Wall Street Journal was that every two to three years, though, I was able to reinvent myself and to try out new things.

Duane: That seems to be the story of your career.

Nathalie: Exactly. Exactly. Curiosity is really what drives me. It’s not necessarily chasing the next big title, it’s always the curiosity that pushes me into the next endeavor. And so in the beginning, I worked on the consumer side on the print side for the Wall Street Journal.

And then they had an opening on the digital ad sales research side. And I still remember I went and had my pitch and I pitched to Andy Sipple [00:06:00] and Mark Fishkin, who were back then leading the marketing and the digital sales organization. And I pitched them why I should have that role as digital ad sales lead.

And my pitch was essentially, I know that I can do this. I know I have no proof of that, but I can do it. And they both gave me a chance and that reinvented my career because all of a sudden I went from print, from traditional print syndicated research, I went to digital and got my footing into digital analytics.

Duane: Fantastic. What a great story. You know, knowing you as I do, I think it is again, this, the story of this is this classic pattern, Natalie, where you go into this, this job, which, you know, you don’t really have the, the, the background for, but you just thrive in these roles. Like you go into these places where you don’t have the background and you just thrive in that environment.

It’s almost like it’s an. [00:07:00] Asset for you, like starting these, these jobs kind of like with a fresh slate seems to have really given you like a little bit of a leg up.

Nathalie: It does, it does. I think the ability to adapt to situations, but also to have some resilience and the willingness just to learn new stuff all the time and thinking through new approaches and exposing myself to.

New environment and challenges that works for me, and it keeps it interesting. The most fascinating thing for me is, is there’s always that pattern of the role needs to be intellectually challenging, and it needs to be a product. That I’m interested in, which is quite funny that I ended up at ESPN because I’m not really a sports fan in the traditional sense, but I like the business of sport, everything that has to do with a sport and around it is fascinating.

However, if you force me to watch a [00:08:00] full game of any Anything I, I kind of like say, Oh, do I have to, can I not just read the summary of the results?

Duane: So let’s talk about that. Then you did this transition and you went to ESPN. How, how did that come about? I mean, you were actually with Dow for many, many, many years.

Nathalie: Correct. Yes. And one of the things that had started to emerge in my last couple of years at the Wall Street Journal was that constant question of how are users moving across platforms. Back then it was between print and digital. And lo and behold, there was no great answer out there in the marketplace.

So we had to come up with our own studies. We had to think through our own thoughts of how can I tie together people that access us on the desktop that access us on mobile apps and also on on print, how do I tie all of that together? Get an understanding of what is really the crossover [00:09:00] and what are the power users and what are the single siloed users?

Sounds familiar, that problem.

Duane: All these years later, it sounds like the problem we’re still dealing with today. Exactly,

Nathalie: but it was really something that fascinated me. And then I went to one of the research conferences and Artie Bulgarin presented together with his team Project Blueprint. And yeah,

Duane: I remember blueprint.

What a, what a landmark project.

Nathalie: It was because in that moment, I realized I cannot do cross media research. When I am not having a, no TV in there and when I don’t know TV, so I took a very consolidated effort and looked what type of openings does ESPN have open. Then I looked in my network of saying, okay, can anybody vouch for me because I know nothing about American sports.

And then I went on and applied and Artie Bulgarin together with Barbara Singer hired [00:10:00] me. Although I was very candid and honest in my interview that I’m not a sports fan. And so I had really a great opportunity to work at ESPN as senior director of emerging platform channels. And that was great because I was overseeing parts of the multimedia ad sales research team, but I also got to play in a new field and that had areas of testing out new measures and metrics that came into the marketplace.

One of them was for example, viewability where you and I worked. Very closely together, but really testing out how are these metrics impacting our ad sales business? What is it that we need to test out? What is it that we need to learn about those metrics and how should we incorporate them into our business flow?

That was part of the role. And it was fascinating. I had, I had a blast. It was really great.

 

Duane: You know, one of the reasons that I come back to this theme about why I say. It’s always been an advantage that you haven’t [00:11:00] had the background into the, the, the roles that you’ve landed in is that one of your real strengths has always been to bring in new angles, new perspectives, you know, like, like the work that I did with you and with ESPN.

We’ll talk a little bit more about that later in the podcast as well, but, you know, people were dealing with viewability. They were looking at, I mean, everybody was looking at things a certain way and, and one of your real strengths. You know, Natalie has been, you bring a different perspective to the party.

Like you come in and you say with with the viewability, you know, one of the things that really impressed me is the way that you look at the financial dimensions of what the consequences were of the kind of things that were happening there. Such a strength, such an asset. And it’s really, I think a big part of the key of why you’ve been so successful in these different roles.

You know, that ability to bring in these new perspectives. I mean, you know, kudos to you. It’s really been, been such an outstanding

Nathalie: job. It is the fun part of really thinking about the different aspects, but I also [00:12:00] have to admit it depends a lot on the environment in which I work in. Now I was lucky enough to work with Artie Bulgarin and he allowed me.

To explore a little bit that is kind of like an important part, the ability for him to having trusted me and saying, okay, she does for a day to day job, but I can also give her a little bit of leeway. And then I will also see how that is ultimately also benefiting our corporation in the larger sense.

And that helped a lot in allowing me also to, to really partner, for example, with our business operations team on viewability and saying, okay, if, if. If we can only count, let’s say, 95 percent of our ad impressions, what does that actually mean in revenue? How is that impacting us? How can it be when you have 10 different vendors you have a range from, let’s say, 30 percent non viewable, To let’s say 5 percent non viewable, how can something like that happen?

And that was all in the [00:13:00] beginning of viewability, when there was a very new concept that was introduced to the marketplace by the Media Rating Council.

Duane: Yeah, it’s a great example. So, you know, we were all shocked. I remember at, at ESPN, we were all shocked the day. We heard you were leaving. It was a huge surprise.

You fit in so well in the ESPN culture there. What happened that, that got you to

Nathalie: leave? Well, one of the things, and I think that was more the structural thought process that ESPN had back, back then, it was the idea of you have traditional research on the left side and on the right side, you have data.

And they were not, they were not thinking about this is basically the same coin, just different sides of it, and you can really work closer together. And during my time when I was there, other groups had made the decision that those two disciplines should not be overseen by RD, for example, but that they [00:14:00] had different leadership.

And I was, during that time, approached by CBS Interactive. And they suggested to me, Hey, we want you to oversee our business intelligence team and the research team. And we want you to kind of like oversee both. I thought, that sounds really interesting. And it is actually more philosophically aligned with where I think the knowledge disciplines need to go.

We cannot pit traditional research. And data against each other, they need to work hand in hand and we cannot really have that divergence in that. And that’s what fascinated me. There was also the part when I had. Interviews with the CBS Interactive leaders, it was fascinating because they were already knee deep in the streaming world when nobody aside from Netflix was actually talking about streaming, they already [00:15:00] had all access up and running.

And that was just intriguing because all of a sudden I needed to think about it’s almost a new medium, a completely new platform, and what type of insights do we need to bring here? How do I approach that? And that was just fascinating for me. It was new learnings, combining that with some of my skill sets.

And also with a brand that I liked, I, I am a little bit snobby when it comes to brands, I feel,

Duane: but, you know, you raise a good point with the, the integration of the data science and the consumer insights, kind of like sides of the business every organization, I think has really had to struggle, you know, every media organization has had to struggle with this question about how do you deal with your consumer insights, People and your data science people, do you have them separate?

Do you do somehow combine them? You know, it’s been a [00:16:00] real challenge. I think for the industry with very different kinds of results, you know, very different results in terms of what effects they’ve had on the structure of their performance ultimately in the market. Well, we’ll talk more about that in the podcast as well.

So then you were at CBS Interactive. How long were you at CBS?

Nathalie: Six years, almost six

Duane: years. Six years. Wow. Time flies. And what happened? You, you were then there for a while and then you, you made this transition to the ANA overseeing kind of like their, their cross platform initiative. How did that come about?

Nathalie: And for me, it was fascinating. First of all, I loved working for CBS and tech. It was, was a ton of fun. And I got the chance to really build in a data and insights framework for all of the CBS digital brands. And that was really the opportunity of a lifetime. And in the end, I was even starting to oversee Pluto BI.

So it was really going deeply in, but what I also realized was [00:17:00] that my job became more and more. Just purely in overseeing an amazing team and that sounds weird, but the day to day became very static. There was not really that, that new challenge of let me build something completely new. I realized that that’s not really my strengths, kind of like that maintaining and, and, and kind of like just saying, let’s maintain that high status.

And of course we’re kind of like increasing, but you had nothing. Completely new to invent in that moment. There was one part, but it was also timing. The ANA approached me, Bill Tucker approached me and said that my name was given for a role where the ANA is focusing very heavily on a cross platform measurement initiative.

And that was intriguing because the notion of cross platform measurement has been still haunting me. It’s still haunting me. It’s like, I still haven’t cracked that, not completely. So there [00:18:00] must be a way. And I was fascinated by the marketers wanting to take charge of that conversation and wanting to be actively involved.

Because up until then, I only saw media companies. Publishers and vendors really trying to, to, to go knee deep into that problem and all of a sudden here you have this really powerful trade organization that has the marketers and that has really that those marketers that are fascinated and interested in making a change in the industry and that intrigued me tremendously.

That really intrigued me tremendously and that was ultimately where I said, okay, I build a great team with CBS Interactive or then Viacom CBS and I’m okay with, with kind of like handing the reins over to an amazing team and focus on a new challenge. That’s, that’s how I ended up at the ANA and it was great.

It was fascinating for me because again, it soothed my [00:19:00] curiosity. The project. Act itself was incredibly fascinating because all of a sudden you’re trying to, to work with all of these groups together. And we have one goal, but we have a multitude of different thoughts and feedback about the approach. It was great.

I, I learned so much about myself. I got a ton of really great exposure with really great marketing leaders. It was fascinating.

Duane: And you had a chance to work with Artie again in that role. Yes,

Nathalie: yes, I had the chance to work with Artie together and he’s just amazing. There are not a lot of media researchers or researchers in general around advertising.

That have such a fundamental knowledge of how advertising works on each and every single layer and having just those conversations with him. I think we’re at the moment industry wide [00:20:00] at a very important point because you’re seeing more and more marketers pushing for performance marketing at the cost of brand marketing.

And I don’t think that we have as an, as a larger industry fully explored yet of what that impact ultimately is and how that will trickle down on the notion of brand equity, which again, then for marketers, that’s what you paid for building the brand equity, the brand loyalty. So just seeing that all as an ecosystem and having those conversations it was a blast.

It was amazing.

Duane: So then you, you made the transition to your current role only very recently as chief research officer at Yahoo. How did that come about?

Nathalie: Ah, yes. So I worked at CBS Interactive for among others for the CEO, Jim Lanzone. And when Jim Lanzone became CEO of Yahoo, he reached out and said, Would you [00:21:00] be interested in coming to Yahoo?

And. I have a sweet spot for Yahoo. It’s, it’s, it’s my longest standing digital relationship with my Yahoo email. That’s my longest standing digital relationship. And there, there was really that moment where I thought, that sounds interesting. And then I started to hear more and more what Yahoo actually does.

I kind of Did my research. You look in all of the different syndicated resources to see how big is the the footprint of, of Yahoo. What is really rolling up under it? And all of a sudden you realize, oh, this is a media company. This is a mail provider. This is a search organization. This has this is a company with an ad tech business.

This is a global organization.

Duane: I think also most people don’t realize how, how big the footprint of Yahoo is in the market. What you rank something like fifth. Is it in the market today? Yes. Yeah. One of the top [00:22:00] five. It’s

Nathalie: amazing. Exactly. Yes. It’s it is really people underestimate that. And you have Yahoo obviously kind of like as, as, as the corporate brand, but you also have AOL is there.

AOL people chuckle about it, but let’s face it. AOL brought people through the internet and we shall not forget that, but it’s, it’s just a great. Corporation with, with so much brand equity, but also with so many different interesting businesses that it’s just fascinating. It really, it’s, it speaks obviously to me because there are areas where I’m still a novice in and where I can say, okay, fine.

Let me, let me learn more about that. How is that business really going? What is the monetization model behind it? How are we really measuring everything? How are we tying all of the metrics? together. And then because Yahoo has Has [00:23:00] its footing really, or was born as a tech corporation, the wealth of data is like, this is like a candy store

Duane: for someone like me.

You know, I remember many, many, many moons ago when we started the, the Disney lab, one of the first studies, in fact, it was the first study that we did, we were doing an analysis of ESPN’s competition and ad models. And I remember how incredibly effective and powerful the Yahoo models were. And the big discussion we had was what the Yahoo knew that we could see it, you know, on the ESPN side when we were analyzing the research, but we didn’t know if Yahoo knew how, how powerful a lot of the new ad formats were at that time.

Nathalie: Exactly, exactly. It was just a great opportunity. I have the chance to work with a global team. The organization is really, we’re having some of the best leaders for each and every single one of those businesses in place and [00:24:00] it’s It’s just fascinating. It’s just really fascinating.

Duane: So Natalie, we’ve done a little bit of a bird’s eye view of your career here. We’ve gone from, you know, you starting off in Germany at Dow. That was your first position, you know, which translated into, you know, your time at the Wall Street Journal in New York, moving on to ESPN, CBS Interactive. The ANA and now in this role at Yahoo.

Now let’s go back and let’s drill deeper on a couple of the key kind of like moments, if you will, the couple of the arenas that you’ve played in. Let’s start with cross platform. That’s such a big part of your career. Such a big part of your career has been really working in the sandbox around. Cross platform it started for you with this fascination you had with project blueprint.

I look back at project blueprint and it’s like, I cannot understand for the life of me how the industry missed that opportunity project project [00:25:00] blueprint was so good, you know, again. Kudos to Artie. He did such a good job, kind of like looking over the horizon and seeing the need for for the measures before anybody else did.

But, but Project Blueprint really kind of like had most of the fundamentals in place. And here it is a decade later, and we’re still trying to figure it out. Why do you think it was that people Missed the boat, you know, with with the opportunity that project blueprint presented and have struggled so much since then to try to, to bring that measure to market.

Nathalie: I thought a lot about that because obviously it is something I still consider it as not fully solved the issue. I think that the methodologies. That we can apply to get to that question have evolved, but the reason why we haven’t found an industry wide solution to that problem is that I think as a larger industry, we’re [00:26:00] making the mistake of confusing the business of accurately counting audiences and understanding audience overlaps with what is the best measurement for my business.

And we’re not keeping those things separately. And so we are ending up converging that and thereby we are ending up with a measurement system that gets dominated by the question of what is best for my business, as opposed to saying what is really just purely scientifically the most accurate way of accurately.

Aggregating audiences and understanding audience overlaps, because there is no way that with all of the smart people in the industry that we cannot figure out the map. There’s no way that we couldn’t figure that one out so far, but we are allowing ourselves to get sidetracked really by the, by the business conversation.

Now. I’m not saying that [00:27:00] that is a trivial problem. The, the opposite is the case. Ultimately, there is a lot of, of money tied to that, but I think we also need to be honest about it from an industry perspective and saying, we could solve for the math problem, we’re just choosing to, to deprioritize the math and focus potentially a little bit more on the, on the monetization models that are impacted by the audience aggregation.

 

Duane: You know, it’s, it’s such a good point. When you look at our industry today, it’s a little bit crazy. You know, once upon a time, we used to talk about reach. We used to talk about frequency. It’s, it’s hard to imagine any media measurement that you would have done that did not have reach and frequency kind of like at the heart and soul of what we do.

But, you But now reach and frequency seem like foreign concepts because of this inability to get good reach and, and frequency, you know, with the, with the, you know, [00:28:00] duplication of audience across platforms, it just seems like such. An incredibly important question, you know, to address before you’re dealing with 101 other kind of like measurement challenges that you have.

It just seems like, you know, we, we haven’t understood as an industry, how important it is to get the fundamentals right before we start worrying about. You know, all of the other kinds of because I think one of the problems that we haven’t come to terms with as an industry is that every brand needs to understand what its key communication objectives are and come up with strategies for measuring against those, you know, best in class methods for measuring against those objectives, but as an industry beyond whatever the needs of a specific brand are, there are some common problems that we all face and, and getting a good read of the experience exposure side of that, not anything more, just the exposure side of that.
That’s got to be like the fundamental [00:29:00] foundation upon which everything else has to then get built.

I look back at project blueprint and it’s like, I cannot understand for the life of me how the industry missed that opportunity project project [00:25:00] blueprint was so good, you know, again. Kudos to Artie. He did such a good job, kind of like looking over the horizon and seeing the need for for the measures before anybody else did.

But, but Project Blueprint really kind of like had most of the fundamentals in place. And here it is a decade later, and we’re still trying to figure it out. Why do you think it was that people Missed the boat, you know, with with the opportunity that project blueprint presented and have struggled so much since then to try to, to bring that measure to market.

Nathalie: Well, yes. If you’re talking with, let’s say measurement purists like you and me, absolutely, absolutely. But I, and I, I think it’s really. It’s a question of agendas that are maybe trying to, to, to converge a little bit too aggressively.

I am concerned that we are moving further and further away from fundamental measures like region frequency. I think there’s a lot of New shiny toys that are kind of like being brought up and everybody says, Ooh, look at this. But in the end, I think it is really the basics of understanding how many people do I reach?

How often do I reach them? And with what type of messaging should I reach them? That’s ultimately what the marketers should stay focused on. But it is also hard for marketers to stay focused when their roles are also [00:30:00] often Focused or, or rushed into get me results quickly, get me kind of like something that I, that showcases to me that this advertising worked, show me a fancy dashboard that showcases something has happened and so marketers and their agencies have to take those limited resources that they have and really figure out how can I make that very, very visible to an increasingly financially challenged business model.

Duane: You know, I think also in line with that, there’s, there’s been a little bit of a problem in that, you know, we’ve moved, moved to this, I say fantasy, forgive me for using that word in this context, because it’s, it’s perhaps a little bit over, you know, over exaggerated, but, but we’ve moved to this fantasy that we can measure things at census level, you know, that we can look at performance data and understand what’s going on without understanding things.

Thank you. The complexities of what happens on the other side of of of a screen, [00:31:00] you know, without understanding that a metric doesn’t mean what we think it means if, you know, it’s easy for us to get very distorted pictures of what’s going on by just looking at the performance metrics and that’s the reason why getting the fundamentals right are so critical because if we don’t have.

If we don’t have a ground truth, if we don’t have something where we know what’s going on, you know, all those measures are great, but you need something to calibrate them against ultimately. And if you don’t have that, you don’t know what’s really going on. You just end up with such distorted pictures.

Nathalie: Well, it’s, it makes me chuckle a little bit because obviously over the past, I want to say three to five years. The industry kind of like went 360 on the whole notion of panels. I think a couple of years ago, we had the first thought process and people saying, Oh, okay. We don’t need panels. We have all census data.

We can just really use the big data sets. And now the pendulum, I think, swung a little bit back and people understand you still need a [00:32:00] calibration panel. You still need to have that, that, that panel that allows you to understand. What happens beyond the data or how do I ensure that I’m really putting the data sources with the appropriate weights together and achieve thereby a balanced view of of everything.

But it, it is an interesting thought process of really seeing how can the industry really think through the concept of, do we need five different calibration panels or Do we need to think as a larger industry about one open and fairly available calibration panel? Is that something where the marketplace needs to think through?

But then you’re looking at other global markets at the concept of a jig. Is that ultimately what you’re, what you’re looking for, but then how does that violate your competition laws here in this country? So there are so many aspects of measurement. Measurement is no [00:33:00] longer purely about the map. Like I said, you have also a very complex ecosystem around it.

We also need to be realistic. True measurement is not cheap. It’s not something that you can just get very cheaply. So someone needs to foot the bill there. And in many cases, a lot of the big media companies have been stuck with those big bills. And that is also money that they have to justify.

So it is such an intriguing and fascinating industry, but it was also never more interesting than it is today.

Duane: And the rumors that I hear are that there’s good cause for us to have positive hope that, that there are solutions to this problem right around the corner.

Nathalie: You and I hear the same, same type of rumors, I think.

Duane: So Natalie, let’s now shift [00:34:00] gears. Let’s talk about The work that we collaborated on on viewability. It was really truly landmark at the time. And again, just for the sake of the audience, we should frame this. This was a long time ago. I mean, it was a decade ago, but truly monumental in terms of what was happening at that time and the debate at that time.

So you know, for the benefit of the audience, let’s first help the audience. You know, some people in the audience don’t come from the ad metric side. So let’s help everybody get on the same page on what. Okay. Happened, what the genesis was in the industry around this idea of having a viewability measure, what, what led to that?

What was the need, if you will, that, that, that that measure was really trying to solve for.

Nathalie: Yeah, the viewability standards were really at, at its core invented or brought into life to ensure that some of the fraudulent players in the marketplace would not charge for ads that are not viewable in a browser space.

Face or on a mobile screen. [00:35:00] There were really kind of like bad, bad players in the marketplace that were trying to charge advertisers money for advertising impressions that were delivered below the fold where nobody could see them or where ad impressions were delivered behind a browser. The pop up so it was really impossible for the end user and consumer to see the ad impression.

Obviously that is something that cannot go on and should not go on. And the media rating council came thereby up with the viewability standards. The viewability standards had two components. There was on the one hand, how much of the ad needs to be visible in your browser space, for example, on your desktop.

And then also needed to be determined how long the ad must be visible in your browsers. So let’s first go to the space or to the, to the size 50 percent of the pixels of the ad [00:36:00] must have been visible on the screen on the desktop side. And then when it came to the time perspective on desktop, it could be one second, whereas on mobile, it needed to be two seconds.

And one of the things that was just seemed odd was the fact that there are different time components, one for desktop and one for mobile. And that became just very, very clearly a little bit of a financial challenge for organizations that had a strong mobile footprint. Just purely for ESPN, it would have meant a very, very strong financial impact for every second that you add, you had the challenge or could end up with 5 percent less billable ad impressions.

And that would have been a severe financial impact that is, that was one of the parts and that raised the question of, [00:37:00] is there a need for a different time requirement? Yeah, so that was

Duane: Yeah, so that was the genesis of the study was, you know, whether mobile and desktop were sufficiently different from one another to justify mobile effectively being penalized, you know, by having, A time threshold that was twice the standard and you know what the research found, which was so fascinating was that mobile was actually far more viewable than desktop.

And it makes sense. You know, your screen is small. You don’t have the space there, you know, for anything else there when the ads. There, it’s kind of like the star, if you will. So people are going to see it. It requires less time to process the message than it did in desktop. So if anything, there should have been a lower standard.

So if it was one second for desktop, it should have been half a second for mobile or let’s face it at least the same, not different. And, and so that’s what created this problem.

Nathalie: Exactly. It was [00:38:00] really an interesting question. What I enjoyed about that was we wanted to take an academic approach to, to measuring that.

And I, I think up until then, I hadn’t seen any research that was really focused on saying, okay, what happens if someone is exposed for half a second to, to an ad versus one second versus three seconds. How’s that impacting, for example, the, the. The ad recall. Yeah.

Duane: And just again, for the benefit of the audience, the design of the study was that the test subjects were exposed to add content that was delivered in different time intervals.

So, as you were saying, half a second. Of an ad exposure, one second of exposure, two seconds of exposure, all the way up to five seconds of exposure that we had in the in the study. And we did that for desktop and we did it for mobile. And we demonstrated quite conclusively that, you know, on mobile, it was far stronger.

You know, the view ability was [00:39:00] much greater on on mobile than it was on desktop. But there was another finding in that study that again, with the benefit of hindsight was truly landmark. And that was this idea that even relatively short exposures in time actually delivered at impact, you know, the, the focus of the study really was on the eye tracking that we did to see what the visual attention was.

But as it happened, we also had ad recall as, as a, a measure there. And we stumbled into this idea that. Ads actually start to deliver an impact even with just half a second of exposure. I mean, that was shocking, right? That was never the intent of the study, but, you know, what it led us to discover was that, you know, two seconds of an ad exposure, for example, really are meaningful.

It is a really meaningful unit of impacts. That ad impact is really front loaded that you start to get an effect very early in the life of the exposure [00:40:00] of the ad.

Nathalie: Yeah, I think the interesting part is, and again, that was 10 years ago. Now you’re thinking about what levels of ad impact are you actually getting?

What what impact are we really generating? Is it ad recall? Can you generate favorability in a very short? First, or is it really more the ad recall? So, so there are also nuances to it that I feel just fascinating to explore and really

Duane: understand. And of course, short ads have become so much more important for the industry.

We have, you know, new formats, new platforms that live off of those short exposures.

Nathalie: Exactly. It’s also a question of, are the consumers just consuming advertising differently now, and are there acceptance for longer ad units? Is that changing overall, and are people tying that now more to specific [00:41:00] viewing platforms?

 

Where they’re, where they’re saying, hey, I’m okay with a longer ad commercial. Whereas on certain platforms, I’m, I’m not willing to accept anything that is more than three seconds or so.

Duane: Natalie, let’s shift gears again. Another. thing that I think is really exciting about your career. And really it started at your time at ESPN, but of course it really went you know into TurboCharge when you, when you went into your CBS role. And that’s that whole question again, about how you bring data science and consumer insights together.

Right first, whether there’s a need to bring it together. I mean, there are, there are people out there. I think who, who don’t believe that, who don’t think that’s a necessity. So some people think that, you know, the data science alone is sufficient to do the job. So why don’t you talk to us a little bit about this?

[00:42:00] Question about, you know, do you need to bring consumer insights and data science together? Can they not just live in separate silos? Do you even need consumer insights? You know, is data science enough? You know, these are the kinds of challenges I think that are that are there for the industry today. Yes, and,

Nathalie: and they are surprisingly persistent in people’s mindset, but in the end, it comes down to data science can answer questions based on anything that generates a machine answer.

That’s basically what it is. If you can get a signal. You can analyze it as a data scientist, but there’s an aspect of the consumer journey of the consumer behavior that is not necessarily producing a very clear signal. And that is really attitudinal signals. It’s, it’s favorability that is not expressed.

Through a click or hovering over an object or staring [00:43:00] at an object. It is really understanding why do I have. A favorability for a certain product, for a certain piece of content. Why is that touching me? Why do I want to laugh or cry about something? The computer can tell me, did I interact with it? And so thereby the data scientists can help me understand which content or which, which features are generating the most measurable interaction.

But consumer insights or user experience research tells you really why is it driving something? Why are people drawn to it or appalled by.

Duane: it? Yeah. And I think it’s a good point because, you know, we want to have insights that ultimately grow out of research that are actionable. And if all we have is.

Kind of like the what, but not the why it’s very hard to get to the insight out of that. And what we end up doing [00:44:00] often is we end up assuming the why, you know, we, we get data that shows a pattern and we look at that pattern and we make an assumption about. the pattern. I’ll give you an example from some of the work we did in, you know, in, in, in the UK.
Actually, we had data off of the B SkyB platform there, which was looking at their interactive TV ads. You know, interactive TV is much, much, much more developed in the UK market even today. And we were looking at, you know, huge volumes of data, huge volumes of data around, you know, what people were doing.

And then we did a very sophisticated process of coding that data, which, which took my research team a solid year. So it was a massive, a massive endeavor. And what we discovered was that a lot of the assumptions in the market around why people were doing what they were doing, what was actually a mistake that an actual fact, some of the market leaders.

Were getting these spectacular results, but it was [00:45:00] despite the fact that they had done something not because of it. So, in particular, the, the big thing that we found was that the call to action that said click here for more information was the kiss of death. It drove response down almost. 500%. But that was the number one advice that the market was being told about what, what they should do.

And the reason was because you had these spectacular leaders who had done it and it was successful, but they were successful despite the fact that they did that, not because of that. And again, this is the kind of thing that happens all the time. You, you, you see the pattern, but you get the wrong assumption about why the pattern was there.

Such a dangerous thing,

Nathalie: really. It is. It is. I believe that when you have both disciplines together and have them work really collaboratively together, you can really also shape where your, your roadmap, your insights roadmap has to go. And you can really evolve it more and more. It’s also both. Disciplines together can evolve each other and [00:46:00] I, I think, especially in the beginning when data science emerged and it was kind of like Canada, like the new shiny toy in the industry.

I think there was a lot of mismanagement. Of of these groups, because they were pitted against each other. They had to fight for resourcing for headcounts and everything. And that prohibited a lot of really great collaboration. What I’ve seen, for example, very often in practice is that for researchers, the intake process of getting.

To work with their partners to asking them, Hey, what is really the business question that you need? And everything is a very thorough process that often helps shaping a project very, very well. Data scientists often don’t have that level of intake process. They get the email of, Hey, can you help me pull this?
Or can you help me with this information? And then they are left to their own vices. And so they are often just. Put into, hey, answer me one question, [00:47:00] and then they’re disappearing out of the life. Whereas the, the researchers in particular user experience researchers are often very closely tied to the product development and are ongoing in the interactively built into the process.

That doesn’t mean that data scientists are not also having that opportunity, but it is a little bit of a different dynamic because a lot of their job is also. Very heavily focused on, on doing it’s, it’s just them and the machine or them and the data. And that’s, that’s a little bit of a different skill set also sometimes.

And if you have them both work together, you help also the data scientists to get closer to, to the requesters motivations. And at the same time, you get also the researcher a little bit more exposed to what additional information can I use to shape my intake. You

 

Duane: know, Natalie, you did such a good job at CBS Interactive in bringing these two [00:48:00] cultures, these two worlds together.
How did you achieve that? How did you do that?

Nathalie: I think I got to a certain degree lucky because I also had the right personalities in the team. All of my teams were driven very heavily by curiosity and they were open to working with each other. In the beginning, there was a little bit of. Well, in research, you only have 2000 records, I have here a million and kind of like just helping them understand a statistical projections.

Here’s really kind of like the basic math on which everything is built and making sure that they’re understanding. You’re both working basically on the same principles. That is you’re taking a sample of data and you’re projecting it out. And that was very, very important part of having people that are willing to listen.
But then also not getting deterred when you’re seeing after month one, two, three, four, six, seven, or ten, you still have individuals that are just not really clicking and vibing with each other and [00:49:00] getting organically together. It’s really being persistent, encouraging those, those pieces. And often, Making very clear suggestions of why don’t you work together?

Let’s, let’s, for example, take something like the Net Promoter Scores, NPS, where you’re pulling basically a sample of your audience and you’re asking them, would you recommend this product? And then you’re building your score. It’s a very long standing traditional tool that is used. When I first started at CBS, we conducted it, but we never tied it to the user behavior data.

Now, once I suggested to them, can we bridge the gap here and can really understand what is the user experience that this person had before they gave us their rating, all of a sudden you can tie behavior as well as their verbatim together and you all of a sudden get that understanding of, oh, that’s the reason why he’s giving that rating [00:50:00] or, oh, That’s the reason why that person is so satisfied.

That is really helping you really bring additional context to it. Now, what fascinated my researcher, as well as my data scientist was. The data scientist was intrigued by how can I solve for connecting those two pieces with each other? And how can I then visualize that behavior? The researcher was just really, really happy to get additional context and insights into how that user behaved.

Duane: So it was really tickling their curiosity and combining them in one project. What a great example. And what do you think the benefit was ultimately of that, of that integration?

Nathalie: It really helped us getting a little bit clearer in understanding certain user patterns and certain user responses, understanding why users abandoned certain experiences, understanding also why people leaned into certain experiences more or [00:51:00] understanding what type of traffic Or app acquisition drivers that we need to use because that yielded more satisfied users because they had apparently a specific expectation when they came from specific channels.

So that was the type of stuff that really helped us to optimize certain consumer experiences and consumer journeys.

Duane: So Natalie, you’re in this new role now, what is on the horizon for you? What, what do you want to achieve now in this new role?

Nathalie: What I really want to achieve is, is I want to see how can I take all of the data wealth that Yahoo has and really build the North Star of analytics and research. At the moment, we are reporting on product developments and features that have already been in place.

We’re being, we’re being brought in and we measure. Whenever a decision already [00:52:00] has been made or when something has been implemented. Yes, we’re doing experimentations. We’re providing also user experience research to that topic. But ultimately, the framework that I’m dreaming off is that every decision at every Yahoo is going to be driven by insights generated and provided by my team.

So before someone makes a decision, I want to be part of the, of the scenario planning, making sure, okay, fine. If you, you have your decisions to make, which different scenarios are coming on you, you define then after that decision. How do we implement it? Really being part of that implementation and then also measuring the success and really doing that cohesively for our search business, for our media business, from sports, news, finance, entertainment, all of them, but also bringing that to our mail where we are already heavily working on and also the ads business of really being that integrated insights [00:53:00] partner that everybody says, you know what?

Duane:They need to be with me in a room whenever I make a decision and we’ll drive and we thrive on that. That is kind of a rush. What a rush.

Nathalie: What I really want to achieve is, is I want to see how can I take all of the data wealth that Yahoo has and really build the North Star of analytics and research. At the moment, we are reporting on product developments and features that have already been in place.

We’re being, we’re being brought in and we measure. Whenever a decision already [00:52:00] has been made or when something has been implemented. Yes, we’re doing experimentations. We’re providing also user experience research to that topic. But ultimately, the framework that I’m dreaming off is that every decision at every Yahoo is going to be driven by insights generated and provided by my team.

So before someone makes a decision, I want to be part of the, of the scenario planning, making sure, okay, fine. If you, you have your decisions to make, which different scenarios are coming on you, you define then after that decision. How do we implement it? Really being part of that implementation and then also measuring the success and really doing that cohesively for our search business, for our media business, from sports, news, finance, entertainment, all of them, but also bringing that to our mail where we are already heavily working on and also the ads business of really being that integrated insights [00:53:00] partner that everybody says, you know what? it is super, super exciting. I am now, I think on month number five. So it is, it is very interesting just learning the whole business, understanding what each and every single one of your team members does.

And really. Preparing them for that unified vision and doing that also globally

Duane: globally. Wow. You know, when you’re talking about Yahoo, you know, you’re talking about Indonesia. I mean, you know, you’re talking real global, not, you know, not just a few kind of like key markets, you’re really talking about a product that has like this just massive global reach.

Nathalie: Exactly. Exactly. And it is fascinating just to see [00:54:00] different user behavior.

Let’s talk

Duane: about that for a second, Natalie, because that’s a really interesting, let’s, let’s explore this. You know, what do you find in this new global kind of like role that you’re in? What, what is most challenging for you about looking at data like this on a global scale? Do you see a lot of variability in, in different markets around the world?

Nathalie: Well, certain user behaviors are slightly different for sure, but I think what, what is actually more interesting about the, the beauty and the challenge of working global is something trivial like logistics. It’s, it’s really how do you make sure that, that all of your team members feel that you are, that you are there for them, even when they are 12 hours ahead.

It’s how do you make sure that you’re striking a little bit of a balance there? So there’s really the logistics of leading a global team and making sure that, that you can also make sure that they feel incorporated, that they can participate in certain parts. And then there’s obviously also the [00:55:00] usage behavior and understanding how do you Prioritize to account for your biggest markets, but at the same time, also not to be disrespectful of your, your smaller markets and scale. That’s the balance you’re, you’re constantly trying to strike that balance.

Duane: So Natalie, we close every episode with this one question, which is what advice would you give to the new generation of researchers that are coming into the industry?

Nathalie: For sure, be open, be curious, and don’t be afraid to ask for help and advice. I sometimes observe people pretending that they all have it figured out already.

Nobody has. I haven’t figured all of it out and I’m sure you haven’t figured it all of it out. It is okay to ask questions and to learn and and really be open to advice and and learnings from all [00:56:00] areas. That’s, that’s the only thing that I could really see as, as something valuable to give.

Duane: You know, I think this idea of curiosity keeps coming up.

So many of our legends talk about the need for curiosity. It’s, it’s clear that that’s something that management really craves and their employees these days.

Nathalie: I think because it is the only thing that will make people ask after the first, Oh, why is this happening to go to the next time? Yeah. Okay, why is that happening?

Only then you can really get to those deeper insights. There’s always that there’s an easy insight. that percolates on the on the surface, but only when you’re drilling a little bit deeper, then all of a sudden you’re, you’re really getting, you’re getting to the real gold.

Duane: Well, Natalie, what a career and you know, people who don’t know you personally won’t know this, but you’re such an awesome person. [00:57:00] You know, I love hanging out with you all the time because you’re just so much fun to be with. You know, you, You have a really fresh perspective on, on things. You always bring a new dimension to looking at the challenges that we’re facing as an industry.

So, you know, thank you for the contributions that you’re making to the industry.

Nathalie: Oh, thank you so much. I, I got lucky. I’m in the in the, in the job that I really love and that excites me. So
thanks again, Natalie, and thank you, our audience for joining us today. This is the first episode in our. Third season of Legends of Media Research.

Duane: Wow. Three seasons already. So thanks for continuing to join us for these episodes. And remember to tell your friends and colleagues about us. And if you’d like, stick around at the end of today’s podcast to learn more about media science. So until next time, I’m Media Science CEO. Dr. Dwayne Baron thanking you for joining us today, and I look [00:58:00] forward to joining you again next time on Legends of Media Research.

Female Voice: Almost every major innovation in the TV advertising industry over the course of the past decade was first tested by media science researchers. Whether you’re talking about video ads on mobile phones, or limited interruption ad pods, or program context effects, or brand integrations, or pause ads, or picture in picture ads, or six second ads, or interactive ad formats.

I mean, the list goes on and on. All were first tested by MediaScience. Media science is the leader in media innovation research. So when you’re looking for media or advertising innovation research, collaborate with media science. Learn more at mediascience.com