Assessing creative work isn’t a straightforward or easy job, especially in today’s ever-evolving landscape. Marketers are creating more content in a growing number of ad formats for more platforms, and the pressure to be original is on. Over the past years we’ve seen a growing interest in the power of creative advertising, and more specifically in the way creativity in award winning campaigns drives market impact. An extensive analysis on creative strategies was presented at Cannes Lions with WARC in 2020 to arrive at an “effectiveness code” that would create a standard, universal definition of advertising effectiveness, In his latest book, Noise: A Flaw in Human Judgment, Daniel Kahneman again applies his insights on human errors and bias to decision making, but this time with a focus on organizations. When human beings make decisions (such as awarding winning creative campaigns), we’re influenced by “noise” – factors that cause decisions that should be identical to vary. Think about two doctors who are given the same information about a patient but make two different diagnoses – and that the reason for this is that the doctors made their diagnoses in the morning and afternoon, respectively, or at the beginning and end of the week. “Avoiding noise in judgment is not really something individuals are going to be very good at,” says Kahneman. Especially at an organizational level, where a standardized, unified response is expected – for example with judges and juries, or in standards for advertising effectiveness. That’s why in the long term, there’s great opportunity to leverage AI to make decisions (although Kahneman cautions that this will also create big challenges for people and organizations, and certainly depends on the domains in which we use AI to make decisions). AI feeds on data and can eliminate “noise” much more efficiently than we can to arrive at a decision. As the Cannes Lions and WARC effectiveness code shows, one of the challenges in capturing creativity is being able to identify the right creative components of an ad that lead to effectiveness. Since 2017, MetrixLab has partnered with the Advertiser Jury Foundation (Stichting Adverteerdersjury Nederland, or SAN), based in the Netherlands, to research and understand the elements that create successful advertising campaigns. Their goal is to celebrate the best creative work by awarding excellent campaigns: Every year, advertisers and agencies submit their best campaigns to be judged by a jury of marketing professionals. Now here’s an opportunity to put the theory of AI-driven decision making to the test: Since AI and machine learning thrive on data, given the right input, can an AI-powered creative testing solution distinguish winning ads? Based on our extensive copy testing experience, we’ve developed a list of over 130 dimensions to code ads against within our ACT Instant solution. The solution’s predictive model compares current ads with years of survey-based research across platforms and formats, predicting creative effectiveness based on these benchmarks. Together with SAN, we recently conducted a meta-study where campaigns submitted to compete for a SAN Accent Award were evaluated via ACT Instant. A selection of TV commercials from both winning and non-winning SAN cases were coded by the ACT Instant algorithms. The more interpretative variables, such as humor and emotion or number of messages, were coded by human advertising experts (the split is roughly 70/30, with algorithms coding the majority of variables). Machine learning and adaptive regression modeling develop performance results and predictive scoring of the ads’ performance. We found that, on average, all submitted SAN cases scored significantly better than the benchmark, which is not surprising given that both winning and non-winning ads were judged to be top examples of creative. But more importantly, we saw that SAN award winning ads outperformed the non-winning ads in our AI-powered analysis. This test case not only gave us an opportunity to validate one of our most innovative solutions with in-market performance – it also contributes to the efficacy of a mounting trend to leverage AI to make marketing predictions and decisions. We saw consistent patterns in our predictions compared with the judgement of seasoned SAN jury members. But according to Ron Schneider, SAN Chairman of the Jury, the point is not to use these AI decision making tools to replace the human element: “Human touch – as in coding by humans on top of AI, is essential. Just like the way SAN jury members look beyond data and take into account innovativeness and execution.” This idea of “augmented intelligence”, the combining of human and artificial learning, is a core premise of the ACT Instant solution and something that industry leaders recognize as critical as we navigate the emergence of various AI based solutions. While the journey is still in its early stages, we see the potential for augmented intelligence solutions like ACT Instant to grow further and be a strong compliment to consumer-based approaches. That’s why we continue to invest in further development of ACT Instant in terms of expansion of global availability and ad format coverage, as well as exploration of new dimensions that would further strengthen the solution’s predictive accuracy. In the same way that judging strong creative is not easy, nor is the process of developing technology that can do so. But as Kahneman predicts, organizations are fast approaching a time where they will need to adapt to more augmented or artificial intelligence. So MetrixLab will continue to advance in these ways to meet the needs of our industry, to and help our clients develop effective, and sometimes, hopefully, award winning advertising.
What makes a winning ad?
Validating a vision for the future of creative testing
Creativity vs. AI: The case for identifying winning creative through human and artificial intelligence combined
