AI vs. Human Intuition: Can AI Predict Design Winners?

You know what I noticed is you have a an eye for design. I do have an eye for design. >> can recognize the difference between a camp poster and a dog breeding saloon salon or something. Yeah. Well, we're millennials. We grew up on Microsoft Paint. We know what things should look like. I have ideas. So that's a big focus for us, too, is we care about which designs are better. We have arguments, this design versus that design. And Adobe customers use AB testing to say, "Okay, well, which one is the best? Which one is going to drive more revenue? Which is going to be more exciting? Conversions?" All these things. And we thought, "Well,

what if we could, during the pre-flight process, that means before the test even goes live, we could use AI to detect, all right, well, what is the best option and what are some recommendations for that?" >> Like a precog. Yes, it's a precog, exactly. Okay, no one here watches movies, that's cool. No, there weren't like six dorks cheering for that. Like I was like racking my brain that one Thank you. I let the millennials in the audience know it. Okay. Definitely. So to show us how we do it, we've got >> report joke. Okay. We've got Project

Face Off and we have Doa. He's right there. Hi Eric. Hi Eliza. Hi Doa. So I just flew in from Switzerland and I love it there. You see, I love it so much that on the left you see a web page that I built to promote tourism in Switzerland. However, I feel the design is a bit bland. So on the right you see three alternatives that might be more engaging, but I'm not entirely sure. What do you all think? So Eric, if you had to pick one, which of these three on the right would you go for? Uh, you know, of these three, I'm a dark mode fan. I'm going to go with the one in the middle, the dark one. The one in the middle. That's a good choice. Eliza, I'm going to ask your opinion in a bit, but first let me give you a little bit more context. Okay. I mean, I'm here. Yeah. So when stuck in a situation like

this, as Eric said earlier, we would run an AB test. Split real users between version A and version B, wait for enough traffic, crunch the stats, and find out who's the winner. This works, but it can take weeks or even months. So if you don't have that kind of time, how can we build our intuition? I'd like to introduce you to Project

Face Off, an AI-powered tool that simulates an AB test in seconds and even explains which variant wins and why. Let me show you. I'll create a new test. Switzerland campaign. You spelled that so quickly. so quickly. >> [laughter] >> [laughter] >> You know, I studied computer science, so Um Fair. >> [laughter] >> [laughter] >> As my control, I'm going to choose the original version. And And as my challenger, I'm going to choose the one that Eric has picked. He picked the incognito porn one. He liked the dark mode. We all know what that means. that means. I thought it looked good. Open closed tabs. Okay. Oh, well. So let's for our primary conversion

goal, let's say bookings for Switzerland trips. All right. So All right. So are you ready? are you ready? Let's go. Let's go. >> Let's go. >> Let's go. Ooh, look at that. So under the hood,

Project Face Off generates user personas that simulate interaction with both designs, like scrolling, clicking, thinking, and they decide whether they would convert. We repeat this until we hit statistical significance. You see, the test is currently running and we're almost done. So Oh, no. Oh, well, it looks like the

control has received more votes, so that's the winner. Sorry, Eric, you didn't Sorry, I thought it was like the cookbook one. But don't you worry because if you

scroll down over here, the AI insights tell us that the control won because the challenger's dark theme hurt readability, while the control's clean itinerary made it easy for users to understand and act. You see, this is great for pre-flight screening. We can now quickly eliminate weaker options before committing to expensive live tests. And we're not done yet, so if you scroll down over here, you see the system has generated a nice PDF report that summarizes all of these insights and it contains even more details in case you're interested. So Eric, do you want me to send this to you by email? Yeah, at Adobe we love PDFs, so please ship it on over. Sounds good. >> But who converts the document to the PDF? How do you outsource that? We do. >> kidding. Okay. All righty. So All righty. So um go Well, before we move on to Eliza, Eric, any final wishes or feedback? So

here's what I've been thinking about and we we talked about this a little bit is that I like comparing AB tests, but what if I have more designs? You showed what, three other options before. Uh-huh. Can you like whip up a feature that adds gives me the chance to add multiple Other letters. We want other letters besides A and B. Multiple. Okay. Well,

you know what I can do? I have You know, we now have Cloud Code that can help us quickly prototype new visualizations for the interface. Do you want me to try that because I actually had it over here, Cloud. So we could prompt something like, "Allow me to upload multiple challengers." Call it tournament mode. Okay, we'll get real real cool with it. Okay. And with the theme Run them all against See? the same Where's your computer now? I study computers for a living, not a spelling bee when I see >> tournament mode, did I hear that right? Yep. >> tournament mode You're the Swiss genius with spelling. >> Okay. >> Okay. So while Claude is working on that,

let's go back to our variants. Eliza, now it's your turn. Okay. So I want to tell you about another great benefit of this tool, which is it allows us to test scenarios that are very difficult to test in real life, like new market probes or highly specific segments. So for the highly specific segment of elder millennials, which I know you're fairly familiar with, elder millennials elder millennials >> these three designs on the right do you think might resonate the most? For my group? Yes. I like the middle one, but I think it's going to be the lower one because it looks like like an early internet blog post, like Slate Atlantic article. Like I trust it. >> [laughter] >> Okay. Which one? The bottom one. >> The bottom one, the heart of Europe? >> Okay, I That was one That one was my favorite, too, so let's go ahead and start a new test and

pull the changes from Claude. Hopefully we have Ooh, you see over here, tournament mode available. That's exciting. Okay. We're going to go ahead and select these three challengers all at once. Okay, that's a good sign. >> everyone in here gets free drinks. >> [cheering] >> All right, I hope you're right. Me, too. Yeah, and if I'm wrong, you still get the free drinks. So win-win. It's okay. So last but not least, let's select from Adobe Experience Platform our target audience, which is elder millennials. >> Don't let me down. Let's go. >> [laughter] >> So ready for the tournament mode? Yeah. Okay, let's go. Not the porn one. >> Wow, look at that visualization. It's like we're watching a real game, right? Are you feeling the heat, everybody? You guys feeling that heat? >> Nice. >> Well, it's getting certainly hot for me, so I need to take this off. And let's take a look. So the first one, it looks like it's going to be inconclusive. >> The second one, it looks like the control is going to win. And the last one, you were right, Eliza. I know my people. Congratulations, you won. Thank you. So now that we know which one is the

best, most promising challenger, let's go ahead and ask AEM agents to use AI to build a functional web page from this mock. And once done, we're going to launch this live AB test on Adobe Target. And just like that, we're good to go. There you have it, Project Face Off, built to help you That's amazing. >> [applause] >> Awesome. Thank you, Doa. >> you move forward with insights and

confidence. If you'd like to use this technology in your own work, please be sure to vote for us. Thank you, everyone. Aw, I love that. We could say that it's predictive analytics in an unpredictable world. That Wow, that sounds like a good tagline. I see our CMO over here. She's ready to take it. >> Take it. I'll take 10%. Fine. That's the discount that we had for your coming show, too. It's a win-win. I felt that one.