משבר אמון ב-AI? הגדילו אוטומציה בלי לאבד לקוחות!

What an absolute pleasure to be here uh and your host for today. Um we've got a fantastic uh panel here. Um but it's an interesting time. Yam kind of asked for a raise of hands this morning and it was great to see how many hands went up when we were talking around who's adopting AI. We're seeing this accelerate in A&Z now twice as fast as we even expected. Um, but we still, as Cat was mentioning before, we still have this this trust gap right now. Um, and we're trying to overcome that. I speak to marketers, sales leaders, business owners all of the time. In fact, even this morning, I've been talking to them about it. And that the one question they keep asking me is, "How do we scale AI responsibly?" Um, and I'm definitely not the right person to answer that, but I have two wonderful people going to be able to help me do that. First of all, we have Erica who's an expert in this space, our chief legal officer who's flown in from San Francisco. Welcome, Erica. Thank you for being with us. Um, and also Matt Fam as well, who's head of product at Mortgage Choice, uh, works in a financial services industry, a heavy regulated industry, you know, managing AI and change is never more prominent in your world. So, thank you. Warm welcome to Erica and Matt, please. Erica, I might start with you. So, I mentioned the trust gap before. Um, from your perspective, why is trust the the limiting factor for AI adoption right now in Australia? Well, for one, I think

just acknowledging Australia sits in a really interesting sort of continuum, I think, between um in the US where I've flown in from where uh frankly everything is more faster, more AI. Um and we are navigating our own challenges of how do we regulate that and can we even come to the table and regulate um and then you have Europe on the other end of the spectrum. Um which Europe, you know, has come out of the gate with a lot of rules, regulations, a big enforcement regime. um Australia has taken more of a principles approach um which I think is a great way to think about it which is like what are the principles that we should all sort of sign up to um and self-regulate around um and how should we think about that and so um I think it's a very practical approach um it's a country that has you know very traditional um uh sectors that have really um you know sensitive customer data and um important confidential customer data. Matt's going to talk a little bit about um his uh sector that has that as well. Um so I think that's like a good grounding to start with. And so you're already dealing with a region where yes, you're all getting pressure to transform with AI and to do that faster. And you're also being asked to do that in a way that doesn't break anything and doesn't you know trip the wires on anything and doesn't you know sort of disrupt um a lot of the the trust that has been built through the types of businesses that um you know are really shaped the landscape here. that is hard a hard hardearned brand trust built over years and you know can disappear in a moment. So um we'll get into more of the details about how to you know like cross that bridge in between um but I think it's something that sits on a lot of our customer shoulders of go faster make AI work for us uh gain efficiencies oh but also you're also responsible for making sure it doesn't break anything and damage the brand that we've already built. Matt, if

you don't mind, just while we talk about the trust gap, as I mentioned before, you work in a highly regulated space, financial services. Has there been any hesitation um from from your your inside your organization or even your customers? >> Yeah, absolutely. Uh particularly with our our brokers um starting to adopt AI uh features within our offering. I think the questions I asking were fairly uh I think natural questions to ask and and probably ones that everyone in this room has also asked themselves as well which is you know is AI going to take over my job that's always the first one that's a whole other topic to talk about but I think the next question that a lot of our brokers are asking around our AI products we're around how can I actually trust them um and as you mentioned financial services is a highly regulated industry so the information we share the advice we give has to be um explainable and our systems need to be able to be auditable as well so that we can prove out, you know, what we're saying is actually accurate. So AI hallucinating is not an excuse for us to get it wrong. At the end of the day, if something does go wrong, it's actually our people, our brokers that are responsible to actually make sure that they're being compliant. So one of the really big questions that were asked was actually around how do I trust these these experiences that you're building? Um because particularly for some of our brokers that have been around for some time. We've got brokers that have been in the business for over 20 years. They're not digitally native people and as a result they're like this AI experience is great. I put in a prompt and this answer comes out. It looks amazing but how do I trust that it's actually accurate? And unlike I guess previous products we've built which are a little bit more static. Think about a calculator 2 + 2= 4. That's very easy to explain. whereas our AI tools are constantly evolving. And so being able to explain the mechanics of how that works and an education component um to be able to go with that has really shifted the way that we go to market with our products and how we educate our brokers with these tools which has absolutely helped with adoption and trust within our systems. >> Yeah, that's interesting. So they're really worried about the accuracy of it as well, >> which is interest like when we the

number funny enough the number one u thing that our customers are worried about in Australia is actually where the data sits, who who controls it, who owns it, what are you actually doing with it right now? And and Erica, for you, what are customers actually asking you about like data and AI right now? And where do you see the biggest misunderstandings around risk versus the actual reality? >> Yeah. Um, and I'll be interested to hear how this resonates with you too, Matt. But I think we spend a lot of time right now um talking about training and um what is an AI model or what is AI technology actually allowed to do with the data that you're giving access to it. Um and so training, you know, if I rewind even a year ago, like we've progressed the conversation because if I rewind a year ago, um the idea of training or the word the word training from a data perspective was just like a non-starter. Um, and I think everyone was still still trying to understand what do these models actually do? How do they relate to the technology that's being built, etc. Now we have a much more nuanced conversation around training and and to be totally candid as well on the legal side of things as we negotiate some of the commercial arrangements, whether it's with our LLM providers that are under the hood of some of our technology, we've seen the language around this even settle a little bit. when we first started having these conversations with, you know, we use Open AI to power some of our technology, we use um anthropic to power some of our technology, when we were talking with their teams and saying, okay, here's what we want to be able to articulate in the contract, there just weren't settled commercial terms around these things. So now we're we're graduating to where there's like more common understanding. And when we talk about training, we really talk about training from the perspective of what would the customer expect the value out of this product, you know, being delivered. What would they expect from it? have we explained it to them in a very clear way and then do they have a choice around that? Um and so I'll give you just a quick example from HubSpot's perspective and how we build products. We use our customers data when they put it into different products to train outcomes that are unique and personalized to them. So, for example, uh you're putting in your data and we're using that to get better and better at understanding your tone or your brand of voice so that we can create content or automatically generated messages that really mirror you. And you feel this, right? Like you use a product and you're like, "Okay, it's it's getting me now. That feels, you know, more and more." Whereas in the beginning, you're like, "Nope, rewrite it. Nope, that's not good." Um, so that's one area that we use data for training. Another area that we use data for training is when we take data and we look at it on a very aggregated and wide scale to understand trends that might be helpful to our customers. So for example, um we know from our vast, you know, 250,000 plus customer data set that these types of deals typically take this amount of time to close and by the way you're behind and we think here are some actions to take. So that's a type of training that we do. We explain that very clearly to our customers and we also offer the ability for customers to opt out of that training. Um, and then last but not least, when we bring any thirdparty models to the game and there's great models out there that are powering the next innovation curve of technology, we do not let them train on our customers data. Um, and so that's something that we have a hard stop on and that gives our customers the confidence that whatever's happening with their data is really in service of the product that they are using from us and the value exchange that they're getting. and any technology that we're bringing to power that along the way doesn't get sort of backdoor access to that as well. >> A lot of that really resonates with um I

guess our our layers of governance and and how we approach these conversations as well from a from a group office perspective when we're selecting third party tools. We're having those exact conversations, you know, understanding where our data sits, what it's been used for, and and making sure that that's compliant particularly with Australian law as well. And that's you know an aspect to navigate particularly with international businesses like HubSpot as well. Um from a broker perspective um things that we had to kind of think about you know in in terms of um the these these uh safety measures that we had to put into place similar similar themes as what you kind of talked about. Um, one of the first things that I'll probably just start off with is that everything that we release um to our network has to go through a really rigorous due diligence process when we're selecting vendors, particularly third party. So before anything actually goes live, there's already been a fairly extensive process to make sure that we're asking those very questions that that you've mentioned. But in terms of safely deploying it to our our network um there's another layer that we have to also start thinking about and I think you touched upon this a little bit um with the importance of using enterprise technology as well. Um and what's been really good about today is actually hearing thematically with a few of the other talks about the importance of using your data and plugging your data into their systems. And so for us as the conversations around AI tools started to emerge and some of our more digitally savvy brokers were like why aren't we using this tool or that tool it was really important for us to think about the enterprise technology that we're using and we're a Google Workspace business. So, Google Gemini made sense for us and actually making sure that um our instance of of Gemini had our data plugged into it so that uh the results were a lot more predictable um than using things off the shelf that's just connected to the internet. And it's really interesting you talk about um I guess the the idea of training as well. Um, often when we release new products to our network, we'll actually do it in pilot groups, so smaller groups and that's more of a a risk issue. In case something goes wrong, we can roll back, make modifications, and deploy again. But in the instance of AI, um, these pilot groups actually help us train the models as well. So there is a dual benefit of being able to actually do that pilot group so that we've got better confidence on the results as we start to spread out to to wider consumption within our network. >> Awesome. I I love what you just described and I think it really brings things to life. When I think about AI,

everybody's having a personal experience of AI inside of work, outside of work. And so, you know, the surround sound of, hey, I my my buddy's using this cool tool or this other company, you know, the IT departments and people in in Matt's role are just getting inundated. And when you think about being able to move fast but with precision. And I think about this in the context of where do where in life do we move fast but with precision. And I think about a racetrack or I think about you know launching a rocket into space. And what are the common patterns that you see? You see something moving fast but you also see a very tight collection of team members that are responsible for making sure everything runs well and they're involved early and you see them you know as a pit crew etc. And that's I think exactly what you're describing which is if you're waiting to kind of a waterfall approach almost to get a piece of technology into your company and you sort of like are are sending it down the assembly line and legal or security or somebody else is at the end. You're almost guaranteeing that you're going to do this in the longest and most painful way possible. Whereas if you you bring that collection of individuals to the start and you start with and this is what I love what you describe start with the value case. What do you expect this technology to do for your business? What is your hypothesis of the value return? And you all probably heard Yamine talk about this of what is the business problem and that everybody's anchored in that. I think you'll see everybody shift to a more business forward posture including the lawyers because they now have context of everything to say, okay, this is what we're shooting for. How do we design a process that allows us to prove that out in a safe way? And then you have everybody collaborating in a mindset of, okay, what kind of data is there? Can we minimize the amount of data that is really high risk until we trust this product more? We get a few rounds on things. But it also keeps the business from just sort of spraying technology into the space which is yes a risk from a security perspective. You expand attack vectors whenever you introduce new modes of technology into a space. But it's also a practical issue. It's a does everybody in the company understand what tools I'm supposed to use for this process? Do I have control of the cost around it? Have I put vendors into my stack now that might not be here in six months and now I have a business continuity issue. So this is a really 360 conversation to have and the more you get that sort of tiger team of the business stakeholder lawyers IT security in the conversation together and and they treat each other as the team that is here to solve speed and precision the more I think you'll get great outcomes. >> I love that. That's it's so true. If you're you're one team, you got to bring people together. they go on the journey together and I think what I'm seeing when I'm talking to businesses at the moment is you've got business leaders not operating with other parts of their flywheel um and they go as right the way through the process and then they go okay we're going to engage legal now and then legal raise all of the concerns so it's great legal are no longer the no it's a part of the team I like that a lot Erica um Matt just on that then so where has embedding legal helped you move faster um or avoid any mistakes at mortgage choice >> mortgage choice as a part of REA Group, which is our parent company. Um, we're really lucky to have an in-house team of um of legal experts um that are well tenured as well. They've been in the business for some time. There's a habit where um we we get lawyers coming into the business and they they tend to stay for quite a long time and as a result, they're they're really embedded in our actual business functions. So, to your point earlier on, um they're not treated as a a separate office where we go ask for permission or forgiveness. they're actually very much a part of um our processes and we really bring them early on in the conversations so that we can really frame and understand the things that we have to be careful about. Um, and to me that that's that's probably more important to be able to have those conversations early cuz as a a product and tech person, one of the the worst things that um has to happen on occasions, thankfully not too often, is actually releasing a product and then having to scale it back because you found a major flaw from a compliance perspective within financial services. Um, it doesn't matter how small it is. um we can't go live with those things or we have to roll back and fix and we have to you know um alert the network that it's happened. Um and we've all seen the impacts of you know things like data breaches in other businesses um and the the trust impact that that actually you know creates. Um having those conversations earlier on is is super important to be able to avoid those very things. >> I mean mortgage is a large organization. We have lots of large businesses in the

room. Erica, though, we have a lot of smaller businesses as well that don't have access to in-house councils and everything. Um, what's the advice you could give for those small businesses as they like navigate through this change? >> Yeah, so a a couple things I would say and and the first maybe I'll just like tick off. We kind of use a framework internally that's a super simplified framework of five questions that I want our teams constantly thinking about when they're thinking about bringing new technology into a space. Um, and it's simplified because it's meant to be able to like give to anybody in the business. Um, because sometimes we have tools that our teams want to use and there's a salesperson or somebody else that has to be the advocate for that tool. So, they may not be a security professional by trade or a lawyer, but they need to be able to interface with the vendor and say, "Hey, this is what my team is going to expect for you to be able to answer. So, can you give me some basic answers around it?" Um, we touched on one of them already, which is, you know, do you use my data to train AI? And they should be able to answer that question really clearly and specifically the same way that I did. Um, the second is, you know, is is my data secure? And that was true before AI. It matters now even after AI. Can they articulate to you how they secure data? And what this typically looks like is usually on a website they will have some area that is a trust center or documentation where they have some explanation of not only how do they secure their products but what standard because there's you know sort of globally accepted standards. What standard do they use to abide by that? Um you can look for words like ISO 2701 sock 2. Those are good indicators that not only is the company engaged in those practices, but they have an auditor that they bring in to reertify them on a rolling basis and say, "Yep, this these this company is abiding by that standard." Um, so that's that's around security. Um, the third is who can access my data. Um, and this is as much as you know who if it matters to you as a small and mediumsized business or not, it's around where is the data. So we talked a little bit about some of our more sophisticated customers really wanting that data to stay in the Australian data center. So you should have some uh understanding of where does the data go and this should be on their website and in documentation. Um but also um who in the company actually has access to the data. So is it just the product teams? Do the support teams get access? You know can they articulate the answer to that question? Um the fourth uh area is are you going to help me comply with laws and regulations? And the easy one to look for here is if they don't have a good answer to do you comply with GDPR run. So that's like an easy test case where you know the EU sets a pretty high bar. GDPR actually has quite a few regulations around how to handle data and they should have some really good explanation of here's how we do it and here's our subprocessor agreements etc. Um, last but not least is uh do you delete my data? So when I ask for specific data to be deleted or when I actually leave, what is your process for that? And what we're typically looking for there is some reasonable explanation of how do they actually delete data? Can they articulate the mechanism that they do that words like uh cryptographic hashing or um rolling data stores? And then you know they may have something in backups but there should be some period of time that they can say at this point in time we've we've deleted all your data. Those are good indicators of good hygiene practices and those are things that we even give that talk track to our um business associates and and people in the company that are trying to run ahead of the backlog of low legal and security and whoever else that it's like if you can get reasonable answers to these questions just from their website that's a good indicator that you're dealing with a mature company. Um, the two other tips that I would give is, um, one, if you're looking at a license, um, and there's tiers, pick the enterprise tier. And the reason that I say that is because it's going to give you the best protections in ter and if you're a business running, um, your business off of specific technology stack, you do not want that tool using your data in a way that you would not be comfortable talking to your customers about. And the enterprise tier tier even though it's more expensive is almost going to have all all the time have the sufficient protections. The reason for that is because AI tools are powered by data. And so if you're talking about a lower tier, you're talking about a pay-to-play model where you're getting a discount but you're paying in the form of giving rights to data that you probably don't want to. Um, last but not least is to think about whether or not there's a delineation between um high-risk data or really anything else. Um, and an easy search for this is actually the EU has a great website where you can go on as a small and medium-sized business. Enter the data um that you're considering dealing with or in a specific vendor and it will sort of give you a decision tree of are you dealing with high-risisk data or are you dealing with lower risk data in which case you don't have to think about as many parameters overall. So, those are just some good tips and tricks as you're kicking the tires on things. And I think if you use those like 90% of the time, even in the absence of a big compliance team or lawyers, you're going to come out on the right side of things. Uh I don't know how you think about my cheat sheet, Matt, but uh you can use it for your teams if you'd like. >> I I think you basically described our process essentially. Um and and I work, you know, in the product side, so on the opposite side of Erica in terms of talking about what we want to do and have these collaborative conversations with our legal teams. And I think what what's really good about thinking about this up front and setting up these processes early uh it makes it so much easier. Um so Eric mentioned you literally have a checklist. You know we have these these documents that asks every vendor that we work with these same questions and often the answers for those questions will lead into what we have to do next as well. Um, if you asked me earlier in my career how I feel about governance and processes and and these types of things, I probably wouldn't have been that excited by it if I'm being honest. But making it easy um by doing it really often um and and building that muscle um just really enables you to move at speed afterwards. So standardizing this process and and you know I mentioned that before not everyone has the luxury of having an in-house legal team. Um, but actually having these processes in place still makes sense and and you know are sensible for any business to think about asking the right questions so that you don't find yourself in a bit of a sticky situation later on and have to kind of rectify. Wonderful. Well, that was a big checklist. So, but don't worry, there's an AI transcript coming out. So, if you didn't catch all of those amazing tips and checklist, you'll be able to get that. Um, basically, Erica, what do regulators

actually care about versus customers right now? Is it is there a difference? and and for you what's the non-negotiables every company needs in place before deploying AI? >> It's a really interesting question because you know oftent times regulators are stepping into the shoes of the consumer. Um so regulators are really thinking about where is there an asymmetry of power and where do I need to step in and protect an end user or consumer of technology that may not have the same bargaining position as a business for example. Um, in our case with HubSpot, our customers are businesses. Um, and so we're often, you know, thinking about, okay, how what obligations do they have? How would they have this conversation with their customer? Um, and honestly, like a good line in the sand, um, that I use frequently when I'm talking with our product development teams and um, and even our chief product officer, uh, I, you know, we'll say, okay, would like this is what we want to build understand. Okay, let's let's think through it through the the lens of the customer and maybe we'll make some tweaks. And then we'll also take a step back and go, okay, would we all feel comfortable sitting across the table from a customer and explaining that this is how we use their data? And that's usually just a really good like brings it home, you know, human level question because if you can articulate that and stand behind it from, you know, a relationship standpoint, that's probably a good litmus test that you're you're on the right track. Um but generally the customers are looking to us to really think about their stewardship obligations and their end users and would they feel comfortable explaining to their customers or prospects that we're using that data in that way in their tool. Um so I think the the interests are are super aligned. Regulators on the other hand when we interact with them they're really looking for us to be thoughtful about the end user. Um, so they care about you all as customers, but they ultimately care about collectively as a customer, as a business using our tools and as a provider providing business tools, are we collectively thinking about the individual end user or consumer in a way that's responsible? And so they're looking for the thoughtfulness of that process. They're looking for exactly what Matt described, which is like, do you have a process where you involve all of the stakeholders upstream at the point of design? Um, and then what are the checklists or you know some of the controls that you've put in place. Doesn't mean that everything goes right 100% of the time. Um, and believe it or not, regulators deal with situations that don't go right 100% of the time, but they're really looking that you are nimble and that you're able to apply a thoughtful framework and then adapt that framework where you see vulnerabilities exist. >> So, I'm hearing governance equals speed and scale. There's definitely no constraint there. So, governance all the way. Now I do want to move though to the fact that at the same time we've got like a decision paralysis. There are so many tools, unclear policies and rising risk, right Matt? No, no doubt in your industry, how do you decide what tools to allow through and what to block? And what does your hard know while assessing approach looks like in practice? Look,

in in terms of tools that we uh release to our network of brokers, um we've got a very firm policy on um what tools are allowed and and what aren't. And because the the risks involved particularly off really accessible AI that looks great and Erica touched upon businesses that are startups that may fold in the future or might not have the same I guess security um processes in place. It it's too dangerous for us to allow um our network of brokers to use. So we we've got a very strict policy in terms of um using software and and platforms that we have heavily vetted um tested gone through pilot groups trained and also one of the things that I think uh we haven't touched upon is the continual governance of these platforms as well. Unlike static products where you can build and and walk away from and until they break you don't have to worry about them as much. um AI products are constantly evolving and constantly learning which I think from a legal perspective would be a massive headache to kind of stay on top of. Um so we have to put processes in place to also look at governing the results of our tools as well to make sure that over time as our AI products evolve they don't start to go into weird directions. So that's a really important part of the process. Um as well in terms of experimentation though um in in head office where we're looking at new technologies all the time there's a real shift in culture I feel um and mortgage choice is a part of the wider business called RA group um and RA owns a business called real estate.com.au Um, which you may have come across. And even though it's a wide and large business, we're starting to see a bit more of a startup vibe with experimentation with with new products. And there's a real encouragement from leadership for our teams to actually think about what is out there um and test and learn and experiment in controlled environments. So there's a there's a real balance between um releasing something to our network to make sure it's safe. Um but then also keeping I guess a finger on the pulse of what's coming up and you know we have processes that are actually put into place now to think about how those start to filter up to go through you know a a due diligence process as well. >> I I would I would just add to that. I think it is a tension point because on the one hand it's like you transform the organization but you're also expected to put tools in the hands of people so they can they can transform themselves. there's that call to action as well. And by the way, you got to keep all this under control at the same time. Um, and so I think what you described of having a process where you know you can sort of get things into experimentation mode and then have a clear understanding of when does it graduate to fully in production. But I would also add if you have that culture, which I think everybody should right now of experimentation and growth, you've also got to garden that process. So you know at what point is somebody going back and going oh those 10 cool to like you know tools that we sort of green lit on this pilot basis is anybody going back and saying is anyone still using these or should we clean them up and you know remove deprecate access to our systems. So you know any time that we're going through transformation experimentation there's got to be a hygiene to it where it's a living process and somebody's going back and editing at any given point. Okay, these are the things that they were fun to tinker with for, you know, 30 days or so, but like let's actually get them out of the environment now because no one's using them and they're just a, you know, point of vulnerability for us. >> Absolutely. Like we're we're not looking at like tech spiral and tech debt growing with these things. There's a process in place even just for that to kind of make sure that to your point like there's not any any shadow, you know, AI being used in the business. um and and and being able to kind of come back and ensure that you know we are staying compliant internally as well because it's just as important in terms of um regulating and make sure that's been governed as well. Yeah. >> Interesting. Well, that's obviously where governance can break down. Um what what risks are companies underestimating, Erica? Especially with things like vibe coding or or unapproved tools. >> I think the vibe coding one is is a big area where um it's you know it's in vogue right now. We want people working with systems and being able to understand how technology can can overlay on them. Um, but also there needs to be a point of view from your organization of like what is the tier 0 or tier one data that your company operates off of day in and day out and is one it's it needs to be accurate at all times. It needs to be governed by permissions. You need to be able to have a third party come in and look at it and validate that you've done all those things correctly. And if you don't do that, it actually damages your relationship with your customers. So I think those are the surface areas where there needs to be a point of view of like, okay, this stuff is not the area that we want to vibe code in. You know, if you have a dashboard of insights that you build for yourself based off of your team's goals, etc., like have at it. Um, but if you have an area where there is customer data, regulated data, confidential information, like that is not an area where I want our teams vibe coding and you know, sort of having at it in a different way. nor would our our customers expect that we're we're doing that as well. Um so I think having a really clear point of view of like where is that delineation sort of you know crossed. Um I think you know the the second area is just making sure that you're constantly checking in on the security um uh pro progress on specific tools. So if you have a security team that means doing regular testing um because the vulnerabilities are changing every day and so you know you don't want to sort of set it and forget it like you want a hygiene process where you're going back and evaluating that tool against a new environment um making sure that you're doing the same level of testing because we're learning each day um how these tools tools are made vulnerable. So I think just just more um fodder for it needs to be an evergreen process that it's a living process um and it doesn't just get put on an assembly line and then left and forget. >> It sounds like many of these issues like

data governance, privacy, they're not new, right? So Erica, what is AI forcing companies to confront that they've been able to ignore for years? >> Oh, I think and and uh Matt can check me on this because he's probably living it. I think the AI is having companies confront their data architecture more than ever. It's so good. Right on Q. We didn't rehearse that. There's violent nodding as soon as I said it. Um but we all know, you know, sort of the the um brittleleness of of constrained data systems. That was true before AI. Like if you tried to join data, you tried to pull insights, it was, you know, the data was never current, it was never right. The reality is AI is only as good as the data that feeds it. And so I think this is really bringing to bear a reckoning moment where companies are like right no matter how cool the tool is, no matter how much it promises, you know, to eliminate, you know, all friction and drive all efficiency. If you don't have the data structures underlying to be able to drop that tool in, then you're not going to be able to power it and the outcomes that come out of it are not going to be trustworthy or even if you can identify this is wrong, you don't know how to fix it and make it right. So I do think this is an area where um the best companies will really invest in their data architecture and their data governance and make sure everything's in the places it needs to be. There's the right classification and there's the right permissions and security on top of that. Then you can drop stuff onto that and let it move pretty quickly because it's got the right rails to operate off of. >> Yeah, I'm getting a bit of PTSD just thinking about this process. Um yeah, RA group acquired Mortgage Choice around 4 and a half years ago. And Mortgage Choice is a 34 year old business that is um you know built up from no less than four different um brokering businesses that it's acquired over time as well. So you can imagine over 34 years the attitude and and standards towards how we actually govern data has shifted significantly. Um and the legislation around responsible use of data has also evolved constantly over that period of time. So one of our initiations with mortgage choice becoming part of REA group was around trying to get a handle of our data um for for just standard business operating reasons. So even before AI was a a huge I guess thing that everyone's focusing on and all-encompassing we were already working through those challenges um just to be able to run the business as efficiently as we wanted to actually run that business. So to come back to your question like has it exposed anything? nothing that we didn't know about, but it's absolutely highlighted uh the importance of of building those foundations in place. Um and investing in in that work because that that type of work is often grueling and grind grinding and sometimes hard to really prove out business value as well. But to be able to enable these AI features off that strong data foundation um is something that you can then start to really prove out the value of of that investment in that work. So nothing new but definitely exposed and one of the good things is with AI it's actually helped us to get a hold of our data as well. So you know conversations that we were having originally around how do we start to consolidate all this information coming from all these different platforms and systems. uh the the estimations were huge in terms of thinking about this from a a traditional I guess delivery process and you know how we approach those problems but AI has helped us actually really speed that up and and things that we estimated originally might take a few years are taking months now which is a a real huge kind of you know benefit for us as a business. >> That's great. There's there's no issue of exposing a challenge then you can solve it and it's better to know about it than not know about it and get called out. So I I agree. Did you want to stay on there? reminds me as you were talking through it reminds me a little bit and I love that like the you you know this is this is where it gets really meta where you can use AI to sort of restructure data or like get the taxonomy correct and it is a supercharge but it still remains true you know it strikes me you have this AI transformation happening now I remember when the cloud transformation was happening and there was the the promise of cloud and you know digitizing your your stacks and the hidden tax was migration and so you know this is this is sort the same where like the promise of the technology sits over here but there is a bridge underneath that you have to sort of traverse to unlock the value of the technology. So for onrem that was like migration to you know you didn't just turn on your cloud instance and it just worked. you had to like get everything over there. And the same thing is true with AI where you know you have this promise of turning on AI you have to do the lift of structuring your data getting it into a clean state you know making sure that it work that your data works for you and I think that's the piece that people are coming um to realize and I'm not sure how many of you have experimented um with AI but I've certainly had um scenarios even within the legal team as we try to unlock the power of AI and you know the the demo's great and then you go to actually do the thing and you realize like ah we don't have that integration or the integration's broken or you know the data is not working. And so that's where the vendors you know that can really um house a lot of your data and you know have it in a secure environment but also know dynamically when to serve it up and when to expose it. And you know HubSpot's one of those platforms where we we endeavor to have a ton of context and data in one place at any given time. It can really spare a lot of the like hidden tax of unleashing the potential of AI. >> So true. I I could talk about this all day but we might have to move on. Let's make this actionable for the leaders in the room here as well. Um Matt, for you,

what's one thing a leader should do in the next 30 days uh if they want to scale AI responsibly? >> Oh, does it have to be one? Can it be two or a couple? >> You can have two. Okay, I can have two. Um I I think the first thing is is not trying to jump towards the solution first. Um think about the problem you're trying to actually solve. uh if your if your social media algorithms are anything like mine, when you're jumping online, there's always someone screaming about what you're doing today is is old and redundant and you need to do the new thing or you're going to be left behind. Um so the first thing is really thinking about the problem that you're trying to solve and anchoring to that before you think about the tools that you want to build on because the reality is is that um you know technology moves really really fast but but trust compounds really slowly. So the businesses that will win will not be the ones that are using the latest and greatest technology. It'll be the ones that are actually maintaining and building the trust um from from their customers over that time. And this is why I think the the governance conversation of thinking about that that early risk and making sure that you are building foundations of trust particularly in an environment where it's it's so dynamic and trust can be fleeting. So really really protecting that I think is an important thing that every business leader you know should be thinking about in this environment. >> And Erica on the flip side what what's the mistakes that you you see businesses make right now that you the leaders here should try to avoid? >> I mean I think the biggest one is if you

are in a company where you have the benefit of security compliance legal etc resources. If you don't know who those people are and you haven't sat down and had a conversation about, okay, this is crazy right now, the pace of things, you know, like if you haven't had that that moment where you've all looked at each other and go, is anybody feeling super stressed about the number of tools that you're trying to put into the system all at once and lock arms together and say, "Here's what we're going to commit to. bringing each other in early. We're going to commit to having regular conversations around this where we re review the tool stack. we're going to commit to having a shared understanding that we communicate with the rest of the org about what are the, you know, non-negotiables versus where are the areas that we can fasttrack things more. Um, if you're not doing that or haven't already done that, I think you haven't really set up the engine for transforming yourselves with AI and being able to move with AI speed. And so I would encourage you to do that. >> Wonderful. Good advice. Thank you. Well, we're getting towards the end of the session now. So it's really clear that trust is what determines whether AI stays an experiment or becomes a scalable growth driver. I think that's very clear after this conversation. So my key takeaways here were seek clarity for why you're using them and and what outcome you're trying to achieve. Bring stakeholders in early and work together as a team. I think that was evident that makes a massive impact. I had a conversation with someone today where they didn't do that earlier and they they regret that. Um and define who and the what and set the guardrails really early. So, with that, I just want to say thank you so much to Erica and Matt for sharing your insight today. Um, we actually have a few minutes left. So, we're actually going to open up for a Q&A. We've got time for one question, maybe two, if there's any brave people in the audience. Would anyone like to ask a question? Here we go. There they all are. We have a roaming mic, I believe. >> Oh, there you go. In the front row here, just please. Thank you. When the company doesn't have an internal legal team, who would you recommend uh as the key stakeholders or how do you recommend to go about it? >> How how big is your company? >> 150 and growing pretty fast. >> Okay. So, you probably have like a head of IT C. Yeah. I I think and and feel free to chime in. I think that tends to be a great person to put the mantle of responsibility on. um but also empower them too right to ask for the right resources and whatever else they need to do to scale things but I think that's a natural place where that person is going to be the bottleneck for can I integrate this can I drop this tool into the environment so I think if if you don't have a legal or security you know team that's the natural place to start and that's always a stakeholder that's like at the center of our conversations even in a larger company >> yeah I'd agree even in a larger company um the stakeholders that I engage with with these conversations are risk, security and technology. Um so often our architects are people that will help frame these things that that are really important to consider. Um risk is more about I guess making sure that we are following that process um ensuring that happen. So that that can be you know someone else as well. Um but I think legal is is definitely your friend. Um, you know, it's in these situations and in particularly in this environment as it's navigating so quickly, being able to have those guardrails in place, um, to think about the the longer term of your business, not just the quick wins and the dopamine hits of like these cool new features that are coming out like tomorrow. That's all we've got time for. Um, I hope you enjoyed the session. Um, can I just ask for a big round of applause for Erica and Matt, please?