In this Chats on the Road to Black Hat USA, hosts Sean and Marco discuss the use of AI in hacking and cybersecurity with guest Frederick Heiding, exploring the potential dangers and ethical considerations in this evolving landscape.
Guest: Fredrik Heiding, Research Fellow at Harvard University [@Harvard]
On Linkedin | https://www.linkedin.com/in/fheiding/
____________________________
Sean Martin, Co-Founder at ITSPmagazine [@ITSPmagazine] and Host of Redefining CyberSecurity Podcast [@RedefiningCyber]
On ITSPmagazine | https://www.itspmagazine.com/itspmagazine-podcast-radio-hosts/sean-martin
Marco Ciappelli, Co-Founder at ITSPmagazine [@ITSPmagazine] and Host of Redefining Society Podcast
On ITSPmagazine | https://www.itspmagazine.com/itspmagazine-podcast-radio-hosts/marco-ciappelli
____________________________
This Episode’s Sponsors
Island.io | https://itspm.ag/island-io-6b5ffd
____________________________
Episode Notes
In this Chats on the Road to Black Hat USA, hosts Sean and Marco discuss the use of AI in hacking and cybersecurity with guest Frederick Heiding, specifically large language models, such as GPT-3 and GPT-4 (ChatGPT). They explore the concept of using AI to create realistic phishing emails that are difficult to detect, and how cybercriminals can exploit this technology to deceive individuals and organizations.
The episode also looks at the ease with which AI can generate content that appears real, making it a powerful tool in the hands of attackers. The trio discuss the potential dangers of AI-powered phishing emails and the need for more sophisticated spam filters that can accurately detect the intent of these emails, providing more granular information and recommended actions for users.
Throughout the episode, there is a recognition of AI as a tool that can be used for both good and bad purposes, emphasizing the importance of ethics and the ongoing race between cybercriminals and cybersecurity professionals. The conversation also touches on the positive applications of AI in detecting and preventing phishing attacks, showcasing the efforts of the "good guys" in the cybersecurity world. They discuss the potential for AI to help in blocking phishing emails and providing more granular information and recommended actions for users.
About the Session
AI programs, built using large language models, make it possible to automatically create realistic phishing emails based on a few data points about a user. They stand in contrast to "traditional" phishing emails that hackers design using a handful of general rules they have gleaned from experience.
The V-Triad is an inductive model that replicates these rules. In this study, we compare users' suspicion towards emails created automatically by GPT-4 and created using the V-triad. We also combine GPT-4 with the V-triad to assess their combined potential. A fourth group, exposed to generic phishing emails created without a specific method, was our control group. We utilized a factorial approach, targeting 200 randomly selected participants recruited for the study. First, we measured the behavioral and cognitive reasons for falling for the phish. Next, the study trained GPT-4 to detect the phishing emails created in the study after having trained it on the extensive cybercrime dataset hosted by Cambridge. We hypothesize that the emails created by GPT-4 will yield a similar click-through rate as those created using V-Triad. We further believe that the combined approach (using the V-triad to feed GPT-4) will significantly increase the success rate of GPT-4, while GPT-4 will be relatively skilled in detecting both our phishing emails and its own.
Stay tuned for all of our Black Hat USA 2023 coverage: https://www.itspmagazine.com/bhusa
____________________________
Resources
Devising and Detecting Phishing: Large Language Models (GPT3, GPT4) vs. Smaller Human Models (V-Triad, Generic Emails): https://www.blackhat.com/us-23/briefings/schedule/#devising-and-detecting-phishing-large-language-models-gpt-gpt-vs-smaller-human-models-v-triad-generic-emails-31659
For more Black Hat USA 2023 Event information, coverage, and podcast and video episodes, visit: https://www.itspmagazine.com/black-hat-usa-2023-cybersecurity-event-coverage-in-las-vegas
Are you interested in telling your story in connection with our Black Hat coverage? Book a briefing here:
👉 https://itspm.ag/bhusa23tsp
Want to connect you brand to our Black Hat coverage and also tell your company story? Explore the sponsorship bundle here:
👉 https://itspm.ag/bhusa23bndl
To see and hear more Redefining CyberSecurity content on ITSPmagazine, visit:
https://www.itspmagazine.com/redefining-cybersecurity-podcast
Are you interested in sponsoring an ITSPmagazine Channel?
👉 https://www.itspmagazine.com/podcast-series-sponsorships
Please note that this transcript was created using AI technology and may contain inaccuracies or deviations from the original audio file. The transcript is provided for informational purposes only and should not be relied upon as a substitute for the original recording as errors may exist. At this time we provide it “as it is” and we hope it can be useful for our audience.
_________________________________________
Sean Martin: Marco,
Marco Ciappelli: Sean,
Sean Martin: beep, beep, beep,
Marco Ciappelli: beep, beep, beep. Are you calling me some space? Are you...
Sean Martin: I'm calling you from, uh, from the road. Ah, on way on my way.
Marco Ciappelli: Uh, that was a beep, beep, beep. I thought it was more of a beep beep. Like a satellite going by, you know?
Sean Martin: It was, or the road runner?
Marco Ciappelli: Uh, . The roadrunner. You know, that runner could, could fit very well here because there's a chat on the road.
To Las Vegas. So usually we would do it in the car, both leaving from LA. So we go through the desert this time. It's not going to happen, but, uh,
but you know, the
road runner makes me think about, you know, you always try a lot of different things that latest technology with Acme, if you think about the cartoon, right.
And, uh, sometimes it fails. Sometimes they have the. Opposite results that what he wants to achieve. So I don't know where I'm going with this, but I think you're reading me.
Sean Martin: I think you're onto something there. And, uh, I mean, for me, you just painted the picture of hacker summer camp in general, I mean, that that's black hat in a nutshell and DEF CON and the whole shooting match where everybody comes together to.
To show what they've done that made something do something that it wasn't supposed to do. Exactly. Oh, usually for good purposes.
Marco Ciappelli: Hopefully they don't blow themselves up. That's right. That's the coyote. That's right. And all it does.
Sean Martin: That's right. But enough about, uh, roadrunners and why the coyotes and... And, uh, the beep beep sounds that they make, um, I'm thrilled to have Frederick Hyding on.
Frederick, thanks for being patient as Marco and I pretend to have fun with each other.
It's good to have you on, man.
Fredrik Heiding: Yeah, thanks. It's great to be here.
Sean Martin: And, uh, you have a session. which is why we're chatting with you. And it's, it's on a topic that's on top of mind for a lot of people and they may not realize why, which is why we, why we're going to be chatting with you today, uh, before we get into the world of AI and large language models and your.
Your comparison to human models, um, a few words about who you are, Frederik, uh, some of the things you work on and, uh, your, your background, if you could, please.
Fredrik Heiding: Yeah, that makes sense. That makes sense. I'm, uh, I'm currently a research fellow in computer science at Harvard University in the last year of my PhD.
I'm also a PhD student from the Royal Institute of Technology in Sweden, uh, where I started my PhD. I'm born in Sweden as well. I've focused, um, you know, I've been researching cyber security for the last five, six years and, um, In the early days, early days, my PhD, I focused a lot of more technical penetration testing.
Uh, created a framework for how to paint this IOT devices. I analyzed a lot of iot devices that has really bad security. Did a pretty cool study where we tested devices from new households with, uh, common embedded devices such as vacuum cleaners, refrigerators, and we found 17 CVEs in 22 devices. It's pretty hard, pretty many cvs.
Some of them are critical rank, you know, nine to eight or 10 and these kind of things. So it was pretty fun study. And we also may have found out that some of these, you know. I said, everyone has people use them all over the world and the smart door locks and just weird things. We can break them to enter the house.
And so basically, I guess a lot of the focus on my, uh, last year has been to find out all the vulnerabilities that exist out there and sort of poke at them. And something I'm really interested in how, how the changing landscape that we're really having technology sort of affects vulnerability. It goes fast in a lot of different ways.
Everything's getting connected. Worked a lot with smart cities too, which is pretty cool here in the U. S. We. I am part of investigating some so smart cities, also health devices, everything's getting connected everything from new air sensors to health devices and there's a massive amount of connectivity and it's pretty dangerous because all of that connectivity is sometimes devices that are cheap, not produced in the U.
S. or in the country. They're not entirely secure. We find it in so many cases. And this is what the problem is that the nature of embedded devices is often that they're producing mass quantity. You don't just want one. air sensor. For example, if you're in a city, you want a lot of them. Yes, I guess I do.
And that's the producer problems. I think it's really interesting to see. But from a technical perspective, how can you, how can you prove that these devices are vulnerable? A lot of people do that. And of course, from a bigger perspective, how can you, how do you solve this? I'm pretty invested now in the Biden's security act that came out of this year, secured by default and secured by design.
It's a 12 point guideline for some best practices and the best practices are actually pretty good. So I'm also starting a project with the Harvard Business School where we sort of try to investigate how this is implemented in practice, because that's, uh, I think if, if people would follow these code guidelines, things would be really good.
European Union had a similar framework earlier this year that they released. I think these are really big acts and me and my department have been saying for a lot of years is that. We are pretty bad at creating software and everyone says, right, and that's, that's the main problem. And that's the problem.
Yeah.
Sean Martin: I was going to say, I'll have to have you back on to, to dig into those acts and the, and the, uh, like the guides, cause I think it's an important discussion. Um, obviously shift left and moving things over before taking care of the problem before it becomes a problem is important. Um, I want to, I want to bring us to the topic at hand, which is large language models and comparing that to human.
And I'm, I'm going to use, uh, use a story. So I don't know if it was one of the first, it certainly felt like one of the first, um, connected door locks on the market I bought many, many, many years ago, and being a security guy, I was worried about. The networking and the vulnerability that that device had, but where I made my final decision was in the physical human based interaction of that device, meaning you when you physically lock a door, it works a certain way and it feels a certain way.
And you can, you can tell that latches the connection of that device to the physical door wasn't quite right. And I felt that it would fail physically From a human perspective. Um, and there are some other things in there as well, uh, using it from a human perspective more. I felt that failure was more likely than even the risk of, of a breach and that device being compromised digitally.
So my point of all of this is We're going to compare the two, but I have a sneaking, sneaky suspicion that, uh, they're still combined. The human's part of it at all, at all times. So, maybe how we, a moment to describe large language models. What, what does that mean for folks who are listening who may not be familiar with it?
And maybe a quick comparison to the human. Centric models that, uh, folks might be a little more familiar with.
Fredrik Heiding: Yeah. That makes sense. Yeah, could I take one minute before that sort of tie in? Because I was digressing a bit. But I'd like to just tie into why I'm going to say something. I think this is so interesting. But basically, after all these pentests, they also work pretty close to industry. And I think this is pretty relevant to mention. I met a lot of companies and They, of course, think it's pretty cool to do this pentest, but many organizations said that, Hey, your work is awesome, but we don't really care because our problem is our users, our employees, they always mess up.
We can't, we can't really, even if we make the fiscal security really good, we can't make the users do the right thing. The users are often the weakest link. And I began slowly researching that, looking into cybersecurity training and more social engineering attacks. And that's, uh, that's. From my perspective, it turned out to be a really big problem for a lot of companies, and that's what made me interested in that.
And that's what brings us to Blackit. We'll talk about how large language models can be used to exploit users. I just want to mention too, it's a collaborative effort. I don't do this alone. I work with a great team. I guess I can mention them after the description of language models, because that makes sense.
So yeah, what are language models and why are they interesting here? So, as I mentioned, a lot of cyberattacks focus on humans. A lot of cyberattacks, of course, focus on technical systems. But that's, that's all well and done. The problem here is that there's something called large language models. And they're neural networks.
Basically, they're a data model that's been trained on huge data sets. So it's a model that learn, and it learns by analyzing vast amounts of data. And what happens when it analyzes all this data? It gets really, really good at creating context that appears real. You can say it didn't learn, if you so will.
It learns how to create content that appears to be created by a human. Uh, specifically textual content for large language models. And what's important here to note, to learn, appears real. The content is not necessarily real. In my research team, we did some tests, or researchers did, is to just try to, you know.
Do something with the research and the research always sites. It's pretty fun. If you ask what is language models to decide things. Maybe you're asking what's What's a deduce attack? And it tells you what a deduce attack is, then you tell them, okay, give me the citation for this, give me the references so I can cite it.
And it gives you a really good reference for it. It looks perfect, but when you look into that reference, it's complete bullshit, it's false. And that's sort of, that's one really interesting thing with large language models, and sometimes it creates content that is real. Oftentimes it is real, but what they do is they create content that appears to be real.
And that's really, really important to remember. And that's, you know, I'm going to talk about my team in a second, but that's why we used to brainstorm and thought about, Hey, how can this be used? And it can be used for physical hacking. I done that myself too, but it's really interesting when it comes to hacking humans, because when hackers exploit the employees or users of humans, they usually do it by no.
To try to show them something that appears real and the user falls for it and they trick it. So, of course, the hypothesis is that large language models are really good at tricking humans. Because they create content that appears real and, um, you know, then humans fall for it. That's a little info about large language models.
Marco Ciappelli: It's more than little because you just highlight the fact that the key in hacking human is appears to be real. Either you want to pretend to be someone you knock on the door with a certain, uh, uniform and people believe what they see. And so in the end, if it looks real, maybe we think that the The model, it's actually thinking, even if I'm really just putting one word after, after another anyway.
But, but what it really matters is the, the result of this. So I'd like to know when you were running this comparison, and I'm sure you were looking at the way that, uh, the attackers can use this. So it's so good to pretend to be real that obviously it's a big asset for cyber criminals. So give us some example of what you're going to be discussing and what.
The research find out about this. I mean, is it really more effective than, than the human manipulation when you do it with the, with the machine?
Fredrik Heiding: Yeah, that's, that's a really good question. There's, there's some things, uh, some things I want to highlight with that. So one thing is that it is really good at creating content that appears real.
But another thing that's worth to mention is that it's also really easy to do this. So when you interact with language models, you usually only have to write one sentence, for example, create a phishing email targeting Harvard students who are frequent members of the Malkin Athletic Center, which is the gym on campus.
And that's, it's not hard to write that. You can write it in pretty bad English, but it's still okay. When you write that sentence, you're going to get a really good phishing email that, you know, it could have a call to action. For example, press this link to get a three months membership. And why is this interesting?
It's interesting because a lot of the phishing emails traditionally up to this point for the last couple of decades, phishing has been around for a long time. They're pretty bad. And when we think about a phishing email, sometimes you think about the Nigerian print scam. There's someone saying, Hey, you know, your grandmother died and you have a billion dollars inheritance.
And it's. Very obvious false. And there's some, there's some benefits with that too. But overall, there's a lot of really bad phishing emails out there. And my hypothesis is that that will change from now on. There will only be very good phishing emails because it makes no sense to create an email from yourself.
And that's, that's a big difference. Like, I don't think we understand how big of a difference that is because phishing emails are very easy to send. It doesn't really cost anything. You have to write the email, but when you have the email, just send it. Um, you know, it doesn't cost anything to send an email except for a minimal bandwidth and so forth.
And imagine, yeah, just that phishing emails are still efficient. People wouldn't use it if they weren't efficient. Everyone, of course, thinks, you know, you and I will never fall for it because you're stupid, but some people fall for them. But even the most phishing emails are pretty bad today. They're still efficient.
So imagine what everyone, it doesn't matter if you're a native English speaker, you almost don't even have to speak English, you can create a native sounding email in seconds that are highly targeting and pretty efficient. So, you know, I think the ease of access is. is a very strong point here that it's going to be super easy to create large scale, not only phishing spear phishing.
Spear phishing is a targeted phishing attack. Instead of just sending random emails, maybe they target me as a black cat speaker. Say, create a, create an email to a black cat speaker saying that, you know, he has to register in this link in order to get this hotel reimbursement. That's a very targeted email.
It's very easy to write that prompt. And the language model will create a super good email. It looks realistic and maybe I would even fall for it. So I think that's, that's one of the points I want to highlight that they are, they make it so easy, which is very dangerous.
Sean Martin: And so let's talk a little bit about, um, spotting these because, I mean, we, we have a hard enough time as individuals, um, not, not recognizing that, that this is an attack, uh, asking us to click on a link.
Organizations have a hard time. Determining whether something is real and should go to its destination or not. Yeah. I'm thinking, I'm thinking even things like, uh, business email compromise, which is a little more, more along the lines of a, of a spearfish attack, right? Yeah, yeah. Um, are we going to find ourselves in a situation where it's hard to spot these things?
And, and because it's so easy, there's going to be so many that, that. Where we might, may have only had to identify and block 10 yesterday where we have 10, 000 to block today.
Fredrik Heiding: Yeah, Sean, that's a perfect point. And yeah, I really believe we will. You're hitting the nail perfectly there. I think there will be a massive amount Yeah, even more than there is today.
I met one interesting professor who said something theoretical There will never be here in practice, I think, but if we just made every email cost 0. 001 dollar to send All this would go away because it would be infeasible. That will never happen. People want to send free emails. But it's free to send. It takes one second to create.
Everyone can do it. And it's a perfect English email. Yeah, I really believe there will be a bomb of emails.
Sean Martin: Can you, can you tell me about the, the process though? Cause I think it's probably easy for most to see the body of the email getting created. And then that naturally the subject line for the email.
You mentioned the call to action with, with a link. Um, Is that, is that a manually connected thing or is the, or is the LLM creating that link as well as it going out and sourcing the, the, the phishing, uh, payload for, for the bad actor? Uh, I'm thinking about the, the from address, the to address, um, the business email compromise it's using maybe.
A slightly different, uh, domain name that looks identical to the real one that it's coming from. Can it get down to that level and are there other things, like metadata and things like that, that it can help tweak to help it bypass detection?
Fredrik Heiding: Yeah, that's super good. And like, I actually really liked the question and that's something that I'm currently in my new research looking into.
And I like to call it like the phishing kill chain or whatever in cyber, you have to call it the cyber kill chain, which is a bunch of steps to hack something. But it's a chain of stuff, right? You, you don't know what to create the email. But before that, there's a first step, which is you want to collect some background information.
You just create some data and that doesn't have to be hard. But I want just a few data points of the user. That makes a massive difference. Do I live in Sweden? Do I live in America? Am I a student? Do I work at Amazon? Whatever. Everyone has big, big digital footprints online. So first you want to collect the data.
Just a few points. Then you want to create the email. Then you want to create a call to action and as I said, all these things can actually be automated and what I really would like to do is I want to pick out all of these things and see which is the most feasible to start because this will be automated into the phishing box.
It's going to be super easy. And yeah, you can already write to an arbitrary large language model. Create, you know, create a language page, you know, in Python, using React, whatever you want, whatever language you want, almost. Create a language, uh, create a landing page that, you know, collects the IP address of the user.
That's not so interesting, but maybe create a landing page that, you know, prompts the user to, to log in, to, uh, to enter their company's, uh, new job portal, whatever. You know, that's easy. What you can even do is that, you know, you can collect some background information. Then maybe you say, hey. This person just uploaded to LinkedIn that they started a job at Google two weeks ago.
Perfect. The language bot takes that information and says, Hey, create a language or a landing page for new Google employees. Ask them to register to get all the company benefits. Paste it in the link. Create a phishing email with that. Sending it out. Participant presses the link. They see this landing page to enter their information.
Once again, this is easy to do. You can automate this so that everyone, what would've taken months to do can take seconds to do, uh, in these targeted attacks and everything, the whole, the whole chain can be automated and even to a, to a future point. When the person enter this webpage, then they can enter to me by chatbot and, and that's what you know, the language models are best at to talk with you.
Then you talk with this person maybe, and you can do this in all kinds of use cases. It's really, there's a bunch of studies, I did some before, but you're trying to make people spot fake webpages, that's almost impossible. If you take, you know, Amazon and you change one letter, it's so hard to recognize because we just don't see these kind of things.
So, yeah, it can be automated. I would definitely like to just pick out every layer, just five phases or whatever you want to say in this process. Where can we stop it? Some of them will be really hard to stop. Maybe some of them we can stop to make this automated chain impossible, but it's harder. It's going to be very easy if we don't add roadblocks.
I think we need to add some roadblocks here.
Marco Ciappelli: All right, so let's talk about that. And I'm sure that's where you guys are going to go into when you do the presentation. But I'm kind of curious, you know, is there always like first there is a technology. We're all excited about it. And then we figure it out.
And this is the history of humanity that, Hey, we can use this hammer, not just to nail something, but to kill someone or do other things. And so, you know, all of a sudden this model are from really cool to very dangerous. And then eventually we say, you know what? Maybe more technology. We need a bigger hammer to slam the small hammer.
My point is, We as, and I say we because we are on the good side, I think, I hope, um, we're not the villain. So what do the, the, the cyber security protector can do with this now that it's in the hand of a very creative cyber criminal mind? Then how do we take it back and say, well, we can still use this technology to detect when something happened or to block it.
Where are we with that? I mean, are you guys going to cover some of this angle?
Fredrik Heiding: That's an awesome question. I really like you saying that. And that's, to some degree, the most important thing to ask. And we will, for sure. Basically, my presentation or our presentation with Blackit will be divided in two parts.
And how can AI be used to hack us? And how can AI be used to help us from being hacked? And I think it's really interesting to cover this. Also, I want to sort of follow up to what you say and mention you some positive notes because sometimes these days AI discussions are really sort of doom and gloom, they're pretty negative and I want to try to shed sort of clear the fog around that a little bit too and say there's, there's a lot of uncertainties, we don't know what happened, but AI is a tool, just like it's a hammer or whatever tool you have, it can be used for bad and for good, there's just so many fantastic use cases with AI.
So like, I really don't want my talk to be, you know, something that says AI is bad, we should be afraid of it or whatever. But AI is a tool. It's a very interesting tool. One of the fastest, you know, growing and evolving tools I've ever seen. It's a tool that can be used for bad and for good. And we did find out it can be used for quite a lot of bad things.
We also found out it can be used for a lot of good things. And that's actually one of the more interesting parts. And primarily we investigated two things. How can AI and specifically large, large language models. be used to detect phishing emails, not just detect them, but to take the intent of the attacker.
And that's something we often focus on. The intent is really interesting because we already have really good spam filters. All the emails have them. And a lot of research has been done in this area too, both by industries and academia. How can we improve the spam filters with large language models? And the spam filters are fairly good.
They block out a lot of, a lot of content. And that's awesome. And that's something that I really want to highlight that, yeah, all these It's a race basically between the bad guys and the good guys. The bad guys try to use the AI to create good phishing emails. The good guys try to use them to create good spam filters in different ways.
And what we have investigated a little bit is not just to create a good spam filter, because a lot of people do that and they do it well, but I'd like a more granular spam filter. I can say not just... This looks like a bad email, but it's like, Hey, this look, it looks like an email that tries to get you to, uh, get a gift card.
Uh, so you should be a little bit careful because XYZ and some of the models, there are a bunch of different large language models and some of them, and not everyone, but some of these vendors. They were really good at not just defining the correct intent, and that's really hard to do, but they do it pretty well, and also giving recommended actions.
So, for example, one email we fed to a language model, say, hey, here's an email, what do you think is the intent of this email? And then the pretty cool, this is a good email that looks like a legit, you know, legit email offering you a gift card at the store. And then, uh, the, the language model say, hey, this, the intent of this email.
appears to be a phishing email on the front. It tries to make you get a gift card, but we believe it really tries to fish you because of X, Y, and Z. And I think that's super cool that they can do that because it's quite sophisticated. And then they'll say, Hey, if you really want this gift card, you might want to go to the official page of this company and check whether the offer is valid.
And if you don't find any information there, send an email to the contact page of this company and see if the gift card is valid. I mean, that's really good advice. I I don't think I could give better advice than that. And I should be some kind of expert in this area. But that's, that's, that's, uh, that's quite impressive.
So there's a lot of really good use cases. And that's, that's something we're investigating to not just have this binary way, because, you know, binary classification is good in saying, you know, this is spam. This is not spam or malicious, not malicious. But language models gives us a way to do this in a much more detailed way.
I'm going to
Marco Ciappelli: go back to, sorry, Sean, just a quick, yeah, because it really kind of made a little light. You know, light bulb blink in my head because it's like, as you said, like if you use it, not just to automatically block everything, but to actually educate the user as you go. And it's like, Hey, before you do this, how about that?
Or this, this, this seems a little suspicious. Uh, what about you do this before you actually act on that? And I think it could become the very human element that comes into play. To educate them instead of just say, here's the easy button to protect you from everything. Sorry, Sean, just my mind going there.
Sean Martin: No, I mean, the cool thing is that there's a lot of opportunity to think of new ways to, to leverage this technology. And I, and, uh, I want to bring it back to your, one of the very first points you made, which was how easy it was. Or is yeah, to just write a simple query or prompt. Right. And I think that's the value of this in defense where today we're trying to build massive filter lists and, and all of these algorithms to try to find.
Context and behavior and all these things that perhaps a simple question as something comes in. Um, if I see if it's heading toward this particular group, is it X, Y and Z? And I'm not going to try to. Pretend to be smart here, but I guess the point is a query or a prompt to look at things as they're happening.
Perhaps with the ability to look deeper into, into the contents based on it's, it's learning, uh, as an LLM can help identify things that no filter and no algorithm outside of an LLM could ever figure out. So I want to, we're coming up close to, uh, our time here today and, uh, And, uh, our, our car to Vegas is going to run out of gas.
We don't, uh, wrap up joking, of course, but what I want to do, I mean, yeah, you have some cool, uh, folks joining you on stage, uh, Bruce Schneier, of course, many folks in the room, Vish, uh, Vishwana and Jeremy Bernstein, uh, the four of you get to talk GPT three, GPT four, and human models, uh, all in the context of, uh, the things we talked about today.
Okay. Um, That's at, uh, that's on Wednesday the 9th, 10 20, uh, level three South Seas CD for folks who know where that is. Um, maybe a quick recap from you or summary from you of the session. What do you hope people to walk away with after spending that time with you on Wednesday?
Fredrik Heiding: Yeah, that makes sense. And I really want to sort of, uh, throw back what you said there and just give a highlight to, to the team because that's, this is really created by everyone.
Uh, and, uh, Jeremy, uh, is a close friend and a postdoc at MIT in neural networks. He really is an expert at, uh, an expert from the machine learning perspective, all of this. And yeah, Bruce Schneier, he's, uh, he's of course a security expert, but he also wrote a book, uh, released earlier this year called The Hacker's Mind, which among other things, he mentions, you know, how emerging technologies are changing the hacker.
landscape to some degree, in a way, and that sort of plays well into the presentation. And Aaron, Aaron wrote a book and released it last year called The Weakest Link, which talks about fishing and why fishing is so successful. So like, all of our combined effort really goes into this presentation. I have some expertise in fishing and, uh, So I really want to highlight how this is a combination of all our works.
And yeah, what I want people to, uh, to come away with, um, first of all, I want to give some sort of specifics to this and I really tried to do it because I think AI is super interesting, large language models and so much talk about it. Now there's a lot of uncertainty. There's some people say a lot of things and then some people don't really believe them.
I want to try to. Give us some concrete information and say, Hey, we actually try this, we knew everything can happen in the future. I think it's pretty fun, you know, sort of pretend, you know, to just look into the future, see what will happen, what can happen, but what we need here is, okay, what can we do?
Like everything is super cool, but if you really take the best AI models out there, the best human model out there, how do they fare? Like how, where are we right now? And one thing I also, I like to highlight with that is that in this type of research field, which is quite rare in this day, but. When you get some results these days, the results are more or less already obsolete because it goes so fast.
And that's what I'm gonna blend, you know, some of our results from the earlier studies. We could work with you right now and these are new results we're getting. But basically just give some clarity on, you know, where are we? Like regardless of all the talk and the hype, these things, what is the exact capabilities right now and what can we expect that this will lead in the lead in the near future and the further away future?
Sean Martin: I love it. And um, Marco loves when I say one more question, because you made a point on charging a fraction of a penny, a fraction of a dollar for sending emails. That's, that's the delivery mechanism, which to your point, I don't think we're actually going to charge for mail, email delivery, but I've seen extremes in the use of The LLM technologies where in some instances, it's completely free.
And then if you try to get advanced with it, it can actually cost a lot of money. So I'm wondering, um, very quickly, if any of the four of you have any, any thoughts on the, the economics of this as a way to perhaps control some of the threat that it brings. And, and I don't know if that impacts then the ability to counter.
Measure some of these things as well.
Fredrik Heiding: Yeah, that's a very interesting point and I, personally, I don't think we have focused too much on it because the problem is that some of these models are already able to download open source, so to speak, so you can just download and have them local and it's very expensive to train them, but running them is not very expensive, which means that it's going to be pretty easy to just run these queries and get the results.
And it's hard to sort of monitor, restrain what you could do, and especially it's hard to You add monetary implications that makes it hard for criminals to use them, but it could make it possible for other people to use it. Like you can, but it's, I think that's pretty hard. There's definitely, you know, you could definitely do it, but I don't have an answer from the top of my mind of how to do that in a good way.
Um, but it is a cool point and I will definitely think of it more in the future.
Sean Martin: And I was driven by Mark when we were having a conversation yesterday about, uh, Yeah, what I
Marco Ciappelli: was thinking is, except that people go into crime because they're actually making good money. It may not last. So if there is somebody that has the budget to pay the extra, he's probably not a regular user.
Fredrik Heiding: Yeah, that's, that's true, actually. And cybercrime is good money these days. So that's, that's definitely a problem.
Marco Ciappelli: We're not actually telling you to go there. We're doing the opposite here. I just want to let you know. Anyway, I'm, I'm excited to, to follow this conversation that you guys are going to have.
And, uh, uh, definitely relevant. Uh, we're talking about that all the time for the good and the bad.
Sean Martin: Yep. Great panel. Great topic at a great event, of course. And, uh, hope everybody enjoys your session, enjoys chatting with you and, uh, meeting the rest of your, uh, rest of your panel. And, uh, obviously, hopefully.
Folks are thinking a bit more about this topic and, uh, we'll join you to, to hear each of your thoughts on it. So, uh, thanks everybody for listening to this. Thanks Frederick for, uh, for sharing your, your thoughts on this topic and, uh, for putting that. That session together. Of course, everybody listening to us.
Uh, we have a lot coming Black Hat this year and Hacker Summer Camp, and, uh, you can follow it all on itspmagazine. com forward slash bh USA and, uh, more chats on the road, keynotes, speakers, panelists. You got it. It's like all going to be there. Links to that will be in the show notes along with, uh, Fredericks and teams, uh, presentation and other stuff from Black Hat.
So keep well, everybody have fun, uh, with AI. Uh, responsibly.
Fredrik Heiding: Perfect. Thank you so much for having me. Yeah.
Marco Ciappelli: Thank you. Thank you, Frederick.