Black Hat 2023 Review: LLMs Everywhere


I attended as many Black Hat briefings as possible this year. As a data scientist, I paid particular attention to the data science, machine learning, and artificial intelligence talks. Before we get into details, let’s address the elephant in the room: ChatGPT and LLMs.

Yes, it did seem like every third talk was about trying to apply a large language model (LLM) to either hack or secure a computer. I don’t doubt that the hacker and security communities will continue to extract value from the compressed knowledge stored in large language models, and made available in user-friendly interfaces such as ChatGPT. But the organizations that extract the most value from LLMs will be the ones that are best able to take advantage of these models’ power for interpreting language. One talk in particular did an excellent job of this.


This Year’s Best Application of Large Language Models (LLMs)

In my opinion, the talk that made the best use of a large language model was “IRonMAN: InterpRetable Incident Inspector Based ON Large-Scale Language Model and Association miNing” by Sian-Yao Huang, Cheng-Lin Yang, and Chung-Kuan Chen at CyCraft Technology. The basic idea is to borrow the strength of LLMs in interpreting natural language, and use that interpretive power to create vector representations of Windows command lines. 

Natural languages are very flexible, with many ways to express the same information. Likewise, command lines have a certain amount of flexibility. The first kind is obfuscation: There are several equivalent, distinct ways to write the same command. This is important in a security context because these obfuscations can evade detection with tools like regular expressions.

The second source of nuance is that the same string can have different meanings in different contexts. As an example, a user could invoke a command foo and also pass an argument named foo to the command; these usages could have the same or distinct meanings. A regular expression would hit on both of those usages, even if our intention is to only capture one of them.

LLMs take (encoded) strings as inputs and yield numerical representations as outputs. The speakers in this talk leverage these numerical outputs, by observing that equivalent command lines tend to be closer to each other than they are to dissimilar command lines, even when obfuscation is used. This gives the model a certain amount of robustness to variations in the input, and allows one to group together command lines from different incidents, facilitating attribution.

I want to emphasize that the main reason this talk is so intriguing to me is that it really leaned on the LLM for the thing that it is best at (interpreting text inputs) and incorporated that utility into a security workflow. Interpreting texts has enormous value for security researchers; using LLMs to do at machine speed what was previously a human-speed task is a big deal.

This talk does not rely on the chat interface at all! Instead, it peeks “under the hood” to work directly with the numerical representations that the model uses to interpret text.

In my humble opinion, the weaker LLM talks focused on the cat-and-mouse aspects of the chatbot interfaces—using ChatGPT to create “black hat” stuff. While the chat interface is impressive, and it can generate some amusing outputs, I don’t see them as a big value-add for security researchers, especially in light of the untrustworthiness of the results. For instance, a recent study found that ChatGPT can generate plausible-but-incorrect answers to Stack Overflow questions 52% of the time. I would expect even worse results if we asked ChatGPT to respond to a security incident.

Looking to the Future

LLMs are a powerful tool, but to truly leverage that power, security researchers will need to think carefully about how to wield that tool. Where are security researchers bottlenecked? Are those bottlenecks related to interpreting large amounts of text? In the security space, I anticipate that LLM methods will find homes in assessing security risks of source code, the code on web pages, and even decompiled binary executable. I hope that we see some of these applications at next year’s Black Hat.

  • There are no suggestions because the search field is empty.
Latest posts
Revving up the Fun: DNSFilter's IndyCar Experience Recap — Long Beach Edition Revving up the Fun: DNSFilter's IndyCar Experience Recap — Long Beach Edition

What a weekend at the Long Beach street circuit! The energy was electric, the excitement palpable, and DNSFilter was at the heart of the action, ensuring our guests had an unforgettable experience with Juncos Hollinger Racing and Romain Grosjean, the #77 driver for Juncos Hollinger.

Securing Public Wireless Networks Securing Public Wireless Networks

In the current era of digital transformation, securing public wireless networks has emerged as a fundamental challenge for IT professionals worldwide. The evolution of technology and the increasing reliance on digital platforms for both business and personal use have made public Wi-Fi networks indispensable. However, greater access creates greater vulnerabilities, making these networks prime targets for cybercriminals. The imperative to secure pu...

How to Secure Public Wi-Fi Networks How to Secure Public Wi-Fi Networks

In the quest to safeguard public Wi-Fi networks from the myriad of cyber threats, certain proactive steps stand out as fundamental. These measures form the backbone of a comprehensive security strategy, ensuring that the network remains robust against unauthorized access, data breaches, and various forms of cyberattacks.

Explore More Content

Ready to brush up on something new? We've got even more for you to discover.