AI and Hardware Hacking on the Rise – Source: www.securityweek.com

AI hacking (both of and with AI), hardware hacking, and AI-assisted hardware hacking are all increasing.

Bugcrowd’s eighth annual Inside the Mind of a Hacker report surveys the thoughts of one of the world’s largest hacker communities. Almost 1,300 hackers took part.

The report covers four primary subject areas: defining a hacker; the motivations for hacking; the rise of hardware hacking; and the effect of AI on hacking. 

The first two subjects are already covered extensively in the SecurityWeek Hacker Conversations series, which discusses the history, mind and motivations of some of the industry’s most famous individual hackers (including HD Moore, Joe Grand, Weld Pond, Space Rogue, and Bugcrowd’s own Casey Ellis). 

For this reason, we’ll concentrate on the last two subjects: hardware hacking and the increasing effect of AI on hacking. By understanding how hackers think and work, we are better able to defend ourselves from truly malicious threat actors.

Hardware hacking. A rise in hardware hacking has followed the growth of cheaply made but overly complex smart devices, which often prioritize speed to market and quantity of features over security. “Software security, no matter how sophisticated, is practically useless if attackers find ways to exploit the physical hardware.” 

Hardware hacking techniques include fault injection, side-channel attacks, and firmware manipulation. The report notes an increasing democratization of such hacking through cheaper tools and traditional hacker creativity. 

Lennert Wouters used fault injection to hack a Starlink satellite dish via a $25 modchip. He temporarily shorted the system, bypassed the security, and gained access to locked areas of the system’s software.

Side-channel attacks are also more widely possible because of the growth in affordable and precise measuring tools. 

While the current common level of hardware hacking will not create the media headlines of x million records of PPI stolen from y by China’s APT z; the consequences for individual victims can be severe. Messing with home heating systems can cause fires, while messing with medical devices can cause deaths.

AI and hacking. AI is still evolving so fast that nobody knows where it is going. The report notes that some hackers are wondering how it will help them, others are wondering whether it will replace them – but all agree that companies using AI now have a new class of threats. “At Bugcrowd, we frame this as the three Ts of AI: AI as a tool, a target, and a threat.”

As a tool, AI increases the speed and sophistication of processes. For adversaries, this is better, faster phishing campaigns. For defenders, this is a more accurate and almost realtime detection of intrusions.

As a target, AI systems provide a new attack vector. Many AI systems have access to company data, while inadequate prompt guardrails give attackers access to that data. Indeed, it is a moot point whether prompt guardrails can prevent prompt injection attacks without eliminating the ‘conversational’ element of LLM technology.

“I achieved a remote code execution (RCE) accidentally while chatting with an AI bot that was misconfigured and had the ability to execute OS commands on the system where it was hosted. It provided me with the command output,” explained one hacker in the report (PDF).

As a threat, AI can both inadvertently and by manipulation cause harm to its users. Inadvertent harm can come from hidden bias within the algorithms. Inflicted harm can come from malicious poisoning of AI model training data to cause erroneous output – and then there’s the future potential of AI to create new and more powerful cyberweapons.

Compared to last year, hackers’ views on AI have remained virtually static, except for one telling area: the agreement that “AI technologies increase the value of hacking” has leapt from 21% of hackers to 71% of hackers. If non-malicious hackers are beginning to see the practical value of AI in their work, malicious threat actors will be doing the same.

Using AI to assist hardware hacking. AI is likely to assist hackers in many ways – including hardware hacking. For side-channel attacks, “AI algorithms can perform complex analyses, discovering minute variations in power consumption, electromagnetic emissions, or timing data from a device.”

For fault-injection attacks, “AI’s ability to quickly process and adapt to complex patterns makes it particularly suited for tackling the intricate timing issues often associated with fault injection techniques. AI can help determine the correct frequency, timing, and intensity of induced faults.”

And “The ability to automate and parallelize attacks enables simultaneous breaches across multiple devices… In the not-so-distant future, devices may be so interconnected that an AI could hack all sorts of devices with just an internet connection.”

Bugcrowd’s 2024 report makes it clear that new developments in technology bring new opportunities for threat actors. Bugcrowd’s friendly hackers have demonstrated this. 

The basic problem is summarized by Brandon Reynolds. “Speaking with the companies developing hardware devices, I find that often, they don’t understand unless someone can handhold them through the process. There are exceptions, but I fear the average internal hardware team is going to be of a much lower caliber than your average hardware-tinkering hacker out there on the Bugcrowd Platform.”

By inference, the same comparison must apply to malicious threat actors, and especially the better-resourced organized or state-sponsored threat actor groups.

Related: Hacker Conversations: Casey Ellis, Hacker and Ringmaster at Bugcrowd

Related: Bugcrowd Raises $102 Million

Related: AI Models in Cybersecurity: From Misuse to Abuse

Related: Simbian Introduces LLM AI Agents to Supercharge Threat Hunting and Incident Response

Related: MITRE Announces AI Incident Sharing Project