Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
In the nearly two years since the explosion in generative AI, boosters of the technology have pointed to its broad application possibilities—notably, its use in developing code.
Petros Efstathopoulos, VP of research at RSA Conference, urges caution. The potential for the technology is high, he told IT Brew at this year’s event in San Francisco, but until AI is mature enough to manage complexity, it will be lacking.
“When you insert prompts into those models and ask questions, they have a difficulty distinguishing between commands and data,” Efstathopoulos said. “The model, as it’s receiving the prompts, cannot always tell which part of the prompt is a command.”
Attack surface. That can lead to “interesting attacks,” Efstathopoulos said, where threat actors can insert commands into data or vice versa. When the AI model can’t necessarily distinguish between the two, that leads to problems. Dedicating resources to fixing these types of problems before the models become too big and too dangerous is essential, he told us.
IT Brew has reported on generative AI’s role in writing code, noting the possibilities for the technology. As Harness CTO Nick Durkin told us: “This can make good developers great. This can make great developers excellent.”
But with that excellence comes the flip side; notably, that attackers can use the technology for their own nefarious ends. “If attacks against you are easier, attacks against your partners and your customers and your ecosystem, your value chain” are easier too, Jeff Pollard, VP and principal analyst at consultancy firm Forrester, told IT Brew last year.
Futurism. Still, Efstathopoulos believes there’s a significant role for generative AI in coding. He likened the adoption of the technology to how machine learning took off a few years ago; as it became widely adopted, experts found that they hadn’t been thinking of implications: privacy, information that could be extracted, and the like.
“We’re seeing a repetition of the story; we’re seeing the enthusiasm, which I fully embrace. I’m a tech person—I want people to be enthusiastic about cutting edge technology,” Efstathopoulos said. “But that enthusiasm comes with certain risks and a certain set of responsibilities that we need to be cognizant of.”