How data debt can impact AI security

Debt’s no laughing matter—especially when it involves the data a company needs to make decisions and build AI tools.

Aaron McCray, field CISO and executive technology strategist at IT provider CDW, defines data debt as “poorly classified, overly permissioned, and unmeshed data that’s grown exponentially over the years.” When organizations don’t have the necessary visibility and management tools to wrangle their data, that debt can become a serious problem.

McCray suggested that most organizations “typically only use about 30% of their data, and the remaining 70% is often unstructured.” That can sometimes lead to issues for the IT team: Some 94% of organizations are struggling to manage their unstructured data, according to a survey of 1,000 companies in the US, UK, France, and DACH (Germany, Austria, Switzerland) regions by Nasuni.

All roads lead to AI. McCray said that issues with data governance at most organizations extend beyond automated tools and platforms, touching fundamentals such as how an enterprise stores and classifies its data.

Mismanaged data can lead to a host of problems, including unauthorized access and leakage when data is improperly used to train a large language model. Although AI can help employees quickly find data that hasn’t been stored properly, a lack of guardrails and organization can also lead to an AI accidentally fetching highly sensitive information and presenting it to the wrong user.

“Users previously had to navigate through maybe dozens of folders to find a sensitive document,” McCray said. “[AI] can now retrieve that same information with just a simple natural language prompt.”

McCray suggested that AI retrieving unauthorized information can highlight organizations’ lack of information management. “AI didn’t really create a security vulnerability; it already existed, and what it did is it democratized it, it made it accessible to everyone,” he said. “That access that used to be hidden and sensitive data that was hard to find and sitting unprotected is now available in the environment.”

Because of that, McCray added, organizations “have to focus on fixing some of the debt, that data debt or that security debt that they’ve let accumulate over the years.”

Success story for operationalizing data. Chris Yule, senior director for Sophos’ Counter Threat Unit, points to his organization’s successful collection and management of threat intelligence data as a long-term focus in support of its cybersecurity mission. Now the company uses the collected information to power its threat graph and utilize AI to further analyze cybersecurity threats.

Sophos had been investing in data governance, so it had years of data processes and supportive tools to meet the rise of AI. “The challenge that we face…is you have a whole bunch of different datasets, that’s third-party reports, it’s telemetry from customers, it’s our own research, and the challenge is making all of that structured and interconnected across different sources of data,” Yule said.

Yule added that having “good governance” and processes around layering AI on top of information systems is a way to prevent AI from hallucinating.

“We talk a lot about using the right tools for the right job, and AI is very effective at some things, but it’s not the solution to everything,” Yule said. “If you rely on AI exclusively, and where it doesn’t have all of those relationships and the deterministic things underneath it to support it, then you will get much messier output. That’s harder to operationalize.”

Is it too late? McCray said that organizations need to address data debt for AI solutions by having a “sense of direction to where we’re going.”

“If my data is not clean and it’s bad data, and it’s bad data then I’m going to get bad analysis and bad output—all of this is very basic and fundamental to think about what an AI prompt is,” McCray said.

Before jumping into an AI-centric revamp, McCray suggested that professionals consider some key points, including:

• Have data identification and classification based on policy and processes governed at the highest levels
• Consider data protection, or safeguarding information from compromise or loss, as a foundation for AI initiatives
• Ensure there’s sufficient visibility for making decisions around data storage, classification, and implementation, with an emphasis on compliance and regulatory obligations