Try this excuse tonight: You didn’t forget the pizza. You just prioritized new information more highly.
That’s how neural networks treat data, according to a March 2023 interview with Forbes, when AI pro Miguel Luengo-Oroz, asked the large language model answer bot about the GPT-3 “Can neural networks forget?”
The model’s human-like response at the time: “Artificial intelligence systems, like neural networks, do not have the ability to forget in the same way that humans do. The network may adjust its weights to better fit the new data, which could result in it making different predictions for the same input.”
More than a year later, after an IT Brew presentation titled When It Comes to AI and Data Privacy, How Careful is too Careful, an attendee asked a similar question to the human speakers:
How do we get AI to “forget” data that is now obsolete or has been requested to be removed for privacy concerns?
IT Brew posed the question separately to industry professionals this week. Below are excerpted responses, edited for length and clarity.
Ashley Casovan, managing director, International Association of Privacy Professionals (IAPP): There’s work that’s being done on this. There are companies that you can submit requests to, as a service. You can pay to have your data removed from the internet, where a lot of these different generative-AI models are learning from.
U Win, technology director, Americas, Designit: Just because one model recognizes that data is deleted doesn’t mean the other 599 recognize that that data has been deleted. So, it’s up to the company [building the model] to make sure that it’s gone.
Casovan: Products like ChatGPT have the ability to filter out information or learn based on some of your inputs. Because of that, there could be the ability to do the reverse and [have it] forget about some of the different aspects of information that might not be relevant to a user. I think, though, that in theory, that’s a lot easier than when that information is already embedded into a series of other information that it’s pulling and generating altogether; it’s really difficult to forget that.
Top insights for IT pros
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.
Paul Pallath, VP, applied AI practice, Searce: That is something that researchers are working on, in terms of how do we translate the static memory wherein LLMs [large language models] do not forget at all versus a transient memory wherein you are degrading the weights of some of those networks.
Pallath: Now adapter models are really understanding the nuances of the language, which is learned by the large language models. But here the content that you’re training on is the content that is specified by your industry, by your company…the relevance comes in when you are able to train these models only to understand, talk, and respond to the context of what your industry solution is about.
Myles Washington III, director, experience design and strategy, Designit: I came from Amsterdam, living in Europe for the last four years. So GDPR [General Data Protection Regulation] at least gives you some structure and support around what that means to have privacy-related data, but also the knowledge and knowing that if you consent to having your data deleted, that’s going to be a requirement governmentally. Here in the US, we don’t necessarily have those stopgaps and security. So it’s incumbent on the companies to be transparent around that data, and getting a company to delete it, and then knowing that they hopefully are not going to share that with a third-party company. It becomes a very complicated breadcrumb.