An invitation to an unusual international scientific conference forced the researchers to find out what would happen if what they had created fell into the wrong hands.
In 2020, Sean Ekins – Executive Director of Collaborations Pharmaceuticals, based in Raleigh, North Carolina (USA) – received an invitation to participate in the Spiez Convergence Conference in Switzerland.
“I had never heard of her before,” Ekins told BBC News Mundo, the BBC’s Spanish-language service.
“Actually what excited me more than anything was the location, because it was in Switzerland, in a beautiful place.”
But the covid-19 pandemic caused the conference to be cancelled. Ekins and his team continued to go about their day-to-day business, searching for rare disease drugs not served by big pharmaceutical companies, as they were considered unprofitable.
His research ranges from ‘a disease that shortens the lives of children and affects around 100 people worldwide’ to infections such as malaria, which, ‘although there are drugs for some cases, lose their effectiveness over time’.
“We’re not going to get rich, but at least I think we’re contributing to society as a whole,” says Ekins.
A year passed and he received another invitation to the conference. But now there was no attraction for the trip, as it would be done through the Zoom application.
Ekins had to pay more attention this time. It was a meeting to evaluate new trends in chemical and biological research that could pose a threat to security.
“They were interested in the misuse of technology, scientific instruments and science in general,” he says.
The organizers asked Ekins to give a presentation, which sounded “very strange. [Aquilo] it forced me to think about how we could abuse our jobs.”
What are they doing
To discover drugs for rare diseases, Collaborations Pharmaceuticals created the MegaSyn artificial intelligence (AI) platform.
The platform is something you don’t buy, but assemble, “like Lego,” explains Ekins. According to him, “all the bricks” needed to build it were obtained from “open source software databases”.
“We then query databases of information to build machine learning models that we plug into MegaSyn,” he explains. “This makes it possible to design molecules with specific properties.”
In other words, the researchers trained artificial intelligence to do the job of a human chemist, but at breakneck speed.
Drugs are basically composed of molecules. MegaSyn not only looks for existing molecules to see if any are useful for fighting disease, but it can also create other molecules, which have not yet existed.
The doctor. Bad’
On the eve of the conference, Ekins and Associate Director Fabio Urbina set to work to see what would happen if MegaSyn fell into the wrong hands. Thus was born the experiment that received the name of “Dr. Malvado Project”.
It was a simple project. To generate a new pharmaceutical product, it is essential to ensure that it is non-toxic. What if that filter changed?
“We just flipped a switch on the model. Instead of ‘non-toxic,’ we said ‘toxic,'” Ekins explains. “Literally, it was about swapping a 1 and a 0 in the program. Simple as that.”
They hit “enter” and let MegaSyn do the work.
The next day they found a list of tens of thousands of molecules and did a search to see if they knew any. And they were amazed at the result.
VX extension
“This simple change of plans had a dramatic effect,” says Ekins.
With the push of a button, MegaSyn had engineered the fearsome VX nerve agent, classified as a weapon of mass destruction and banned by the UN.
It is one of the deadliest substances ever produced. Less than one drop on the skin is enough to kill a person.
They also confirmed that MegaSyn “succeeded in finding hitherto unknown precursor molecules – starting points for the development of more powerful mass chemical weapons.”
Most troubling was learning that although Dr. Although it was a theoretical experiment, other people, with other intentions, “might be doing the same, if they haven’t already.”
“You don’t need a PhD, just some programming knowledge, a basic laptop…everything else is available online for free,” he said.
“If some chemist had this in his hands and wanted to convert these molecules into weapons, because no one knows they exist, it would be impossible to track and detect them.”
the dilemma
To warn the well-meaning of the danger is also to inform the bad-minded of the possibilities.
Ekins and Urbina chose not to give too much detail about Dr. Malvado’s findings in his Spiez talk. They presented a generic description of their work, offering just enough information to tip the experts.
But, however disguised it was, the potential negative use of its technology was what caused the biggest impact.
Shortly thereafter, a summary article of the conference, published in the scientific journal Nature Machine Intelligence, attracted the attention of newspapers and magazines such as The Washington Post and The Economist. Discussions have erupted between experts in chemical weapons, artificial intelligence, the pharmaceutical industry and even between philosophers.
Ekins and his team were even summoned to a meeting with the Office of Science and Technology Policy and the National Security Council at the White House.
“It was surreal,” she recalls.
Despite initial fears that talking about the subject might serve more to inspire than prevent, Ekins had to take the risk of raising awareness, not just because of the interest it aroused, but because he “couldn’t sleep soundly anymore.”
But he has never been willing to share the list of tens of thousands of possible toxic molecules generated by MegaSyn, not even with the White House.
What to do?
Even with all the risks, no one can stop progress. Platforms like MegaSyn are immensely beneficial and Ekins knows this better than most people.
One strategy to prevent misuse is to “educate people about the issues.”
“Now that the government is aware, we have been asked to speak on behalf of government agencies,” he says.
“In May, for example, we will be going to a conference in Brazil, speaking on behalf of the US State Department to academics and industry professionals about potential issues surrounding Generative AI.”
And did these activities help Ekins sleep better? Well, not much.
“Since we made Project Dr. Malvado, generative AI has gotten a lot more public with things like ChatGPT and DALL-E,” he says. “Chat applications have even been shown to project molecules.”
“Barriers to entry today are much lower than they were a year and a half ago. It’s crucial that we educate scientists who haven’t had the same experience as us,” insists Ekins.
But the specialists consulted by the press say there is no reason to launch so many alarms. They emphasize that the identification of new molecules is only the first step on a long journey towards the production of a new drug… or a new weapon. And that most of these paths lead to failure.
On both sides of the debate the same analogy often emerges.
The cake
For some experts — those for whom nothing can keep them up at night — what MegaSyn provided was an ingredients list. And it would take chefs with all the Michelin stars and a kitchen full of cooks to make the perfect pie.
“A year ago, that would have suited me,” says Ekins. “There have always been people making drugs and weapons illegally, and obviously that’s a niche market.”
“But we’ve increased the potential number of pastry chefs,” he said.
For Ekins, technologies like MegaSyn produce not only ingredients, but recipes as well. And if the recipes don’t work, they help perfect them.
“Once the recipe is provided, anyone can bake this cake,” she says. “Some will do better than others, but the risk is always there.”
But he points out that there is something that could help us sleep better.
Several entities in countries such as Switzerland are very committed to this issue. There are discussions to see how bodies such as the Organization for the Prohibition of Chemical Weapons (OPCW), which regulates molecules, can include AI in their legal framework.
“We can use these technologies to defend ourselves. [Podemos] use the same tools to verify which chemicals could be used to prepare new toxic molecules and, therefore, control them”, concludes Sean Ekins.
– This text was published at https://www.bbc.com/portuguese/articles/c51ppklppyko
Source: Terra
Rose James is a Gossipify movie and series reviewer known for her in-depth analysis and unique perspective on the latest releases. With a background in film studies, she provides engaging and informative reviews, and keeps readers up to date with industry trends and emerging talents.
