Alessandro Curione
Artificial intelligence is back in the news
OpenAI’s ChatGPT and image generation systems such as MidJourney and Stable Diffusion have attracted a lot of people who are interested in and talking about advanced AI. Which is a good thing. It wouldn’t be pretty if the transformative changes that would take place in the next two or three decades came as a surprise to most of us.
One company that has been a leader in advanced AI longer than most is IBM. Alessandro Curione, a senior executive at IBM, joined London Futurists Podcast To discuss IBM’s current projects in artificial intelligence, quantum computing, and related fields. Alessandro has worked with IBM for 25 years. He is an IBM Fellow, Director of IBM Research, and Vice President for Europe and Africa.
IBM’s Great Challenges
IBM has been inventing the future of computing for 70 years. It has staged a series of impressive grand challenges, such as Deep Blue beating Garry Kasparov at a game of chess in 1996, and Watson beating Ken Jennings in a Jeopardy TV competition in 2011. More recently, in 2018, the company has developed a machine capable of holding with it. Debating with a world champion in debate.
The first of these machines were rule-based, while the later ones used deep learning, which creates models trained on large amounts of data. Another paradigm shift is happening now, with the arrival of large language models (LLMs), or core models, that use a technique called self-moderating to do the training. The system would take an enormous amount of sentences—hundreds of billions of them—from the web, randomly hide one word from each sentence, and try to guess the word. Over time, the system builds a model of which words go into which sentences. Automating the training process is a huge advance, made possible by the vast amounts of data and computing power available today.
It turns out that this methodology is not limited to text. It can be used on any type of structured data, including images, video, or computer code. or data streams generated by industrial processes. Or the language of science: translating molecules into symbols.
narrower focus
IBM builds large language paradigms, but for specific applications rather than general purpose use, like ChatGPT. For example, systems are being built for majoring in organic chemistry, and in business. The weakness of general purpose systems is that they are shallow. They can answer most questions at a high level, but if you dig deeper, they will be lost. More specialized machines can go deeper and are less brittle. Specialization often means you can get better quality data, and you can remove bias more easily.
One of the reasons ChatGPT performs better than GPT-3 is because of Reinforcement Learning with Human Feedback (RLHF). OpenAI, the company that created these systems, employed large numbers of people to comment on the system’s output, labeling biased or offensive clips accordingly. This prompts the joke that AI does not mean artificial intelligence, but for affordable Indians, but humans are used during training, not in operation.
IBM hopes to prove that it can develop a large model in a given domain, which can then be trained on proprietary data of client organizations within that domain. This would be a significant cost and sustainability improvement over the old approach, which involved developing a new model for each application.
More efficient chip designs
Another area where IBM is looking to improve the efficiency and sustainability of AI and computing is chip design. Large language models approximate the scale of computation going on inside the human brain, but they use the same energy as a small city, while the brain uses the same energy as a light bulb.
Curioni says IBM is taking three steps to reduce the power demand of advanced AI systems. The first step is neuromorphic chips, such as IBM’s True North and Loihi’s, that are modeled more closely on human neurons than traditional chip designs. Their calculations are less accurate and more analog.
The second step is memristors, where processes and memory are stored on the same chip, reducing the energy spent retrieving and re-storing data between computations.
The third step is the enormity of neural networks, which transmit information only when its own function is required, whereas in conventional chips, each neuron transmits information all the time.
Together, these three steps can provide improvements of 2 to 4 orders of magnitude in energy efficiency.
A breakthrough in quantum computing
IBM may not currently be seen as the world leader in artificial intelligence, but the field it is generally recognized for being in the number one spot is quantum computing, along with Google and Microsoft. It has just announced a breakthrough in quantum encryption that will enable data transmitted today to remain secure, even as quantum computers are built that can break today’s encryption. Quantum computers running Shor’s algorithm can efficiently parse numbers and, when scaled, will be able to factor out very large numbers, which classical machines cannot do within reasonable timescales.
What IBM and a number of academic partners have done is develop a new type of cryptography called secure quantum cryptography. It is based on high-dimensional lattice encryption, and is believed to be unbreakable by quantum computers. Over the past decade, a large research program has been conducted to evaluate many potential types of secure quantum cryptography, and last July, four algorithms emerged as the most powerful. Three of those four were developed at Curioni’s lab in Zurich, and the winner has just been chosen.
The next step is to migrate the data from the old forms of encryption to this new form. This task has become urgent. There was a panic in December 2022 when a team of Chinese researchers announced that they had already figured out how to hack existing encryption technologies. Their paper has been dubbed the “quantum apocalypse”. They soon realized they weren’t all the way in, but it might not be long before someone made it – perhaps as soon as two or three years later. The US government has directed that all of its agencies must be quantum secure by 2025, and other governments and companies are doing the same. Perhaps the IBM hack came at the right time.
London Futurists Podcast