How Alexnet transformed AI and computer vision forever

What is Alexnet?

Alexnet is an artificial neuronal network created to recognize the content of photographic images. It was developed in 2012 by then: the University of Postgraduate Students of Toronto Alex Krizhevsky and Ilya Sutskever And his Faculty Advisor, Geoffrey Hinton.

Hinton is considered one of the parents of deep learning, the type of artificial intelligence that uses neural networks and is the basis of the current main AI. Simple three -layer neuronal networks with only one layer of adaptive weights were built for the first time in the late 1950s, mainly by Cornell researcher Frank Rosenblatt, but it was discovered that they had limitations. [This explainer gives more details on how neural networks work.] In particular, researchers needed networks with more than one layer of adaptive weights, but there was no good way to train them. In the early 1970s, IA researchers had largely rejected neural networks.

Frank Rosenblatt [left, shown with Charles W. Wightman] He developed the first artificial neuronal network, The Perceptron, in 1957.Division of Rare and Manuscript Collections/Library of the University of Cornell

In the 1980s, the research of the neuronal network was revived outside the community of Cognitive scientists from the University of California, San Diego, under the new name of “Connectism”. After finishing your pH.D. At the University of Edinburgh in 1978, Hinton had become a postdoctoral fellow at UCSD, where he collaborated with David Rumelhart and Ronald Williams. The three rediscovered the backpropagation algorithm to train neural networks, and in 1986 they published two articles that show that allowed neural networks to learn multiple layers of characteristics for language and vision tasks. The backpropagation, which is essential for deep learning today, uses the difference between the current output and the desired output of the network to adjust the weights in each layer, from the backward layer to the input layer.

In 1987, Hinton joined the University of Toronto. Far from the centers of the traditional AI, Hinton’s work and those of his graduate students made Toronto a deep learning research center in the coming decades. A postdoctoral student from Hinton was Yann LecunNow scientist Chief of Meta. While working in Toronto, Lecun showed that when the backpropagation was used in “convolutional” neuronal networks, they became very good to recognize the numbers written by hand.

Imagenet and GPU

Despite these advances, neural networks could not constantly overcome other types of automatic learning algorithms. They needed two developments from outside the AI ​​to pave the way. The first was the appearance of much greater amounts of data for training, made available through the web. The second was enough computational power to perform this training, in the form of 3D graphic chips, known as GPU. For 2012, time was Maduro for Alexnet.

The Imagenet of Fei-Fei LI image data set, completed in 2009, was fundamental in Alexnet’s training. Here, li [right] Talk to Tom Kalil in the computer history museum.Douglas Fairbairn/Computing History Museum

The necessary data to train Alexnet were found in ImagenetA project initiated and directed by Stanford’s professor Fei-Fei Li. As of 2006, and against conventional wisdom, Li imagined a set of image data that cover each noun in the English language. She and her postgraduate students began to collect images found on the Internet and classify them using a taxonomy provided by WordnetA word database and their relationships with each other. Given the enormity of their task, LI and its collaborators finally recruited the task of labeling images to the workers, using Amazon Turkish Mechanical Platform.

After 2009, Imagenet was larger than any previous image data set by several magnitude orders. Li hoped that their availability stimulated new advances, and she began a competence In 2010 to encourage research teams to improve their image recognition algorithms. But in the next two years, the best systems only made marginal improvements.

The second condition necessary for the success of neuronal networks was economic access to large amounts of calculation. Neural network training implies many repeated matrix multiplications, preferably made in parallel, something that GPUs are designed to do. Nvidiaco -founded by CEO Jensen HuangHe had led the way in the 2000s to make GPUs more generalizable and programmable for applications beyond the 3D graphics, especially with the CUDA PROGRAMMING SYSTEM Launched in 2007.

Both Imagenet and Cuda were, as the same neuronal networks, quite niche developments that expected the correct circumstances to shine. In 2012, Alexnet gathered these elements (deep neuronal networks, large data sets and GPUs) for the first time, with restless results. Each of these needed the other.

How Alexnet was created

At the end of the 2000s, Hinton graduate students at Toronto University were beginning to use GPU to train neuronal networks for the recognition of images and voice. His first successes came in voice recognition, but the success in the recognition of images would point to deep learning as a possible general purpose solution for AI. A student, Ilya Sutskever, believed that the performance of the neural networks would be climbed with the amount of data available, and the arrival of Imagenet provided the opportunity.

A milestone occurred in 2011, when Dannet, a neuronal network trained in GPU created by Dan CIREALMAN And others in the Jürgen Schmidhuber laboratory in Switzerland, they won 4 image recognition contests. However, these results were in smaller data sets and could not move the computer vision field. Imagenet, which was much bigger and more integral, was different. That same year, Sutskever convinced his fellow graduated student of Toronto Alex Krizhevsky, who had a great ability to expel the maximum performance of the GPUs, to train a convolutional neural network for Imagenet, with Hinton as principal researcher.

Alexnet used NVIDIA GPU with CUDA CUDA trained in the Imagenet data set. The CEO of Nvidia, Jensen Huang, was appointed 2024 CHM member for his contributions to computer graphics and IA chips.Douglas Fairbairn/Computing History Museum

Krizhevsky had already written the CUDA code for a convolutional neuronal network using the NVIDIA GPU CUDA-CONVNETtrained in the much smaller CIFFAR-10 image data set. Cuda-Convnet extended with multiple GPU support and other features and trained again in Imagenet. The training was done on a computer with two Nvidia cards in Krizhevsky’s room at his parents’ house. In the course of next year, he constantly modified the parameters of the network and recovered it until he achieved higher performance to his competitors. The network would finally be called Alexnet, after Krizhevsky. Geoff Hinton summarized the Alexnet project in this way: “Ilya thought we should do it, Alex did it and obtained the Nobel Prize. “

Krizhevsky, Sutskever and Hinton wrote a Document on Alexnet That was published in the fall of 2012 and presented by Krizhevsky at a computer vision conference in Florence, Italy, in October. The veteran computer vision researchers were not convinced, but Lecun, who was at the meeting, declared a turning point for AI. He was right. Before Alexnet, almost none of the main computer vision documents used neural networks. After that, almost all of them would.

Alexnet was just the beginning. In the next decade, neural networks would advance to synthesize credible human voices, overcome champions players and generate works of art, which culminate with the launch of Chatgpt in November 2022 by OpadaiA company co -founded by Sutskever.

Launch the Alexnet source code

In 2020, I contacted Krizhevsky to ask about the possibility of allowing ChM to release Alexnet’s source code, due to its historical importance. He connected to Hinton, who worked on Google at that time. Google owned Alexnet, after having acquired Dnnresearch, the company owned by Hinton, Sutskever and Krizhevsky. Hinton rolled the ball by connecting ChM to the right team on Google. ChM worked with Google team for five years to negotiate the launch. The team also helped us identify the specific version of the Alexnet source code to launch; There have been many Alexnet versions over the years. There are other code repositories called Alexnet in Github, but many of these are recreations based on the famous document, not on the original code.

CHM is proud to present the source code to the 2012 version of Alexnet, which transformed the field of artificial intelligence. You can access the source code in ChM github page.

This publication appeared originally in the Computer History Museum Blog.

This article was updated on March 25, 2025.

Expressions of gratitude

A special thanks to Geoffrey Hinton for providing his appointment and reviewing the text, Cade Metz and Alex Krizhevsky for additional clarifications, and David Bieber and the rest of the Google team for his work in obtaining the launch of the Source Code.