How To Prepare
How to prepare
Teams at the IOAI will be expected to have knowledge of the following areas of AI, programming languages, and frameworks.
Python
Python is the main programming language used in the development of machine learning models. It is a high-level programming language, preferred by AI practitioners for its intuitive syntax and wide community support, among other things.
Check out a Python tutorial which uses Google Colab, an interactive computing platform, where you can execute Python code directly in a browser. Google Colab supports simple code like the one presented in the tutorial, as well as much more complex code for building and training machine learning models.
PyTorch
PyTorch is a widely used machine learning framework, based on Python. It provides high-level functionalities like building a machine learning model from pre-configured blocks, loading data from files of various formats, executing model training with just a few lines, and many more.
Notice that at the top of every tutorial page, there is a “Run in Google Colab” link that opens up the tutorial in Google Colab and allows you to interact with the code.
Machine Learning
While AI has evolved a lot in recent years, the fundamental principles of machine learning remain unchanged, so this course from 2018 is still a great resource. Focus specifically on lectures 1 through 5 and 11 through 13.
Deep Learning
Deep learning is what makes AI possible. This Coursera course from Andrew Ng and colleagues will give you the best introduction to the subject.
Once you’ve gained some skill with PyTorch (see below), also check out this interactive textbook on DL: https://d2l.ai/.
Natural Language Processing
The “Speech and Language Processing” textbook by Dan Jurafsky and James H. Martin is the Bible of NLP. Focus on Part 1.
Word Embeddings
This is a practical implementation of the word2vec algorithm you learn about in Chapter 6 of the textbook above. Word embeddings are a prerequisite for any machine learning performed over text. Use this tutorial to better understand the concept of word embeddings, and the use of Pytorch and Google Colab for building and training machine learning models.
Transformers
The main model architecture used in both text and image processing these days is the transformer architecture. It revolutionized the field of NLP because it is highly parallelizable and in that way able to consume much larger amounts of data than was ever possible before.
The main Python library for working with this architecture is called transformers and many existing models using the architecture can be found in the HuggingFace.co hub.
Generative Modeling
Generative AI is what really caught the eye of the public and brought AI into production overnight. This repository by Andrej Karpathy is a great practical introduction to the workings of generative language models.
Computer Vision
A course on deep learning for CV from 2015, good for getting a sense of the fundamentals in the field.
Recent Developments in CV
The Deep Learning 1 course here covers key concepts in recent CV research like ResNet, adversarial attacks and the vision transformer.