Markdown programming in Google Colab
Some questions require contestants to write their solutions in Google Colab text cells using Markdown (for example, to enter mathematical equations). Therefore, contestants need to know how to enter text, write code snippets, and typeset mathematical formulas.
Mathematical foundations for AI
- Linear algebra (e.g., affine transformations, matrix decompositions, eigenvalues, and eigenvectors)
- Probability and statistics (e.g., Bayes' rule and Hoeffding's inequality)
- Derivatives in multivariable calculus
- Convex optimization (e.g., gradient descent algorithms and duality)
Basic coding
- Python
- NumPy
- pandas
- matplotlib.pyplot
- seaborn
- scikit-learn
Machine learning
- Supervised learning (e.g., linear regression, logistic regression, support vector machine, decision trees, kNN, ensemble learning, bias-variance tradeoff, cross-validation, loss functions)
- Unsupervised learning (e.g., k-means clustering, principal component analysis)
Advanced coding for deep learning (PyTorch)
In USA-NA-AIO, deep learning problems must be programmed with PyTorch rather than TensorFlow. This is consistent with IOAI requirements and current trends in academia and industry.
Deep learning foundations
- Multilayer perceptron models
- Essential layers (e.g., affine transformations, batch normalization, and dropout)
- Forward propagation, backpropagation, and their mathematical computations (by hand)
Transformers
Note: Transformers are the foundation of many modern AI technologies. Therefore, contestants must have a thorough understanding of transformers.
- Attention mechanisms
- Transformer architecture
- Applications (e.g., natural language processing, vision transformers, and graph neural networks)
Natural language processing
- Tokenization
- Word embeddings
- Transformers
- Pre-training
- Fine-tuning
Computer vision and generative AI
- Convolutional neural network
- Object detection
- UNet
- Autoencoder
- Variational autoencoder
- Generative adversarial network
- Denoising diffusion probabilistic models
- Stable diffusion
For the topics above, contestants should understand both theory and programming.
For example, in theory, students need to know how to derive an estimator in a linear regression model and how to prove whether a matrix is a valid kernel (i.e., positive definite).
For programming, contestants need to know how to build a fully connected neural network from scratch and understand the reasoning behind each step.
A strong contestant should perform well in both theory and programming. Knowing theory without knowing how to program (e.g., not knowing how to use NumPy to build a principal component analysis (PCA) class from scratch) is insufficient. Likewise, knowing how to program without understanding the theory (e.g., only knowing how to use the scikit-learn PCA class as a black box without understanding how the PCA eigenvalue problem is derived) is also insufficient.
In summary, the AI Olympiad is neither a pure math Olympiad nor a pure coding Olympiad. To perform well, you need to excel in both mathematics and programming.