USA-NA-AIO Syllabus

• Markdown programming in Google Colab

  • There are questions that contestants shall write down their solutions in Google Colab text cells with markdown programming (such as math equations). Therefore, contestants need to know how to use it to text words, write snippets of code, and enter math formulae.

• Mathematical foundations for AI

  • Linear algebra (e.g., affine transformations, matrix decompositions, eigenvalues and eigenvectors)

  • Probability and statistics (e.g., Bayes' rule, Hoeffding's inequality)

  • Derivatives in multi-variable calculus

  • Convex optimization (e.g., gradient descent algorithm, duality)

• Basic coding

  • Python

  • NumPy

  • pandas

  • matplotlib.pyplot

  • seaborn

  • scikit-learn

• Machine learning​

  • Supervised learning (e.g., linear regression, logistic regression, support vector machine, decision trees, kNN, ensemble learning, bias-variance tradeoff, cross-validation, loss functions)

  • Unsupervised learning (e.g., k-means clustering, principal component analysis)

• Advanced coding for deep learning (PyTorch)​

  • PyTorch (In USA-NA-AIO, deep learning problems shall be programmed with Pytorch, not TensorFlow. This is consistent with IOAI's requirement and modern trend in academia and industry.)

• Deep learning foundation​

  • Multi-layer perceptron model​

  • Essential layers (e.g., affine transformation, batch normalization, dropout)

  • Forward propagation and backpropagation and their mathematical computations (by hand)

• Transformers (Note: Transformers is the foundation of many modern AI technologies. Therefore, contestants shall have very solid and deep understandings of transformers.)

  • Attention mechanisms​

  • Transformers architecture

  • Applications (e.g., natural language processing, vision transformers, graph neural networks)

• Natural language processing​

  • Tokenization​

  • Word embeddings

  • Transformers

  • Pre-training

  • Fine-tuning

• Computer vision and generative AI

  • Convolutional neural network​

  • Object detection

  • UNet

  • Autoencoder

  • Variational autoencoder

  • Generative adversarial network

  • Denoising diffusion probabilistic models

  • Stable diffusion

For the topics above, contestants should know both theory and programming.

For example, for theory, students need to know how to derive an estimator in a linear regression model and how to prove whether a matrix is a valid kernel (that is, positive definite).

For programming, contestants need to know how to build a fully-connected neural network from scratch and the reasoning behind each step.

A strong contestant should do well in both theory and programming. Knowing theory without knowing how to program (e.g., do not know how to use NumPy to build a principal component analysis (PCA) class from scratch) is not good. Knowing how to program without knowing theory (e.g., only know how to use scikit-learn PCA class as a black box and do not know how the PCA eigenvalue equation is established) is also not good.

To summarize, AI Olympiad is neither a pure math Olympiad nor a pure coding Olympiad. To do well in our AI Olympiad, you need to be do well in both math and coding.