everything is always wip and I am [foolish](https://news.stanford.edu/2005/06/12/steve-jobs-2005-graduates-stay-hungry-stay-foolish/) (or at least try to be). I took down this website in 2025 and am gradually reuploading; let me know at [[about me#^socials]] if links break. check out [[default math notation]] for a lot of this to make sense. # statistics and classical machine learning - [[tutorial on variational autoencoders]] - ==[[Hebbian learning]]== - ==[[Covers theorem]]== - [[machine learning]] - [[reinforcement learning]] and [[reinforcement learning algorithm]] - [[generative model]] - [[universality of ridge regression]] - [[proof of asymptotic universality of ridge regression]] - **[[rotation trick|rotational invariance of Gaussians]]** - [[generalized linear model]] - [[exponential family]] - [[kernel method]] - **[[geometry of random variables]]** - [[the Fisher information matrix is the Hessian of the KL divergence]] - [[partition function]] - [[visualization of lasso and ridge behaviour]] - [[stochastic domination]] - [[artificial neural network]] - [[Fisher score]] ## Hopfield networks ![[tutorial on Hopfield networks]] ![[gradient descent#gradient descent]] # deep learning and optimization - [[continuous optimization]] - ==[[autodiff]]== - [[optimization]] - [[convex program]] - *[[NumPy]]* - ==[[Lagrangian duality]]== - [[backpropagation]] - [[deep learning]] - [[double descent]] - [[2021JumperEtAlHighlyAccurateProtein|AlphaFold 2]] - [[video]] - [[image generation]] - [[code generation]] - [[audio tasks]] - [[context-free text representations]] - [[efficient deep learning]] - [[finetuning]] - [[quantization]] - [[efficient training]] - [[data parallelism]] ## scaling - *[[scaling law]]* - [[capabilities of large language models]] - [[inverse scaling]] - [[2023WeiEtAlInverseScalingCan|Inverse scaling can become U-shaped]] ## AI safety - [[AI safety]] - [[we need to get superhuman intelligence right]] - [[Goodharts law]] - [[reward model]] - [[intent alignment]] - [[adversarial attack]] - [[2023AkashOvertonWindowWidens|The Overton Window widens]] - [[prosaic superhuman intelligence]] ## interpretability - [[interpretability]] (please try out TransformerLens) - [[transformer interpretability]] - [[interpretability]] - [[2022CovertEtAlExplainingRemovingUnified|Explaining by Removing]] ![[2023WangEtAlConceptAlgebraTextcontrolled#concept algebra]] ## Transformers and language models - [[reward model]] (includes RLHF, RLAIF finetuning) - [[decoder-only transformer]] - [[natural language processing]] - [[large language model]] - [[capabilities of large language models]] - [[interpretability]] - [[concept vector]] - [[2017VaswaniEtAlAttentionAllYou|Transformer paper and resources]] - [[2022BaiEtAlConstitutionalAIHarmlessness|Constitutional AI]] - [[the llama craze]] - [[compute-optimal training]] (20 tokens per parameter) ## deep learning theory - [[deep learning]] - [[resolvent]] # math - ==[[Gram-Schmidt procedure]]== - [[derivative]] - [[Taylor approximation]] - [[matrix multiplication]] - [[implementation of matrix multiplication]] - [[linearly separable]] - [[distance to hyperplane]] - [[continuous]] - [[principal component|PCA]] - [[singular value decomposition|SVD]] - [[Householder transformation]] - [[circular law]] # kernels - [[kernel]] - [[kernel method]] - [[kernel smoothing]] - [[reproducing kernel Hilbert space]] ![[orthogonal projection#orthogonal projection]] # philosophy - [[Hamming question]] - [[psychology and epistemology]] - [[pedagogy]] - [[pedagogical communication]] - [[abstract]] - [[personal knowledge management]] - [[PKM tools]] # programming - [[web development]] # course notes See [[my courses]] # misc - *[[Internet garden]]* - [[blogs and personal sites]] - *[[information retrieval and open resources and search engines]]* - [[setting up remote desktop on vast ai]] - [[typesetting]] - [[research]] - [[research tools]] - [[research advice and motivation]] (from others, not me)