Active learning notes

Papers on Active learning

Methods/Algos

1. uncertainty-based sampling: least confident (Lewis and Catlett), max margin and max entropy

2. committee-based algorithms: vote entropy, consensus entropy and max disagreement (Cohn et al.)

3. multilabel strategies: SVM binary minimum (Brinker), max loss, mean max loss, (Li et al.) MinConfidence, MeanConfidence, MinScore, MeanScore (Esuli and Sebastiani)

4. expected error reduction: binary and log loss (Roy and McCallum)

5. Bayesian optimization: probability of improvement, expected improvement and upper confidence bound (Snoek et al.)

6. batch active learning: ranked batch-mode sampling (Cardoso et al.)

7. information density framework (McCallum and Nigam)

8. stream-based sampling (Atlas et al.)

9. active regression with max standard deviance sampling for Gaussian processes or ensemble regressors

How to estimate uncertainty in Deep Learning networks

• Excellent tutorial from AGW on Bayesian Deep Learning

• This tut is inspired by his publication Bayesian Deep Learning and a Probabilistic Perspective of Generalization

• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (Gal and Ghahramani, 2016) This describes Monte-Carlo Dropout, a way to estimate uncertainty through stochastic dropout at test time

• Bayesian Uncertainty Estimation for Batch Normalized Deep Networks (Teye et al. 2018) This describes Monte-Carlo BatchNorm, a way to estimate uncertainty through random batch norm parameters at test time

• Bayesian Deep Learning and a Probabilistic Perspective of Generalization (Gordon Wilson and Izmailov, 2020) Presentation of multi-SWAG a mix between VI and Ensembles.

• Advances in Variational inference (Zhang et al, 2018) Gives a quick introduction to VI and the most recent advances.

• A Simple Baseline for Bayesian Uncertainty in Deep Learning (Maddox et al. 2019) Presents SWAG, an easy way to create ensembles.

Bayesian active learning

• Deep Bayesian Active Learning with Image Data (Gal and Islam and Ghahramani, 2017) Fundamental paper on how to do Bayesian active learning.

• Sampling bias in active learning (Dasgupta 2009) Presents sampling bias and how to solve it by combining heuristics and random selection.

• Bayesian Active Learning for Classification and Preference Learning (Houlsby et al. 2011) Fundamental paper on one of the main heuristic BALD.

GP and active learning

• Exploration of lattice Hamiltonians for functional and structural discovery via Gaussian process-based exploration–exploitation

• Active Learning for Deep Gaussian Process Surrogates

• Actively learning GP dynamics

General

• Review paper: https://arxiv.org/abs/2009.00236

• Book on Active learning: https://www.manning.com/books/human-in-the-loop-machine-learning

• “Towards Robust Deep Active Learning for Scientific Computing” https://arxiv.org/abs/2201.12632

• Deep bayesian active learning + image https://arxiv.org/abs/1703.02910

• A Comparative Survey of Deep Active Learning: https://arxiv.org/abs/2203.13450

Tutorials/worksops

• Google’s Active Learning Playground: This is a python module for experimenting with different active learning algorithms.

• deep-active-learning: Python implementations of the following active learning algorithms

• PyTorch Active Learning: Library for common Active Learning methods

• active-learning-workshop:KDD 2018 Hands-on Tutorial: Active learning and transfer learning at scale with R and Python. PDF

Codebases

1. Modal

Built on scipy+sklearn modAL-python/modAL

2. gpax

Gaussian processes + active learning! Very new, uses JAX ❤️ + numpyro Problem: unstable/

ziatdinovmax/gpax

3. Scikit-activeml https://www.preprints.org/manuscript/202103.0194/v1 https://pypi.org/project/scikit-activeml/

Also built on scipy+sklearn

3. Baal

A ” Bayesian active learning library” Built on pytorch Seems like focus on images https://baal.readthedocs.io/en/latest/notebooks/compatibility/sklearn_tutorial/ https://baal.readthedocs.io/en/latest/ https://arxiv.org/abs/2006.09916

4. DeepAL ej0cl6/deep-active-learning

5. adaptive

Adaptive sampling technique for ‘learning’ functional representation of the data https://adaptive.readthedocs.io/en/latest/index.html python-adaptive/adaptive

6. AliPy Agnostic of pytorch/sklearn/tflow NUAA-AL/alipy

7. libact Features the active learning by learning meta-strategy that allows the machine to automatically learn the best strategy on the fly.

Works with sklearn

Hasnt been updated in 2 years ntucllab/libact

8. botorch A bayesian optimization framework built on pytorch https://botorch.org/