SysCon machine learning reading group

A weekly machine learning reading group. Each session is 1h. Sessions are typically on Thursdays. Each week we take turn to pick a paper to read, which we then go through together and discuss.

Contact person: Daniel Gedon.

Detailed internal information.

Paper List 2024

Week	Paper	Venue
3	Open Sesame! Universal black box jailbreaking of large language models	arxiv, 2023
4	SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models	EMNLP, 2023
5	Mass-Editing Memory in a Transformer	ICLR, 2023
6	Flow Matching for Generative Modeling	ICLR, 2023
7	Diffusion-GAN: Training GANs with Diffusion	ICLR, 2023
8	Transformers learn in-context by gradient descent	ICML, 2023
9	Learning Unforeseen Robustness from Out-of-distribution Data Using Equivariant Domain Translator	ICML, 2023
10	Segment Anything	arxiv, 2023
11	Learning Efficient and Robust Ordinary Differential Equations via Invertible Neural Networks	ICML, 2022
12	Fourier Neural Operator for Parametric Partial Differential Equations	ICLR, 2021
13	(Easter)	-
14	LoRA: Low-Rank Adaptation of Large Language Models	ICLR 2022
15	Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction	arxiv, 2024
16	Fine-Tuning Language Models with Just Forward Passes	NeurIPS, 2023

Paper List 2023

Week	Paper	Venue
3	Out-Of-Distribution Detection Is Not All You Need	AAAI 2023
4	Gradient Descent Happens in a Tiny Subspace	arxiv, 2018
5	Everything is Connected: Graph Neural Networks	arxiv, 2023
6	The Forward-Forward Algorithm: Some Preliminary Investigations	arxiv, 2022
7	Neural Networks Trained with SGD Learn Distributions of Increasing Complexity	arxiv, 2022
8	Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting	NeurIPS 2020
9	Unsupervised Out-of-Distribution Detection with Diffusion Inpainting	arxiv, 2023
10	How Powerful are Graph Neural Networks?	ICLR 2019
11	Why AI is Harder Than We Think	arxiv, 2021
12	Resurrecting Recurrent Neural Networks for Long Sequences	arxiv, 2023
13	PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics	KDD 2021
14	Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis	MIDL 2023
15	(Easter)	-
16	A Deep Conjugate Direction Method for Iteratively Solving Linear Systems	arxiv, 2022
17	Assaying Out-Of-Distribution Generalization in Transfer Learning	NeurIPS 2022
18	I2SB: Image-to-Image Schrödinger Bridge	ICML 2023
19	High-Resolution Image Synthesis with Latent Diffusion Models	CVPR 2022
20	-	-
21	Consistency Models	ICML 2023
22	Simplified State Space Layers for Sequence Modeling	ICLR 2023
23	Mechanism of Feature Learning in Deep Fully Connected Networks and Kernel Machines that Recursively Learn Features	arxiv, 2022
24	Transport with Support: Data-Conditional Diffusion Bridges	arxiv, 2023
	-
	(Summer break)
	-
34	Loss Landscapes are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent	ICLR 2023
35	A Law of Data Separation in Deep Learning	PNAS, 2023
36	-	-
37	Efficient Formal Safety Analysis of Neural Networks	NeurIPS 2018
38	Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style	NeurIPS 2021
39	-	-
40	-	-
41	-	-
42	-	-
43	-	-
44	A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle	EAAMO 2021
45	Considerations for Addressing Bias in Artificial Intelligence for Health Equity	npj Digital Medicine, 2023
46	Large Language Models Propagate Race-based Medicine	npj Digital Medicine, 2023
47	Algorithmic Fairness In Artificial Intelligence For Medicine And Healthcare	Nature Biomedical Engineering, 2023
48	Reclaiming AI as a theoretical tool for cognitive science	PsyArXiv
49	On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? parrot emoji	FAccT, 2021

Paper List 2022

Week	Paper	Venue
7	Energy-based Out-of-distribution Detection	NeurIPS 2020
8	Deep Learning Through the Lens of Example Difficulty	NeurIPS 2021
9	Transformers Can Do Bayesian Inference	ICLR 2022
10	The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers	ICLR 2021
11	Comparing Elementary Cellular Automata Classifications with a Convolutional Neural Network	ICAART 2021
12	Random Synaptic Feedback Weights Support Error Backpropagation for Deep Learning	Nature Communications, 2016
13	Selective Classification for Deep Neural Networks	NeurIPS 2017
14	Uncalibrated Models Can Improve Human-AI Collaboration	arxiv, 2022
15	Conformalized Quantile Regression	NeurIPS 2019
16	-	-
17	-	-
18	As simple as ABC: a straightforward method for evaluating performance of machine learning models for classification	-
19	-	-
20	Open-Set Recognition: a Good Closed-Set Classifier is All You Need?	ICLR 2022
21	When are Bayesian model probabilities overconfident?	arxiv, 2020
22	Shaking the foundations: delusions in sequence models for interaction and control	arxiv, 2021
23	Weakly-Supervised Disentanglement Without Compromises	ICML 2020
24	Greedy Bayesian Posterior Approximation with Deep Ensembles	TMLR, 2022
25	Linear Time Sinkhorn Divergences using Positive Features	NeurIPS 2020
	-
	(Summer break)
	-
35	On the Information Bottleneck Theory of Deep Learning	ICLR 2018
36	Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection	NeurIPS 2021
37	Learning to learn by gradient descent by gradient descent	NeurIPS 2016
38	Adversarial Examples Are Not Bugs, They Are Features	NeurIPS 2019
39	Generative Modeling by Estimating Gradients of the Data Distribution	NeurIPS 2019
40	Mechanistic models versus machine learning, a fight worth fighting for the biological community?	Biology Letters, 2018
41	RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection	NeurIPS 2022
42	Pseudo-Spherical Contrastive Divergence	NeurIPS 2021
43	Multi-scale Feature Learning Dynamics: Insights for Double Descent	ICML 2022
44	Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution	ICLR 2022
45	Learning deep representations by mutual information estimation and maximization	ICLR 2019
46	Prioritized Training on Points that are learnable, Worth Learning, and Not Yet Learnt	ICML 2022
47	Collocation based training of neural ordinary differential equations	Statistical Applications in Genetics and Molecular Biology, 2021
48	Denoising Diffusion Implicit Models	ICLR 2021
49	Physics-Informed Neural Networks for Cardiac Activation Mapping	Frontiers in Physics, 2020
50	Continuous Time Analysis of Momentum Methods	JMLR, 2020

Paper List 2021

Week	Paper	Venue
2	Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention	ICML 2020
3	Getting a CLUE: A Method for Explaining Uncertainty Estimates	ICLR 2021
4	No MCMC for Me: Amortized Sampling for Fast and Stable Training of Energy-Based Models	ICLR 2021
5	Meta Pseudo Labels	CVPR 2021
6	On the Origin of Implicit Regularization in Stochastic Gradient Descent	ICLR 2021
7	Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision	ICML 2021
8	Neural Relational Inference for Interacting Systems	ICML 2018
9	Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations	AISTATS 2022
10	Unsupervised Learning of Visual Features by Contrasting Cluster Assignments	NeurIPS 2020
11	Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability	ICLR 2021
12	Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling	NeurIPS 2020
13	Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling	ICML 2021
14	Q-Learning in enormous action spaces via amortized approximate maximization	arxiv, 2020
15	Learning Mesh-Based Simulation with Graph Networks	ICLR 2021
16	-	-
17	Stiff Neural Ordinary Differential Equations	arxiv, 2021
18	PixelTransformer: Sample Conditioned Signal Generation	ICML 2021
19	-	-
20	-	-
21	-	-
22	-	-
23	Meta-Learning Bidirectional Update Rules	ICML 2021
24	Deconstructing the Regularization of BatchNorm	ICLR 2021
	-
	(Summer break)
	-
34	DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation	CVPR 2019
35	Differentiable Particle Filtering via Entropy-Regularized Optimal Transport	ICML 2021
36	Revisiting the Calibration of Modern Neural Networks	NeurIPS 2021
37	NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis	ECCV 2020
38	Hierarchical VAEs Know What They Don't Know	ICML 2021
39	Information Dropout: Learning Optimal Representations Through Noisy Computation	arxiv, 2016
40	SMD-Nets: Stereo Mixture Density Networks	CVPR 2021
41	Learning to Simulate Complex Physics with Graph Networks	ICML 2020
42	Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets	ICLR Workshops 2021
43	Deep Classifiers with Label Noise Modeling and Distance Awareness	TMLR, 2022
44	-	-
45	Transferring Inductive Biases through Knowledge Distillation	arxiv, 2020
46	Masked Autoencoders Are Scalable Vision Learners	arxiv, 2021
47	On the Importance of Gradients for Detecting Distributional Shifts in the Wild	NeurIPS 2021
48	An Information-theoretic Approach to Distribution Shifts	NeurIPS 2021
49	Periodic Activation Functions Induce Stationarity	NeurIPS 2021
50	Efficiently Modeling Long Sequences with Structured State Spaces	ICLR 2021

Paper List 2020

Week	Paper	Venue
2	Z-Forcing: Training Stochastic Recurrent Networks	NeurIPS 2017
3	Multiplicative Interactions and Where to Find Them	ICLR 2020
4	A Primal-Dual link between GANs and Autoencoders	NeurIPS 2019
5	Modelling heterogeneous distributions with an Uncountable Mixture of Asymmetric Laplacians	NeurIPS 2019
6	Uncertainty Decomposition in Bayesian Neural Networks with Latent Variables	arxiv, 2017
7	Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning	ICML 2018
8	Convolutional Conditional Neural Processes	ICLR 2020
9	Bayesian Deep Learning and a Probabilistic Perspective of Generalization	arxiv, 2020
10	Batch Normalization Biases Deep Residual Networks Towards Shallow Paths	NeurIPS 2020
11	-	-
12	Conservative Uncertainty Estimation By Fitting Prior Networks	ICLR 2020
13	Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning	ICLR 2020
14	Normalizing Flows: An Introduction and Review of Current Methods	TPAMI, 2020
15	Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration	NeurIPS 2019
16	How Good is the Bayes Posterior in Deep Neural Networks Really?	ICML 2020
17	-	-
18	-	-
19	Stable Neural Flows	arxiv, 2020
20	Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One	ICLR 2020
21	-	-
22	BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning	ICLR 2020
23	Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors	ICML 2020
24	End-to-End Object Detection with Transformers	ECCV 2020
25	Joint Training of Variational Auto-Encoder and Latent Energy-Based Model	CVPR 2020
	-
	(Summer break)
	-
36	Denoising Diffusion Probabilistic Models	NeurIPS 2020
37	Gated Linear Networks	arxiv, 2020
38	Uncertainty Estimation Using a Single Deep Deterministic Neural Network	ICML 2020
39	Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness	NeurIPS 2020
40	-	-
41	Satellite Conjunction Analysis and the False Confidence Theorem	2019
42	Implicit Gradient Regularization	ICLR 2021
43	-	-
44	-	-
45	Approximate Inference Turns Deep Networks into Gaussian Processes	NeurIPS 2019
46	VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models	ICLR 2021
47	Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images	ICLR 2021
48	Rethinking Attention with Performers	ICLR 2021
49	-	-
50	Dissecting Neural ODEs	NeurIPS 2020
51	Score-Based Generative Modeling through Stochastic Differential Equations	ICLR 2021

Paper List 2019

Week	Paper	Venue
2	Neural Processes	ICML Workshops 2018
3	How Does Batch Normalization Help Optimization?	NeurIPS 2018
4	An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling	arxiv, 2018
5	A Complete Recipe for Stochastic Gradient MCMC	NeurIPS 2015
6	Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models	NeurIPS 2018
7	A Simple Baseline for Bayesian Uncertainty in Deep Learning	NeurIPS 2019
8	Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks	ICML 2019
9	Language Models are Unsupervised Multitask Learners	2019
10	Coupled Variational Bayes via Optimization Embedding	NeurIPS 2018
11	A recurrent neural network without chaos	ICLR 2017
12	Evaluating model calibration in classification	AISTATS 2019
13	Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling	ICLR 2019
14	Stochastic Gradient Descent as Approximate Bayesian Inference	JMLR, 2017
15	-	-
16	-	-
17	-	-
18	Visualizing the Loss Landscape of Neural Nets	NeurIPS 2018
19	-	-
20	-	-
21	-	-
22	Attention Is All You Need	NeurIPS 2017
23	LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving	CVPR 2019
	-
	(Summer break)
	-
40	Trellis Networks for Sequence Modeling	ICLR 2019
41	Variational Inference with Normalizing Flows	ICML 2015
42	Improving Variational Inference with Inverse Autoregressive Flow	NeurIPS 2016
43	Neural Autoregressive Flows	ICML 2018
44	-	-
45	Weight Uncertainty in Neural Networks	ICML 2015
46	Learning nonlinear state-space models using deep autoencoders	CDC 2018
47	Learning Latent Dynamics for Planning from Pixels	ICML 2019
48	Dream to Control: Learning Behaviors by Latent Imagination	ICLR 2020

Paper List 2018

Week	Paper	Venue
38	Gaussian Process Behaviour in Wide Deep Neural Networks	ICLR 2018
39	Neural Autoregressive Flows	ICML 2018
40	-	-
41	On gradient regularizers for MMD GANs	NeurIPS 2018
42	-	-
43	Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes	ICLR 2019
44	Conditional Neural Processes	ICML 2018
45	The Lottery Ticket Hypothesis: Finding Small, Trainable Neural Networks	ICLR 2019
46	Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)	ICML 2018
47	Stable Recurrent Models	ICLR 2019
48	A Probabilistic U-Net for Segmentation of Ambiguous Images	NeurIPS 2018
49	Evidential Deep Learning to Quantify Classification Uncertainty	NeurIPS 2018
50	Neural Ordinary Differential Equations	NeurIPS 2018