Dimitris Bertsimas @dbertsim

MIT professor, analytics, optimizer, Machine Learner, entrepreneur, philatelist mit.edu/~dbertsim/ Belmont, MA Joined August 2017

Tweets

53
Followers

3K
Following

93
Likes

18

arXiv math.OC Optimization and Control @mathOCb

2 years ago

Dimitris Bertsimas, Georgios Margaritis: Global Optimization: A Machine Learning Approach arxiv.org/abs/2311.01742 arxiv.org/pdf/2311.01742

0 15 60 6K 28

JAMA Surgery @JAMASurgery

2 years ago

Interpretable machine learning methodologies are powerful tools to diagnose and remedy system-related bias in care, such as disparities in access to postinjury rehabilitation care. ja.ma/3P8Sdjc @hayfarani @dbertsim @AnthonyGebran @LMaurerMD

0 8 18 17K 5

Wes Gurnee @wesg52

2 years ago

This paper would not have been possible without my coauthors @NeelNanda5, Matthew Pauly, Katherine Harvey, @mitroitskii, and @dbertsim or all the foundational and inspirational work from @ch402, @boknilev, and many others! Read the full paper: arxiv.org/abs/2305.01610

2 3 44 4K 12

Wes Gurnee @wesg52

2 years ago

Precision and recall can also be helpful guides, and remind us that it should not be assumed a model will learn to represent features in an ontology convenient or familiar to humans.

2 1 23 4K 2

Download Image

Wes Gurnee @wesg52

2 years ago

While we found tons of interesting neurons with sparse probing, it requires careful follow up analysis to draw more rigorous conclusions. E.g., athlete neurons turn out to be more general sport neurons when analyzing max average activating tokens.

1 2 22 4K 1

Download Image

Wes Gurnee @wesg52

2 years ago

What happens with scale? We find representational sparsity increases on average, but different features obey different scaling dynamics. In particular, quantization and neuron splitting: features both emerge and split into finer grained features.

1 3 19 4K 1

Download Image

Wes Gurnee @wesg52

2 years ago

Results in toy models from @AnthropicAI and @ch402 suggest a potential mechanistic fingerprint of superposition: large MLP weight norms and negative biases. We find a striking drop in early layers in the Pythia models from @AiEleuther and @BlancheMinerva.

1 3 30 5K 3

Download Image

Wes Gurnee @wesg52

2 years ago

Early layers seem to use sparse combinations of neurons to represent many features in superposition. That is, using the activations of multiple polysemantic neurons to boost the signal of the true feature over all interfering features (here “social security” vs. adjacent bigrams)

1 4 37 5K 3

Download Image

Wes Gurnee @wesg52

2 years ago

But what if there are more features than there are neurons? This results in polysemantic neurons which fire for a large set of unrelated features. Here we show a single early layer neuron which activates for a large collection of unrelated n-grams.

1 3 38 7K 3

Download Image

Wes Gurnee @wesg52

2 years ago

Neural nets are often thought of as feature extractors. But what features are neurons in LLMs actually extracting? In our new paper, we leverage sparse probing to find out arxiv.org/abs/2305.01610. A 🧵:

11 127 695 227K 424

Download Image

Wes Gurnee @wesg52

2 years ago

One large family of neurons we find are “context” neurons, which activate only for tokens in a particular context (French, Python code, US patent documents, etc). When deleting these neurons the loss increases in the relevant context while leaving other contexts unaffected!

3 12 112 91K 23

Download Image

Dimitris Bertsimas @dbertsim

3 years ago

Reducing overall deaths and increase access for patients waiting for lung transplants. prnewswire.com/news-releases/…

0 0 14 2K 1

Dimitris Bertsimas @dbertsim

3 years ago

As part of HIAS and together with Professor Georgios Stamou from NTUA, Greece we are offering a course on Universal AI (in English, free of charge) aicourse2023.hellenic-ias.org on July 3-5, 2023 in Athens, Greece. Prospective participants can declare their interest in the website.

0 3 19 2K 1

Ryan Cory-Wright @RyanCoryWright

3 years ago

Delighted to share that our paper "A new perspective on low-rank optimization" has just been accepted for publication by Math Programming! Valid & often strong lower bounds on low-rank problems via a generalization of the perspective reformulation from mixed-integer optimization

Ryan Cory-Wright @RyanCoryWright

4 years ago

2 0 15 0 0

1 1 16 0 0

Ryan Cory-Wright @RyanCoryWright

3 years ago

📢New preprint alert! arxiv.org/abs/2303.07695 We use sampling schemes and clustering to improve the scalability of deterministic Bender's decomposition on data-driven network design problems, while maintaining optimality. w/ @dbertsim, Jean Pauphilet, and Periklis Petridis

1 2 19 3K 4

Dimitris Bertsimas @dbertsim

3 years ago

The paper presents a novel holistic deep learning framework that improves accuracy, robustness, sparsity, and stability over standard deep learning models, as demonstrated by extensive experiments on both tabular and image data sets. arxiv.org/abs/2110.15829

0 1 35 4K 3

Dimitris Bertsimas @dbertsim

3 years ago

My book with David Gamarnik “Queueing Theory: Classical and Modern Methods” was published. It was a long journey that lasted two decades but both of us are delighted with the journeys completion. For more details see dynamic-ideas.com/books/quueing-…

5 22 185 0 21

Download Image

npj Digital Medicine @npjDigitalMed

3 years ago

The Holistic AI in Medicine (HAIM) framework from @dbertsim et al. in @AIHealthMIT is a pipeline to receive multimodal patient data + use generalizable pre-processing + #machinelearning modelling stages adaptable to multiple health related tasks. nature.com/articles/s4174…

0 4 10 0 2

Download Image

Stefanos Kechagias @StefanosKe

3 years ago

If you are into #MachineLearning and #Statistics check this out. I would also highly recommend the Machine Learning under a modern optimization lens book by @dbertsim and Dunn. Here are two two teaser must watch imo youtube videos youtu.be/7w9aRrYgGEs youtu.be/jJgdJaCo568

Christoph Molnar 🦋 christophmolnar.bsky.social @ChristophMolnar

3 years ago

10 104 545 0 380

0 2 2 0 1

MIT Jameel Clinic for AI & Health @AIHealthMIT

3 years ago

pipelines that can consistently be applied to train multimodal AI/ML systems & outperform their single-modality counterparts has remained challenging. #JameelClinic faculty lead @dbertsim, executive director @ifuentes3, postdoc @lrsoenksen, Yu Ma, @CynthiaZeng1,... (2/4)