Michael Hu

I am a third-year PhD student at the NYU Center for Data Science, working with Kyunghyun Cho and Tal Linzen. I work on algorithms that optimize the training data of language models. I am supported by an NSF Graduate Research Fellowship.

In my spare time, I enjoy cooking, running, and playing basketball.

Previously, I completed a BSE at Princeton CS, where I spent two lovely years working with Karthik Narasimhan and Tom Griffiths.

news

Jul 17, 2024	New preprint: “The importance of human-scale language modeling for psycholinguistics.”
Nov 21, 2023	“Latent State Models of Training Dynamics” accepted to TMLR.
Dec 9, 2022	“Using natural language and program abstractions to instill human inductive biases in machines” received an Outstanding Paper Award at NeurIPS 2022! 🏅
Sep 1, 2022	Started a PhD at the NYU Center for Data Science as an NSF GRFP Scholar.
Oct 6, 2021	“Safe RL with Natural Language Constraints” accepted to NeurIPS 2021 as a spotlight presentation!

selected publications

2023

TMLR

Latent State Models of Training Dynamics

Michael Y. Hu, Angelica Chen, Naomi Saphra, and 1 more author

Transactions on Machine Learning Research, 2023

Website

2022

NeurIPS

Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines

Sreejan Kumar, Carlos G. Correa, Ishita Dasgupta, and 7 more authors

NeurIPS, 2022

Abs arXiv

Strong inductive biases give humans the ability to quickly learn to perform a variety of tasks. Although meta-learning is a method to endow neural networks with useful inductive biases, agents trained by meta-learning may sometimes acquire very different strategies from humans. We show that co-training these agents on predicting representations from natural language task descriptions and programs induced to generate such tasks guides them toward more human-like inductive biases. Human-generated language descriptions and program induction models that add new learned primitives both contain abstract concepts that can compress description length. Co-training on these representations result in more human-like behavior in downstream meta-reinforcement learning agents than less abstract controls (synthetic language descriptions, program induction without learned primitives), suggesting that the abstraction supported by these representations is key.

2021

NeurIPS

Safe Reinforcement Learning with Natural Language Constraints

Tsung-Yen Yang, Michael Hu, Yinlam Chow, and 2 more authors

NeurIPS, 2021

Abs arXiv Website

While safe reinforcement learning (RL) holds great promise for many practical applications like robotics or autonomous cars, current approaches require specifying constraints in mathematical form. Such specifications demand domain expertise, limiting the adoption of safe RL. In this paper, we propose learning to interpret natural language constraints for safe RL. To this end, we first introduce HazardWorld, a new multi-task benchmark that requires an agent to optimize reward while not violating constraints specified in free-form text. We then develop an agent with a modular architecture that can interpret and adhere to such textual constraints while learning new tasks. Our model consists of (1) a constraint interpreter that encodes textual constraints into spatial and temporal representations of forbidden states, and (2) a policy network that uses these representations to produce a policy achieving minimal constraint violations during training. Across different domains in HazardWorld, we show that our method achieves higher rewards (up to11x) and fewer constraint violations (by 1.8x) compared to existing approaches. However, in terms of absolute performance, HazardWorld still poses significant challenges for agents to learn efficiently, motivating the need for future work.