Sean Welleck's Home Page
Home | Podcast
E-mail: wellecks {@ | at} uw.edu
Links: [google scholar][twitter][blog][github]
I am a Postdoctoral Scholar at the University of Washington and Young Investigator at AI2, working with Yejin Choi.
Previously, I earned a PhD at New York University, advised by Kyunghyun Cho and Zheng Zhang.
My research focuses on machine learning methods for generating and reasoning with natural language. This includes:
- Developing and understanding learning [1, 2, 3], decoding [1, 2, 3, 4], and evaluation [1, 2] methods for neural sequence generation.
- Bridging symbolic structure and neural methods in commonsense [1, 2, 3, 4] and mathematical [1, 2, 3] reasoning.
My long-term goal is enabling high-impact applications in scientific discovery and education [1, 2] through advances in these areas.
News
Preprints
- NaturalProver: Grounded Mathematical Proof Generation with Language Models
S. Welleck*, J. Liu*, X. Lu, H. Hajishirzi, Y. Choi.
arXiv preprint 2022.
- Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
J. Jung, L. Qin, S. Welleck, F. Brahman, C. Bhagavatula, R. Le Bras, Y. Choi.
arXiv preprint 2022.
- COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics
L. Qin, S. Welleck, D. Khasabi, Y. Choi.
arXiv preprint 2022.
[code]
Publications
- NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics
X. Lu, S. Welleck, P. West, L. Jiang, J. Kasai, D. Khasabi, R. Le Bras, L. Qin, Y. Yu, R. Zellers, N. Smith, Y. Choi.
NAACL 2022.
[code]
- Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
P. West, C. Bhagavatula, J. Hessel, J. Hwang, L. Jiang, R. Le Bras, X. Lu, S. Welleck, Y. Choi.
NAACL 2022.
[code]
- Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
D. Khasabi, S. Lyu, S. Min, L. Qin, K. Richardson, S. Singh, S. Welleck, H. Hajishirzi, T. Khot, A. Sabharway, Y. Choi.
NAACL 2022.
- Generated Knowledge Prompting for Commonsense Reasoning
J. Liu, A. Liu, X. Lu, S. Welleck, P. West, R. Le Bras, Y. Choi, H. Hajishirzi.
ACL 2022.
[code]
- Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics
S. Welleck, P. West, J. Cao, Y. Choi.
AAAI 2022.
[code][slides][talk]
- Towards Grounded Natural Language Proof Generation
S. Welleck, J. Liu, J. Han, Y. Choi.
MathAI4Ed Workshop at NeurIPS 2021 (Contributed Talk).
[poster][slides]
- NaturalProofs: Mathematical Theorem Proving in Natural Language
S. Welleck, J. Liu, R. Le Bras, H. Hajishirzi, Y. Choi, K. Cho.
NeurIPS 2021 Datasets and Benchmarks (Oral (Top 1%)).
[data/code][talk]
- An Information Divergence Measure Between Neural Text and Human Text
K. Pillutla, S. Swayamdipta, R. Zellers, J. Thickstun, S. Welleck, Y. Choi, Z. Harchaoui.
NeurIPS 2021 (Oral, Outstanding Paper Award (Top 0.1%)).
[code][press]
- Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral
L. Liu, K. Pillutla, S. Welleck, S. Oh, Y. Choi, Z. Harchaoui.
NeurIPS 2021.
- Mode recovery in neural autoregressive sequence modeling
I. Kulikov, S. Welleck, K. Cho.
SPNLP 2021.
[code]
- Order and Learning in Sequential Neural Structured Prediction
S. Welleck.
PhD Thesis, New York University.
[slides]
- MLE-guided parameter search for task loss minimization in neural sequence modeling
S. Welleck, K. Cho.
AAAI 2021.
[code][poster][talk]
- Consistency of a Recurrent Language Model With Respect to Incomplete Decoding
S. Welleck*, I. Kulikov*, J. Kim, R. Pang, K. Cho.
EMNLP 2020.
[code][talk]
- A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models
E. Mansimov, A. Wang, S. Welleck, K. Cho.
arXiv pre-print 2020.
[code]
- Making Inconsistent Dialogue Unlikely with Unlikelihood Training
M. Li, S. Roller, I. Kulikov, S. Welleck, Y.L. Boureau, K. Cho, J. Weston.
ACL 2020.
- Neural Text Generation with Unlikelihood Training
S. Welleck*, I. Kulikov*, S. Roller, E. Dinan, K. Cho, J. Weston.
ICLR 2020.
[code]
- Non-Monotonic Sequential Text Generation
S. Welleck, K. Brantley, H. Daume III, K. Cho.
ICML 2019.
[code] [slides] [poster]
- Sequential Graph Dependency Parser
S. Welleck, K. Cho.
RANLP 2019.
[slides]
- Dialogue Natural Language Inference
S. Welleck, J. Weston, A. Szlam, K. Cho.
ACL 2019.
[dataset][poster][press]
- Loss Functions for Multiset Prediction
S. Welleck, Z. Yao, Y. Gai, J. Mao, Z. Zhang, K. Cho.
NeurIPS 2018.
NVIDIA AI Labs Pioneering Research Award 2018.
[poster]
- Saliency-based Sequential Image Attention with Multiset Prediction
S. Welleck, J. Mao, K. Cho, Z. Zhang.
NeurIPS 2017.
NVIDIA AI Labs Pioneering Research Award 2017.
[poster][press]
- Efficient AUC Optimization for Information Ranking Applications
S. Welleck.
ECIR 2016.
Teaching
- Guest Lecture: Neural sequence generation (DATA 598) [slides]
University of Washington
March 2022.
- Deep Learning (DS-GA 1008)
New York University
Fall 2020
- Deep Learning for NLP
African Master's Program in Machine Intelligence
March 2020
- Introduction to Machine Learning (CSCI-UA 0473)
New York University
Spring 2020
- NLP with Representation Learning (DS-GA 1011)
New York University
Fall 2019
Past
- Facebook, AI Research Team (FAIR), May. 2019 - Sep. 2019
- Facebook, AI Research Team (FAIR), May. 2018 - Jan. 2019
- Primer AI, Feb. 2016 - Aug. 2016
- IBM, Sep. 2014 - Feb. 2016
- University of Pennsylvania (Computer Science, MSE), May. 2013 - May. 2014
- University of Pennsylvania (Computer Science, BSE), Sep. 2009 - Feb. 2013
