Sean Welleck's Home Page

I am a Postdoctoral Scholar at the University of Washington and Young Investigator at AI2, working with Yejin Choi.
Previously, I earned a PhD at New York University, advised by Kyunghyun Cho and Zheng Zhang.

My research focuses on machine learning methods for generating and reasoning with natural language. This includes:

• Developing and understanding learning [1, 2, 3], decoding [1, 2], and evaluation [1, 2] methods for neural sequence generation.
• Bridging symbolic structure and neural methods in commonsense [1, 2, 3] and mathematical [1, 2] reasoning.

News

• [11.2021] Poster on generalization in symbolic mathematics at BlackboxNLP [poster].
• [10.2021] New paper on systematic generalization in symbolic mathematics.
• [10.2021] New paper on symbolic knowledge distillation of commonsense knowledge.
• [10.2021] New paper on generated knowledge prompting for commonsense reasoning.
• [10.2021] NaturalProofs accepted for an oral presentation at NeurIPS 2021 (Datasets and Benchmarks) [slides].
• [09.2021] Mauve, a divergence measure for text generation, accepted to NeurIPS 2021 (Oral).
• [09.2021] Our paper on divergence frontiers accepted to NeurIPS 2021.
• [09.2021] Talk at AITP 2021 on NaturalProofs [slides].
• [08.2021] Co-organizing the MATH-AI for Education workshop at NeurIPS 2021.
• [06.2021] New paper on an information-divergence measure for neural text generation.
• [06.2021] New multi-domain NaturalProofs for theorem proving in natural language [code].
• [06.2021] New paper on divergence frontiers for evaluating generative models.
• [06.2021] New paper on mode recovery in neural sequence modeling accepted to SPNLP.
• [03.2021] New paper on mathematical theorem proving in natural language.
• [02.2021] Presented our paper on MLE-guided parameter search at AAAI 2021.
• [12.2020] Defended my PhD thesis, Order and Learning in Sequential Neural Structured Prediction.
• [11.2020] Presented our paper on consistency of language models at EMNLP 2020.

Papers

• S. Welleck, P. West, J. Cao, Y. Choi. Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics.
  arXiv preprint 2021.
• P. West, C. Bhagavatula, J. Hessel, J. Hwang, L. Jiang, R. Le Bras, X. Lu, S. Welleck, Y. Choi. Symbolic Knowledge Distillation: from General Language Models to Commonsense Models.
  arXiv preprint 2021.
• J. Liu, A. Liu, X. Lu, S. Welleck, P. West, R. Le Bras, Y. Choi, H. Hajishirzi. Generated Knowledge Prompting for Commonsense Reasoning.
  arXiv preprint 2021.
• S. Welleck, J. Liu, R. Le Bras, H. Hajishirzi, Y. Choi, K. Cho. NaturalProofs: Mathematical Theorem Proving in Natural Language.
  NeurIPS 2021 Datasets and Benchmarks (Oral).
  [data/code][talk]
• K. Pillutla, S. Swayamdipta, R. Zellers, J. Thickstun, S. Welleck, Y. Choi, Z. Harchaoui. An Information Divergence Measure Between Neural Text and Human Text.
  NeurIPS 2021 (Oral).
  [code]
• L. Liu, K. Pillutla, S. Welleck, S. Oh, Y. Choi, Z. Harchaoui. Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral.
  NeurIPS 2021.
• I. Kulikov, S. Welleck, K. Cho. Mode recovery in neural autoregressive sequence modeling.
  SPNLP 2021.
  [code]
• S. Welleck. Order and Learning in Sequential Neural Structured Prediction.
  PhD Thesis, New York University.
  [slides]
• S. Welleck, K. Cho. MLE-guided parameter search for task loss minimization in neural sequence modeling.
  AAAI 2021.
  [code][poster][talk]
• S. Welleck*, I. Kulikov*, J. Kim, R. Pang, K. Cho. Consistency of a Recurrent Language Model With Respect to Incomplete Decoding.
  EMNLP 2020.
  [code][talk]
• E. Mansimov, A. Wang, S. Welleck, K. Cho. A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models.
  arXiv pre-print 2020.
  [code]
• M. Li, S. Roller, I. Kulikov, S. Welleck, Y.L. Boureau, K. Cho, J. Weston. Making Inconsistent Dialogue Unlikely with Unlikelihood Training.
  ACL 2020.
• S. Welleck*, I. Kulikov*, S. Roller, E. Dinan, K. Cho, J. Weston. Neural Text Generation with Unlikelihood Training.
  ICLR 2020.
  [code]
• S. Welleck, K. Brantley, H. Daume III, K. Cho. Non-Monotonic Sequential Text Generation.
  ICML 2019.
  [code] [slides] [poster]
• S. Welleck, K. Cho. Sequential Graph Dependency Parser.
  RANLP 2019.
  [slides]
• S. Welleck, J. Weston, A. Szlam, K. Cho. Dialogue Natural Language Inference.
  ACL 2019.
  [dataset][poster][press]
• S. Welleck, Z. Yao, Y. Gai, J. Mao, Z. Zhang, K. Cho. Loss Functions for Multiset Prediction.
  NeurIPS 2018.
  NVIDIA AI Labs Pioneering Research Award 2018.
  [poster]
• S. Welleck, J. Mao, K. Cho, Z. Zhang. Saliency-based Sequential Image Attention with Multiset Prediction.
  NeurIPS 2017.
  NVIDIA AI Labs Pioneering Research Award 2017.
  [poster][press]
• S. Welleck. Efficient AUC Optimization for Information Ranking Applications.
  ECIR 2016.

Teaching

• Deep Learning (DS-GA 1008)
  New York University
  Fall 2020
• Deep Learning for NLP
  African Master's Program in Machine Intelligence
  March 2020
• Introduction to Machine Learning (CSCI-UA 0473)
  New York University
  Spring 2020
• NLP with Representation Learning (DS-GA 1011)
  New York University
  Fall 2019

Past

• Facebook, AI Research Team (FAIR), May. 2019 - Sep. 2019
• Facebook, AI Research Team (FAIR), May. 2018 - Jan. 2019
• Primer AI, Feb. 2016 - Aug. 2016
• IBM, Sep. 2014 - Feb. 2016
• University of Pennsylvania (Computer Science, MSE), May. 2013 - May. 2014
• University of Pennsylvania (Computer Science, BSE), Sep. 2009 - Feb. 2013