JPH :)

Jack Parker-Holder jack.parker-holder@spc.ox.ac.uk | |


About
I am a third year DPhil student at St Peter's College, Oxford, where I am part of the Machine Learning Research Group, advised by Stephen Roberts.

I am interested in ways to use reinforcement learning (RL) to train generally capable agents. I believe this will be made possible with an open-ended learning process, where environments constantly propose new challenges [1, 2] and agents adapt their configurations to solve them [3, 4, 5]. To make this process unbounded, I am excited by methods to actively encourage diversity [8, 9], while I believe we can eventually use world models to learn simulators purely from offline data [10, 11].

I have been incredibly fortunate to spend summers interning at FAIR London with Tim Rocktäschel and Ed Grefenstette and Aspect Capital in the ML group. Before coming to Oxford I was a VP in the Quantitative Research team at JPM in New York. While in America I studied for a Master's part-time at Columbia where I discovered the joys of machine learning research! I am originally from the UK, and studied Maths at Exeter.


News
  • [6/2022] Our new work on offline RL from pixels won an Outstanding Paper Award at L-DOD.

  • [5/2022] Two papers accepted to ICML 2022... see you in Baltimore :)

  • [4/2022] We are organizing the first workshop on Agent Learning in Open-Endedness at ICLR 2022, come along!

  • [3/2022] New work on multi-task RL was given an honorable mention for best paper at AISTATS!

  • [3/2022] We released ACCEL, a new algorithm for open-ended learning, check it out!

  • [9/2021] Three papers accepted to NeurIPS 2021! I'm grateful to work with such great people :)

  • [6/2021] I am interning at Facebook AI Research with Tim Rocktäschel and Ed Grefenstette.

  • [11/2020] PB2 was included into Ray Tune! Check out the blog post.

  • [9/2020] Three papers accepted to NeurIPS 2020. Thank you to my amazing collaborators!!

  • [6/2020] Three papers accepted to the main conference at ICML 2020.

  • [2/2020] New work on model-based RL, Ready Policy One, was covered by VentureBeat (here).

Conference Papers
Evolving Curricula with Regret-Based Environment Design
Jack Parker-Holder*, Minqi Jiang*, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
The International Conference on Machine Learning (ICML), 2022
(Paper) (Website) (Paper Review) (Video Interview)


From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
Krzysztof Choromanski*, Han Lin*, Haoxian Chen*, Tianyi Zhang, Arijit Sehanobish, Valerii Likhosherstov, Jack Parker-Holder, Tamas Sarlos, Adrian Weller, Thomas Weingarten
The International Conference on Machine Learning (ICML), 2022
(ArXiv)


Revisiting Design Choices in Offline Model Based Reinforcement Learning
Cong Lu*, Philip Ball*, Jack Parker-Holder, Michael Osborne, Stephen Roberts
International Conference on Learning Representations (ICLR), 2022 (Spotlight)
Reinforcement Learning for Real Life Workshop @ ICML, 2021 (Spotlight)
(Paper)


Towards an Understanding of Default Policies in Multitask Policy Optimization
Ted Moskovitz, Michael Arbel, Jack Parker-Holder, Aldo Pacchiano
Artificial Intelligence and Statistics (AISTATS), 2022 (Oral, Best Paper Honorable Mention)
(Paper)


On-the-fly Strategy Adaptation for ad-hoc Agent Coordination
Jaleh Zand, Jack Parker-Holder, Stephen Roberts
International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022
Cooperative AI Workshop, NeurIPS, 2021


Lyapunov Exponents for Diversity in Differentiable Games
Jonathan Lorraine, Paul Vicol, Jack Parker-Holder, Tal Kachman, Luke Metz, Jakob Foerster
International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022
Beyond first-order methods in ML systems workshop, ICML, 2021
(Paper)


Same State, Different Task: Continual Reinforcement Learning without Interference
Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, Stephen Roberts
AAAI Conference on Artificial Intelligence, 2022
Workshop on Continual Learning, ICML, 2020
(Paper)


Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL
Jack Parker-Holder, Vu Nguyen, Shaan Desai, Stephen Roberts
Advances in Neural Information Processing Systems (NeurIPS), 2021
(ArXiv)


Replay-Guided Adversarial Environment Design
Minqi Jiang*, Michael Dennis*, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
Advances in Neural Information Processing Systems (NeurIPS), 2021
(ArXiv)


Tactical Optimism and Pessimism for Deep Reinforcement Learning
Ted Moskovitz, Jack Parker-Holder, Aldo Pacchiano, Michael Arbel, Michael I. Jordan
Advances in Neural Information Processing Systems (NeurIPS), 2021
(ArXiv)


MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, Tim Rocktäschel
NeurIPS (Datasets and Benchmarks Track), 2021
(Paper)


Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment
Philip J Ball*, Cong Lu*, Jack Parker-Holder, Stephen Roberts
The International Conference on Machine Learning (ICML), 2021
Workshop on Self-Supervision in Reinforcement Learning, ICLR, 2021 (Spotlight)
(Paper) (Website)


Towards Tractable Optimism in Model-Based Reinforcement Learning
Aldo Pacchiano*, Philip Ball*, Jack Parker-Holder*, Krzysztof Choromanski, Stephen Roberts
Uncertainty in Artificial Intelligence (UAI), 2021
(ArXiv)


Effective Diversity in Population-Based Reinforcement Learning
Jack Parker-Holder*, Aldo Pacchiano*, Krzysztof Choromanski, Stephen Roberts
Advances in Neural Information Processing Systems (NeurIPS), 2020 (Spotlight)
The Fourth Lifelong Machine Learning Workshop at ICML, 2020
(ArXiv) (Talk) (Code)


Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits
Jack Parker-Holder, Vu Nguyen, Stephen Roberts
Advances in Neural Information Processing Systems (NeurIPS), 2020
The 7th ICML Workshop on Automated Machine Learning (AutoML), 2020 (Contributed Talk)
(ArXiv) (Blog) (Code)


Ridge Rider: Finding diverse solutions by following eigenvectors of the Hessian
Jack Parker-Holder*, Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair HP Letcher, Alex Peysakhovich, Aldo Pacchiano, Jakob Foerster*
Advances in Neural Information Processing Systems (NeurIPS), 2020
Beyond First Order Methods in ML Systems, ICML, 2020 (Spotlight)
(Arxiv) (Blog)


Ready Policy One: World Building Through Active Learning
Philip Ball*, Jack Parker-Holder*, Aldo Pacchiano, Krzysztof Choromanski, Stephen Roberts
The International Conference on Machine Learning (ICML), 2020
(ArXiv) (Media) (Code)


Learning to Score Behaviors for Guided Policy Optimization
Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Anna Choromanska, Krzysztof Choromanski, Michael I. Jordan
The International Conference on Machine Learning (ICML), 2020
(ArXiv) (Code)


Stochastic Flows and Geometric Optimization on the Orthogonal Group
Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas Sindhwani
The International Conference on Machine Learning (ICML), 2020
(ArXiv)



Practical Nonisotropic Monte Carlo Sampling in High Dimensions via Determinantal Point Processes

Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*
Artificial Intelligence and Statistics (AISTATS), 2020
(ArXiv)


From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization
Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Vikas Sindhwani
Advances in Neural Information Processing Systems (NeurIPS), 2019
(ArXiv) (Code)


Provably Robust Blackbox Optimization for Reinforcement Learning
Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang, Deepali Jain, Yuxiang Yang, Atil Iscen, Jasmine Hsu, Vikas Sindhwani
The Conference on Robot Learning (CoRL), 2019 (Spotlight)
(ArXiv)



Workshop Papers
Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations
Cong Lu, Philip J. Ball, Tim G. J. Rudner, Jack Parker-Holder, Michael A. Osborne, Yee Whye Teh
L-DOD, RSS, 2022 (Outstanding Paper Award)
(ArXiv) (Code)


Grounding Aleatoric Uncertainty in Unsupervised Environment Design
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster
Deep RL Workshop, NeurIPS, 2021


Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay
Iryna Korshunova, Minqi Jiang, Jack Parker-Holder, Tim Rocktäschel, Edward Grefenstette
Deep RL Workshop, NeurIPS, 2021
I (Still) Can’t Believe It’s Not Better Workshop, NeurIPS, 2021


Taming the Herd: Multi-Modal Meta-Learning with a Population of Agents
Robert Müller, Jack Parker-Holder, Aldo Pacchiano
The Fourth Lifelong Machine Learning Workshop at ICML, 2020
(Paper)


Reinforcement Learning with Chromatic Networks for Compact Architecture Search
Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Deepali Jain, Yuxiang Yang
The ICLR Workshop on Neural Architecture Search, 2020
(ArXiv)



Preprints
Unlocking Pixels for Reinforcement Learning via Implicit Attention
Krzysztof Choromanski*, Deepali Jain*, Jack Parker-Holder*, Xingyou Song*, Valerii Likhosherstov, Anirban Santara, Aldo Pacchiano, Yunhao Tang, Adrian Weller
(ArXiv)



Employment
Facebook AI Research - Intern, FAIR Labs - (2021)


Aspect Capital - Intern, Machine Learning Research - (2020)


JPMorgan Chase - Vice President, Quantitative Research - (2012-2019)



Education
University of Oxford - DPhil Machine Learning - (2019-)


Columbia University - MA QMSS - (2016-2018)


University of Exeter - BSc Mathematics - (2009-2012)



Additional Information
  • Reviewing AISTATS: '21, ICLR: '21, '22 (Highlighted Reviewer), ICML: '21, NeurIPS: '21 (Top 8%), '22.

  • Interests In normal times I practice Brazilian Jiu Jitsu (blue belt from Ailson Henrique Brites).