Jack Parker-Holder jack.parker-holder@spc.ox.ac.uk | |

I am a third year DPhil student at St Peter's College, Oxford, where I am part of the Machine Learning Research Group, advised by Stephen Roberts.

I am interested in ways to use reinforcement learning (RL) to train generally capable agents. I believe this will be made possible with an open-ended learning process, where environments constantly propose new challenges [1, 2] and agents adapt their configurations to solve them [3, 4, 5]. To make this process unbounded, I am excited by methods to actively encourage diversity [8, 9], while I believe we can eventually use world models to learn simulators purely from offline data [10, 11].

I have been incredibly fortunate to spend summers interning at FAIR London with Tim Rocktäschel and Ed Grefenstette and Aspect Capital in the ML group. Before coming to Oxford I was a VP in the Quantitative Research team at JPM in New York. While in America I studied for a Master's part-time at Columbia where I discovered the joys of machine learning research! I am originally from the UK, and studied Maths at Exeter.

News

[6/2022] Our new work on offline RL from pixels won an Outstanding Paper Award at L-DOD. [5/2022] Two papers accepted to ICML 2022... see you in Baltimore :) [4/2022] We are organizing the first workshop on Agent Learning in Open-Endedness at ICLR 2022, come along! [3/2022] New work on multi-task RL was given an honorable mention for best paper at AISTATS! [3/2022] We released ACCEL, a new algorithm for open-ended learning, check it out! [9/2021] Three papers accepted to NeurIPS 2021! I'm grateful to work with such great people :) [6/2021] I am interning at Facebook AI Research with Tim Rocktäschel and Ed Grefenstette. [11/2020] PB2 was included into Ray Tune! Check out the blog post. [9/2020] Three papers accepted to NeurIPS 2020. Thank you to my amazing collaborators!! [6/2020] Three papers accepted to the main conference at ICML 2020. [2/2020] New work on model-based RL, Ready Policy One, was covered by VentureBeat (here).

Conference Papers

Evolving Curricula with Regret-Based Environment Design Jack Parker-Holder*, Minqi Jiang*, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel The International Conference on Machine Learning (ICML), 2022 (Paper) (Website) (Paper Review) (Video Interview)

From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers Krzysztof Choromanski*, Han Lin*, Haoxian Chen*, Tianyi Zhang, Arijit Sehanobish, Valerii Likhosherstov, Jack Parker-Holder, Tamas Sarlos, Adrian Weller, Thomas Weingarten The International Conference on Machine Learning (ICML), 2022 (ArXiv)

Revisiting Design Choices in Offline Model Based Reinforcement Learning Cong Lu*, Philip Ball*, Jack Parker-Holder, Michael Osborne, Stephen Roberts International Conference on Learning Representations (ICLR), 2022 (Spotlight) Reinforcement Learning for Real Life Workshop @ ICML, 2021 (Spotlight) (Paper)

Towards an Understanding of Default Policies in Multitask Policy Optimization Ted Moskovitz, Michael Arbel, Jack Parker-Holder, Aldo Pacchiano Artificial Intelligence and Statistics (AISTATS), 2022 (Oral, Best Paper Honorable Mention) (Paper)

On-the-fly Strategy Adaptation for ad-hoc Agent Coordination Jaleh Zand, Jack Parker-Holder, Stephen Roberts International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022 Cooperative AI Workshop, NeurIPS, 2021

Lyapunov Exponents for Diversity in Differentiable Games Jonathan Lorraine, Paul Vicol, Jack Parker-Holder, Tal Kachman, Luke Metz, Jakob Foerster International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022 Beyond first-order methods in ML systems workshop, ICML, 2021 (Paper)

Same State, Different Task: Continual Reinforcement Learning without Interference Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, Stephen Roberts AAAI Conference on Artificial Intelligence, 2022 Workshop on Continual Learning, ICML, 2020 (Paper)

Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL Jack Parker-Holder, Vu Nguyen, Shaan Desai, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2021 (ArXiv)

Replay-Guided Adversarial Environment Design Minqi Jiang*, Michael Dennis*, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel Advances in Neural Information Processing Systems (NeurIPS), 2021 (ArXiv)

Tactical Optimism and Pessimism for Deep Reinforcement Learning Ted Moskovitz, Jack Parker-Holder, Aldo Pacchiano, Michael Arbel, Michael I. Jordan Advances in Neural Information Processing Systems (NeurIPS), 2021 (ArXiv)

MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, Tim Rocktäschel NeurIPS (Datasets and Benchmarks Track), 2021 (Paper)

Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment Philip J Ball*, Cong Lu*, Jack Parker-Holder, Stephen Roberts The International Conference on Machine Learning (ICML), 2021 Workshop on Self-Supervision in Reinforcement Learning, ICLR, 2021 (Spotlight) (Paper) (Website)

Towards Tractable Optimism in Model-Based Reinforcement Learning Aldo Pacchiano*, Philip Ball*, Jack Parker-Holder*, Krzysztof Choromanski, Stephen Roberts Uncertainty in Artificial Intelligence (UAI), 2021 (ArXiv)

Effective Diversity in Population-Based Reinforcement Learning Jack Parker-Holder*, Aldo Pacchiano*, Krzysztof Choromanski, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2020 (Spotlight) The Fourth Lifelong Machine Learning Workshop at ICML, 2020 (ArXiv) (Talk) (Code)

Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits Jack Parker-Holder, Vu Nguyen, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2020 The 7th ICML Workshop on Automated Machine Learning (AutoML), 2020 (Contributed Talk) (ArXiv) (Blog) (Code)

Ridge Rider: Finding diverse solutions by following eigenvectors of the Hessian Jack Parker-Holder*, Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair HP Letcher, Alex Peysakhovich, Aldo Pacchiano, Jakob Foerster* Advances in Neural Information Processing Systems (NeurIPS), 2020 Beyond First Order Methods in ML Systems, ICML, 2020 (Spotlight) (Arxiv) (Blog)

Ready Policy One: World Building Through Active Learning Philip Ball*, Jack Parker-Holder*, Aldo Pacchiano, Krzysztof Choromanski, Stephen Roberts The International Conference on Machine Learning (ICML), 2020 (ArXiv) (Media) (Code)

Learning to Score Behaviors for Guided Policy Optimization Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Anna Choromanska, Krzysztof Choromanski, Michael I. Jordan The International Conference on Machine Learning (ICML), 2020 (ArXiv) (Code)

Stochastic Flows and Geometric Optimization on the Orthogonal Group Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas Sindhwani The International Conference on Machine Learning (ICML), 2020 (ArXiv)

Practical Nonisotropic Monte Carlo Sampling in High Dimensions via Determinantal Point Processes Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang* Artificial Intelligence and Statistics (AISTATS), 2020 (ArXiv)

From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Vikas Sindhwani Advances in Neural Information Processing Systems (NeurIPS), 2019 (ArXiv) (Code)

Provably Robust Blackbox Optimization for Reinforcement Learning Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang, Deepali Jain, Yuxiang Yang, Atil Iscen, Jasmine Hsu, Vikas Sindhwani The Conference on Robot Learning (CoRL), 2019 (Spotlight) (ArXiv)

Workshop Papers

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations Cong Lu, Philip J. Ball, Tim G. J. Rudner, Jack Parker-Holder, Michael A. Osborne, Yee Whye Teh L-DOD, RSS, 2022 (Outstanding Paper Award) (ArXiv) (Code)

Grounding Aleatoric Uncertainty in Unsupervised Environment Design Minqi Jiang, Michael Dennis, Jack Parker-Holder, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster Deep RL Workshop, NeurIPS, 2021

Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay Iryna Korshunova, Minqi Jiang, Jack Parker-Holder, Tim Rocktäschel, Edward Grefenstette Deep RL Workshop, NeurIPS, 2021 I (Still) Can’t Believe It’s Not Better Workshop, NeurIPS, 2021

Taming the Herd: Multi-Modal Meta-Learning with a Population of Agents Robert Müller, Jack Parker-Holder, Aldo Pacchiano The Fourth Lifelong Machine Learning Workshop at ICML, 2020 (Paper)

Reinforcement Learning with Chromatic Networks for Compact Architecture Search Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Deepali Jain, Yuxiang Yang The ICLR Workshop on Neural Architecture Search, 2020 (ArXiv)

Preprints

Unlocking Pixels for Reinforcement Learning via Implicit Attention Krzysztof Choromanski*, Deepali Jain*, Jack Parker-Holder*, Xingyou Song*, Valerii Likhosherstov, Anirban Santara, Aldo Pacchiano, Yunhao Tang, Adrian Weller (ArXiv)

Employment

Facebook AI Research - Intern, FAIR Labs - (2021)

Aspect Capital - Intern, Machine Learning Research - (2020)

JPMorgan Chase - Vice President, Quantitative Research - (2012-2019)

Education

University of Oxford - DPhil Machine Learning - (2019-)

Columbia University - MA QMSS - (2016-2018)

University of Exeter - BSc Mathematics - (2009-2012)

Additional Information

Reviewing AISTATS: '21, ICLR: '21, '22 (Highlighted Reviewer), ICML: '21, NeurIPS: '21 (Top 8%), '22. Interests In normal times I practice Brazilian Jiu Jitsu (blue belt from Ailson Henrique Brites).