Jack Parker-Holder jparkerholder@gmail.com | |

I am a Research Scientist at Google DeepMind in the Open-Endedness Team and an Honorary Lecturer at University College London, where I am part of UCL DARK. I am interested in building open-ended systems that are worth running forever. For an example of my recent work, see AdA.

Before joining Google DeepMind was a DPhil student at St Peter's College, Oxford, where I was part of the Machine Learning Research Group, advised by Stephen Roberts. I also completed internships at FAIR London with Tim Rocktäschel and Ed Grefenstette and Aspect Capital in the ML group. In a previous life I had a seven year finance career, as both a Quantitative Researcher and ETF Trader at J.P. Morgan in New York. While in America I studied for a Master's part-time at Columbia where I discovered the joys of machine learning research! I am originally from the UK, and studied Maths at Exeter.

Aside from AI research, I also enjoy parenting a daughter and a dog, travel, Brazilian Jiu Jitsu and supporting Chelsea FC.

News

[10/2023] I have joined UCL as an Honorary Lecturer in Computer Science :) [9/2023] Two papers accepted to NeurIPS 2023, back to New Orleans again! [7/2023] The Agent Learning in Open-Endedness workshop is back for version 2 at NeurIPS 2023! [6/2023] AdA was accepted as an Oral at ICML 2023! [1/2023] Our work on AdA was covered in the New Scientist (here). [1/2023] One paper accepted to ICLR 2023! [9/2022] Two papers accepted to NeurIPS 2022, looking forward to NOLA! [7/2022] I have joined DeepMind as a Research Scientist in the Open-Endedness team :) [6/2022] Our new work on offline RL from pixels won an Outstanding Paper Award at L-DOD. [5/2022] Two papers accepted to ICML 2022... see you in Baltimore :) [4/2022] We are organizing the first workshop on Agent Learning in Open-Endedness at ICLR 2022, come along! [3/2022] New work on multi-task RL was given an honorable mention for best paper at AISTATS! [3/2022] We released ACCEL, a new algorithm for open-ended learning, check it out! [9/2021] Three papers accepted to NeurIPS 2021! I'm grateful to work with such great people :) [6/2021] I am interning at Facebook AI Research with Tim Rocktäschel and Ed Grefenstette. [11/2020] PB2 was included into Ray Tune! Check out the blog post. [9/2020] Three papers accepted to NeurIPS 2020. Thank you to my amazing collaborators!! [6/2020] Three papers accepted to the main conference at ICML 2020. [2/2020] New work on model-based RL, Ready Policy One, was covered by VentureBeat (here).

Conference Papers

Synthetic experience replay Cong Lu*, Philip J. Ball*, Yee Whye Teh, Jack Parker-Holder Advances in Neural Information Processing Systems (NeurIPS), 2023 (Paper) (Code)

Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design Matthew T. Jackson, Minqi Jiang, Jack Parker-Holder , Risto Vuorio, Chris Lu, Gregory Farquhar, Shimon Whiteson, Jakob Foerster Advances in Neural Information Processing Systems (NeurIPS), 2023 (Paper) (Code)

Human-Timescale Adaptation in an Open-Ended Task Space Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, Lei Zhang The International Conference on Machine Learnning (ICML), 2023 (Oral) (Paper) (Website) (New Scientist)

MAESTRO: Open-ended environment design for multi-agent reinforcement learning Mikayel Samvelyan, Akbir Khan, Michael Dennis, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Roberta Raileanu, Tim Rocktäschel The International Conference on Learning Representations (ICLR), 2023 (Paper) (Website)

Learning Genneral World Models in a Handful of Reward Free Deployments Yingchen Xu*, Jack Parker-Holder*, Aldo Pacchiano*, Philip J. Ball*, Oleh Rybkin, Stephen J. Roberts, Tim Rocktäschel, Edward Grefenstette Advances in Neural Information Processing Systems (NeurIPS), 2022 (Paper) (Website)

Evolving Curricula with Regret-Based Environment Design Jack Parker-Holder*, Minqi Jiang*, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel The International Conference on Machine Learning (ICML), 2022 (Paper) (Website) (Paper Review) (Video Interview)

From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers Krzysztof Choromanski*, Han Lin*, Haoxian Chen*, Tianyi Zhang, Arijit Sehanobish, Valerii Likhosherstov, Jack Parker-Holder, Tamas Sarlos, Adrian Weller, Thomas Weingarten The International Conference on Machine Learning (ICML), 2022 (ArXiv)

Revisiting Design Choices in Offline Model Based Reinforcement Learning Cong Lu*, Philip Ball*, Jack Parker-Holder, Michael Osborne, Stephen Roberts International Conference on Learning Representations (ICLR), 2022 (Spotlight) Reinforcement Learning for Real Life Workshop @ ICML, 2021 (Spotlight) (Paper)

Towards an Understanding of Default Policies in Multitask Policy Optimization Ted Moskovitz, Michael Arbel, Jack Parker-Holder, Aldo Pacchiano Artificial Intelligence and Statistics (AISTATS), 2022 (Oral, Best Paper Honorable Mention) (Paper)

On-the-fly Strategy Adaptation for ad-hoc Agent Coordination Jaleh Zand, Jack Parker-Holder, Stephen Roberts International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022 Cooperative AI Workshop, NeurIPS, 2021

Lyapunov Exponents for Diversity in Differentiable Games Jonathan Lorraine, Paul Vicol, Jack Parker-Holder, Tal Kachman, Luke Metz, Jakob Foerster International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022 Beyond first-order methods in ML systems workshop, ICML, 2021 (Paper)

Same State, Different Task: Continual Reinforcement Learning without Interference Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, Stephen Roberts AAAI Conference on Artificial Intelligence, 2022 Workshop on Continual Learning, ICML, 2020 (Paper)

Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL Jack Parker-Holder, Vu Nguyen, Shaan Desai, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2021 (ArXiv)

Replay-Guided Adversarial Environment Design Minqi Jiang*, Michael Dennis*, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel Advances in Neural Information Processing Systems (NeurIPS), 2021 (ArXiv)

Tactical Optimism and Pessimism for Deep Reinforcement Learning Ted Moskovitz, Jack Parker-Holder, Aldo Pacchiano, Michael Arbel, Michael I. Jordan Advances in Neural Information Processing Systems (NeurIPS), 2021 (ArXiv)

MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, Tim Rocktäschel NeurIPS (Datasets and Benchmarks Track), 2021 (Paper)

Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment Philip J Ball*, Cong Lu*, Jack Parker-Holder, Stephen Roberts The International Conference on Machine Learning (ICML), 2021 Workshop on Self-Supervision in Reinforcement Learning, ICLR, 2021 (Spotlight) (Paper) (Website)

Towards Tractable Optimism in Model-Based Reinforcement Learning Aldo Pacchiano*, Philip Ball*, Jack Parker-Holder*, Krzysztof Choromanski, Stephen Roberts Uncertainty in Artificial Intelligence (UAI), 2021 (ArXiv)

Effective Diversity in Population-Based Reinforcement Learning Jack Parker-Holder*, Aldo Pacchiano*, Krzysztof Choromanski, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2020 (Spotlight) The Fourth Lifelong Machine Learning Workshop at ICML, 2020 (ArXiv) (Talk) (Code)

Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits Jack Parker-Holder, Vu Nguyen, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2020 The 7th ICML Workshop on Automated Machine Learning (AutoML), 2020 (Contributed Talk) (ArXiv) (Blog) (Code)

Ridge Rider: Finding diverse solutions by following eigenvectors of the Hessian Jack Parker-Holder*, Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair HP Letcher, Alex Peysakhovich, Aldo Pacchiano, Jakob Foerster* Advances in Neural Information Processing Systems (NeurIPS), 2020 Beyond First Order Methods in ML Systems, ICML, 2020 (Spotlight) (Arxiv) (Blog)

Ready Policy One: World Building Through Active Learning Philip Ball*, Jack Parker-Holder*, Aldo Pacchiano, Krzysztof Choromanski, Stephen Roberts The International Conference on Machine Learning (ICML), 2020 (ArXiv) (Media) (Code)

Learning to Score Behaviors for Guided Policy Optimization Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Anna Choromanska, Krzysztof Choromanski, Michael I. Jordan The International Conference on Machine Learning (ICML), 2020 (ArXiv) (Code)

Stochastic Flows and Geometric Optimization on the Orthogonal Group Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas Sindhwani The International Conference on Machine Learning (ICML), 2020 (ArXiv)

Practical Nonisotropic Monte Carlo Sampling in High Dimensions via Determinantal Point Processes Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang* Artificial Intelligence and Statistics (AISTATS), 2020 (ArXiv)

From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Vikas Sindhwani Advances in Neural Information Processing Systems (NeurIPS), 2019 (ArXiv) (Code)

Provably Robust Blackbox Optimization for Reinforcement Learning Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang, Deepali Jain, Yuxiang Yang, Atil Iscen, Jasmine Hsu, Vikas Sindhwani The Conference on Robot Learning (CoRL), 2019 (Spotlight) (ArXiv)

Workshop Papers

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations Cong Lu, Philip J. Ball, Tim G. J. Rudner, Jack Parker-Holder, Michael A. Osborne, Yee Whye Teh L-DOD, RSS, 2022 (Outstanding Paper Award) (ArXiv) (Code)

Grounding Aleatoric Uncertainty in Unsupervised Environment Design Minqi Jiang, Michael Dennis, Jack Parker-Holder, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster Deep RL Workshop, NeurIPS, 2021

Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay Iryna Korshunova, Minqi Jiang, Jack Parker-Holder, Tim Rocktäschel, Edward Grefenstette Deep RL Workshop, NeurIPS, 2021 I (Still) Can’t Believe It’s Not Better Workshop, NeurIPS, 2021

Taming the Herd: Multi-Modal Meta-Learning with a Population of Agents Robert Müller, Jack Parker-Holder, Aldo Pacchiano The Fourth Lifelong Machine Learning Workshop at ICML, 2020 (Paper)

Reinforcement Learning with Chromatic Networks for Compact Architecture Search Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Deepali Jain, Yuxiang Yang The ICLR Workshop on Neural Architecture Search, 2020 (ArXiv)

Preprints

Unlocking Pixels for Reinforcement Learning via Implicit Attention Krzysztof Choromanski*, Deepali Jain*, Jack Parker-Holder*, Xingyou Song*, Valerii Likhosherstov, Anirban Santara, Aldo Pacchiano, Yunhao Tang, Adrian Weller (ArXiv)

Employment

Facebook AI Research - Intern, FAIR Labs - (2021)

Aspect Capital - Intern, Machine Learning Research - (2020)

JPMorgan Chase - Vice President, Quantitative Research - (2012-2019)

Education

University of Oxford - DPhil Machine Learning - (2019-)

Columbia University - MA QMSS - (2016-2018)

University of Exeter - BSc Mathematics - (2009-2012)

Additional Information

Reviewing AISTATS: '21, ICLR: '21, '22 (Highlighted Reviewer), ICML: '21, NeurIPS: '21 (Top 8%), '22. Interests In normal times I practice Brazilian Jiu Jitsu (blue belt from Ailson Henrique Brites).