JPH :)

Jack Parker-Holder jparkerholder@gmail.com | |


About
I am a Research Scientist at Google DeepMind in the Open-Endedness Team and an Honorary Lecturer at University College London, where I am part of UCL DARK. I am interested in training world models from Internet scale data, providing unlimited training environments for embodied AGI. For an example of my recent work, see Genie.

Before joining Google DeepMind was a DPhil student at St Peter's College, Oxford, where I was part of the Machine Learning Research Group, advised by Stephen Roberts. I also completed internships at FAIR London with Tim Rocktäschel and Ed Grefenstette and Aspect Capital in the ML group. In a previous life I had a seven year finance career, as both a Quantitative Researcher and ETF Trader at J.P. Morgan in New York. While in America I studied for a Master's part-time at Columbia where I discovered the joys of machine learning research! I am originally from the UK, and studied Maths at Exeter.

Aside from AI research, I also enjoy parenting a daughter and a dog, travel, Brazilian Jiu Jitsu and supporting Chelsea FC.


News
  • [2/2024] We introduced Genie.

  • [10/2023] I have joined UCL as an Honorary Lecturer in Computer Science :)

  • [9/2023] Two papers accepted to NeurIPS 2023, back to New Orleans again!

  • [7/2023] The Agent Learning in Open-Endedness workshop is back for version 2 at NeurIPS 2023!

  • [6/2023] AdA was accepted as an Oral at ICML 2023!

  • [1/2023] Our work on AdA was covered in the New Scientist (here).

  • [1/2023] One paper accepted to ICLR 2023!

  • [9/2022] Two papers accepted to NeurIPS 2022, looking forward to NOLA!

  • [7/2022] I have joined DeepMind as a Research Scientist in the Open-Endedness team :)

  • [6/2022] Our new work on offline RL from pixels won an Outstanding Paper Award at L-DOD.

  • [5/2022] Two papers accepted to ICML 2022... see you in Baltimore :)

  • [4/2022] We are organizing the first workshop on Agent Learning in Open-Endedness at ICLR 2022, come along!

  • [3/2022] New work on multi-task RL was given an honorable mention for best paper at AISTATS!

  • [3/2022] We released ACCEL, a new algorithm for open-ended learning, check it out!

  • [9/2021] Three papers accepted to NeurIPS 2021! I'm grateful to work with such great people :)

  • [6/2021] I am interning at Facebook AI Research with Tim Rocktäschel and Ed Grefenstette.

  • [11/2020] PB2 was included into Ray Tune! Check out the blog post.

  • [9/2020] Three papers accepted to NeurIPS 2020. Thank you to my amazing collaborators!!

  • [6/2020] Three papers accepted to the main conference at ICML 2020.

  • [2/2020] New work on model-based RL, Ready Policy One, was covered by VentureBeat (here).

Conference Papers
Synthetic experience replay
Cong Lu*, Philip J. Ball*, Yee Whye Teh, Jack Parker-Holder
Advances in Neural Information Processing Systems (NeurIPS), 2023
(Paper) (Code)


Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design
Matthew T. Jackson, Minqi Jiang, Jack Parker-Holder , Risto Vuorio, Chris Lu, Gregory Farquhar, Shimon Whiteson, Jakob Foerster
Advances in Neural Information Processing Systems (NeurIPS), 2023
(Paper) (Code)


Human-Timescale Adaptation in an Open-Ended Task Space
Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, Lei Zhang
The International Conference on Machine Learnning (ICML), 2023 (Oral)
(Paper) (Website) (New Scientist)


MAESTRO: Open-ended environment design for multi-agent reinforcement learning
Mikayel Samvelyan, Akbir Khan, Michael Dennis, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Roberta Raileanu, Tim Rocktäschel
The International Conference on Learning Representations (ICLR), 2023
(Paper) (Website)


Learning Genneral World Models in a Handful of Reward Free Deployments
Yingchen Xu*, Jack Parker-Holder*, Aldo Pacchiano*, Philip J. Ball*, Oleh Rybkin, Stephen J. Roberts, Tim Rocktäschel, Edward Grefenstette
Advances in Neural Information Processing Systems (NeurIPS), 2022
(Paper) (Website)


Evolving Curricula with Regret-Based Environment Design
Jack Parker-Holder*, Minqi Jiang*, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
The International Conference on Machine Learning (ICML), 2022
(Paper) (Website) (Paper Review) (Video Interview)


From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
Krzysztof Choromanski*, Han Lin*, Haoxian Chen*, Tianyi Zhang, Arijit Sehanobish, Valerii Likhosherstov, Jack Parker-Holder, Tamas Sarlos, Adrian Weller, Thomas Weingarten
The International Conference on Machine Learning (ICML), 2022
(ArXiv)


Revisiting Design Choices in Offline Model Based Reinforcement Learning
Cong Lu*, Philip Ball*, Jack Parker-Holder, Michael Osborne, Stephen Roberts
International Conference on Learning Representations (ICLR), 2022 (Spotlight)
Reinforcement Learning for Real Life Workshop @ ICML, 2021 (Spotlight)
(Paper)


Towards an Understanding of Default Policies in Multitask Policy Optimization
Ted Moskovitz, Michael Arbel, Jack Parker-Holder, Aldo Pacchiano
Artificial Intelligence and Statistics (AISTATS), 2022 (Oral, Best Paper Honorable Mention)
(Paper)


On-the-fly Strategy Adaptation for ad-hoc Agent Coordination
Jaleh Zand, Jack Parker-Holder, Stephen Roberts
International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022
Cooperative AI Workshop, NeurIPS, 2021


Lyapunov Exponents for Diversity in Differentiable Games
Jonathan Lorraine, Paul Vicol, Jack Parker-Holder, Tal Kachman, Luke Metz, Jakob Foerster
International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), 2022
Beyond first-order methods in ML systems workshop, ICML, 2021
(Paper)


Same State, Different Task: Continual Reinforcement Learning without Interference
Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, Stephen Roberts
AAAI Conference on Artificial Intelligence, 2022
Workshop on Continual Learning, ICML, 2020
(Paper)


Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL
Jack Parker-Holder, Vu Nguyen, Shaan Desai, Stephen Roberts
Advances in Neural Information Processing Systems (NeurIPS), 2021
(ArXiv)


Replay-Guided Adversarial Environment Design
Minqi Jiang*, Michael Dennis*, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, Tim Rocktäschel
Advances in Neural Information Processing Systems (NeurIPS), 2021
(ArXiv)


Tactical Optimism and Pessimism for Deep Reinforcement Learning
Ted Moskovitz, Jack Parker-Holder, Aldo Pacchiano, Michael Arbel, Michael I. Jordan
Advances in Neural Information Processing Systems (NeurIPS), 2021
(ArXiv)


MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, Tim Rocktäschel
NeurIPS (Datasets and Benchmarks Track), 2021
(Paper)


Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment
Philip J Ball*, Cong Lu*, Jack Parker-Holder, Stephen Roberts
The International Conference on Machine Learning (ICML), 2021
Workshop on Self-Supervision in Reinforcement Learning, ICLR, 2021 (Spotlight)
(Paper) (Website)


Towards Tractable Optimism in Model-Based Reinforcement Learning
Aldo Pacchiano*, Philip Ball*, Jack Parker-Holder*, Krzysztof Choromanski, Stephen Roberts
Uncertainty in Artificial Intelligence (UAI), 2021
(ArXiv)


Effective Diversity in Population-Based Reinforcement Learning
Jack Parker-Holder*, Aldo Pacchiano*, Krzysztof Choromanski, Stephen Roberts
Advances in Neural Information Processing Systems (NeurIPS), 2020 (Spotlight)
The Fourth Lifelong Machine Learning Workshop at ICML, 2020
(ArXiv) (Talk) (Code)


Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits
Jack Parker-Holder, Vu Nguyen, Stephen Roberts
Advances in Neural Information Processing Systems (NeurIPS), 2020
The 7th ICML Workshop on Automated Machine Learning (AutoML), 2020 (Contributed Talk)
(ArXiv) (Blog) (Code)


Ridge Rider: Finding diverse solutions by following eigenvectors of the Hessian
Jack Parker-Holder*, Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair HP Letcher, Alex Peysakhovich, Aldo Pacchiano, Jakob Foerster*
Advances in Neural Information Processing Systems (NeurIPS), 2020
Beyond First Order Methods in ML Systems, ICML, 2020 (Spotlight)
(Arxiv) (Blog)


Ready Policy One: World Building Through Active Learning
Philip Ball*, Jack Parker-Holder*, Aldo Pacchiano, Krzysztof Choromanski, Stephen Roberts
The International Conference on Machine Learning (ICML), 2020
(ArXiv) (Media) (Code)


Learning to Score Behaviors for Guided Policy Optimization
Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Anna Choromanska, Krzysztof Choromanski, Michael I. Jordan
The International Conference on Machine Learning (ICML), 2020
(ArXiv) (Code)


Stochastic Flows and Geometric Optimization on the Orthogonal Group
Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas Sindhwani
The International Conference on Machine Learning (ICML), 2020
(ArXiv)



Practical Nonisotropic Monte Carlo Sampling in High Dimensions via Determinantal Point Processes

Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*
Artificial Intelligence and Statistics (AISTATS), 2020
(ArXiv)


From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization
Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Vikas Sindhwani
Advances in Neural Information Processing Systems (NeurIPS), 2019
(ArXiv) (Code)


Provably Robust Blackbox Optimization for Reinforcement Learning
Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang, Deepali Jain, Yuxiang Yang, Atil Iscen, Jasmine Hsu, Vikas Sindhwani
The Conference on Robot Learning (CoRL), 2019 (Spotlight)
(ArXiv)



Workshop Papers
Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations
Cong Lu, Philip J. Ball, Tim G. J. Rudner, Jack Parker-Holder, Michael A. Osborne, Yee Whye Teh
L-DOD, RSS, 2022 (Outstanding Paper Award)
(ArXiv) (Code)


Grounding Aleatoric Uncertainty in Unsupervised Environment Design
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Heinrich Küttler, Edward Grefenstette, Tim Rocktäschel, Jakob Foerster
Deep RL Workshop, NeurIPS, 2021


Return Dispersion as an Estimator of Learning Potential for Prioritized Level Replay
Iryna Korshunova, Minqi Jiang, Jack Parker-Holder, Tim Rocktäschel, Edward Grefenstette
Deep RL Workshop, NeurIPS, 2021
I (Still) Can’t Believe It’s Not Better Workshop, NeurIPS, 2021


Taming the Herd: Multi-Modal Meta-Learning with a Population of Agents
Robert Müller, Jack Parker-Holder, Aldo Pacchiano
The Fourth Lifelong Machine Learning Workshop at ICML, 2020
(Paper)


Reinforcement Learning with Chromatic Networks for Compact Architecture Search
Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Deepali Jain, Yuxiang Yang
The ICLR Workshop on Neural Architecture Search, 2020
(ArXiv)



Preprints
Unlocking Pixels for Reinforcement Learning via Implicit Attention
Krzysztof Choromanski*, Deepali Jain*, Jack Parker-Holder*, Xingyou Song*, Valerii Likhosherstov, Anirban Santara, Aldo Pacchiano, Yunhao Tang, Adrian Weller
(ArXiv)



Employment
Facebook AI Research - Intern, FAIR Labs - (2021)


Aspect Capital - Intern, Machine Learning Research - (2020)


JPMorgan Chase - Vice President, Quantitative Research - (2012-2019)



Education
University of Oxford - DPhil Machine Learning - (2019-)


Columbia University - MA QMSS - (2016-2018)


University of Exeter - BSc Mathematics - (2009-2012)



Additional Information
  • Reviewing AISTATS: '21, ICLR: '21, '22 (Highlighted Reviewer), ICML: '21, NeurIPS: '21 (Top 8%), '22.

  • Interests In normal times I practice Brazilian Jiu Jitsu (blue belt from Ailson Henrique Brites).