Jack Parker-Holder jack.parker-holder@spc.ox.ac.uk | |

I am a second year DPhil student at St Peter's College, Oxford, where I am part of the Machine Learning Research Group, advised by Stephen Roberts.

I am interested in ways to improve generalization in RL. This can be viewed from many different lenses: 1) At the algorithm level where I am interested in ways to make RL algorithms work "out of the box" by adapting their behavior on-the-fly. This can be through learning hyperparameters, exploration strategies, or even update rules. 2) At the agent level, where I am working on ways to train agents which can generalize beyond a single deterministic instantiation of an environment. My work so far has focused on producing a diverse set of policies (here and here), with the aim to improve robustness by dynamically selecting from these at test time. 3) Beyond a single environment. In an ideal world, agents can also quickly adapt to new tasks, potentially chaining together primitives (or options). They may also need to learn these tasks continually.

Before coming to Oxford I was a VP in the Quantitative Research team at JPM in New York. While in America I studied for a Master's part-time at Columbia where I discovered the joys of machine learning research! I am originally from the UK, and studied Maths at Exeter.

I am immensely privileged to have worked with some fantastic people, in alphabetical order: Philip Ball, Krzysztof Choromanski, Aldo Pacchiano and Yunhao Tang.

News

[11/2020] PB2 was included into Ray Tune! Check out the blog post. [9/2020] Three papers accepted to NeurIPS 2020. Thank you to my amazing collaborators!! [6/2020] Three papers accepted to the main conference at ICML 2020. [2/2020] New work on model-based RL, Ready Policy One, was covered by VentureBeat (here).

Conference Papers

Effective Diversity in Population-Based Reinforcement Learning Jack Parker-Holder*, Aldo Pacchiano*, Krzysztof Choromanski, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2020 (Spotlight) The Fourth Lifelong Machine Learning Workshop at ICML, 2020 (ArXiv) (Talk) (Code)

Provably Efficient Online Hyperparameter Optimization with Population-Based Bandits Jack Parker-Holder, Vu Nguyen, Stephen Roberts Advances in Neural Information Processing Systems (NeurIPS), 2020 The 7th ICML Workshop on Automated Machine Learning (AutoML), 2020 (Contributed Talk) (ArXiv) (Blog) (Code)

Ridge Rider: Finding diverse solutions by following eigenvectors of the Hessian Jack Parker-Holder*, Luke Metz, Cinjon Resnick, Hengyuan Hu, Adam Lerer, Alistair HP Letcher, Alex Peysakhovich, Aldo Pacchiano, Jakob Foerster* Advances in Neural Information Processing Systems (NeurIPS), 2020 Beyond First Order Methods in ML Systems, ICML, 2020 (Spotlight) (Arxiv) (Blog)

Ready Policy One: World Building Through Active Learning Philip Ball*, Jack Parker-Holder*, Aldo Pacchiano, Krzysztof Choromanski, Stephen Roberts to appear in The International Conference on Machine Learning (ICML), 2020 (ArXiv) (Media) (Code)

Learning to Score Behaviors for Guided Policy Optimization Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Anna Choromanska, Krzysztof Choromanski, Michael I. Jordan to appear in The International Conference on Machine Learning (ICML), 2020 (ArXiv) (Code)

Stochastic Flows and Geometric Optimization on the Orthogonal Group Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, Mrugank Akarte, Jack Parker-Holder, Jacob Bergquist, Yuan Gao, Aldo Pacchiano, Tamas Sarlos, Adrian Weller, Vikas Sindhwani to appear in The International Conference on Machine Learning (ICML), 2020 (ArXiv)

Practical Nonisotropic Monte Carlo Sampling in High Dimensions via Determinantal Point Processes Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang* to appear in Artificial Intelligence and Statistics (AISTATS), 2020 (ArXiv)

From Complexity to Simplicity: Adaptive ES-Active Subspaces for Blackbox Optimization Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang*, Vikas Sindhwani Advances in Neural Information Processing Systems (NeurIPS), 2019 (ArXiv) (Code)

Provably Robust Blackbox Optimization for Reinforcement Learning Krzysztof Choromanski*, Aldo Pacchiano*, Jack Parker-Holder*, Yunhao Tang, Deepali Jain, Yuxiang Yang, Atil Iscen, Jasmine Hsu, Vikas Sindhwani The Conference on Robot Learning (CoRL), 2019, Spotlight (ArXiv)

Workshop Papers

UNCLEAR: A Straightforward Method for Continual Reinforcement Learning Samuel Kessler, Jack Parker-Holder, Philip Ball, Stefan Zohren, Stephen Roberts Workshop on Continual Learning, ICML, 2020 (Paper)

Taming the Herd: Multi-Modal Meta-Learning with a Population of Agents Robert Müller, Jack Parker-Holder, Aldo Pacchiano The Fourth Lifelong Machine Learning Workshop at ICML, 2020 (Paper)

Reinforcement Learning with Chromatic Networks for Compact Architecture Search Xingyou Song, Krzysztof Choromanski, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamas Sarlos, Deepali Jain, Yuxiang Yang The ICLR Workshop on Neural Architecture Search, 2020 (ArXiv)

Preprints

Towards Tractable Optimism in Model-Based Reinforcement Learning Aldo Pacchiano*, Philip Ball*, Jack Parker-Holder*, Krzysztof Choromanski, Stephen Roberts (ArXiv)

Employment

Aspect Capital - Intern, Machine Learning Research - (2020)

JPMorgan Chase - Vice President, Quantitative Research - (2012-2019)

Education

University of Oxford - DPhil Machine Learning - (2019-)

Columbia University - MA QMSS - (2016-2018)

University of Exeter - BSc Mathematics - (2009-2012)