Ioannis Exarchos

Hi, I’m Ioannis Exarchos, though I usually go by Yannis. I am a Data & Applied Scientist at Microsoft. Prior to that I was a postdoctoral research fellow at the computer science department of Stanford University. I received a PhD degree in 2017 from the Georgia Institue of Technology, and also hold two MS degrees, in Mathematics and Aerospace Engineering, also from Georgia Tech. My initial degree was in Mechanical Engineering, obtained at the University of Patras, Greece (where I graduated at the top of my class and was appointed valedictorian). My doctoral thesis was in the field of stochastic optimal control, under the guidance of profs. Panagiotis Tsiotras and Evangelos Theodorou. After the completion of my PhD I spent some time developing machine learning and control algorithms for a neuroscience/neurosurgery lab at Emory University, and then joined Stanford University to work on control and deep reinforcement learning for robotics with prof. C. Karen Liu. My research interests lie at the intersection of stochastic control, reinforcement learning, optimization, and deep learning, particularly in applications within robotics and neuroscience. I also have some background in differential game theory. During my PhD studies I was an Onassis Foundation scholar.

In my free time I enjoy reading books (both non-fiction as well as classic literature; see my reading journal here!), Jiu Jitsu and yoga to offset the time spent sitting at a desk, and playing the piano whenever I get the chance.

news

June 2021	I joined Microsoft as a Data & Applied Scientist!
May 2021	Two more papers accepted at RSS and ICML!
March 2021	Our paper on policy transfer via kinematic domain randomization and adaptation has been accepted at ICRA 2021!

selected publications

ICLR

NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Control

Exarchos, Ioannis, Pereira, Marcus A, Wang, Ziyi, and Theodorou, Evangelos A

In International Conference on Learning Representations (ICLR) 2021

Abs HTML

In this work we propose the use of adaptive stochastic search as a building block for general, non-convex optimization operations within deep neural network architectures. Specifically, for an objective function located at some layer in the network and parameterized by some network parameters, we employ adaptive stochastic search to perform optimization over its output. This operation is differentiable and does not obstruct the passing of gradients during backpropagation, thus enabling us to incorporate it as a component in end-to-end learning. We study the proposed optimization module’s properties and benchmark it against two existing alternatives on a synthetic energy-based structured prediction task, and further showcase its use in stochastic optimal control applications.
Automatica

Stochastic optimal control via forward and backward stochastic differential equations and importance sampling

Exarchos, Ioannis, and Theodorou, Evangelos A

Automatica 2018

Abs HTML

The aim of this work is to present a novel sampling-based numerical scheme designed to solve a certain class of stochastic optimal control problems, utilizing forward and backward stochastic differential equations (FBSDEs). By means of a nonlinear version of the Feynman–Kac lemma, we obtain a probabilistic representation of the solution to the nonlinear Hamilton–Jacobi–Bellman equation, expressed in the form of a system of decoupled FBSDEs. This system of FBSDEs can be solved by employing linear regression techniques. The proposed framework relaxes some of the restrictive conditions present in recent sampling based methods within the Linearly Solvable Optimal Control framework, and furthermore addresses problems in which the time horizon is not prespecified. To enhance the efficiency of the proposed scheme when treating more complex nonlinear systems, we then derive an iterative algorithm based on Girsanov’s theorem on the change of measure, which features importance sampling. This scheme is shown to be capable of learning the optimal control without requiring an initial guess.