Karan Tandon

I am a Research Software Engineer 2 at Microsoft Research India. My primary research interests lie in systems for ML and ML for systems, with a broader focus on systems. I am also open to exploring new areas.

I am actively seeking PhD opportunities. Please feel free to reach out for collaboration or potential openings.

Currently, I am working on Project Reward Copilot, automating interpretable reward function design for RL-driven systems optimizations, with Dr. Nagarajan Natarajan and Dr. Mayukh Das.

Previously, I have worked on research projects including:

Project Network Brain, focusing on geo-distributed training of large language models (LLMs) and output sequence length prediction for LLMs, with Dr. Venkat Padmanabhan, Dr. Debopam Bhattacherjee and Dr. Rohan Gandhi.
Project OPPerTune, a framework enabling apps in deployment to continuously optimize performance by tuning configuration parameters, with Dr. Nagarajan Natarajan and Dr. Ranjita Bhagwan.
Project CEISER, focusing on cost-effective inference serving of machine learning (ML) models, with Dr. Jayashree Mohan and Dr. Anand Iyer.

Before joining Microsoft Research, I was a Software Engineer at Microsoft India in the Devices team. I completed my B.Tech. in Computer Science and Engineering from IIIT Delhi in 2021, with my thesis on client-independent IP geolocation advised by Prof. Sambuddho Chakravarty and Prof. Mukulika Maity.

Selected Publications

OPPerTune: Post-Deployment Configuration Tuning of Services Made Easy

Gagan Somashekar^*, Karan Tandon^*, Anush Kini, and 6 more authors

In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), Apr 2024

Abs PDF Video

Real-world application deployments have hundreds of interdependent configuration parameters, many of which significantly influence performance and efficiency. With today’s complex and dynamic services, operators need to continuously monitor and set the right configuration values (configuration tuning) well after a service is widely deployed. This is challenging since experimenting with different configurations post-deployment may reduce application performance or cause disruptions. While state-of-the-art ML approaches do help to automate configuration tuning, they do not fully address the multiple challenges in end-to-end configuration tuning of deployed applications. This paper presents OPPerTune, a service that enables configuration tuning of applications in deployment at Microsoft. OPPerTune reduces application interruptions while maximizing the performance of deployed applications as and when the workload or the underlying infrastructure changes. It automates three essential processes that facilitate post-deployment configuration tuning: (a) determining which configurations to tune, (b) automatically managing the scope at which to tune the configurations, and (c) using a novel reinforcement learning algorithm to simultaneously and quickly tune numerical and categorical configurations, thereby keeping the overhead of configuration tuning low. We deploy OPPerTune on two enterprise applications in Microsoft Azure’s clusters. Our experiments show that OPPerTune reduces the end-to-end P95 latency of microservice applications by more than 50% over expert configuration choices made ahead of deployment. The code and datasets used are made available at https://aka.ms/OPPerTune.
Reward Copilot for RL-driven Systems Optimization

Karan Tandon, Manav Mishra, Gagan Somashekar, and 2 more authors

In NeurIPS 2024 Workshop on Machine Learning for Systems, Dec 2024

Abs PDF

Systems optimization problems such as workload auto-scaling, kernel parameter tuning, and cluster management arising in large-scale enterprise infrastructure are becoming increasingly RL-driven. While effective, it is difficult to set up the RL framework for such real-world problems — designing correct and useful reward functions or state spaces is highly challenging and needs a lot of domain expertise. Our proposed novel reward co-pilot solution can help design suitable and interpretable reward functions guided by client-provided specifications for any RL framework. Using experiments on standard benchmarks as well as systems-specific optimization problems, we show that our solution can return reward functions with a certain (informal) feasibility certificate in addition to pareto-optimality.