Open to Robotics & AI roles

Sai Sampath Kedari

Robotics and AI researcher, working on reinforcement learning and moving toward robot learning, with a foundation in probabilistic state estimation and optimal control.

I am a founding researcher at an early-stage AI startup in the Bay Area, and previously worked on robust estimation for robotics at the University of Michigan. My implementations, notes, and reports live on GitHub.

Research & Projects

I build the stack from scratch: probability and Monte Carlo, state estimation, the theory of decisions, and reinforcement learning, with deep RL and robot learning next.

Up next

Robot Learning

Planned

Next up: manipulation-first robot learning, toward a physical OpenARM arm.

Imitation LearningAction Chunking TransformerDiffusion PolicyDecision TransformerSim-to-RealOpenARM

Repo coming soon

Deep Reinforcement Learning

Planned

Next up: deep reinforcement learning, toward MuJoCo continuous control and a manipulation milestone.

REINFORCEGAEPPOTRPODQNDDPGTD3SACMuJoCo

Repo coming soon

Built from scratch

Reinforcement Learning

In progress

A from-scratch walk through reinforcement learning, built method by method from Sutton and Barto.

Mountain Car: a learned semi-gradient Sarsa policy with tile coding, building momentum to reach the goal.

Covered

Tabular prediction: Monte Carlo, TD(0), n-step TD, TD(λ)
Tabular control: Sarsa, Q-learning, n-step Sarsa, Sarsa(λ)
Function approximation: gradient MC, semi-gradient TD and Sarsa, tile coding

In progress

Chapter 13 policy gradients next, then model-based planning, off-policy with approximation, and eligibility traces.

Reproduced

Blackjack (Fig 5.2), Windy Gridworld (15-step optimal path), 1000-state Random Walk (Figs 9.1, 9.2), Mountain Car (Fig 10.1 cost-to-go).

12 derivation reports · learned-policy animation

$$ \nabla_\theta J(\theta) = \mathbb{E}_{\pi_\theta}\!\big[\, \nabla_\theta \log \pi_\theta(a\mid s)\, q_{\pi_\theta}(s,a) \,\big] $$

The 12 derivation reports

GitHub Notebooks

Bayesian Filtering and Smoothing

Done

How a robot tracks its hidden state under noisy sensors, derived and implemented filter by filter from Sarkka.

Extended Kalman Filter tracking a nonlinear pendulum: true state versus estimate, with the covariance evolving over time.

Covered

Gaussian filters: Kalman, Extended Kalman, Unscented, Gauss-Hermite
Particle filters: bootstrap, EKF-proposal, UKF-proposal, with self-normalized weights
Resampling and ESS degeneracy diagnostics

Failure modes characterized

EKF linearization breakdown, particle weight degeneracy, UKF sigma-point collapse.

Companion · Bayesian Inference

System identification as parameter inference, on an SIR model, with identifiability, Laplace approximation, and DRAM-MCMC.

14 reports · pendulum filtering animations

More filters: unscented and Gauss-Hermite

Unscented KF

Gauss-Hermite KF

$$ p(\mathbf{x}_k \mid \mathbf{y}_{1:k}) \propto p(\mathbf{y}_k \mid \mathbf{x}_k)\, p(\mathbf{x}_k \mid \mathbf{y}_{1:k-1}) $$

GitHub Reports Bayesian-Inference

Monte Carlo Statistical Methods

Done

Sampling from hard distributions and estimating what you cannot compute by hand, built from Casella and Robert.

DRAM (delayed-rejection adaptive Metropolis) exploring a curved banana target, learning its covariance as it goes.

Covered

Sampling: inverse-transform, accept-reject
Importance sampling: standard and self-normalized
Variance reduction: control variates, stratified sampling
MCMC: Metropolis-Hastings, Adaptive Metropolis, Delayed Rejection, DRAM

Diagnostics

Burn-in, mixing, autocorrelation, integrated autocorrelation time, effective sample size.

13 reports · 16 notebooks · DRAM-on-banana animation

$$ \alpha(\mathbf{x},\mathbf{x}') = \min\!\left(1,\; \frac{p(\mathbf{x}')\, q(\mathbf{x} \mid \mathbf{x}')}{p(\mathbf{x})\, q(\mathbf{x}' \mid \mathbf{x})}\right) $$

GitHub Reports Notebooks

Sequential Decision Making

In progress

The theory under reinforcement learning, worked out from the ground up before any code.

Covered

MDPs: finite and infinite horizon, the induced stochastic process
Dynamic programming: backward induction, optimality equations, value iteration
Foundations: Banach fixed-point, Bellman operators, the policy-improvement theorem

Source

Synthesized from Puterman, Bertsekas, and Sutton and Barto.

11 LaTeX reports

$$ T^\star v = \max_{d}\big[\, r_d + \lambda P_d v \,\big] $$

GitHub Read the notes