Open to Robotics & AI roles

Sai Sampath Kedari

Robotics and AI researcher working across reinforcement learning, deep RL, and robot learning, with a strong foundation in applied mathematics and statistical inference.

I am a founding researcher at an early-stage AI startup in the Bay Area and previously worked on robust state and parameter estimation for robotics at the University of Michigan. My implementations, derivations, experiments, and reports are available on GitHub.

Research & Projects

I build the stack from scratch: probability and Monte Carlo, state estimation, the theory of decisions, and reinforcement learning, with deep RL and robot learning next.

Current work

Robot Learning

Planned

Next up: manipulation-first robot learning, toward a physical OpenARM arm.

Imitation LearningAction Chunking TransformerDiffusion PolicyDecision TransformerSim-to-RealOpenARM

Repo coming soon

Deep Reinforcement Learning

In progress

Deep reinforcement learning, toward MuJoCo continuous control and a manipulation milestone.

REINFORCEGAEPPOTRPODQNDDPGTD3SACMuJoCo

GitHub

Earlier work

Reinforcement Learning

Done

A from-scratch walk through reinforcement learning, built method by method from Sutton and Barto.

Mountain Car: a learned semi-gradient Sarsa policy with tile coding, building momentum to reach the goal.

Covered

Tabular prediction: Monte Carlo, TD(0), n-step TD, TD(λ)
Tabular control: Sarsa, Q-learning, n-step Sarsa, Sarsa(λ)
Function approximation: gradient MC, semi-gradient TD and Sarsa, tile coding

Reproduced

Blackjack (Fig 5.2), Windy Gridworld (15-step optimal path), 1000-state Random Walk (Figs 9.1, 9.2), Mountain Car (Fig 10.1 cost-to-go).

16 derivation reports · learned-policy animation

$$ \nabla_\theta J(\theta) = \mathbb{E}_{\pi_\theta}\!\big[\, \nabla_\theta \log \pi_\theta(a\mid s)\, q_{\pi_\theta}(s,a) \,\big] $$

The 16 derivation reports

GitHub Notebooks

Bayesian Filtering and Smoothing

Done

How a robot tracks its hidden state under noisy sensors, derived and implemented filter by filter from Sarkka.

Extended Kalman Filter tracking a nonlinear pendulum: true state versus estimate, with the covariance evolving over time.

Covered

Gaussian filters: Kalman, Extended Kalman, Unscented, Gauss-Hermite
Particle filters: bootstrap, EKF-proposal, UKF-proposal, with self-normalized weights
Resampling and ESS degeneracy diagnostics

Failure modes characterized

EKF linearization breakdown, particle weight degeneracy, UKF sigma-point collapse.

Companion · Bayesian Inference

System identification as parameter inference, on an SIR model, with identifiability, Laplace approximation, and DRAM-MCMC.

14 reports · pendulum filtering animations

More filters: unscented and Gauss-Hermite

Unscented KF

Gauss-Hermite KF

$$ p(\mathbf{x}_k \mid \mathbf{y}_{1:k}) \propto p(\mathbf{y}_k \mid \mathbf{x}_k)\, p(\mathbf{x}_k \mid \mathbf{y}_{1:k-1}) $$

GitHub Reports Bayesian-Inference

Monte Carlo Statistical Methods

Done

Sampling from hard distributions and estimating what you cannot compute by hand, built from Casella and Robert.

DRAM (delayed-rejection adaptive Metropolis) exploring a curved banana target, learning its covariance as it goes.

Covered

Sampling: inverse-transform, accept-reject
Importance sampling: standard and self-normalized
Variance reduction: control variates, stratified sampling
MCMC: Metropolis-Hastings, Adaptive Metropolis, Delayed Rejection, DRAM

Diagnostics

Burn-in, mixing, autocorrelation, integrated autocorrelation time, effective sample size.

13 reports · 16 notebooks · DRAM-on-banana animation

$$ \alpha(\mathbf{x},\mathbf{x}') = \min\!\left(1,\; \frac{p(\mathbf{x}')\, q(\mathbf{x} \mid \mathbf{x}')}{p(\mathbf{x})\, q(\mathbf{x}' \mid \mathbf{x})}\right) $$

GitHub Reports Notebooks

Sequential Decision Making

In progress

The theory under reinforcement learning, worked out from the ground up before any code.

Covered

MDPs: finite and infinite horizon, the induced stochastic process
Dynamic programming: backward induction, optimality equations, value iteration
Foundations: Banach fixed-point, Bellman operators, the policy-improvement theorem

Source

Synthesized from Puterman, Bertsekas, and Sutton and Barto.

11 LaTeX reports

$$ T^\star v = \max_{d}\big[\, r_d + \lambda P_d v \,\big] $$

GitHub Read the notes