Writing
Notes & essays
Notes where I work through the math behind my projects, re-explained with intuition.
Jun 2026
Markov Decision Processes: The Formal Object
A careful, ground-up tour of the MDP as a mathematical object: states, actions, transitions, rewards, the Markov property, and the full hierarchy of decision rules and policies, from history-dependent randomized down to stationary deterministic.
Jun 2026
The Banach Fixed-Point Theorem and Why Value Iteration Converges
Why iterative policy evaluation converges: the Bellman operator is a contraction on a Banach space, and the Banach fixed-point theorem turns that single fact into existence, uniqueness, and a convergent algorithm.
Coming soon
More notes on the way
I am converting my derivation reports across reinforcement learning, sequential decision making, Bayesian filtering, and Monte Carlo into posts.