Merge pull request #21 from amitfishy/quarto

risk sensitive mdps corrections

Author: adityam

risk-sensitive/risk-sensitive-mdps.qmd

@@ -113,16 +113,16 @@ $$ λ(x) = \begin{cases}
 Let $w \colon \ALPHABET X \to \reals$ be a bounded function and $ν \in
 Δ(\ALPHABET X)$.  Then,
 $$
-  \log \sum_{x \in \ALPHABET X} ν(x) \exp( w(x)) = 
-  \sup_{μ \in Δ(\ALPHABET X)} \Bigl\{
-     \sum_{x \in \ALPHABET X}  μ(x) w(x) - 
-    I(μ \| ν)
+  \log \sum_{x \in \ALPHABET X} ν(x) \exp( θw(x)) = 
+  \inf_{μ \in Δ(\ALPHABET X)} \Bigl\{
+     \sum_{x \in \ALPHABET X}  μ(x) w(x) + 
+    \frac{1}{θ} I(μ \| ν)
   \Bigr\},
 $$
-where the supremum is attained at the unique probability measure $μ^*$
+where the infimum is attained at the unique probability measure $μ^*$
 given by
 $$
-  μ^*(x) = \frac{e^{θv(x)}}{\int e^{θv(x)}ν(x) dx} ν(x).
+  μ^*(x) = \frac{e^{θw(x)}}{\sum_{x \in \ALPHABET X} e^{θw(x)}ν(x)} ν(x).
 $$
 :::
 
@@ -138,9 +138,9 @@ See, for example, @Follmer2010.
 Using @lem-legendre-mutual-info, the dynamic program of \\eqref{eq:avg} can be
 written as
 $$ \begin{equation}
-  J + v(x) = \min_{u \in \ALPHABET U} \sup_{μ \in Δ(\ALPHABET X)}
+  J + v(x) = \min_{u \in \ALPHABET U} \inf_{μ \in Δ(\ALPHABET X)}
   \Bigl\{
-    c(x,u) + \sum_{y \in \ALPHABET X} μ(y) v(y) - \frac{1}{θ}
+    c(x,u) + \sum_{y \in \ALPHABET X} μ(y) v(y) + \frac{1}{θ}
     I(μ \| P(\cdot | x, u) ) 
   \Bigr\}.
 \end{equation} $$