Tips & Tricks

Table of Contents

Log-sum-exp

Why

Improves numerical stability.

How

tricks_and_treats_7d07e3772d1404ab1743ae681a3a9b069c47f23e.png

  • tricks_and_treats_6e9d20113f7ef024a71fc3edcc10e77417d06393.png usually chosen to be tricks_and_treats_2bc0de83811a897bbeafc643d9c08a05533ddc40.png
  • I can see that the introduction of tricks_and_treats_6e9d20113f7ef024a71fc3edcc10e77417d06393.png will make it more stable, by decreasing how much tricks_and_treats_e9f4d218474bc10aa94958ec30139aee865c0173.png affects the sum, but at the same time you will get even smaller exponentials due to tricks_and_treats_e9f4d218474bc10aa94958ec30139aee865c0173.png. In my head that would imply even more instability, no? I guess the constant tricks_and_treats_6e9d20113f7ef024a71fc3edcc10e77417d06393.png somehow provides more stability than this additional instability..