==⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠== You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
Embedded Files
05d8083c1c14c2d090af47770f66375ab344ed26: \(V_\psi(s)\)
355937cbe392596c38078c32c3efa479dfc45ecc: \(Q_\theta(s, a)\)
bc4968d7479dcc7355a7a54425f3e776bd05c5f9: \(Q_{\hat \theta}(s, a)\)
4af55410ca5ec89d70e92b3f2833cf6a72cb28e7: \(\nabla_\psi L(\psi)\)
24bd225c245cd4a71b531707aa7739a6793081ff: \(\nabla_\theta L(\theta)\)
bc3e5263404d0d04df64905f71c67cbaf5f4061e: \(\hat \theta \leftarrow \theta\)
%%
Drawing
N4KAkARALgngDgUwgLgAQQQDwMYEMA2AlgCYBOuA7hADTgQBuCpAzoQPYB2KqATLZMzYBXUtiRoIACyhQ4zZAHoFAc0JRJQgEYA6bGwC2CgF7N6hbEcK4OCtptbErHALRY8RMpWdx8Q1TdIEfARcZgRmBShcZQUebQBGAGZtHho6IIR9BA4oZm4AbXAwUDBSiBJuCAAzNgB2HgBZABUADXi00shYRErCfWikfjLMbniANgAGbQBORImAVnieeIAW
FenpiYm+IsgYbmceRLiVgA5psZ4VxcT6o6HIChJ1UZ4pnlPJxOvExPn1v4PKQIQjKaTceYnY63FbbWpra7TIHWZTBbgTIHMKCkNgAawQAGE2Pg2KRKgBiLZUiYdMqaXDYXHKHFCDjEIkkskSbHWZhwXCBHK0yBVQj4fAAZVgaIkkgZGkCwogWJx+IA6s9JKNMdi8QgpTAZehBB4lSywRxwnk0PEgWx+dg1PsbVsgSy2ZbmNbUBwhOLMQgEMRuKd6
mMVkDGCx2FwbSsdp0GExWJwAHKcMQQyETP5zWpAoRwYi4KBB0a1MbjK7bY6nRJAwjMAAiGVLwbQVQIYSBzOEcAAksRvfkALpAzTCNkAUWCWRyw92xUXkAqElxAAUqm8GtMGvMOPhpgBFYhNIxHloAGX7ADVhWVuuJ0AKcVRFwBfB5LzplVfofRwPo2AUEYCCXpgN7YAAYgASpoABWADyRhNvMhBNPeXTwE+EAvmwb4/u+uxjrsEBEBwuLcL6/qkS
SjJlh2XYIECorBIOlQLMQpwTHW2DxHxKzYDwxATJsuBVCstRSRMVRjGMtzzLgmjfCsQY8GMSrMO4T6FD+tqLvExENmyWCVLgNIseQWTsWg1H4DqJZCN6ECIGyhAcMoSrYDicBUX6+BFJ+RQlL+7b/lATQNAAEqQmCIZI66XgA0ggiHMDw2DOC066EEqj69P0yiDKRIxoM4sJTEssyhtMtTTNxvxAs6qCHPG2hnHm0yQps4ZJECTzEC8NrTHEkLfK
c8SnCs8RdRcQKSCCYJQBCtTtTNHywvME0fPWpEokaGKkSqeocqSFLUlsSr0oyvasuyxJndy5AcHyArZMtLFipK0o4XK2AKsVibHeqmrakdur4gaRrKsSFRusIFpWmDib2gyTqjK6pHusQnrenZAYMag8S1IkmyJKc8y1PpiZRimsZE9skbJjG6YcJmaAbYsW3XAWRYloTxOVpcpOJOGJMNs2raE52+DdqRt0DkOBQkYmE53TOmTvQuP4hYmf4QBu
W4TDue4Hsep7nlet6YdA2FmaQr4QB+X666FlTYFAUUAKqkMoXvECsUVRRMFDxBK/brgAWgAgtH/Y2/lEh4QRnREZ0KtlORlG2f5QJ0fiYUy3LiasQgNnoH88yzLU2CaAgpM8FXYzYOTEy1KcLcZYkCCdpJ0zEFU2DXAg2DYJp2kFAZX6GenxnEKZSftJZuDWWF+Pg45zmuY4HleT5fnioFQy6+UYUG0IiSRxHuANDxEpGLgiFRbUl5QcQ65UECif
oH0AxKqVLVxhTBGsscm6l1jzHmOTJq3B1jaFDPMCYYx/iJCWOcaa/VQY2lOHEeIPF6pXCWOpUWYx5qLXBGgSY2hajbA2NNGEyxlgRj2h5A6OpVSEgelydAlILpXQZEybGp1uHQGeq9QUH1SKsW+oaX68oRCAzKMDBAGpBpahtOwvUUMcImjhljBGkhcbIzKKjR0sAMaHUTNjIxOcaJA0DGFeIW0q5bW+NTMotMYyjHkkzaMaYMxPh4LVOs0x4hON
5sWNs5YhbViuCTSxv5JbBCiYxWWzF5YskVsODOkA1bTlnFrSeOtlyn0qABICIEwIQWgnBJCKE0IYS/FhHoScHb4SdoRF2JT9Ye29r7f2gdg6h3DlHWO8cmm2xac+NpKdShp1KDksi7ls4+lzrRNg9FC5MU+mxM+mgh4XFOMQeEdViCjxJpA3AtRcD/HjPMKo3cpJjE0CJeY2B7nTHHgQHSU8DJGVIu5eeADcKpGXqvA+9kN5QCcpUbe7lPJAm8va
CFR9goArPobbcu59yHhPGeC8147xfztq0x2QIAGHAmPEbQ4Z5jyQIbcS4OCYFlTeKcbQkIoFoLGKE347jHhYNQLmbQbdKyLEObUSEpwyGggoagd4FwFhtxoTmLqdL8wsNRE+BJAgIacM5BSKoPFsCXXHAI26bJhGVB5C9fkEilTSO0ZUXRwZNEgzUdwBMSi9VOokC6s0BibFEztA6dGLodUQAVoObJYKy5rzWXrEywLcCJADXdINrtJmBN2EFexA
siHHB4JKjVNNmacFGH1Uinj/FsyfDNUSEx4w4JLYklsyTpbbNInk4gGs5y5AhRE/mjiKxVlhEcDaedlkDvWZs7gRd0mJjgGwdy/a0C6U6OuzoOrSgTEXDksAm7SjCtFU4maYwEEfCac4BVkxEHSVVZAsYtQ93/MTPgUIUAiT6H0GoNs65l1ClsZCoGURSBQAAEKAvhdwTNGQ+3lwgPoCK0VYrxUSilNKGUso5XvNUDZMK0CVQpkW7YpwyNt1uMTJ
Ei5I24F8mgeYMxJjqXkqsHiPE6w5qBNkYgkG3K7zQLBgpOQEM1HqM0NouGajYAI/KhIWwxr1BoXEpIokvy0fo6gaYHKuMbzA9HGZC1cDxrsWUHjBnXxGbPsnJUQQJwUA7Wk1FpQT763KcBUC4FIKwQQshVC6E8okumWSkqowcHUO6uAgh0124ssATQ+BbdVifCkv8HimCPUczmBy0m8SNiTSkvUGVS1PV4PapsC4iCcGk0psiVh2q3X6sejwo1Hd
TVdvNUIrh1qxF2veg6r6vrjSw1deDDhqihq8Ea0NmGpp4Z+EMUjDRpFTFhoZhGqNSs12LKqFZON07E1ArMisNNHoluoEzd/Hgum83DpWKLduPF5i+Lpp6+qL2WYBIhN8JBUD7gAqSQgFJqB53jknD24Tq7VmmcgIWSJAsR2XDHZJDBtEp1Abzhsguc7O2LoA1Dw9h6wDbuJ3ur8RPDjZagbVOY+XQxSS9Z0ZYUx1iiWQTxI4XVn0/hHK+zOH6v0/
pkEGf9K6DveoFBBqDAmLs0bg+9UTdRGitCXjR6Tsnr0irbv8OqODaoLB5bMRu6nlB0e4NpxuVKqXcVDKLEa0wbtmbZHxnenlBNy8hwh3EF8r7rhvnfB+T8X5vw/lJ/D3pNdUeml1J9pN5LtzmjR03mmyszSWKz8B3xFjXdTg5fThmQgmeA074gFn8JWftiFxMdn8KObCM578bsJC9J9n7AOQcQ5hwjjHOOgWpm4Rmf/A457qHcXbjgpIPAgmrDi6
TGYFwwGIMgWEqlGXJvLG06GLY3xfi/bwcwxMC1ZXLQ5kseBhappBPhHMRnuF6voka1aiQ5JWsmosh1m6XWDVPV5H1oUOyZFoZ/VGsJt1EpsxstEfpnURs01EYvRjFIBVtzFw0exMlo1lZY1y515DsF5nx5hTscZztLsgsc85kCYwoupd9fgqYPty00AHtaCOBWZ2YhVxg5J1IcwJY20gc68F06Rwde1CkMdSI4ch1olR1thJJFhJ0KJxdIB85eCg
Ql0V1tYN1lwSdd0edydlxDgZp4EVUd85hkF98r1GFz8p9L9ap7ttgX1Z5aIBcDAhc/18c5DlRQMpd+M3dZcfwMBPcMUfdr5b5Th75H5n5X535P41dw90RtBxpIFKMqZYR254xSEk8zc0BN925z0glG4pIppHdIAeMXdoN3cfD5cRMz4kNIoYo4oEpkpUp0pMpspcp1N1dvR3gkgKYqVkENh6gRokETd0iGYUgppthLh/hPg6xRICi3DJcy8KAK9h
Dq82R5jFjgt2luN8B7NeCG9XMz5JAYIpxwNphUwCQYI+8cJf4ioh8MjGNjdSJmpnB25qESZ7tJU18wDOitcLofjdpD9yET9UBYR2ozgi0tpLgkFzh3jNU2EID8Qn8eEfj+FP9wcETRFf83p/8pFBsoDZR5FFQQDBV+U3COEZtgD9EFsg1iSkDmo8EI1rFztsClEHFPUi0p9GDRhRZGDmC614RzgujiTRDgdBYJCwTSYuCpYtk0lUC+x0Dtswd1ZI
cY00dZCljM4sdFDsTdlKgVhxJIFpoJg8B5gR5zhjkJgEARplItw6xEhsBZIrkMpNAPgEBagvkjRD1+UwAZ4Fk55cDcINJMCi8HJoUt4eNoNEV94gNdj0VKgYB9AKBo4YBLwKAmh+xsBagbwKA1QJh4IOA1RnB4gEVSJv5yhCpFFIAKVNhqEON4wLgn1Ji4tDgtpqyq5RJKYqZuIZ9SIBpJskhqVQkKwKZZg/gSZwxis5U2pUFC0thxhayJ0YSGs4
SmsRFeFqRkTBFUTusf9bVMTJES4cTZFKg/oAZNI9VQCECSTIDDy/UYD5s4DvRqTQ1kD1t5szt4C1SBAWS0A1gaEZpJVnsq0y16Zxg/iPEgKeTYEKZQwjhUFB1hTEdqx7tz0AK9ZAdgdQcMlZSttUBRwFT8lNZ5wMCVSVkmT5CNSpTi4yhS4EMrgXkp83krg3kA4rl4hNAuUaFrkrkSZphcAn1ZgeJ4gqgqg3Sfk9Jp4+cVwk0zJXTAzXCsRN5YUw
zd4IzkUoyyC0U9Yz4pwABHaOFYRCGCXEWoCgcDCUJoSgG8JoKCGAKCSOS8C4gqP+clA4VBRjNBO4H8+SW9Rs6YFYWIrYYciaFI0JA/MoHsz4s/YtBEc4SEO4W/I/Erb87TRBY4c9VnSYEmYk/aRckDDhNE1cqkdci1e6b/dAG1cRfrAAmbY8hRU88bIk6bXE4bObCk+8i8mkixV8wg986HYvT8gWc4GaeqeoFCsCvxemZYUayAatJgr7G0Iw0JT4
J9OChHGJWEc9OkiU9tCivgyATbZU1WAQpUoit9dHXqzHWdVJSikUMUfbCQfZbuSEeSWEFYESESV62SVne5KmbAJ9WuPUiSJBeIcs5UCeNdX5PSCS8oKSpOU4B1PbLAhNCXEMxSzwveVS3qwKcAFWXCOAOAKUfmGDIoaABaLISoIgErIYBgQgBAEyzrTc0qiAF/ISlm2kCAGTB2BXUsfQKUPKrclrY1drMoDmiRfsbm8Demu6NE8qv/PcyAEWrmzI
KCA8oA284mhWkTbm3mvUc8jmKmjWqAMWzIbWyGJq2bPRYWkQUW7mmCQNc7dxdmq2xW/QRCJ82kzGS2zmzWpWzgKAKCFeMUZqUax2r2w27mqCX2iUQgUCT1BJEO62zIJoLAKAaOCm+mMiHuOW+O52gm/PSzQvcXbO72/QKcFYgvYzSvDY9Wp24u+YpoILCAbGNmrSHEcUFobgeEPy+4pRJFcUAATQhC2GoXUjpW7sgCMDYAMCJppgICEDrQbyLrDs
yFtvTXO0bvBzZuZBICjpjo5jjq3uIClAQE02DoPoaDYHnlLqUmCB2qpoPoRJPnA2JDPl9npAAAplh8xeBiZqBv6v6ph5gABKJUGCBAZQP0AUSoV+3AD+uYX+o4DEXgOBuTIBjpMAXNeWmuqAE21KOAKAGMPGJG6oPbUBkyUgEo7wp3a+wmbEOexFIgTTWh3aiADgFeJ8Jhu0IQKAcidh0gOeheuweCEeXICUVhuAc+y+1hzQG+nHaU4mhkfBxgJo
Se/Aaeh8BusIYID2LxRFJyKAAweuqZUisici2R664hgwCUDIbRugkHXHfnLEaObRpRlRlFOZcAXNaoW670YAIid8IAA=
%%