Skip to content

Commit 664579e

Browse files
authored
Update notes to be clearer
1 parent 09f049d commit 664579e

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,11 +30,11 @@ Used in [Memory-efficient episodic control reinforcement learning with dynamic o
3030

3131
Notes about this codebase based on conversations with Robert Kirk
3232

33-
- This uses brute-force kNN search, as opposed to the approximate kNN search used in the original works
34-
- This uses separate hash functions, as in the [original works](https://github.com/EndingCredits/Neural-Episodic-Control/issues/4)
35-
- The keys and values in the NEC table are not updated - this is a bug!
36-
- Key/value gradients are taken from the first instance of each updated key, but averaging would make more sense
37-
- Despite the bug, this can still reproduce the results from the NEC paper (hence those updates may not be important)
33+
- This uses brute-force kNN search, as opposed to the approximate kNN search used in the original works. Without knowing the full details of DeepMind's k-d tree, especially how they might update the tree with new data, I chose to stick to the exact version to be safe.
34+
- As in the [original works](https://github.com/EndingCredits/Neural-Episodic-Control/issues/4), this uses a hash of the state in order to detect exact state matches.
35+
- Key/value gradients for the DND are taken from the first instance of each updated key, but averaging would make more sense.
36+
- The gradients for the keys and values in the DND do not get applied properly (effectively, the gradients are zero) - this is a bug! Despite the bug, this codebase can still reproduce the results from the NEC paper (hence those updates may not be important).
37+
3838

3939
## Acknowledgements
4040

0 commit comments

Comments
 (0)