- I love the “it is unclear why” in this title. So many times readers/reviewers want everything neatly tied up by page 8. Identifying a problem is work! It’s hard!
- 1/ We found that deep sequence models memorize atomic facts "geometrically" -- not as an associative lookup table as often imagined. This opens up practical questions on reasoning/memory/discovery, and also poses a theoretical "memorization puzzle."
