AI

NN Memory

• Memory: 3.6 bytes per parameter
• Double Descent: At first, the model tries to just memorize the data, so it gradually improves. But once its memory fills up, it has to slowly start discarding what it memorized and actually learn the underlying logic of the subject. As a result, the model keeps getting better over time, then suddenly its quality drops, and after that, it begins improving again.

Singularity

• Researcher: Ray Kurzweil
• Reference: Based on Moore's law

Godfathers

• AI

  

• Convolutional NN