GPT-3
basics https://en.wikipedia.org/wiki/GPT-3
Real, Live GPT-3 Apps You Can Use
Language Models are Few-Shot Learners
GPT-3: Language Models are Few-Shot Learners Grigory Sapunov
video GPT-2: Why Didn't They Release It? - Computerphile
- distillation
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, Jeff Dean
Applications
GPT-3 bottleneck is training data | François Chollet and Lex Fridman
GPT-2
https://github.com/openai/gpt-2
- Words of wisdom from GPT-2
GPT-2 is an application that produces novel text based on a text prompt (Here’s how to install it).
Most of what it generates hovers on the border of sense and nonsense. Here are four pieces of advice it has given to me. — MF
• “If we cannot make sense of the present, we cannot plan for the future.”
• “The more we change to meet the challenges in our lives, not to stay where we are, the more successful we will be.”
• “If it is an animal you have seen, do you think it will be like you, too? This is the first statement of my doctrine, that all things are one thing. Do you want me to prove it, or to let it pass?” —Philosopher Epicurus (384-399 A.D.)
• “Your gut tells you it’s not right to continue investing in some idea–it’s too risky, because its consequences are hard to understand. Your gut is right about one thing, though: there is no way of knowing it’s not wrong to spend your energy making the best decision for yourself and your goals.”