it's already a great piece, but if i had to make a few suggestions they would be:
1. another paragraph on scaling laws and the degree to which the future turns on how long they continue. perhaps some discussion of what it mean to be "bitter lesson pilled" and the degrees thereof
A good primer. One quibble. The so called Scaling Laws are not laws as regards LLM’s. The idea of scaling as regards model capabilities Open AI and others declared victory a little early and claimed that all we need to do is throw more compute at it and will reach AGI. Whatever that really means. But this scaling is not very cost efficient unlike network effects.
This was wonderful. Thanks for writing it up—immediately sending it to my smart, non-AI-pilled friends :).
Thank you, Simone!
very good primer for the uninitiated!
it's already a great piece, but if i had to make a few suggestions they would be:
1. another paragraph on scaling laws and the degree to which the future turns on how long they continue. perhaps some discussion of what it mean to be "bitter lesson pilled" and the degrees thereof
2. the doomer/accelerationist axis
Thank you! Great suggestions.
A good primer. One quibble. The so called Scaling Laws are not laws as regards LLM’s. The idea of scaling as regards model capabilities Open AI and others declared victory a little early and claimed that all we need to do is throw more compute at it and will reach AGI. Whatever that really means. But this scaling is not very cost efficient unlike network effects.