AI is Climbing the Wall -- Fast
You said: Somebody is reporting the Ilya is saying that AI is hitting a wall due to diminishing returns on data size. I absolutely disagree with this. Review what I have to say, let's discuss and put it into a compelling reply: I think that getting discouraged by concentration on limitations of LLMs and raw scaling is what will kill the ones who lose the race. Just dumping data in the hopper has been remarkably successful up until now, but that data's problem is not lack of size it is lack of quality. Further, the way models are both built and deployed is sub-optimal. The compute we have in place is already well beyond what I would consider ultimately necessary for spectacularly powerful inference well beyond what we see today. Beyond that, there is a lot of manual work in the pits when it comes to building things that will be sorted out and automated in fairly short order. We are just coming in to the part of the curve where AI itself will take on a rapidly increasing role i...