And as for the previous point.
Quote: “Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training — the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures — have plateaued.” So yeah, finally they’ve started admitting how things actually are. And I think Ilya understands damn well that we need other methods and algorithms for the next step. Transformers have pretty much shown what they’re capable of in full.
Original (RU)
Ну и касательно предыдущего тезиса.
Цитата: "Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued." Вообщем наконец-то начали признавать как обстоят дела. И думаю что Илья хорошо понимает что нужны другие методы и алгоритмы для следующего шага. Трансформеры в полной мере примерно показали себя.