Ilya Sutskever (co-inventor of AlexNet – convolutional neural network): the era of simply scaling AI has come to an end

He said that the results from scaling pre-training of AI models have reached a plateau.

“Now we are back in the era of miracles and discoveries. Everyone is looking for something new,” said Ilya Sutskever.

Sutskever declined to share more details about how his team is solving this problem, only mentioning that SSI is working on an alternative approach to scaling pre-training.

What does this mean for the industry?

1. Simply increasing computing power and data volume no longer produces the same results

2. New approaches and breakthrough ideas are needed

3. Researchers are looking for alternative methods to improve AI models