Author: tomr Format: TextSam Altman posted stream of conversation videos on his X account, where they all recognize that the training data may be exhausted and that data efficiency should be considered in the earnest and this stream contains this [[https://x.com/sama/status/1910363431141802416/quotes]] as well (I don't know how to grab individual tweet, but these are the quotes that wrap the video in this tweet).
I think that this conversation deserves attention for at least 2 points:
1) this is the first time (AFAK), when major AI company recognizes that there may be some principled way to the A*I;
2) this can be important for the academic community as well - if the big companies start to consider the development of A*I as an approximation of Solomonoff induction and AIXI then this will really (re)ignite theoretical work (and I don't see the problem if that is automated). And this will be really exciting.
Sorry for sharing this. If this is not appropriate place for the celebration then it is OK to delete it.
But well - maybe this can start new entry in these pages.
There have already been some work:
[[https://arxiv.org/abs/2408.12065v1]] Transformers As Approximations of Solomonoff Induction
[[https://openreview.net/forum?id=tJDlRzQh7x]] Neural Networks and Solomonoff Induction (Marcus Hutter is one of the co-author, but community didn't accept the paper)
And I can guess, that the expression involving all the theories somehow can be expressed as some functional over category of theories. So - this can reinvigorate the category theory as well. And if we are striving for the utmost expressivity (going beyond FOL and HOL), then we should consider category of homotopy type theories as well and hence - the contextual categories and infinity category theory.
Sam Altman posted stream of conversation videos on his X account, where they all recognize that the training data may be exhausted and that data efficiency should be considered in the earnest and this stream contains this //x.com/sama/status/1910363431141802416/quotes (https) as well (I don't know how to grab individual tweet, but these are the quotes that wrap the video in this tweet).
I think that this conversation deserves attention for at least 2 points: 1) this is the first time (AFAK), when major AI company recognizes that there may be some principled way to the A*I; 2) this can be important for the academic community as well - if the big companies start to consider the development of A*I as an approximation of Solomonoff induction and AIXI then this will really (re)ignite theoretical work (and I don't see the problem if that is automated). And this will be really exciting.
Sorry for sharing this. If this is not appropriate place for the celebration then it is OK to delete it.
But well - maybe this can start new entry in these pages.
And I can guess, that the expression involving all the theories somehow can be expressed as some functional over category of theories. So - this can reinvigorate the category theory as well. And if we are striving for the utmost expressivity (going beyond FOL and HOL), then we should consider category of homotopy type theories as well and hence - the contextual categories and infinity category theory.