Added two articles

Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković,

*Categorical Deep Learning: An Algebraic Theory of Architectures*[arXiv:2402.15332]Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Liò, Paolo Di Lorenzo, Vasileios Maroulas, Nina Miolane, Farzana Nasrin, Karthikeyan Natesan Ramamurthy, Bastian Rieck, Simone Scardapane, Michael T. Schaub, Petar Veličković, Bei Wang, Yusu Wang, Guo-Wei Wei, Ghada Zamzmi,

*Position Paper: Challenges and Opportunities in Topological Deep Learning*[arXiv:2402.08871]

Position Paper: Challenges and Opportunities in Topological Deep Learning

]]>Added breakdown of Neural Network Gaussian process (NNGP) results, Neural tangent kernel (NTK) theory and more recent approaches to QFT (Neural Network field theory, NNFT, and the latest paper in that direction).

One could also think of making a separate page for Neural tangent kernel theory and moving some of the large width build-up there. There, there’s some obvious reference links one could add and I might later at one point. (In the style I’d like to further clean up the somewhat over reliance on brackets in my paragraph and remove the explanation-by-comparison to classical mechanics by the actual formulas, albeit even the Wikipedia breakdown isn’t that bad.) There were already references in that direction, but no main text. Feel free to alter any running text.

I’m personally mostly interested in the field theory and stochastics stuff, but the article could also bridge to information geometry results

]]>Added some references for *topological* deep learning

Ephy R. Love, Benjamin Filippenko, Vasileios Maroulas, Gunnar Carlsson,

*Topological Deep Learning*(arXiv:2101.05778)Mathilde Papillon, Sophia Sanborn, Mustafa Hajij, Nina Miolane,

*Architectures of Topological Deep Learning: A Survey on Topological Neural Networks*(arXiv:2304.10031)Mustafa Hajij et al.,

*Topological Deep Learning: Going Beyond Graph Data*(pdf)

Since revision 4 the Idea-section starts out with

A

neural networkis a class of functions used…

This seems a little strange. Maybe what is meant is:

Neural networksare a class of functions used…

But either way, the sentence conveys no information about the nature of neural networks.

]]>adding information about how neural networks are related to differential equations/dynamical systems.

Anonymous

]]>added pointer to today’s:

- Daniel A. Roberts, Sho Yaida, Boris Hanin,
*The Principles of Deep Learning Theory*, Cambridge University Press 2022 (arXiv:2106.10165)

Good. I have added these further references in this direction:

Further discussion under the relation of renormalization group flow to bulk-flow in the context of the AdS/CFT correspondence:

Yi-Zhuang You, Zhao Yang, Xiao-Liang Qi,

*Machine Learning Spatial Geometry from Entanglement Features*, Phys. Rev. B 97, 045153 (2018) (arxiv:1709.01223)W. C. Gan and F. W. Shu,

*Holography as deep learning*, Int. J. Mod. Phys. D 26, no. 12, 1743020 (2017) (arXiv:1705.05750)J. W. Lee,

*Quantum fields as deep learning*(arXiv:1708.07408)Koji Hashimoto, Sotaro Sugishita, Akinori Tanaka, Akio Tomiya,

*Deep Learning and AdS/CFT*, Phys. Rev. D 98, 046019 (2018) (arxiv:1802.08313)

]]>

Added a small mention of the relation with renormalisation group flow

]]>I will indeed add something like this, but have not had a chance yet. Am prioritising the fundamental functionality first; I wouldn’t be averse to adding something sooner, but I need to think it through a bit (I’d rather not put in place some quick hack that people get used to, which later causes problems!).

For example, I think we probably should stick to BibTex’s convention of having `article`

only refer to published articles, because this allows validation: we can require that a journal, etc, is given, and in fact such a requirement is already implemented. Of course we could make up our own new document type such as ’preprint’ or something (but allow use of ’misc’ or ’unpublished’, converting to ’preprint’ behind the scenes if for example the arXiv field is present).

Thanks. In fact `article`

would be more sensible. To refer to an arXiv preprint as a “miscellaneous” reference is a weird anachronism!

So I am happy to stick with your (currently) supported fields!

But when I go to the edit pane you made, all I get to see is a big white box and no indication what to do.

If you could just make the edit pane show a rudimentary template of fields into which the user could then type their data, that would already get us started!

]]>Yes, it only accepts the ’article’ document type for now. But completely agree this is exactly the kind of thing the bibliography is for :-)! I need to complete the ability to edit references in the bibliography, and then I will add support for all the common document types. Have had to focus on other things in the nLab software recently, but I’ll work on this when I have a chance.

]]>I see. That reminds me that we should use Richard’s new bibtex-like functionality to harmonize formatting. Maybe once that is a little more convenient to use: I just tried to offer it the bibtex data as produced by the arXiv

```
@misc{spivak2021learners,
title={Learners' languages},
author={David I. Spivak},
year={2021},
eprint={2103.01189},
archivePrefix={arXiv},
primaryClass={math.CT}
}
```

but it does not swallow that.

]]>But I would always punctuate after a title

- Corfield, D. 2003. Towards a philosophy of real mathematics. CUP.

or something like that.

]]>Sorry for raising a trivial point on formatting:

In a reference, let’s not have a comma before the parenthesis with the arXiv number, it doesn’t seem to be needed. What do you think?

]]>Added two more category-theoretic treatments

David Spivak,

*Learners’ languages*, (arXiv:2103.01189)G.S.H. Cruttwell, Bruno Gavranović, Neil Ghani, Paul Wilson, Fabio Zanasi,

*Categorical Foundations of Gradient-Based Learning*, (arXiv:2103.01931)

Removed the redirect to ’machine learning’, as this is far more general.

]]>Added two more category-theoretic treatments

David Spivak,

*Learners’ languages*, (arXiv:2103.01189)G.S.H. Cruttwell, Bruno Gavranović, Neil Ghani, Paul Wilson, Fabio Zanasi,

*Categorical Foundations of Gradient-Based Learning*, (arXiv:2103.01931)

Removed the redirect to ’machine learning’, as this is far more general.

]]>Added that article in #3.

]]>What I don’t understand yet in HSTT 18 is where the non-linear activiation functions are in the story, i.e. how is what they have different from a discretized solution of a differential equation. But I don’t really have time to look into this properly.

]]>I never got round to looking at

- Brendan Fong, David Spivak, Rémy Tuyéras,
*Backprop as Functor: A compositional perspective on supervised learning*, (arXiv:1711.10455)

added these references on the learning algorithm as analogous to the AdS/CFT correspondence:

Yi-Zhuang You, Zhao Yang, Xiao-Liang Qi,

*Machine Learning Spatial Geometry from Entanglement Features*, Phys. Rev. B 97, 045153 (2018) (arxiv:1709.01223)W. C. Gan and F. W. Shu,

*Holography as deep learning*, Int. J. Mod. Phys. D 26, no. 12, 1743020 (2017) (arXiv:1705.05750)J. W. Lee,

*Quantum fields as deep learning*(arXiv:1708.07408)Koji Hashimoto, Sotaro Sugishita, Akinori Tanaka, Akio Tomiya,

*Deep Learning and AdS/CFT*, Phys. Rev. D 98, 046019 (2018) (arxiv:1802.08313)

Stub. For the moment just for providing a place to record this reference:

- Jean Thierry-Mieg,
*Connections between physics, mathematics and deep learning*, Letters in High Energy Physics, vol 2 no 3 (2019) (doi:10.31526/lhep.3.2019.110)