Not signed in (Sign In)

Start a new discussion

Not signed in

Want to take part in these discussions? Sign in if you have an account, or apply for one below

  • Sign in using OpenID

Site Tag Cloud

2-category 2-category-theory abelian-categories adjoint algebra algebraic algebraic-geometry algebraic-topology analysis analytic-geometry arithmetic arithmetic-geometry bundles calculus categorical categories category category-theory chern-weil-theory cohesion cohesive-homotopy-type-theory cohomology colimits combinatorics complex-geometry computable-mathematics computer-science constructive cosmology deformation-theory descent diagrams differential differential-cohomology differential-equations differential-geometry digraphs duality elliptic-cohomology enriched fibration finite foundations functional-analysis functor galois-theory gauge-theory gebra geometric-quantization geometry graph graphs gravity grothendieck group group-theory harmonic-analysis higher higher-algebra higher-category-theory higher-differential-geometry higher-geometry higher-lie-theory higher-topos-theory homological homological-algebra homotopy homotopy-theory homotopy-type-theory index-theory integration integration-theory internal-categories k-theory lie lie-theory limit limits linear linear-algebra locale localization logic manifolds mathematics measure-theory modal modal-logic model model-category-theory monads monoidal monoidal-category-theory morphism motives motivic-cohomology nforum nlab noncommutative noncommutative-geometry number number-theory of operads operator operator-algebra order-theory pages pasting philosophy physics pro-object probability probability-theory quantization quantum quantum-field quantum-field-theory quantum-mechanics quantum-physics quantum-theory question representation representation-theory riemannian-geometry scheme schemes set set-theory sheaf simplicial space spin-geometry stable-homotopy-theory string string-theory superalgebra supergeometry svg symplectic-geometry synthetic-differential-geometry terminology theory topology topos topos-theory type type-theory universal variational-calculus

Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.

Welcome to nForum
If you want to take part in these discussions either sign in now (if you have an account), apply for one now (if you don't).
  1. This is a brief description of the construction that started appearing in category-theoretic accounts of deep learning and game theory. It appeared first in Backprop As Functor (https://arxiv.org/abs/1711.10455) in a specialised form, but has slowly been generalised and became a cornerstone of approaches unifying deep learning and game theory (Towards Foundations of categorical Cybernetics, https://arxiv.org/abs/2105.06332), (Categorical Foundations of Gradient-based Learning, https://arxiv.org/abs/2103.01931).

    Our group here in Glasgow is using this quite heavily, so since I couldn’t find any related constructions on the nLab I decided to add it. This is also my first submission. I’ve read the “HowTo” page, followed the instructions, and I hope everything looks okay.

    There’s quite a few interesting properties of Para, and eventually I hope to add them (most notably, it’s an Para is an oplax colimit of a functor BM -> Cat, where B is the delooping of a monoidal category M).

    A notable thing to mention is that I’ve added some animated GIF’s of this construction. Animating categorical concepts is something I’ve been using as a pedagogical tool quite a bit (more here https://www.brunogavranovic.com/posts/2021-03-03-Towards-Categorical-Foundations-Of-Neural-Networks.html) and it seems to be a useful tool getting the idea across with less friction. If it renders well (it seems to) and is okay with you, I might add more to the Optics section, and to the neural networks section (I’m hoping to get some time to add our results there).

    Bruno Gavranović

    v1, current

    • CommentRowNumber2.
    • CommentAuthorbgavran
    • CommentTimeJul 22nd 2021
    Hi, I am Bruno Gavranović, the autor of the above post. I am realising that on the nforum I am registered as bgavran, but I put my full name as the author of the post above. From what I can tell, I need to add the full name to my account here, which I've done so now.

    Hopefully that fixes it! Apologies, I am still trying to understand how the system works
    • CommentRowNumber3.
    • CommentAuthorbgavran
    • CommentTimeJul 22nd 2021

    Added reference

    diff, v2, current

    • CommentRowNumber4.
    • CommentAuthorDavid_Corfield
    • CommentTimeJul 22nd 2021

    A little editing. It be good to link the page to other related concepts, and have links back.

    diff, v3, current

    • CommentRowNumber5.
    • CommentAuthorDavid_Corfield
    • CommentTimeJul 22nd 2021

    Or maybe lenses aren’t specifically a related concept, but anyway we tend to have a section for these.

    Like the graphics, by the way.

    • CommentRowNumber6.
    • CommentAuthorDavid_Corfield
    • CommentTimeJul 22nd 2021

    Added more links

    diff, v3, current

    • CommentRowNumber7.
    • CommentAuthorUrs
    • CommentTimeJul 22nd 2021

    Is it clear that you want the “para construction” instead of the standard Kleisli category of a reader comonad?

    The latter has

    • a fixed paramter object PP;

    • morphisms from AA to BB are of the form P×AfBP \times A \xrightarrow{\;\; f \;\;} B;

    • composition from AA to BB to CC is much as in “para”, but precomposed with the diagonal morphism:

      P×AΔ×idP×P×Aid×f 1P×Bf 2C P \times A \xrightarrow{ \Delta \times id } P \times P \times A \xrightarrow{ id \times f_1 } P \times B \xrightarrow{\;\; f_2 \;\;} C

    Do you have considered this? If so, could you say in which examples/applications this is insufficient?

    My gut feeling would be that this Reader-comonad Kleisli category is the relevant one in applications, as you will want to be parameterizing over a fixed set of parameters, not regard every imaginable set as a potential set of parameters in a given application. No?

    • CommentRowNumber8.
    • CommentAuthorrongmin
    • CommentTimeJul 22nd 2021
    • (edited Jul 23rd 2021)

    Re. #2:

    Hi, Bruno! Finally, we have animated GIFs on the nLab, and I’d like to second David Corfield’s sentiment in #5.

    Re. #7:

    you will want to be parameterizing over a fixed set of parameters, not regard every imaginable set as a potential set of parameters in a given application.

    In the original construction of Fong et al., the underlying category is the category of Euclidean spaces. The dimension of the parameter object can vary, since it is the number of edges connecting two layers of a neural network, and this can vary in most neural network architectures. Hence, the parameter object need not be fixed.

    Is it clear that you want the “para construction” instead of the standard Kleisli category of a reader comonad?

    It could be, though the comonad would be formed from different objects in the same underlying category. Perhaps one could link the page to Kleisli category, though it would make the page easier to find and reference if it was renamed to, say, “Para”. The bicategory could perhaps be called the bicategory of “parametric morphisms”, in a nod to parametric right adjoints (and to avoid the s/z choice in spelling).

    Since differential categories appear in this context, I’d also like to highlight a paper by Wallbridge1 that gives a model of differential linear logic using, in part, the Kleisli category of the jet comonad. Wallbridge was motivated by potential computational applications, so this should be interesting for Bruno and others.


    1. James Wallbridge, Jets and differential linear logic, Mathematical Structures in Computer Science, Vol 30, Issue 8, September 2020, pp. 865 - 891. (arXiv:1811.06235, journal

    • CommentRowNumber9.
    • CommentAuthorUrs
    • CommentTimeJul 23rd 2021

    Renaming the entry to something descriptive like “parametric morphisms” would be good.

    • CommentRowNumber10.
    • CommentAuthorbgavran
    • CommentTimeJul 24th 2021
    • (edited Jul 24th 2021)

    Re #7

    As rongmin pointed out, indeed the Kleisli category of the reader comonad is not what we want (aside: I’ve seen that called also a CoKleisli category, is there a consensus on how to call these things?) in machine learning/game theory. The parameter space of a neural network/economic agent is the data of the morphism itself, and is not fixed beforehand. Even more, when composing two agents together, we don’t want the composite agent to copy the strategy and use it twice: we want each individual agent to be able to choose their own strategy.

    But nonetheless, these constructions are related. If we think of the reader comonad as an oplax functor 1Cat1 \to \mathbf{Cat}, then its Kleisli category is the oplax colimit of that functor. Similarly, we recover Para(𝒞)\mathbf{Para}(\mathcal{C}) as an oplax colimit of the composite oplax functor

    B𝒞BB[𝒞,𝒞]Cat\mathbf{B}\mathcal{C} \xrightarrow{\mathbf{B}\otimes'} \mathbf{B}[\mathcal{C}, \mathcal{C}] \hookrightarrow \mathbf{Cat}

    where B(𝒞)\mathbf{B}(\mathcal{C}) is the delooping of a monoidal category and I’m using \otimes' to mean :𝒞×𝒞𝒞\otimes : \mathcal{C} \times \mathcal{C} \to \mathcal{C} under the tensor-hom adjunction.

    I think this is the idea behind graded comonads, but I’m not entirely sure. I also think special care needs to be taken to get 2-cells in Para(𝒞)\mathbf{Para}(\mathcal{C}) to come up nicely that way (or we can just think of Para(𝒞)\mathbf{Para}(\mathcal{C}) as a category, appropriately quotiented).

    • CommentRowNumber11.
    • CommentAuthorUrs
    • CommentTimeJul 24th 2021

    Thanks! That would be good to discuss in the entry, to show that there is category theory behind these definitions.

    • CommentRowNumber12.
    • CommentAuthorDavid_Corfield
    • CommentTimeJul 24th 2021

    I think this is the idea behind graded comonads…

    So we’ve got how to generalize monad to graded monad, say as lax 2-functor from the delooping of MM to CatCat, BMB M rather than 1\mathbf{1}.

    At comonad we don’t give alternative definitions. What is it, oplax 2-functor?

    • CommentRowNumber13.
    • CommentAuthorbgavran
    • CommentTimeJul 25th 2021

    At comonad we don’t give alternative definitions. What is it, oplax 2-functor?

    Indeed, it should be an oplax 2-functor.

    • CommentRowNumber14.
    • CommentAuthorrongmin
    • CommentTimeJul 26th 2021
    • (edited Jul 26th 2021)

    Re. #10:

    I’ve seen that called also a CoKleisli category, is there a consensus on how to call these things?

    I think some people would like to call the Kleisli category of a comonad a “co-Kleisli category” to allow them to omit specifying a comonad, whereas others want to fix the name “Kleisli category” and specify whether it is of a monad or a comonad.

    If we think of the reader comonad as an oplax functor 1Cat1 \to \mathbf{Cat}, then its Kleisli category is the oplax colimit of that functor. Similarly, we recover Para(𝒞)\mathbf{Para}(\mathcal{C}) as an oplax colimit of the composite oplax functor B𝒞BB[𝒞,𝒞]Cat\mathbf{B}\mathcal{C} \xrightarrow{\mathbf{B}\otimes'} \mathbf{B}[\mathcal{C}, \mathcal{C}] \hookrightarrow \mathbf{Cat}

    Very interesting.

    I think this is the idea behind graded comonads,

    and graded monads1 are apparently also known as “parametric monads”, so could we say that Para(𝒞)\mathbf{Para}(\mathcal{C}) is the Kleisli category of some 𝒞\mathcal{C}-parametric comonad?

    we can just think of Para(𝒞)\mathbf{Para}(\mathcal{C}) as a category, appropriately quotiented

    I think experience has shown that we get more interesting things if we don’t take quotients, but it is more expedient to do so.


    1. The graded monad entry mentions coeffects, but there’s nothing there at the moment, so I googled and found this website about context-aware programming languages by Tomas Petricek

Add your comments
  • Please log in or leave your comment as a "guest post". If commenting as a "guest", please include your name in the message as a courtesy. Note: only certain categories allow guest posts.
  • To produce a hyperlink to an nLab entry, simply put double square brackets around its name, e.g. [[category]]. To use (La)TeX mathematics in your post, make sure Markdown+Itex is selected below and put your mathematics between dollar signs as usual. Only a subset of the usual TeX math commands are accepted: see here for a list.

  • (Help)