Not signed in (Sign In)

Not signed in

Want to take part in these discussions? Sign in if you have an account, or apply for one below

  • Sign in using OpenID

Site Tag Cloud

2-category 2-category-theory abelian-categories adjoint algebra algebraic algebraic-geometry algebraic-topology analysis analytic-geometry arithmetic arithmetic-geometry book bundles calculus categorical categories category category-theory chern-weil-theory cohesion cohesive-homotopy-type-theory cohomology colimits combinatorics complex complex-geometry computable-mathematics computer-science constructive cosmology deformation-theory descent diagrams differential differential-cohomology differential-equations differential-geometry digraphs duality elliptic-cohomology enriched fibration foundation foundations functional-analysis functor gauge-theory gebra geometric-quantization geometry graph graphs gravity grothendieck group group-theory harmonic-analysis higher higher-algebra higher-category-theory higher-differential-geometry higher-geometry higher-lie-theory higher-topos-theory homological homological-algebra homotopy homotopy-theory homotopy-type-theory index-theory integration integration-theory k-theory lie-theory limits linear linear-algebra locale localization logic mathematics measure-theory modal modal-logic model model-category-theory monad monads monoidal monoidal-category-theory morphism motives motivic-cohomology nforum nlab noncommutative noncommutative-geometry number-theory of operads operator operator-algebra order-theory pages pasting philosophy physics pro-object probability probability-theory quantization quantum quantum-field quantum-field-theory quantum-mechanics quantum-physics quantum-theory question representation representation-theory riemannian-geometry scheme schemes set set-theory sheaf sheaves simplicial space spin-geometry stable-homotopy-theory stack string string-theory superalgebra supergeometry svg symplectic-geometry synthetic-differential-geometry terminology theory topology topos topos-theory tqft type type-theory universal variational-calculus

Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.

Welcome to nForum
If you want to take part in these discussions either sign in now (if you have an account), apply for one now (if you don't).

nLab > nLab Technical Matters: Scraping nLab

Bottom of Page

1 to 7 of 7

  1.  
    • CommentRowNumber1.
    • CommentAuthorDaniel Geisler
    • CommentTimeApr 22nd 2024
    Hello, my name is Daniel Geisler and I'm new here. I'm providing software support to Valeria de Paiva. I'm working with the nLab corpus from a couple of years ago, but nLab is 90% larger now. I'm using Python software to scrape nLab so a current corpus of nLab can be built. This is just a heads up that I am scraping nLab. I'll try to access nLab with a short list of pages for debugging, but I just had to run upto XYZ before I generated an error.
    • CommentRowNumber2.
    • CommentAuthorUrs
    • CommentTimeApr 22nd 2024

    Hi Daniel,

    thanks for writing in; sounds interesting.

    In case there is anything concerning the nLab’s server, let me know and I can bring you in contact with our technical team.

    • CommentRowNumber3.
    • CommentAuthorDmitri Pavlov
    • CommentTimeApr 22nd 2024

    Re #1: A repository with the source code of all nLab pages is available here: https://github.com/ncatlab/nlab-content, and a repository with the compiled HTML code of all nLab pages is available here: https://github.com/ncatlab/nlab-content-html.

    • CommentRowNumber4.
    • CommentAuthorDaniel Geisler
    • CommentTimeApr 24th 2024
    Thank you Dmitri. I was able to scrape nLab a couple of days before by using the Python code from the previous scrape. Next task is to get a version of LaTeXML running on my computer and then use spaCy to build the corpus conll file. Then I can run statistics using UD stats.py.
    • CommentRowNumber5.
    • CommentAuthorzskoda
    • CommentTimeApr 24th 2024

    What does it mean “to scrape” in this context ?

    • CommentRowNumber6.
    • CommentAuthorUrs
    • CommentTimeApr 24th 2024

    Wikipedia: Web scraping

  7. I need to create documentation for how to create a nLab corpus from scratch. Both an overview of the project as well as software installation and configuration. If I created a nLab corpus page or category on nLab then other interested parties could contribute information or questions. Does this sound like a good approach?

1 to 7 of 7

  1.