Want to take part in these discussions? Sign in if you have an account, or apply for one below
Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.
1 to 31 of 31
I have finally completed an initial implementation of something I have long been planning (I began over 2 years ago!), and which has long been wished for: a proper referencing mechanism for the nLab.
It is far from complete, but the basics are there, and it can be taken into use immediately. I have given an example in the current version of the Sandbox of how to use \cite and \bibitem for items that have already been added to the central nLab bibliography. To add a new reference to the bibliography, one does the same kind of thing as for creating a page, etc: one just writes \bibitem{MyExpectedCitationKey} on the page, it will show up as MyExpectedCitationKey? when you render, and the ? is a link to the page for creation. After you save the reference, you will be taken to a little page which displays the reference, which can also be viewed directly at /nlab/reference/show/MyCitationKey. For example, the two references I have added can be viewed here and here.
I think we would all agree that the value of having a central bibliography rather than referencing on individual pages is potentially considerable, so I hope people will take this into use! The idea is that \cite and \bibitem should essentially reproduce what is typically done by hand currently.
I have had to make some choices along the way. For example, a ’citation key’ is generated from the bibliography entry. It follows the rule: concatenation of surnames of all authors plus year. It is not perfect, and I am open to suggestions, but it should be OK for now, and is similar to what we currently do. To get this to work well, it is important that the author field of the bibliography entry is entered in a certain format. This is explained in the instructions on the reference creation page. Links to nLab pages are allowed.
You will see that I suggest to create a bibliography entry by fetching a BibTex entry from Zentralblatt and pasting it in, modifying the author field. Zentralblatt has always been openly available with some search restrictions, but since this year is now completely open. Thus I think we should use it rather than MathSciNet, and should ideally not use home-grown BibTex files, although there is little that can be done to stop this currently (it is not a disaster if people do this, but having a canonical source is preferable). Zentralblatt plan to create APIs for interaction with their data, which would probably be very useful for us in the future.
Currently I have only implemented rendering of ’article’ type BibTex entries! Obviously I will implement the other most common types asap, but ’article’ should be enough for people to try things out. Again, I have followed how Zentralblatt styles the rendering, but I am open to other ways to do it. There are a million other things we can later do with this, e.g. generate the references rather than manually typing ’bibitem’, but it’s a start. One thing we will definitely need is some kind of search functionality for bibliography, or at least (to begin with) a list of the entire bibliography. Again, I will add something like this soon. Also editing of references will be needed. Maybe they should be announced at the nForum when created/edited. Etc.
This looks extremely good, thanks a lot!
Concerning the last paragraph, a really useful bit of functionality would be to list articles that cite a given bibliographic entry on that entry’s page.
What is supposed to happen when we need to cite more than one paper by the same author published in the same year?
Also, in case there is an arXiv version and a journal version, which one of the two (typically different) years are we supposed to use?
Great!
It is really nice and amazing how much you do to improve the software.
It has been voiced many times since the beginning of the Lab by various people that having help from a central bibliography would be helpful for many purposes.
On the other hand, I am afraid (what can be justified or not, I don’t know) that
previous literature records in all their styles and with all possible comments may get overwritten. I hope this would not happen. What I mean: I find it convenient to cite in custom format at different places. For example if some page is short and has little content and/or some reference is of central importance I write full names of the authors (even if several), multiple sources (say editions in two languages if useful) and so on. If it is one of many bibentries in a long list, then I write in a short form with abbreviations and even merge many entries. For example, when a single author writes a series of 5 papers on the topic I write the name only once and then list titles/references in a sequence without starting a new paragraph. I think I have personally created huge amount of literature items within Lab (especially in comparison to my more modest contribution to real content), most notably in areas where my own learning, studying and writing is in the initial phase (for instance at tau-function, slope etc.) so I am afraid to write the real content while I feel I need to have noted good references listed at this stage.
Writing bibitems only into a central bibliography makes it now harder to use the sourcecode to copy its parts (say blocks of bibliography LaTeX record on some topic) to other wikis, say personal Labs or to LaTeX presentations. I also find it more scary to correct the bibentry. If I write an entry of a topic where I am somewhat knowledgeable I will adapt the references and so on by the best standards I consider appropriate. But if I make the improvements (say adding a link to the French translation of the book in brackets) to a central version it will affect pages written by very different authors with different style conventions and needs/audience. So I will hesitate to improve the central entry the ways I did not hesitate in the single page version. Thus I think it is good to have it optional but not mandatory. I did raise these same issues around 2011/2012 when there was some early discussion on a possible similar system. In LaTeX I always use manually adding references to each single paper (different journals have different formats and it is actually quicker for short papers). I had recently the first collaboration in which the coauthor was using separate BiBTeX file and it made me much less effective in adding the new items (I know more references than my younger coauthor), so we eventually went back to the manual addition of bibitem entries with full record.
I also noticed that the search button within Lab does not find the key phrases from the bibliography if it is in this format. I searched for “characterisation of the category” and got the following output: compactum, filtrality, pretopos with the reference in the classical format, but not Sandbox where the reference is listed in the new format. On the other hand, Sandbox is indexed and you find it by looking for say “delete the following”.
Partial solution to 2: to have intermediate source available to the editors/source-viewers. I suppose that you have source, than using bibdatabase you create an intermediate source and then you go to the rendering.
Partial solution to 1: nonmandatory status of using the central database for all bibliography records would give some space for the custom/manual records in some parts of some entries. This depends both on automatic design conventions of the system and on our policies. Similarly, some optionality also existed before about having markup for citing within the page just for some entries, not for all. As a general principle, uniformity has its advantages and its costs. The best IMHO is to have the possibility of automatic uniformity, but having it optional.
I would also like to hear what about the interoperability with other wikis in the system, say with personal labs.
What do people think ?
@2
Also, in case there is an arXiv version and a journal version, which one of the two (typically different) years are we supposed to use?
I think if we had a search option for within the bibliography database/reference pages, then we could always find if there is essentially our entry in the literature and then the citation name convention is not that critical and in such boundary cases can be decided case by case. If one does not check into the database then the multiples will inevitably happen anyway as spelling norm is understanding will be routinely happening (Rham, deRham, DeRham…, then papers with original name in another language and couple of translations with different spelling); besides there are some papers with really many authors.
I mean if I use searchbox within say https://ncatlab.org/nlab/reference or within https://ncatlab.org/nlab/reference/show/MarraReggio2020 than these searchboxes could be for reference pages only, not for the main lab or whatever other solution is (minimal solutions not adding much new complications to regular Lab pages are probably the best). I do not know what is planned. Should the page https://ncatlab.org/nlab/reference/show/MarraReggio2020 have a regular edit button ? Are there limitations on the complexity of content in bibliography entry (special symbols, links, markup links, inline formulas etc.).
Also the rendering and the sourcecode for the link does not need to be the same, just like we can do with markup links. I am not sure how far is that desirable to other people.
A bit off topic: As far as style I also think that nowdays it is superfluous to list links to journal pages whenever doi is listed (unless it is an alternative page or something like that, arXiv and free alternatives are important). I sometimes still make exception (stable URLs of euclid when it is free etc. though the latter may have the same doi, namely it is encouraging for a newcomer to click if it is a known name of a free journal page)
Would it be possible to list names in the normal order, i.e., Richard Garner instead of Garner, Richard?
Would it be possible to also extract the DOI from zbMATH data and make it available on the reference page as a hyperlink?
Nice, thanks, Richard, for doing this!
Just began trying to explore the functionality:
How to edit the bibentries?
The example at ncatlab.org/nlab/reference/show/vandenBergGarner2011 has no “edit” button, and replacing “show” by “edit” in the url does not lead anywhere.
Thanks for the feedback! I should by the way have emphasised that this is a relatively large piece of functionality, and there will be teething problems! I prefer to release early and add/fix things though; the kind of feedback given here so far is perfect for that. In haste, but will reply to a few things:
Concerning the last paragraph, a really useful bit of functionality would be to list articles that cite a given bibliographic entry on that entry’s page.
Great idea, this is exactly the kind of thing a central bibliography can enable. I will add that as soon as time allows.
What is supposed to happen when we need to cite more than one paper by the same author published in the same year?
In these early stages I am so far ignoring this problem (but am aware of it!). I am open to suggestions. I was thinking maybe to add the first few non-trivial words of the title or something if and only if that is required. One of course does not want the citation key to be too long in general.
Also, in case there is an arXiv version and a journal version, which one of the two (typically different) years are we supposed to use?
I’d suggest to use the published year as it appears in Zentralblatt. If one actually copies from Zentralblatt and then modifies, this will be handled. I need to add something to handle arXiv references cleanly. The bibliography database structure is flexible, and allows for ’custom fields’. I am thinking that the arXiv reference and year can be such fields. In general, the philosophy will be to store as much info as possible about a reference in the bibliography. We do not have to use it all when rendering (e.g. Zentralblatt typically has a few extra fields which I am storing but ignoring when rendering currently).
Would it be possible to list names in the normal order, i.e., Richard Garner instead of Garner, Richard?
Yes, we can render them that way, I will change that. It is important (for now at least) for indexing that they are entered in the Garner, Richard format, though. If someone knows of the algorithm BibTex uses to parse an author field into surnames and first names, I can implement it, but if not, until I figure out an algorithm myself, we will need to help the parser :-).
Would it be possible to also extract the DOI from zbMATH data and make it available on the reference page as a hyperlink?
The DOI is implemented as a field in the database of the bibliography. However, I think currently it is not present in the BibTex generated by Zentralblatt. The DOI is available on the Zentralblatt page though, so we can add to the instructions that it should be fetched and entered manually. When Zentralblatt add APIs, it should be possible to fetch it automatically. I could in principle hack this already now, but it should be fine as a manual step for now.
How to edit the bibentries?
As mentioned in #1, edit is not implemented yet. If one follows the instructions and uses Zentralblatt as the canonical source for the reference, editing should be relatively rare, but is certainly needed, as there are a couple of manual steps even with the Zentralblatt approach, and additional data can be added manually. This editing functionality is one of the things that needs to be done in the immediate next stages, as mentioned in #1.
I should clarify that although the references look (by design) like ordinary pages, they are implemented independently (completely so in the case of rendering). I have not read carefully through Zoran’s feedback yet (will do so later), but this answers one or two of his questions I think.
It will be this evening European time at the earliest before I work more on this, unless someone discovers a heinous bug that is affecting other things!
I have now implemented a dynamic search functionality for the bibliography, which is necessary really for it to be usable. It’s available at https://ncatlab.org/nlab/reference/all. It’s not really designed to bring up all possible references at once, but if you wish to do so so that you can play about with it, you can use a regex such as .*.*. If you go to this link, you will get the search page with this search expression already entered for you.
The main substantial missing piece now is the ability to edit, which I’ll work on next, along with the rendering of more types of bibliography entries.
Dmitri will be happy that this page is HTML5, unlike the rest of the nLab; it was some work to get Rails to not use the default layout, but worth it compared to not being able to use innerHTML, which I consider indispensable for this kind of thing: imagine trying to create a DOM Element without it with MathML involved!!
Hi Richard,
I just tried to create my first bibitem. I thought I’d try adding
But I admit I get stuck right away:
The instructions tell me to check for my item with “Zentralblatt”. That comes as a surprise on my side. (It does not seem to know Witten’s article, for one. Or maybe I am not searching the right way. I never used Zentralblatt. If I need review of an article, I read it myself. :-)
But I guess the idea is just that Zentralnlatt produces the kind of bib-tex code which is guaranteed to be understood by your sript? In principle I could add bib-tex code by hand?
Could we (the users) maybe not have to produce any bib-tex code at all, but have the script present us with a form in which we fill out relevant fields?
zbMATH does know about Witten’s article: https://zbmath.org/?q=an:0990.81663
Thanks. I see I left out the word “dynamics” in the search query, and that is enough to break the search.
Okay, so I have Zentralblatt’s bib-tex data for Witten’s article now. But I see it has neither the arXiv number nor the doi.
I’d suggest we remove reference to Zentralblatt from our bibitem mechanism. Is it even a requirement at this point? Could I just paste my own bib-tex code into the box? Maybe I’ll try later, got to do something else now.
But I guess the idea is just that Zentralnlatt produces the kind of bib-tex code which is guaranteed to be understood by your sript? In principle I could add bib-tex code by hand?
Yes, the idea is that Zentralblatt should be a help, not a hindrance. The point is just that it contains the BibTex for a reference, as you say. You can certainly add it by hand, there is no requirement to use Zentralblatt. I will tweak the instructions as soon as I get a chance. Once Zentralblatt make APIs available, one could search Zentralblatt from within the nLab (without needing to mention Zentralblatt explicitly), which should ease the process. There are some advantages to having a canonical source such as Zentralblatt and adding in additional information, but it is not essential: again, once APIs are available, it will be possible to add Zentralblatt data to a manual entry.
Could we (the users) maybe not have to produce any bib-tex code at all, but have the script present us with a form in which we fill out relevant fields?
The thought is that many nLab users will have a reference either in their own BibTex files or readily be able to look one up on MathSciNet or Zentralblatt or whatever, and that it would be quicker to copy and paste it than to manually fill out all fields. However, we should certainly be able to have both options available eventually, i.e. both a graphical user interface and the possibility to enter BibTex code.
There are fields ’arxiv’ and ’doi’ which the parser recognises (contrary to standard BibTex). I’d suggest to enter both as links using Markdown, i.e. the same way you would on the nLab. If people typically have the arXiv and DOI fields present in a certain format in their own BibTex files, I can eventually add additional parsing to try to automate this step. Right now the arXiv and DOI links will not be shown when the reference is rendered, but that is trivially fixed, and I can do so later. Just to re-iterate again as well that only rendering of articles has been added so far, though again it is fairly trivial to extend to other document types.
Just to re-iterate that there will be lots of backward and forwarding until we get something that works well, but in the end the benefits will be enormous: once a reference has been added once it can be cited everywhere, and any reference update will update all citations; it will save time once we have a substantial library built up, because people can just cite rather than have to add the reference details; there will be consistent formatting of references across the nLab; it provides semantic benefits (we can look up which pages a reference is cited on); etc.
Thanks, Richard. And thanks for bearing with me.
Maybe to combine the best of both worlds:
If the edit pane would not appear empty but with a template bibtex code showing the supported fields, then I could decide whether to
write over it by copy-and-pasting an existing bibtex-entry, grabbed from whichever source of bibtex-entries,
or else fill in data by hand into the respective fields.
Good idea! The only issue is that there are loads of supported fields, many of which are not typically used, so maybe we should only show some of them by default, or something. I can add something like that as soon as I get a chance.
I have now added arXiv and DOI links to the rendering of articles when present in the bibliography. See the Sandbox for examples. I have also fixed 1. of #5 now. It is no longer required to use Surname, First Names as the format in the author field; instead the parser relies on BibTex’s mechanism for indicating surname using curly brackets, which Zentralblatt seems to always use too.
I have also tweaked the instructions for submitting a reference as requested in #11; hopefully it is better now.
Still only articles can be rendered, and no possibility to edit yet. Also no sorting of the items shown in the search mechanism of the bibliography.
I took the liberty of adding an article to an actual page, namely the first reference (currently) at subtractive category, which is Janelidze2005. There is some weird bug in the rendering of the DOI link, which I’ll fix when I get a chance; it seems specific to something in that DOI.
Thanks! Would it be possible to automatically link authors’ names to their nLab pages, at least if they exist?
There is a bug with zbMATH entry https://www.zbmath.org/?q=an:1209.18001, copy-pasting it produces the following error:
An author must be given in the form: first names/initials {surname} This is not the case for: J. {Ad'amek
Apparently, the parser fails to recognize accents.
While we are on it, I have a suggestion based on my experience with this system so far: rather than require users to copy-paste BibTeX entries manually, present them with a simple form to fill in standard IDs, something like:
zbMATH id:
arXiv id:
doi:
Two other databases that are useful in practice are EuDML and Project Euclid.
zbMATH, arXiv, and CrossRef (DOI) all provide BibTeX code. Additionally, arXiv and DOI have a public API.
For myself, I wrote a script, available here: https://dmitripavlov.org/scripts/mathid
The script mathid accepts a title and an optional author name as a parameter, and produces as an output zbMATH id, arXiv id, DOI, MR number (the latter only if you have a subscription). If more than one article matches the title, the user is presented with a list of matches and is asked to select one.
I could provide assistance in embedding this script into the nLab, this could greatly simplify adding bibliography entries.
Hi Dmitri, thanks very much for continuing to try this, it is very helpful!
Re #17: the nLab code was behaving more or less correctly here, I currently do not wish to allow use of LaTeX code for diacritic marks/accents, as I do not have a parser for it (maybe one day I’ll add one, but it’s not a priority); one can always manually replace these by the actual characters, since the nLab understands unicode. I have now tried to improve the error message to indicate this.
Incidentally rendering of the reference you tried to add would not have worked, because only articles are supported for the moment, but of course I intend to change that asap.
Re #18: thanks for the suggestion, something like that sounds like a good idea. Let’s wait a little as there are plenty of more basic things to do; I’d like the functionality fully functioning first (edits possible, all of the most common document types supported, etc), and then there are certainly lots of things we can do to make it more user-friendly/quicker.
There is some weird bug in the rendering of the DOI link, which I’ll fix when I get a chance; it seems specific to something in that DOI.
I think the doi number is of interest just for hyperlinking and copying using mouse with preferred rendering something like doi, displaying clearly the functional link but without the very number displayed (one can always use mouse and hover to access, see or copy in the cases of need). Long version takes space, attention and clarity to the page (and burden for printers). This is the way I used to write,
Unlike doi, I do find having arXiv number far more often useful in explicit full print (and on printouts). It is shorter, more uniformly formatted, more human readable, more often copied, communicated by voice and even remembered in human communication and non-automatic access and gives a partly meaningful and useful info (notably month and year or publication) and I often recognize it to distinguish clustered papers by the same author. Say,
or a similar idea.
Re #19: Thanks for the feedback! I don’t have a strong preference regarding DOI myself, I agree that it does look a bit ugly in some cases. Basically I typically try to copy what Urs usually does, as he adds the majority of references to the nLab; if Urs is OK with just showing DOI in the way you suggest, we can do that. How unique is the first part of the URL, the first six digits after the ’doi:’, would it be worth displaying these (edit: not unique at all I see, so probably not worth it)?
The arXiv links are currently shown in more or less the way you suggest, see the Sandbox for example.
I’ve not had time to look into the bug regarding DOI rendering yet in that particular reference.
However, I have now added something which is so far invisible from the end-user point of view, but which is crucial to an edit functionality and also to implementing Dmitri’s suggestion in #2, namely storing where references are used. See github.
Implementing this functionality is a bit like writing a new mini-nLab from scratch; one has all the same ingredients (rendering (including LaTeX), editing, cross-linking, etc), just slightly more limited in their scope!
Just to say that historically, I had adopted the habit of making those number codes arXiv:..., doi:..., jstor:... visible, years ago, because Zoran had been amplifying demand for this, for offline use.
The qualification in #19, that this demand only applies to arXiv-numbers, I hear now for the first time! I could have saved myself some keystrokes, over the years :-)
But seriously, we should either display them all or none. The idea that there are people who memorize and communicate through arXiv numbers but then not through other codes I find too far fetched for us to need to accomodate for.
Wikipedia, for what it’s worth, does display all these number codes. So why not stick to it.
Sorry, I am not particularly against uniformity in this respect, which Urs pointed out. Choose whatever you like, I am not particular about it ! Urs is possibly more visionary in seeing the big picture.
And, indeed, I was voiced about using arXiv numbers offline long ago and possibly in some other case, I do not recall about doi, but I certainly did record a lot of doi numbers myself in silent format and also added doi to references put in the lab by many other people. It used to be even more important to have visible for offline in less technological times ten years ago. Doi was less important at the time (we did not have scihub which changed that much for those beyond paywall and doi is the way to go there) and having arxiv number was saving writing a year for unpublished papers and of the original preprint when the published version differs much. Year is useful to glance when reading.
It was just a preference which I thought was reasonable in my perception of common usage and balance of readability. I did see a (maybe indeed minor) difference between arXiv number where the first 4 digits (and in earlier days classification) have some meaning, unlike doi etc. And some journals do publish reference lists in such format where hyperlinks are just from title of the paper and arXiv is explicit. It is also a common habit that we list arXiv numbers in our preprints while other numbers only exceptionally. But if you see this as inessential for our use, you can impose abstract uniformity as far as I am concerned. What I find regretful however is that some commercial journals are erasing arxiv references in bibliographies of published papers making it harder to know if the paper is readily available or not at the time of offline reading, even when the arXiv version differs substantially.
(I was not kidding about everyday use of arXiv numbers, it was not usually meant to remember the whole number (except here or there) but month of the year much of the people I know around do know for lots of papers – approximate or exact year and possibly month of arXiv for articles they recently discussed. These are people more limited to more narrow specialization than polymaths like Urs :) so I understand this to be foreign to somebody working on so massive knowledge base. Wikipedia indeed does record all kind of things in multiple formats, even Croatian wikipedia not the same way as English. I find wikipedia references hard to use for copying and reusing, especially because of source code format there which is strange. Sorry, just a feeling.)
OK, let’s keep the display of the DOI for now! One benefit of a central bibliography will of course be that we will be able change the rendering of something across the entire nLab with just a few keystrokes.
I support showing identifiers like arXiv:⋯ doi:⋯ MR⋯ Zbl⋯ etc., in their standard formats. I have scripts that take such identifiers and (a) download the PDF file of the article and save it under a meaningful name in my library and (b) produce a formatted bibliographic entry for inclusion into my papers. Other mathematicians may have their own use for such identifiers. For example, both Sci-Hub and Library Genesis allow you to download a paper using a doi number.
I wanted to create an entry on EI-category for a book (key: tomDieck1987). There was some error message, which I, unfortunately did not copy, and since then I keep getting the message “Citation key already used: tomDieck1987” but there seems to be not citation entry.
Hi Daniel, thanks for taking the new bibliography into use! It is still under development; what happened in this case was that the book was successfully added to the bibliography, but (as mentioned earlier in the thread) I had previously only implemented rendering of bibliography items of ’article’ document type.
However, I have now made rendering work for items of ’book’ type as well. Pushing the boundaries of the functionality of something is not a bad way to get me to implement something!!!
At EI-category you can see the reference for the book you added now rendering. Another book (I’m not sure if you or another added it) in the bibliography which is not yet used on a page is VeblenWhitehead1932. I have also added Quillen1967, and replaced the pre-existing refererence to it at model category, as well as used the new \cite syntax to link to it, replacing the pre-existing manual link.
As a reminder of the functionality that remains to be added, the most important one is that it is not possible currently to edit references, though the database groundwork for this is done. Along similar lines, if one adds a \bibitem to a page for an item not yet in the bibliography, and then adds the requested item, the page is not updated automatically to make the reference render, one currently has to do render manually (e.g. make a trivial edit).
Apart from those two things, it mainly remains to render other document types (incollection and misc/unpublished are probably the most important), and then to add various bells and whistles on top to make things more user friendly, e.g. adding a template to the edit pane when one creates a reference.
Thank you for raising this, I realised at work today that there was a case I had forgotten to test! I have now fixed it.
(You will not be able to add exactly this entry because of the LaTeX code for diacritic marks, but the error message you receive should be good enough now for you to understand that this is an issue! If you or somebody else is willing to write a little Python function which replaces the LaTeX code for diacritic works with unicode symbols, so that we can automate this, I will add it to the code; if not, I will aim to add it myself one day, but there are other things with higher priority for the moment. The LaTeX ampersand in the publisher name also would need to be replaced by an appropriate unicode symbol.)
The implementation of the bibliography never left the prototype stage. In the interest of maintainability (and possible future transitions), we have removed this feature. It remains available in branch bibliography in the nlab repository.
All relevant uses of \cite and \bibitem have been replaced appropriately.
1 to 31 of 31