Want to take part in these discussions? Sign in if you have an account, or apply for one below
Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.
1 to 28 of 28
There was a query at HowTo on downloading the nLab. It’s no longer relevant since when updating the software, I re-enabled “all pages” and “recently revised” (but not export) since I wanted a clean sheet to measure the performance better. I’ve replied at HowTo but I’m copying it here since I’ll remove that query box after a short time as I don’t think that HowTo is a good place for query boxes.
The following command line magic depends on the list of all pages which no longer works. Is there any other way to download a copy of the nLab?
Andrew: The “all pages” is currently reenabled, but there are alternative methods depending on exactly what you want to do with the copy. Best route would be to join us in the nForum and discuss it so that we can come up with the best method.
(For housekeeping, I’ll delete this query box in a few days.)
I have deleted this query box. (Six years seems long enough to count as “a few days”.)
Although it does seem that the method suggested at HowTo doesn’t currently work. At least, http://ncatlab.org/nlab/list gives me a bad gateway.
It seems like the most recent discussion about downloading the nlab was this one. I didn’t really follow the conclusion; what should be put on the HowTo?
I don’t know about any of this, except that since many months Bas Spitters has been highlighting by email that there is no means anymore to download the nLab. Bas in turn was writing in reaction to some people he is in contact with, people with interest in web-based mathematics, who requested instructions for such downloads. Adeel has not been reacting to inquiries to this extent. I suppose he is busy with more important tasks.
We need somebody with time and energy who could lend Adeel a hand with being system administrator.
Some of you must be in closer contact with students of computer science than I am. Could it be an interesting thesis project for a student with interest in web technology to look into some aspects of the Lab installation?
As far as I understand, Instiki stores all articles internally in an SQL database.
At one point in the past this SQL database was periodically exported to a Bazaar repository, which could then be pulled externally, thus providing a convenient way to fetch (and backup) the entire nLab: once the database was downloaded, one simply had to do bzr pull once in a while to keep it updated. This is what I did, anyway.
It seems to me that after the mgiration to a different server the Bazaar export feature was dropped (it is not in the Instiki code as far as I can see).
Perhaps the SQL database could be made available via HTTP as a read-only file? This would require very little effort.
Alternatively, if somebody gives me the SQL file, I can write a very short script that would periodically export the database to a git repository, which can then be made publicly accessible in the same manner as the bzr repository, in particular it can be updated with git pull. This requires very little effort and can be done very quickly.
Urs, what do you think?
Urs, what do you think?
Thanks for offering help! You should please contact Adeel Khan with this, let him know what you offer to do and what you need for that. If you don’t have his email address, let me know.
I emailed Adeel and you, let’s see how it works out.
To state the obvious, one should certainly not expose the SQL database itself (via HTTP or any other way). If a copy of it is made on the server every so often (via a cron job, say), and this copy is made available, that would be perfectly fine of course.
Dmitri, did you manage to get this to work?
I talked to Adeel about this, he is currently traveling.
Once he is back, we will implement it.
I apologize for the delay on this. I finally set this up again, using a git repository instead of the bazaar repository we had before. Anyone interested in using this should e-mail me their SSH public key (follow these instructions) and I’ll give you access. You will then be able to clone the git repository from nlab-git@saunders.phil.cmu.edu:nlab-content. To save server resources you can download a “bare” version of the repository with
git clone --bare nlab-git@saunders.phil.cmu.edu:nlab-content
and then run
git clone nlab-content.git nlab-content
rm nlab-content.git
cd nlab-content
git remote set-url origin nlab-git@saunders.phil.cmu.edu:nlab-content
to get a “normal” repository with a working tree. Use git pull inside the working directory to sync it with the server (you can set up a https://help.ubuntu.com/community/CronHowto to do this every day, for example).
Thanks, Adeel!!
I have copied your instructions over to How to download a local copy of the nLab?
Yes, there is a cronjob running on the server to commit any new changes every hour.
Yes, there is a cronjob running on the server to commit any new changes every hour.
Could that be what causes the nLab to apparently hang when I try to submit an edit, or is that some other problem? Canceling and then re-editing with the text that you have hopefully saved seems to work.
Right now I had hanging submit that eventually returned the page
Secure Connection Failed
The git repository is now mirrored at GitHub: https://github.com/ncatlab/nlab-content.
This means in particular that you can skip sending me your public key and just run
git clone git@github.com:ncatlab/nlab-content.git
It also means that there will always be a relatively reliable mirror of the nLab on GitHub, unless something breaks at some point.
Yes, there is a cronjob running on the server to commit any new changes every hour.
Could that be what causes the nLab to apparently hang when I try to submit an edit, or is that some other problem?
Were you eventually able to save your changes or is the problem still persistent? The cronjob takes very little server resources, so more likely the reason was different; around that time I was running a script on the server to replace “bad” author names from the git repository, which GitHub wasn’t accepting as valid (in this case, “bad” meant that they contained certain invalid characters).
Probably a better command for cloning is
git clone https://github.com/ncatlab/nlab-content.git/
The native git protocol is faster than ssh.
I’ve added metadata files to the repository. It’s only the page title, let me know if you need anything else.
I looked into the unicode issue Dmitri mentioned. It turns out that the database encoding was set to latin1 for some reason. I’ve converted everything over to utf8 and I’m now setting up the git repository again from scratch (it will take a day or two I guess).
After that I’ll look into adding html format files. I think it’s best to put them in a separate branch.
Ok, the new repository has been set up (at the same URL: https://github.com/ncatlab/nlab-content). If you’ve already downloaded it, you should delete your local copy and clone it again; git pull will give you a lot of errors.
There’s now a (separate) HTML repository at https://github.com/ncatlab/nlab-content-html.
1 to 28 of 28