Want to take part in these discussions? Sign in if you have an account, or apply for one below
Vanilla 1.1.10 is a product of Lussumo. More Information: Documentation, Community Support.
When I use google to search for an nLab entry, it lately tells me, that the robot.txt forbids to give a description of the appropriate nLab entry. Is that on purpose? Why?
I’ve gotten this as well, but sporadically. Certainly the robots.txt should not bar google from searching pages. But it does bar google from searching all the non-content pages (edit, new, …). It could be that this is a side-effect of the issue we were finding with google trying to index links of the form /new/existing+page
.
If you can spot some sort of pattern I could investigate further.
I spot this when Google gives me the new
page. This is a phenomenon that has been discussed earlier and made us change how we handle hits to such pages.
1 to 3 of 3