[ntp:questions] Re: Lists of public NTP servers

Steve Kostecke kostecke at ntp.org
Wed Nov 12 16:45:05 UTC 2003


On 2003-11-12, Dale R. Worley <worley at theworld.com> wrote:

> Brad Knowles <brad.knowles at skynet.be> writes:
>
>> Once again, I will say that anything in the public twiki space is
>> totally open to world-wide access, including any and all known search
>> engines.
>
> I fear that this conversation is veering severely from what should be
> the actual point: Despite that the TWiki can theoretically be indexed
> by any spider, in practice it is not.

The NTP TWiki is, with a few exceptions, open to spiders. I have tested
this with wget (which, by default, obeys robots.txt directives).

The NTP TWiki was last visited by crawler10.googlebot.com at
12/Nov/2003:04:16:05 +0000. But we're not appearing in the Google search 
results yet.

<snip>

> Much better to say "once we get the performance problem fixed and
> allow the spiders in, anything in the public twiki space will be open
> ...".

I have, in fact, said exactly this earlier in the thread.

BTW: On November 5, 2003, I modified the restrictions in the TWiki's
robots.txt file. You must have missed my post to this newsgroup
(<slrnbqi7ch.mha.kostecke at stasis.kostecke.net>) announcing this fact.

-- 
Steve Kostecke <kostecke at ntp.org>
The Official NTP Documentation and FAQ are linked from
http://www.ntp.org/documentation.html and http://twiki.ntp.org
*** ACCEPT NO SUBSTITUTES ***



More information about the questions mailing list