Git documentation at kernel.org

[Petr Onderka]

Hi,

since the hacking of kernel.org, the online version of git
documentation [1] is not available. I realize that I can use local
version of the documentation and there is also at least one mirror
[2]. But I think it's very useful to have an "official" version of the
documentation online. And, more importantly, there is now lots of dead
links all over the Internet to the kernel.org version of the
documentation.

Can someone with the ability to do so restore the documentation at the
old location?

Thanks.

Petr Onderka

[1]: http://www.kernel.org/pub/software/scm/git/docs/git.html
[2]: http://schacon.github.com/git/git.html

[Clemens Buchacher]

Hi,

Please restore access to the following files when possible. Some sites
are referencing those, including kernel.org itself:

http://www.kernel.org/pub/software/scm/git/docs/git.html
and references therein

http://www.kernel.org/pub/software/scm/git/docs/howto/using-merge-subtree.html
referenced by
https://git.wiki.kernel.org/articles/g/i/t/GitFaq_ebc3.html#How_do_I_clone_a_subdirectory.3F

Also, it would be great if the git wiki could be made editable again.

Thank you for your consideration.

Regards,
Clemens

[Junio C Hamano]

Post by Clemens Buchacher
Please restore access to the following files when possible. Some sites
http://www.kernel.org/pub/software/scm/git/docs/git.html

The pages reachable from this used to be living documents in that every
time the 'master' branch was updated at k.org, automatically a server side
hook script generated a new set of HTML pages and updated them.

My understanding is that we do not want to run such random server side
hooks at k.org, so it no longer can be a living document anymore.

It might be a workable short term workaround to redirect

http://www.kernel.org/pub/software/scm/git/docs/$anything

to

http://schacon.github.com/git/$anything

although that would not give you an access to the list of documentations
for older releases, e.g.

http://www.kernel.org/pub/software/scm/git/docs/v1.6.0/git.html

Post by Clemens Buchacher
Also, it would be great if the git wiki could be made editable again.

Amen.

[Matthieu Moy]

Post by Junio C Hamano

Post by Clemens Buchacher
Please restore access to the following files when possible. Some sites
http://www.kernel.org/pub/software/scm/git/docs/git.html

The pages reachable from this used to be living documents in that every
time the 'master' branch was updated at k.org, automatically a server side
hook script generated a new set of HTML pages and updated them.

Is it possible to have the static HTML uploaded from another machine,
not necessarily for each push, but e.g. for every release?

I don't think anyone cares about having the very latest documentation
there, but it would still be great to have an official place to point to
when writing documentation on the web about such or such command.

[Theodore Tso]

Post by Matthieu Moy
=20
Is it possible to have the static HTML uploaded from another machine,
not necessarily for each push, but e.g. for every release?
=20
I don't think anyone cares about having the very latest documentation
there, but it would still be great to have an official place to point=

to

Post by Matthieu Moy
when writing documentation on the web about such or such command.

I think that's a great idea.

I used to use that service quite a lot, so I'd be willing to push the t=
ar balls
to kernel.org, since I have PGP key set up for kernel.org uploads (so d=
o
other people, and if someone else wants to do it, I'm happy to let them=
get
all the glory :-) Most of the infrastructure to do this has been impl=
emented,
except for the part where the tar ball gets unpacked in the correct dir=
ectory.

This would satisfy the security concerns, and it wouldn't be hard, but =
it would
require some implementation work. Anyone have some perl hacking time =
to
take a look at:=20

git://git.kernel.org/pub/scm/utils/kup/kup.git

=85 and add a "UNPACK pathanme" to the kup-server file, and work with t=
he
sysadmins at kernel.org to get it reviewed and accepted?

-- Ted

[Konstantin Ryabitsev]

This would satisfy the security concerns, and it wouldn't be hard, but it would
require some implementation work. Anyone have some perl hacking time to
git://git.kernel.org/pub/scm/utils/kup/kup.git

 and add a "UNPACK pathanme" to the kup-server file, and work with the
sysadmins at kernel.org to get it reviewed and accepted?

I have a few comments off the top of my head:

1. "kup rm" will need to be modified, as it currently only allows
deleting things that have a matching signature. The alternative
is for UNPACK to create a foo.tar.manifest file that will be
consulted upon "kup rm" to clean up any unpacked contents upon
the deletion of the source archive. Note, that there are many,
many gotchas with this solution -- e.g. .manifest should
probably contain checksums, too, as there are bound to be
conditions when two tarballs reference the same files, and you
want to make sure that you delete files matching the contents of
the old tarball, not the newer one, etc.
2. I would suggest that UNPACK ignores any directory structure in
the archive, and only copies over files matching a restricted
set of extensions (.html, .txt, .jpg, .png) into the same dir as
the original tarball. Basically, untar into a temporary
directory, then find any files matching the above set of
extensions, copy them into another temporary location, force
permissions to 0644, and then move them into the final "live"
location in the same dir with the tarball (with the
corresponding .manifest, if that solution used). There should be
logic to make sure that we never overwrite any files that have a
matching .sign file.
3. There should be some support to ensure that the unpack process
is terminated if unpacked content size reaches a certain limit,
or if it is taking too long to complete.

Best regards,

[Ted Ts'o]

How about this as something *way* simpler? Define a way of marking
the top of a particular directory hierarchy as a tree. Then the
*only* way of updating that tree is all or nothing. That is, someone
submits a signed tarball; then after the signed tarball has its
signature checked, it gets unpacked into a dir.new, and then we rename
dir to dir.old, rename dir.new to dir, and then dir.old gets removed.

That way there's no conflicts between directories that are managed via
the kup-servers PUT and DELETE commands, and those where they get
uploaded as a single tarball to create or replace a specific directory
hierarcy, or which can be deleted only as a entire directory hierarcy.

What do you think?

- Ted

[Junio C Hamano]

Post by Ted Ts'o
How about this as something *way* simpler? Define a way of marking
the top of a particular directory hierarchy as a tree. Then the
*only* way of updating that tree is all or nothing. That is, someone
submits a signed tarball; then after the signed tarball has its
signature checked, it gets unpacked into a dir.new, and then we rename
dir to dir.old, rename dir.new to dir, and then dir.old gets removed.
That way there's no conflicts between directories that are managed via
the kup-servers PUT and DELETE commands, and those where they get
uploaded as a single tarball to create or replace a specific directory
hierarcy, or which can be deleted only as a entire directory hierarcy.
What do you think?

That would not work very well without changing the historical directory
structure (which I think was the point of this discussion "please keep
these stale links alive").

The toplevel index.html in the pub/software/scm/git/docs/ directory and
its pointees were the set of docs for the latest version, and older
versions were rooted at pub/software/scm/git/docs/vX.Y.Z/. Links that
point at software/scm/git/docs/git-cat-file.html still need to work, and
the path needs to be updatable without having to include the preformatted
documentation for all the historical versions in the same tarball.

[Ted Ts'o]

Post by Junio C Hamano
That would not work very well without changing the historical directory
structure (which I think was the point of this discussion "please keep
these stale links alive").
The toplevel index.html in the pub/software/scm/git/docs/ directory and
its pointees were the set of docs for the latest version, and older
versions were rooted at pub/software/scm/git/docs/vX.Y.Z/. Links that
point at software/scm/git/docs/git-cat-file.html still need to work, and
the path needs to be updatable without having to include the preformatted
documentation for all the historical versions in the same tarball.

Hmm... good point. That does make it hard. I could imagine making it
work by having separate hierarchies, and then using apache rewrite
rules so that anything that doesn't begin with vX.Y.Z in the top level
of software/scm/git/docs/* gets redirected to LATEST/*, where LATEST is
a symlink that is managed via kup.

I don't know if the k.org folks would consider that acceptable, though.

- Ted

[Junio C Hamano]

Post by Ted Ts'o
Hmm... good point. That does make it hard. I could imagine making it
work by having separate hierarchies, and then using apache rewrite
rules so that anything that doesn't begin with vX.Y.Z in the top level
of software/scm/git/docs/* gets redirected to LATEST/*, where LATEST is
a symlink that is managed via kup.

We could move vX.Y.Zs out of scm/git/docs/ hierarchy. The existing links
from external sites rarely point at a documentation page of a specific
version, I suspect.

People can be trained to look at scm/git/old-docs/vX.Y.Z when they want to
see how older command set looked like, even those who know that in olden
days they would have consulted scm/git/docs/vX.Y.Z for that information;
they are much less of a problem than existing pages whose links want to
stay working.

[Jeff King]

Post by Konstantin Ryabitsev
1. "kup rm" will need to be modified, as it currently only allows
deleting things that have a matching signature. The alternative
is for UNPACK to create a foo.tar.manifest file that will be
consulted upon "kup rm" to clean up any unpacked contents upon
the deletion of the source archive. Note, that there are many,
many gotchas with this solution -- e.g. .manifest should
probably contain checksums, too, as there are bound to be
conditions when two tarballs reference the same files, and you
want to make sure that you delete files matching the contents of
the old tarball, not the newer one, etc.

For this particular use case, I don't know if that would be necessary.
According to Junio, previously:

The k.org site kept these files under /pub/software/scm/git/docs/. The
in-development "master" version of pages were placed directly
underneath that directory, and the documentation pages for older
versions were kept in vX.Y.Z subdirectory of that directory.

If we tweak that slightly to "all versions are kept in vX.Y.Z
subdirectory, and the root version is simply a symlink or redirect to
the latest vX.Y.Z", then there is no deletion required. The pusher is
always adding new versions, and updating a link[1].

But even if it would be sufficient for this use case, kup developers
may not want such a half-implemented scheme in their protocol.

-Peff

[1] There is a slight complication that the subdirectories live _inside_
of the root directory, so it is not implementable with a single
symlink. You could get around that with a few clever http redirects.

[Junio C Hamano]

Post by Matthieu Moy

Post by Junio C Hamano

Post by Clemens Buchacher
Please restore access to the following files when possible. Some sites
http://www.kernel.org/pub/software/scm/git/docs/git.html

The pages reachable from this used to be living documents in that every
time the 'master' branch was updated at k.org, automatically a server side
hook script generated a new set of HTML pages and updated them.

Is it possible to have the static HTML uploaded from another machine,
not necessarily for each push, but e.g. for every release?

It would probably be possible, but I do not have that much time and
patience to sign 600+ files in the preformatted HTML tree one-by-one and
upload them using kup.

[Jeff King]

Post by Junio C Hamano
It might be a workable short term workaround to redirect
http://www.kernel.org/pub/software/scm/git/docs/$anything
to
http://schacon.github.com/git/$anything
although that would not give you an access to the list of documentations
for older releases, e.g.
http://www.kernel.org/pub/software/scm/git/docs/v1.6.0/git.html

If there is interest in this, we would be happy to host the
documentation. Let me know if that is the case, and we can give it a
much better URL than schacon.github.com. However, I tend to think that
since the project is hosted[1] at kernel.org, the official documentation
site should be there as well.

-Peff

[1] Of course, git being git, it is not really hosted _anywhere_ in
particular. But convention thus far has said that the kernel.org
repository is the official one.

[Matthieu Moy]

Post by Jeff King
If there is interest in this, we would be happy to host the
documentation. Let me know if that is the case, and we can give it a
much better URL than schacon.github.com. However, I tend to think that
since the project is hosted[1] at kernel.org, the official documentation
site should be there as well.

kernel.org is probably the most "official" place for developers, but for
Git users, http://git-scm.com/ is most likely the best entry point. If
it were not for historical reasons, I think http://git-scm.com/docs/ or
so would be the most natural URL to host official docs.

[Jeff King]

Post by Matthieu Moy

Post by Jeff King
If there is interest in this, we would be happy to host the
documentation. Let me know if that is the case, and we can give it a
much better URL than schacon.github.com. However, I tend to think that
since the project is hosted[1] at kernel.org, the official documentation
site should be there as well.

kernel.org is probably the most "official" place for developers, but for
Git users, http://git-scm.com/ is most likely the best entry point. If
it were not for historical reasons, I think http://git-scm.com/docs/ or
so would be the most natural URL to host official docs.

Good point. That is probably the best place to host it.

As far as historical reasons, perhaps the right answer is to put the
documentation where it makes sense to go _now_, and ask kernel.org to
issue http redirects for http://kernel.org/pub/software/scm/git/docs.

-Peff

[Scott Chacon]

Hey,

Post by Jeff King

Post by Matthieu Moy
kernel.org is probably the most "official" place for developers, but for
Git users, http://git-scm.com/ is most likely the best entry point. If
it were not for historical reasons, I think http://git-scm.com/docs/ or
so would be the most natural URL to host official docs.

Good point. That is probably the best place to host it.
As far as historical reasons, perhaps the right answer is to put the
documentation where it makes sense to go _now_, and ask kernel.org to
issue http redirects for http://kernel.org/pub/software/scm/git/docs.

I would be happy to set this up. I'm currently in the process of
revamping the website and this is one of the things I'm planning on
doing anyways - not just hosting the generated docs, but also making
them searchable and whatnot.

Actually, as long as I'm on this, what do people think about git-scm
hosting the wiki as well? As far as I can tell, it was down for
months and now it's back in some sort of weird read-only state. If I
imported everything into a different wiki and hosted it on git-scm
would that be acceptable?

Also, something that I realized I am not willing to maintain any more
is the Git Community Book. It was an experiment at reorganizing some
of the docs, but instead I spent my time on Pro Git, which is CC
licensed. Would anyone object to me removing the community book from
the git-scm site and more tightly integrating the Pro Git content?
It's more up to date and better content, I feel - I would rather have
one book to maintain than two. However, since it is a commercial
product (albeit a Creative Commons licensed one), I wasn't sure if
people would have an issue with it.

Scott

[Jeff King]

Post by Scott Chacon

Post by Jeff King
Good point. That is probably the best place to host it.
As far as historical reasons, perhaps the right answer is to put the
documentation where it makes sense to go _now_, and ask kernel.org to
issue http redirects for http://kernel.org/pub/software/scm/git/docs.

I would be happy to set this up. I'm currently in the process of
revamping the website and this is one of the things I'm planning on
doing anyways - not just hosting the generated docs, but also making
them searchable and whatnot.

That sounds great to me. I'd like to be link-compatible with the old
kernel.org docs section (even if through redirects) so that old links
work (assuming kernel.org gives us a wholesale redirect). Which means
importing all of the docs for released versions. I don't know if the old
kernel.org doc tree was saved anywhere, but if I understand correctly,
they are identical to what's in the "git-htmldocs" repository (which I
_thought_ Junio wasn't going to keep updating, but it seems pretty up to
date).

Post by Scott Chacon
Actually, as long as I'm on this, what do people think about git-scm
hosting the wiki as well? As far as I can tell, it was down for
months and now it's back in some sort of weird read-only state. If I
imported everything into a different wiki and hosted it on git-scm
would that be acceptable?

I'd really love it if the wiki was converted to something that was
git-backed. But I suspect some people might complain about switching off
of mediawiki. IIRC, gollum supports some mediawiki syntax, but I don't
know how much conversion work there would be.

Post by Scott Chacon
Also, something that I realized I am not willing to maintain any more
is the Git Community Book. It was an experiment at reorganizing some
of the docs, but instead I spent my time on Pro Git, which is CC
licensed. Would anyone object to me removing the community book from
the git-scm site and more tightly integrating the Pro Git content?
It's more up to date and better content, I feel - I would rather have
one book to maintain than two. However, since it is a commercial
product (albeit a Creative Commons licensed one), I wasn't sure if
people would have an issue with it.

I can't remember anybody mentioning the Git Community Book here in the
past few years. New users typically come with a "I read this in Pro Git
and I don't understand..." question, and experienced users recommend or
link to Pro Git. So I think the world would be a less confusing place
with just the one source.

-Peff

[Junio C Hamano]

Post by Jeff King
That sounds great to me. I'd like to be link-compatible with the old
kernel.org docs section (even if through redirects) so that old links
work (assuming kernel.org gives us a wholesale redirect). Which means
importing all of the docs for released versions. I don't know if the old
kernel.org doc tree was saved anywhere, but if I understand correctly,
they are identical to what's in the "git-htmldocs" repository (which I
_thought_ Junio wasn't going to keep updating, but it seems pretty up to
date).

I have been updating htmldocs/manpages repositories on "unless I forget"
basis every time the public 'master' gets updated, so they are updated as
frequently as they used to back when they were autogenerated, "unless I
forget".

But the contents of htmldocs does _not_ match what used to be at k.org in
that its git.html does not have links to documentation pages for older
releases, iow, formatted without "stalenotes" defined.

This is because formatting with "stalenotes" needs to make an assumption
on the filesystem layout that I cannot enforce to the users of htmldocs
repository. They will get one tarball for one version, and it is up to
them where they extract these tarballs. They need to extract the tarball
of an older release vX.Y.Z in vX.Y.Z subdirectory next to the git.html of
the latest living document to match the layout, but otherwise the links
created by "stalenotes" will become dangling.

[Jeff King]

Post by Junio C Hamano

Post by Jeff King
That sounds great to me. I'd like to be link-compatible with the old
kernel.org docs section (even if through redirects) so that old links
work (assuming kernel.org gives us a wholesale redirect). Which means
importing all of the docs for released versions. I don't know if the old
kernel.org doc tree was saved anywhere, but if I understand correctly,
they are identical to what's in the "git-htmldocs" repository (which I
_thought_ Junio wasn't going to keep updating, but it seems pretty up to
date).

[...]
But the contents of htmldocs does _not_ match what used to be at k.org in
that its git.html does not have links to documentation pages for older
releases, iow, formatted without "stalenotes" defined.

Ah, that makes sense. We might have to just rebuild the old versions
with stalenotes, then (our doc toolchain is so finicky that I worry
about minor incompatibilities in building old versions with a newer
toolchain, but it is probably good enough).

-Peff

[Konstantin Ryabitsev]

Post by Jeff King
As far as historical reasons, perhaps the right answer is to put the
documentation where it makes sense to go _now_, and ask kernel.org to
issue http redirects for http://kernel.org/pub/software/scm/git/docs.

I think that should be fine, unless John objects. The easiest would be
to preserve the same directory structure, so we do a dir-level redirect
instead of creating one-off redirects for each page.

Best,

[Neal Kreitzinger]

Post by Petr Onderka
since the hacking of kernel.org, the online version of git
documentation [1] is not available. I realize that I can use local
version of the documentation and there is also at least one mirror
[2]. But I think it's very useful to have an "official" version of the
documentation online. And, more importantly, there is now lots of dead
links all over the Internet to the kernel.org version of the
documentation.
Can someone with the ability to do so restore the documentation at the
old location?

Additional info:
http://article.gmane.org/gmane.comp.version-control.git/185302

v/r,
neal