[PATCH 4/4] Documentation: implement linkgit macro for Asciidoctor

[brian m. carlson]

AsciiDoc uses a configuration file to implement macros like linkgit,
while Asciidoctor uses Ruby extensions. Implement a Ruby extension that
implements the linkgit macro for Asciidoctor in the same way that
asciidoc.conf does for AsciiDoc.

Signed-off-by: brian m. carlson <***@crustytoothpaste.net>
---
Documentation/extensions.rb | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
create mode 100644 Documentation/extensions.rb

diff --git a/Documentation/extensions.rb b/Documentation/extensions.rb
new file mode 100644
index 0000000..c33a50d
--- /dev/null
+++ b/Documentation/extensions.rb
@@ -0,0 +1,39 @@
+require 'asciidoctor'
+require 'asciidoctor/extensions'
+
+module Git
+ module Documentation
+ class LinkGitProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+ use_dsl
+
+ named :chrome
+
+ def process(parent, target, attrs)
+ if parent.document.basebackend? 'html'
+ generate_html(parent, target, attrs)
+ elsif parent.document.basebackend? 'docbook'
+ generate_docbook(parent, target, attrs)
+ end
+ end
+
+ private
+
+ def generate_html(parent, target, attrs)
+ section = attrs.has_key?(1) ? "(#{attrs[1]})" : ''
+ prefix = parent.document.attr('git-relative-html-prefix') || ''
+ %(<a href="#{prefix}#{target}.html">#{target}#{section}</a>\n)
+ end
+
+ def generate_docbook(parent, target, attrs)
+ %(<citerefentry>
+<refentrytitle>#{target}</refentrytitle><manvolnum>#{attrs[1]}</manvolnum>
+</citerefentry>
+)
+ end
+ end
+ end
+end
+
+Asciidoctor::Extensions.register do
+ inline_macro Git::Documentation::LinkGitProcessor, :linkgit
+end

[brian m. carlson]

Asciidoctor takes slightly different arguments from AsciiDoc in some
cases. It has a different name for the HTML backend and the "docbook"
backend produces DocBook 5, not DocBook 4.5. Also, Asciidoctor does not
accept the -f option. Move these values into variables so that they can
be overridden by users wishing to use Asciidoctor instead of Asciidoc.

Signed-off-by: brian m. carlson <***@crustytoothpaste.net>
---
Documentation/Makefile | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index cea0e7a..00c50bf 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -97,6 +97,9 @@ man7dir = $(mandir)/man7

ASCIIDOC = asciidoc
ASCIIDOC_EXTRA =
+ASCIIDOC_HTML = xhtml11
+ASCIIDOC_DOCBOOK = docbook
+ASCIIDOC_CONF = -f asciidoc.conf
MANPAGE_XSL = manpage-normal.xsl
XMLTO = xmlto
XMLTO_EXTRA =
@@ -304,13 +307,13 @@ clean:

$(MAN_HTML): %.html : %.txt asciidoc.conf
$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) -b xhtml11 -d manpage -f asciidoc.conf \
+ $(ASCIIDOC) -b $(ASCIIDOC_HTML) -d manpage $(ASCIIDOC_CONF) \
$(ASCIIDOC_EXTRA) -agit_version=$(GIT_VERSION) -o $@+ $< && \
mv $@+ $@

$(OBSOLETE_HTML): %.html : %.txto asciidoc.conf
$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) -b xhtml11 -f asciidoc.conf \
+ $(ASCIIDOC) -b $(ASCIIDOC_HTML) $(ASCIIDOC_CONF) \
$(ASCIIDOC_EXTRA) -agit_version=$(GIT_VERSION) -o $@+ $< && \
mv $@+ $@

@@ -323,13 +326,13 @@ manpage-base-url.xsl: manpage-base-url.xsl.in

%.xml : %.txt asciidoc.conf
$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) -b docbook -d manpage -f asciidoc.conf \
+ $(ASCIIDOC) -b $(ASCIIDOC_DOCBOOK) -d manpage $(ASCIIDOC_CONF) \
$(ASCIIDOC_EXTRA) -agit_version=$(GIT_VERSION) -o $@+ $< && \
mv $@+ $@

user-manual.xml: user-manual.txt user-manual.conf
$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- $(ASCIIDOC) $(ASCIIDOC_EXTRA) -b docbook -d article -o $@+ $< && \
+ $(ASCIIDOC) $(ASCIIDOC_EXTRA) -b $(ASCIIDOC_DOCBOOK) -d article -o $@+ $< && \
mv $@+ $@

technical/api-index.txt: technical/api-index-skel.txt \
@@ -338,7 +341,7 @@ technical/api-index.txt: technical/api-index-skel.txt \

technical/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../
$(patsubst %,%.html,$(API_DOCS) technical/api-index $(TECH_DOCS)): %.html : %.txt asciidoc.conf
- $(QUIET_ASCIIDOC)$(ASCIIDOC) -b xhtml11 -f asciidoc.conf \
+ $(QUIET_ASCIIDOC)$(ASCIIDOC) -b $(ASCIIDOC_HTML) $(ASCIIDOC_CONF) \
$(ASCIIDOC_EXTRA) -agit_version=$(GIT_VERSION) $*.txt

XSLT = docbook.xsl
@@ -386,14 +389,15 @@ howto-index.txt: howto-index.sh $(wildcard howto/*.txt)
mv $@+ $@

$(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt
- $(QUIET_ASCIIDOC)$(ASCIIDOC) $(ASCIIDOC_EXTRA) -b xhtml11 $*.txt
+ $(QUIET_ASCIIDOC)$(ASCIIDOC) $(ASCIIDOC_EXTRA) -b $(ASCIIDOC_HTML) $*.txt

WEBDOC_DEST = /pub/software/scm/git/docs

howto/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../
$(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
- sed -e '1,/^$$/d' $< | $(ASCIIDOC) $(ASCIIDOC_EXTRA) -b xhtml11 - >$@+ && \
+ sed -e '1,/^$$/d' $< | \
+ $(ASCIIDOC) $(ASCIIDOC_EXTRA) -b $(ASCIIDOC_HTML) - >$@+ && \
mv $@+ $@

install-webdoc : html

[Junio C Hamano]

Post by brian m. carlson
Asciidoctor takes slightly different arguments from AsciiDoc in some
cases. It has a different name for the HTML backend and the "docbook"
backend produces DocBook 5, not DocBook 4.5. Also, Asciidoctor does not
accept the -f option. Move these values into variables so that they can
be overridden by users wishing to use Asciidoctor instead of Asciidoc.
---

I think it makes sense to make these customizable, but I wonder if
it makes the result easier to maintain if we make units of logical
definitions larger, e.g.

ASCIIDOC = asciidoc
TXT_TO_MANHTML = $(ASCIIDOC) -b xhtml11 -d manpage $(ASCIIDOC_CONF)
TXT_TO_ARTICLE = $(ASCIIDOC) -b docbook -d article
...

ASCIIDOC_EXTRA may want to apply all of them, even though I see that
we do not feed it to OBSOLETE_HTML right now. It may also be that
$(ASCIIDOC_CONF) and -agit_version=$(GIT_VERSION) could be shared
among the ones that currently do not have.

Then the above would become something like:

ASCIIDOC = asciidoc
ASCIIDOC_COMMON = $(ASCIIDOC) \
$(ASCIIDOC_EXTRA) $(ASCIIDOC_CONF) -agit_version=$(GIT_VERSION)
TXT_TO_MANHTML = $(ASCIIDOC_COMMON) -b xhtml11 -d manpage
...

and would further simplify this part

Post by brian m. carlson
$(MAN_HTML): %.html : %.txt asciidoc.conf
- $(ASCIIDOC) -b xhtml11 -d manpage -f asciidoc.conf \
+ $(ASCIIDOC) -b $(ASCIIDOC_HTML) -d manpage $(ASCIIDOC_CONF) \

into just

$(TXT_TO_MANHTML) -o $@+ $<

After all, our output formats are fairly limited, I would think.
Are there too many different variants and exceptions to make such an
approach infeasible?

[brian m. carlson]

Post by Junio C Hamano
I think it makes sense to make these customizable, but I wonder if
it makes the result easier to maintain if we make units of logical
definitions larger, e.g.
ASCIIDOC = asciidoc
TXT_TO_MANHTML = $(ASCIIDOC) -b xhtml11 -d manpage $(ASCIIDOC_CONF)
TXT_TO_ARTICLE = $(ASCIIDOC) -b docbook -d article

Looking at the code, it seems the most reusable unit is something like
the following:

TXT_TO_HTML = $(ASCIIDOC) -b xhtml11 $(ASCIIDOC_CONF)
TXT_TO_XML = $(ASCIIDOC) -b docbook $(ASCIIDOC_CONF)

that is, omitting the -d argument from the variable. Using -d would
mean we'd have to have a variable for each of the seven locations we
call $(ASCIIDOC).

Post by Junio C Hamano
ASCIIDOC = asciidoc
ASCIIDOC_COMMON = $(ASCIIDOC) \
$(ASCIIDOC_EXTRA) $(ASCIIDOC_CONF) -agit_version=$(GIT_VERSION)
TXT_TO_MANHTML = $(ASCIIDOC_COMMON) -b xhtml11 -d manpage
...
and would further simplify this part

Post by brian m. carlson
$(MAN_HTML): %.html : %.txt asciidoc.conf
- $(ASCIIDOC) -b xhtml11 -d manpage -f asciidoc.conf \
+ $(ASCIIDOC) -b $(ASCIIDOC_HTML) -d manpage $(ASCIIDOC_CONF) \

into just
After all, our output formats are fairly limited, I would think.
Are there too many different variants and exceptions to make such an
approach infeasible?

I'm on board with the $(ASCIIDOC_COMMON) idea, but to minimize the
number of variables, I think we should leave the -d out of the macro
itself. I'll re-roll over the next couple of days.

[brian m. carlson]

The documentation for git-imap-send uses block delimiters with
mismatched lengths, which Asciidoctor doesn't support. As a result, the
page is misrendered. Adjust the delimiters so that they are of the same
length.

Signed-off-by: brian m. carlson <***@crustytoothpaste.net>
---
Documentation/git-imap-send.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-imap-send.txt b/Documentation/git-imap-send.txt
index 7d991d9..c7c0d21 100644
--- a/Documentation/git-imap-send.txt
+++ b/Documentation/git-imap-send.txt
@@ -97,7 +97,7 @@ Using direct mode:
host = imap://imap.example.com
user = bob
pass = p4ssw0rd
-..........................
+.........................

Using direct mode with SSL:

@@ -109,7 +109,7 @@ Using direct mode with SSL:
pass = p4ssw0rd
port = 123
sslverify = false
-..........................
+.........................


EXAMPLE

[brian m. carlson]

AsciiDoc specification states that in two-line titles, the title
underline has to be the same length as the title text, plus or minus two
characters. Asciidoctor, however, requires that this must be plus or
minus one character. Adjust the underlines to be the same length as the
title text to improve compatibility.

Signed-off-by: brian m. carlson <***@crustytoothpaste.net>
---
Documentation/git-prune-packed.txt | 2 +-
Documentation/git-quiltimport.txt | 2 +-
Documentation/git-stage.txt | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-prune-packed.txt b/Documentation/git-prune-packed.txt
index 6738055..9fed59a 100644
--- a/Documentation/git-prune-packed.txt
+++ b/Documentation/git-prune-packed.txt
@@ -1,5 +1,5 @@
git-prune-packed(1)
-=====================
+===================

NAME
----
diff --git a/Documentation/git-quiltimport.txt b/Documentation/git-quiltimport.txt
index a356196..d64388c 100644
--- a/Documentation/git-quiltimport.txt
+++ b/Documentation/git-quiltimport.txt
@@ -1,5 +1,5 @@
git-quiltimport(1)
-================
+==================

NAME
----
diff --git a/Documentation/git-stage.txt b/Documentation/git-stage.txt
index ba3fe0d..25bcda9 100644
--- a/Documentation/git-stage.txt
+++ b/Documentation/git-stage.txt
@@ -1,5 +1,5 @@
git-stage(1)
-==============
+============

NAME
----

[Junio C Hamano]

Thanks, obviously correct.

[Junio C Hamano]

This series is designed to implement the changes necessary to build Git
using Asciidoctor instead of AsciiDoc.
The first two patches are bug fixes. Asciidoctor is stricter about
title underline lengths ($B!^(B 1 character instead of 2) and requires
matching delimiter lengths[0]. They're needed regardless of whether the
other two patches are accepted because git-scm.com uses Asciidoctor to
render the documentation, so we might as well render it correctly.
Even with these patches, Asciidoctor warns about everyday.txt and
user-manual.txt. I'm not sending patches for these right now because
I've seen recent series including those and don't want to cause a
merge conflict.

Sounds good.

The second two patches implement some basic support for building with
Asciidoctor. The first of these moves some items into variables due to
some differences between the AsciiDoc and Asciidoctor command lines.
The user can then override these values when invoking make.
The final patch adds support for the linkgit macro. Asciidoctor uses
Ruby extensions to implement macro support, unlike AsciiDoc, which uses
a configuration file.

What I do not understand is that 3/4 lets you drop inclusion of
asciidoc.conf which contains a lot more than just linkgit:
definition.

For now I'll queue only the first two, which unquestionably take us
in the right direction.

Thanks.

[brian m. carlson]

Post by Junio C Hamano

The second two patches implement some basic support for building with
Asciidoctor. The first of these moves some items into variables due to
some differences between the AsciiDoc and Asciidoctor command lines.
The user can then override these values when invoking make.
The final patch adds support for the linkgit macro. Asciidoctor uses
Ruby extensions to implement macro support, unlike AsciiDoc, which uses
a configuration file.

What I do not understand is that 3/4 lets you drop inclusion of
definition.

Asciidoctor just doesn't understand the -f argument, so trying to pass
it is going to fail. For Asciidoctor, you're going to want to do
something like "-I. -rasciidoctor/extensions -rextensions" there
instead.

As for the rest of the asciidoc.conf file, the DocBook manpage header
declarations are implemented automatically by Asciidoctor after my
recent patches. The paragraph hacks do not appear to be necessary with
Asciidoctor, so they've been omitted.

That leaves the attributes. All but litdd are built-in to Asciidoctor,
and I can reroll with a modification to extensions.rb that implements
that one.

[Jakub Narębski]

Post by Junio C Hamano

The second two patches implement some basic support for building wi=

th

Post by Junio C Hamano

Asciidoctor. The first of these moves some items into variables du=

e to

Post by Junio C Hamano

some differences between the AsciiDoc and Asciidoctor command lines=

=2E

Post by Junio C Hamano

The user can then override these values when invoking make.
The final patch adds support for the linkgit macro. Asciidoctor us=

es

Post by Junio C Hamano

Ruby extensions to implement macro support, unlike AsciiDoc, which =

uses

Post by Junio C Hamano

a configuration file.

What I do not understand is that 3/4 lets you drop inclusion of
definition.

Asciidoctor just doesn't understand the -f argument, so trying to pas=

s

it is going to fail. For Asciidoctor, you're going to want to do
something like "-I. -rasciidoctor/extensions -rextensions" there
instead.
As for the rest of the asciidoc.conf file, the DocBook manpage header
declarations are implemented automatically by Asciidoctor after my
recent patches. The paragraph hacks do not appear to be necessary wi=

th

Asciidoctor, so they've been omitted.
That leaves the attributes. All but litdd are built-in to Asciidocto=

r,

and I can reroll with a modification to extensions.rb that implements
that one.

Would it be possible to automatically convert asciidoc.conf file to=20
Asciidoctor extension?

--=20
Jakub Nar=C4=99bski

[brian m. carlson]

Post by Jakub Narębski

Post by brian m. carlson

Post by Junio C Hamano
What I do not understand is that 3/4 lets you drop inclusion of
definition.

Asciidoctor just doesn't understand the -f argument, so trying to pass
it is going to fail. For Asciidoctor, you're going to want to do
something like "-I. -rasciidoctor/extensions -rextensions" there
instead.
As for the rest of the asciidoc.conf file, the DocBook manpage header
declarations are implemented automatically by Asciidoctor after my
recent patches. The paragraph hacks do not appear to be necessary with
Asciidoctor, so they've been omitted.
That leaves the attributes. All but litdd are built-in to Asciidoctor,
and I can reroll with a modification to extensions.rb that implements
that one.

Would it be possible to automatically convert asciidoc.conf file to
Asciidoctor extension?

It is in theory possible, but it's going to result in a lot of messy
code. I'm also not sure that Junio wants more than the minimal amount
of Ruby possible, since the goal has been to move away from scripting
languages and to C.

[Jeff King]

This series is designed to implement the changes necessary to build Git
using Asciidoctor instead of AsciiDoc.

Thanks. I had always taken the attitude that we wrote for the original
Python AsciiDoc, and that using AsciiDoctor was a choice that
git-scm.com made, and something they would have to deal with as far as
compatibility (AFAIK, AsciiDoctor grew out of git-scm.com's home-grown
asciidoc parser).

What's the status on AsciiDoc versus AsciiDoctor? The latter seems more
actively developed these days, but perhaps that is just my perception.
The incompatibilities seem fairly minimal (if those first two patches
are the extent of it, I have no problem at all trying to remain
compatible with both). Would it ever make sense to switch to AsciiDoctor
as our official command-line build program? I know it is supposed to be
much faster (though a lot of the slowness in our build chain is due to
docbook, not asciidoc itself).

Specifically I'm not excited about getting into a state where we have to
maintain both an asciidoc.conf file _and_ ruby extensions for
asciidoctor. I don't mind if somebody wants to step up and keep the
asciidoctor bits in sync with the asciidoc.conf, but I feel like one of
them needs to be considered the "master".

-Peff

[Junio C Hamano]

Post by Jeff King
Specifically I'm not excited about getting into a state where we have to
maintain both an asciidoc.conf file _and_ ruby extensions for
asciidoctor. I don't mind if somebody wants to step up and keep the
asciidoctor bits in sync with the asciidoc.conf, but I feel like one of
them needs to be considered the "master".

My so-far-unstated inclination, since seeing the patch to fix the
unbalanced example block separators from Brian (which was outside
and before this four-patch series), has been to keep our Makefile in
Documentation/ aware only of AsciiDoc while maintaining *.txt files
in a state so that AsciiDoctor could also be used to process them,
if people want to futz with their copies of Documentation/Makefile.

I do not mind to have the machinery to run AsciiDoctor too much in
my tree. It may make it easier for those who use it to spot places
in *.txt that need (in)compatibility workarounds between the two
formatters than keeping it outside.

But somebody needs to maintain that machinery and that will not be
me.

Thanks.

[brian m. carlson]

Post by Junio C Hamano

Post by Jeff King
Specifically I'm not excited about getting into a state where we have to
maintain both an asciidoc.conf file _and_ ruby extensions for
asciidoctor. I don't mind if somebody wants to step up and keep the
asciidoctor bits in sync with the asciidoc.conf, but I feel like one of
them needs to be considered the "master".

My so-far-unstated inclination, since seeing the patch to fix the
unbalanced example block separators from Brian (which was outside
and before this four-patch series), has been to keep our Makefile in
Documentation/ aware only of AsciiDoc while maintaining *.txt files
in a state so that AsciiDoctor could also be used to process them,
if people want to futz with their copies of Documentation/Makefile.
I do not mind to have the machinery to run AsciiDoctor too much in
my tree. It may make it easier for those who use it to spot places
in *.txt that need (in)compatibility workarounds between the two
formatters than keeping it outside.

I'd be happy if you simply picked up patch 3 and left out patch 4. It
gets us most of the way there, which is good enough for most things.
It's even possible to handle the litdd attribute on the command line, so
the only thing we'd really lose is the linkgit links.

Alternately, I'm happy to be responsible for maintaining the
extensions.rb file. The asciidoc.conf file has not had a substantive
(non-comment) change since 2012, and it has not had a change that would
require an update to the extensions since 2010. I don't anticipate that
keeping it up-to-date will require a significant amount of work. We can
even drop it into contrib if you think that's a better place.

It's really up to you which you'd prefer.

[Junio C Hamano]

Post by brian m. carlson

Post by Junio C Hamano

Post by Jeff King
Specifically I'm not excited about getting into a state where we have to
maintain both an asciidoc.conf file _and_ ruby extensions for
asciidoctor. I don't mind if somebody wants to step up and keep the
asciidoctor bits in sync with the asciidoc.conf, but I feel like one of
them needs to be considered the "master".

I do not mind to have the machinery to run AsciiDoctor too much in
my tree. It may make it easier for those who use it to spot places
in *.txt that need (in)compatibility workarounds between the two
formatters than keeping it outside.

Alternately, I'm happy to be responsible for maintaining the
extensions.rb file.

Let's see how well the patches 3 and 4 work for other people with
AsciiDoctor and then decide to go in that direction.

I do not forsee that changes to allow our *.txt to be used with
AsciiDoctor interfere with what GitHub folks do with their own
documentation toolchain, but I am not sure how the AsciiDoctor
specific alternative build infrastructure we would eventually ship
would interact with them---maybe they are not affected at all, or
maybe they can even take advantage of it.

Thanks.

[Thomas Braun]

Post by Jeff King
What's the status on AsciiDoc versus AsciiDoctor? The latter seems more
actively developed these days, but perhaps that is just my perception.
The incompatibilities seem fairly minimal (if those first two patches
are the extent of it, I have no problem at all trying to remain
compatible with both). Would it ever make sense to switch to AsciiDoctor
as our official command-line build program? I know it is supposed to be
much faster (though a lot of the slowness in our build chain is due to
docbook, not asciidoc itself).

Just recently we added the AsciiDoc toolchain to our git-for-windows/sdk
(formerly known as msysgit). So I'm not really fond of switching now to
something different again.

Remaining compatible with both would therefore be my choice.

Thomas

[brian m. carlson]

Post by Thomas Braun

Post by Jeff King
What's the status on AsciiDoc versus AsciiDoctor? The latter seems more
actively developed these days, but perhaps that is just my perception.
The incompatibilities seem fairly minimal (if those first two patches
are the extent of it, I have no problem at all trying to remain
compatible with both). Would it ever make sense to switch to AsciiDoctor
as our official command-line build program? I know it is supposed to be
much faster (though a lot of the slowness in our build chain is due to
docbook, not asciidoc itself).

I don't think there's a lot of benefit for us to switch, and I say that
being a contributor to Asciidoctor. It's useful to be able to build Git
with both simply to find incompatibilities that we're going to need to
fix anyway, due to the fact that Asciidoctor is used for the website.

And yes, those first two patches are it, as far as I'm aware.

Post by Thomas Braun
Just recently we added the AsciiDoc toolchain to our git-for-windows/sdk
(formerly known as msysgit). So I'm not really fond of switching now to
something different again.
Remaining compatible with both would therefore be my choice.

That's my goal. I simply wanted the ability to support both AsciiDoc
and Asciidoctor without making major changes to the codebase. Hence,
moving the calls into variables.

[Philip Oakley]

This series is designed to implement the changes necessary to build
Git
using Asciidoctor instead of AsciiDoc.

[..]

Even with these patches, Asciidoctor warns about everyday.txt and
user-manual.txt. I'm not sending patches for these right now because
I've seen recent series including those and don't want to cause a
merge conflict.

Does the new version for giteveryday.txt and everyday.txt which
graduated to master, 1cb3324 (Merge branch 'po/everyday-doc',
2014-10-16) format OK?

i.e. does 'git help everyday' now correct the Asciidoctor warnings.

I don't have access to Asciidoctor (on MsysGit), but did make sure the
header underlines were updated.
--
Philip