What is the best way to backport a feature?

[Peter Weseloh]

Hi,

Suppose I have the following situation:

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ \ \ /
F1--F2--M1--F3--M2 Feature_A

Now I want to backport "Feature_A" to the "Release_1.0" branch so that it gets
included into the next minor release, i.e. I want to apply the commits F1, F2
and F3 onto the "Release_1.0" branch.
I cannot just merge "Feature_A" into "Release_1.0" because that would also bring
in the merges M1 and M2 so a lot of other stuff from the Mainline.

I played with cherry-pick but that means I have to manually find the commits F1,
F2 and F3 (which in reality could be many more if Feature_A is big) which is not
very nice.

I also tried 'rebase -i' but that means I have to manually delete all the lines
for changesets from the mainline. Also not very nice.

Is there a better way? To me this scenario sounds not unusual but I could not
find a solution.

Thanks,
Peter

[Björn Steinbrink]

Post by Peter Weseloh
o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ \ \ /
F1--F2--M1--F3--M2 Feature_A
Now I want to backport "Feature_A" to the "Release_1.0" branch so tha=

t

Post by Peter Weseloh
it gets included into the next minor release, i.e. I want to apply th=

e

Post by Peter Weseloh
commits F1, F2 and F3 onto the "Release_1.0" branch.
Is there a better way? To me this scenario sounds not unusual but I
could not find a solution.

What's unusual there is that you merged from Mainline to Feature_A.
Usually, the history would look like this:

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ /
F1-----F2------F3 Feature_A

And then you could easily use rebase to get the job done.

Had you known beforehand that Feature_A is a candidate for backporting,
you would have even branch from an older commit like this:

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ /
F1--------F2-------F3 Feature_A

Then you could easily merge Feature_A to Release_1.0 as well, without
merging anything unrelated.

But that's just for the future...

Given you current history, you could use format-patch + am like this:

git format-patch --stdout --first-parent Mainline..Feature_A > fa.mbox
git checkout Release_1.0
git am -3 fa.mbox

The --first-parent options make it follow the first parent of the merge
commits only, so the whole stuff on the Mainline branch is ignored. And
you just get F1, F2 and F3 in fa.mbox, which you then apply using am.

A long time ago, I hacked the --first-parent thing into rebase, but (of
course) the first iteration of the patch wasn't quite perfect and as
I've not been scratching my own itch there, I never got around to
actually polish the patch so it could get into git.git. Maybe you want
to pick it up?

http://thread.gmane.org/gmane.comp.version-control.git/62782

Bj=F6rn

[Pascal Obry]

Post by Björn Steinbrink
What's unusual there is that you merged from Mainline to Feature_A.

Right, I missed that! It is indeed very unusual at the point that I=20
missed to read it properly :) So my reply is wrong.

Pascal.

--=20

--|------------------------------------------------------
--| Pascal Obry Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--| http://www.obry.net - http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

[Peter Weseloh]

Hi Bj=F6rn,

=46irst of all thank you very much for your quick reply (actually my
thanks go to all that have replied so far).
Note that at the moment it is just a brain exercise. We are currently
using CVS (yes, I know) and want to switch to something else and I'm
trying to push for git. During our discussions this scenario came up
and I could not give a simple answer. That's why I thought I'd better
ask the experts.

Post by Björn Steinbrink
What's unusual there is that you merged from Mainline to Feature_A.
=A0 o--o--o =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Release_1.0
=A0/ =A0 =A0\ =A0\
=A0o-o-o--o--o-o-o-o-o-o---o--o Mainline
=A0 =A0 =A0\ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 /
=A0 =A0 =A0 F1-----F2------F3 =A0 =A0 =A0Feature_A
And then you could easily use rebase to get the job done.

But on the other hand the intermediate merges from the Mainline make
for much simpler merges, right?.
If merging is done only when Feature_A is ready it might become a real
pain. It might take several month to complete it and the mainline
might have changed a lot.

Post by Björn Steinbrink
Had you known beforehand that Feature_A is a candidate for backportin=

g,

Post by Björn Steinbrink
=A0 o--o--o =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Release_1.0
=A0/ =A0 =A0\ =A0\
=A0o-o-o--o--o-o-o-o-o-o---o--o Mainline
=A0\ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 /
=A0 F1--------F2-------F3 =A0 =A0 =A0Feature_A
Then you could easily merge Feature_A to Release_1.0 as well, without
merging anything unrelated.
But that's just for the future...

Yes, sure. If I would know the future already today I would not need
to do any coding anymore :-) But seriously our policy is clear:
Bugfixes (and small enhancements) go to the release branch to end up
in the next minor release. The release branch gets merged with the
mainline so that it is always a superset. Big new features are
developed in seperated branches and are finaly merged to the mainline
to end up in the next major release. But every now and then the
managment is so excited about such a new feature that they want it in
the next minor release. That's life.

Post by Björn Steinbrink
git format-patch --stdout --first-parent Mainline..Feature_A > fa.mbo=

x

Post by Björn Steinbrink
git checkout Release_1.0
git am -3 fa.mbox
The --first-parent options make it follow the first parent of the mer=

ge

Post by Björn Steinbrink
commits only, so the whole stuff on the Mainline branch is ignored. A=

nd

Post by Björn Steinbrink
you just get F1, F2 and F3 in fa.mbox, which you then apply using am.

Ah, great! I played with format-patch + am but missed the
'--first-parent' option. I will give it a try. Thanks a lot!

Post by Björn Steinbrink
A long time ago, I hacked the --first-parent thing into rebase, but (=

of

Post by Björn Steinbrink
course) the first iteration of the patch wasn't quite perfect and as
I've not been scratching my own itch there, I never got around to
actually polish the patch so it could get into git.git. Maybe you wan=

t

Post by Björn Steinbrink
to pick it up?
http://thread.gmane.org/gmane.comp.version-control.git/62782

In case we go for git this might very well be the case.

Post by Björn Steinbrink
Bj=F6rn

Peter

[Johannes Sixt]

[please keep the Cc list]

Post by Peter Weseloh
But on the other hand the intermediate merges from the Mainline make
for much simpler merges, right?.
If merging is done only when Feature_A is ready it might become a real
pain. It might take several month to complete it and the mainline
might have changed a lot.

Incidentally, Junio has blogged about this just the other day:

http://gitster.livejournal.com/42247.html

Basically, as soon as you merge Mainline into Feature_A, you change the topic
of Feature_A from "Feature for Release_1.0" to "Feature for this Mainline".
Clearly, this topic is not suitable for Release_1.0 anymore.

There is a way around this that doesn't sacrifice the topic-oriented nature of
the branch: You keep developing Feature_A as planned for Release_1.0 and when
you notice that merging this feature to Mainline will become increasingly
complex, you fork off a new branch Feature_A_for_Release_2.0 from Mainline
and merge Feature_A into this new branch:

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-X-o---o--o Mainline
\ \
F1 o--o Feature_A_for_Release_2.0
\ / /
F2--------F3-F4 Feature_A

The fork point X must be in Release_2.0.

-- Hannes

[Peter Weseloh]

Post by Johannes Sixt
[please keep the Cc list]

Sorry!

Post by Johannes Sixt
http://gitster.livejournal.com/42247.html
Basically, as soon as you merge Mainline into Feature_A, you change t=

he topic

Post by Johannes Sixt
of Feature_A from "Feature for Release_1.0" to "Feature for this Main=

line".

Post by Johannes Sixt
Clearly, this topic is not suitable for Release_1.0 anymore.
There is a way around this that doesn't sacrifice the topic-oriented =

nature of

Post by Johannes Sixt
the branch: You keep developing Feature_A as planned for Release_1.0 =

and when

Post by Johannes Sixt
you notice that merging this feature to Mainline will become increasi=

ngly

Post by Johannes Sixt
complex, you fork off a new branch Feature_A_for_Release_2.0 from Mai=

nline

Post by Johannes Sixt
=A0 o--o--o =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Release_1.0
=A0/ =A0 =A0\ =A0\
=A0o-o-o--o--o-o-o-o-X-o---o--o Mainline
=A0 =A0 =A0\ =A0 =A0 =A0 =A0 =A0 =A0 \
=A0 =A0 =A0 F1 =A0 =A0 =A0 =A0 =A0 =A0o--o =A0 =A0 Feature_A_for_Rele=

ase_2.0

Post by Johannes Sixt
=A0 =A0 =A0 =A0\ =A0 =A0 =A0 =A0 =A0 / =A0/
=A0 =A0 =A0 =A0 F2--------F3-F4 =A0 =A0 =A0Feature_A
The fork point X must be in Release_2.0.

That makes perfect sense. I will discuss your suggestion with my
colleagues and will send them the link you mentioned. It's just that
branching and especially merging with CVS is so painful that they
might get scared :-). With git that's completly different, of course.

Thanks a lot,
Peter

[Björn Steinbrink]

What's unusual there is that you merged from Mainline to Feature_A=

=2E

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ /
F1-----F2------F3 Feature_A
And then you could easily use rebase to get the job done.

=20
But on the other hand the intermediate merges from the Mainline make =

for

much simpler merges, right?.
If merging is done only when Feature_A is ready it might become a rea=

l pain.

That's usually more often true with CVS or SVN than with git, but ...

It might take several month to complete it and the mainline might hav=

e

changed a lot.

=2E.. over such a long timeframe, yes, things might become ugly. OTOH s=
uch
a long timeframe might also mean that the topic branch actually does to=
o
much. Splitting such a large thing into more manageable pieces would
help there, as you could merge completed topic branch to your mainline
branch earlier and more often.

Had you known beforehand that Feature_A is a candidate for backport=

ing,

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ /
F1--------F2-------F3 Feature_A
Then you could easily merge Feature_A to Release_1.0 as well, witho=

ut

merging anything unrelated.
But that's just for the future...

Yes, sure. If I would know the future already today I would not need =

to do

any coding anymore :-)

I meant something like "I just said that, so you can avoid problems in
the future" ;-) But yeah, knowing beforehand that things should go into
a maintenance branch isn't common, unless it's about a bugfix.

Given you current history, you could use format-patch + am like thi=
git format-patch --stdout --first-parent Mainline..Feature_A > fa.m=

box

git checkout Release_1.0
git am -3 fa.mbox
The --first-parent options make it follow the first parent of the m=

erge

commits only, so the whole stuff on the Mainline branch is ignored.=

And

you just get F1, F2 and F3 in fa.mbox, which you then apply using a=

m.

Ah, great! I played with format-patch + am but missed the '--first-pa=

rent'

option. I will give it a try. Thanks a lot!

Well, it's a rev-list option, which might work by accident. Junio
recently said that the fact that format-patch accepts path limiting is
by accident, might be true for --first-parent as well... No clue. Junio=
?

Bj=F6rn

[Pascal Obry]

Post by Peter Weseloh
Hi,
o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ \ \ /
F1--F2--M1--F3--M2 Feature_A
Now I want to backport "Feature_A" to the "Release_1.0" branch so tha=

t it gets

Post by Peter Weseloh
included into the next minor release, i.e. I want to apply the commit=

s F1, F2

Post by Peter Weseloh
and F3 onto the "Release_1.0" branch.
I cannot just merge "Feature_A" into "Release_1.0" because that would=

also bring

Post by Peter Weseloh
in the merges M1 and M2 so a lot of other stuff from the Mainline.
I played with cherry-pick but that means I have to manually find the =

commits F1,

Post by Peter Weseloh
F2 and F3 (which in reality could be many more if Feature_A is big) w=

hich is not

Post by Peter Weseloh
very nice.
I also tried 'rebase -i' but that means I have to manually delete all=

the lines

Post by Peter Weseloh
for changesets from the mainline. Also not very nice.
Is there a better way? To me this scenario sounds not unusual but I c=

ould not

Post by Peter Weseloh
find a solution.

In such a case I would use a rebase onto:

$ git co Feature_A
$ git rebase --onto Release_1.0 F1 F3

Then

$ git co Release_1.0
$ git merge Feature_A

Pascal.

--=20

--|------------------------------------------------------
--| Pascal Obry Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--| http://www.obry.net - http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

[Michael J Gruber]

Post by Peter Weseloh
Hi,
o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o---o--o Mainline
\ \ \ /
F1--F2--M1--F3--M2 Feature_A
Now I want to backport "Feature_A" to the "Release_1.0" branch so that it gets
included into the next minor release, i.e. I want to apply the commits F1, F2
and F3 onto the "Release_1.0" branch.
I cannot just merge "Feature_A" into "Release_1.0" because that would also bring
in the merges M1 and M2 so a lot of other stuff from the Mainline.
I played with cherry-pick but that means I have to manually find the commits F1,
F2 and F3 (which in reality could be many more if Feature_A is big) which is not
very nice.
I also tried 'rebase -i' but that means I have to manually delete all the lines
for changesets from the mainline. Also not very nice.
Is there a better way? To me this scenario sounds not unusual but I could not
find a solution.

The problem is that you've been a bad boy to begin with ;)

Seriously, I suggest reading up on "topic branches". Feature_A should
have been based off the common merge base of Mainline and Release_1.0,
and, even more importantly, there should not have been any merges from
Mainline into Feature_A. So, that branch is not at all what one would
call a feature branch/topic branch. Hopefully, this scenario is very
uncommon :)

I assume you have to deal with the given structure anyhow, and merge
will not help. The only solution is to try and replay your Feature_A
commits on top of the release branch. (Since you have merged Feature_A
into Mainline already, you probabably don't want to redo that branch and
merge.)

I you have many commits to deal with I suggest finding a good
semi-automated way to list the commits you are after, such as git
rev-list --no-merges sha1..Feature_A (with sha1 being the fork point). A
good way to find out could be git log --no-merges sha1..Feature_A.

Then, try and cherry-pick those onto the release branch. Alternatively,
you can use format-patch/am, or in fact try with rebase (I thought it
would ignore merges), which basically does what cherry-pick does.

Cheers,
Michael

[Greg A. Woods]

Hmmm.... this topic seems in part to be very close to my thread
"git merge" merges too much!

At Sun, 29 Nov 2009 18:02:32 +0100, Michael J Gruber <***@drmicha.warpmail.net> wrote:
Subject: Re: What is the best way to backport a feature?

Post by Michael J Gruber
Seriously, I suggest reading up on "topic branches". Feature_A should
have been based off the common merge base of Mainline and Release_1.0,
and, even more importantly, there should not have been any merges from
Mainline into Feature_A. So, that branch is not at all what one would
call a feature branch/topic branch. Hopefully, this scenario is very
uncommon :)

IFF I understand the original post correctly then actually in my
experience this scenario is VERY common in some environments!

This is exactly how large projects which use the likes of CVS (and SVN?)
manage longer-running development branches for important new features.

One excellent example is NetBSD (and perhaps the other BSDs too).

A developer creates a "working" branch from the trunk, then begins to
make changes and commits to that branch. Periodically the (entire)
trunk is merged again to the working branch. I think this is somewhat
equivalent to using "git rebase" to re-apply the feature branch to a new
fork point from the trunk. However the actual branch base point remains
at the original point -- it is only the delta between the last merge
from trunk and the current head of the trunk which is merged onto the
feature working branch. I think this is what people in the CVS world
mean when they say they want the tool to remember the point on the
source branch from where they did the last merge. They've got their
work-flow "backwards", but this is the best they can do with CVS.

These periodic merges from the trunk mean that once the feature is
finished the delta between the trunk and the new feature branch is going
to be just the new feature, and so merging that delta alone to the trunk
as one commit adds the new feature to the trunk with few or no
conflicts, and the feature working branch can finally be "closed".

I'm guessing that people moving to Git from CVS may choose to stick with
this pattern where they periodically merge-from-master to keep
long-running feature branches as close to in-sync with the master branch
as possible (to avoid final merge conflicts). Ideally, IIUC, perhaps
they should use rebase instead.

Perhaps this "mess" can indeed be cleaned up using "git rebase -i" so
that the final version of the feature branch can be back-ported more
easily (though one will still need to use git-cherry-pick or git-am to
do the back-port to the previous release branch). The result of the
cleanup, before the merge of Feature_A to 1.0 might look more like this:

o--o--o Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o-o-o---------------o Mainline
\ /
F1'--F2'--F3' Feature_A

and then after the merge of Feature_A to Release_1.0:

o--o--o--F1''--F2''--F3'' Release_1.0
/ \ \
o-o-o--o--o-o-o-o-o-o-o-o---------------o Mainline
\ /
F1'--F2'--F3' Feature_A