today I'm thinking about the tradeoffs of using `git rebase` a bit. I think the goal of rebase is to have a nice linear commit history, which is something I like.

but what are the *costs* of using rebase? what problems has it caused for you in practice? I'm really only interested in specific bad experiences you've had here -- not opinions or general statements like “rewriting history is bad”

@Anarcat thanks, this is great

@b0rk
When rebase caused conflicts with the target branch, I naturally solved that conflict. But when I then had to rebase onto that same branch again (develop moves rapidly and this feature branch was taking a few days to complete), I had to solve the same conflict again. And again the next day…

@b0rk git rebase should be banned in any sensible team. I think it is highly indicative of very poor development workflows. You should never need it or at least if it is ever needed that it is only done by the head SCM with multiple signoffs and blood sacrifices and promises that this should never come to pass again.

@nf3xn can you share some bad experiences you’ve had with rebases? i’ve heard your opinion a million times (and i’m sure it’s for good reason!!) but what I really need to know is what bad things happened to you that made you have that opinion

@b0rk We (https://github.com/RobotLocomotion/drake/) use a pure-rebase workflow.
The major cost I've seen is that it plays badly with some tools, and github is sort of top-of-the-list, though reviewable as well: Which commit's message gets to be the rebased PR commit's message is hard to predict and we've had a lot of chained commit messages sneak into history.
If you use commit metadata like co-authored-by it's super easy to lose that; we have external contributors for whom that's professionally damaging.

@nonnihil thank you!

@b0rk I think there are harmful and non-harmful ways of using "rebase." When preparing your set of patches for submission upstream, rebase is essential -- you can split large patches into logically smaller ones, fix typos, reorder commits for better clarity and squash some of them together if they really don't need to be two separate actions. All of this is easily accomplished with "git rebase -i". It is also useful to be able to rebase your work on the newer version of the main branch right before you submit your changes upstream.

However, many other applications of rebase can be sloppy and harmful, especially when used without proper understanding of the effects it will have on the project history. I many situations, merge commits are preferable to rebases.

@b0rk rebasing feels scarier than merging. I would tell you why if I knew!

@sarajw no that’s still helpful thanks! i think there are a lot of very good reasons it feels scary

@sarajw
@b0rk Seconded!

Also, if you had to fix any conflicts during earlier merges, when you do a rebase, you will often have to fix them again.

@sarajw
I find it scarier because it rewrites history. I've watched enough science fiction to know that's dangerous and often ends in some sort of parable.
@b0rk

@b0rk If you have many local commits and have a key component that needs rebasing, this can be much work. I once needed to rebase 1 month of dev work and it took like 3 hours. A merge would have been much quicker but policy states keep the history nice.

@b0rk I've sometimes had to give up on rebasing because some change would change every single commit on a branch, and just merging it would have been much easier. In any other case, I do prefer rebasing, though.

@kescher thanks that makes sense!

@monsieuricon @b0rk I rely on "git rebase -i" so much that I can't imagine life without it, particularly for larger 80, 100 commits patch sets where you tweak and re-tweak things as you work with the changes. You can even automate what you do within the interactive rebase.

@b0rk your branch is behind 10 commits on the same file, then you rebase and the base has many commits on that file also, it's painful to rebase. My go to solution is to abort the rebase, squash all my commits into 1 as I say good bye to all my well-written commit messages as I can't stand fixing the same diff over and over

@bini yeah that’s happened to me too , i wish there were a better way to keep the nice commits

@bini @b0rk i never had a situation bad enough to actually learn how it works, but there is git rerere for that situation https://www.git-scm.com/book/en/v2/Git-Tools-Rerere 😅

@tshirtman @b0rk WOW, I had no idea this existed. I will certainly try this out next time that happens. Thank you very much!

Also, the name of this feature is awesome

@b0rk Coordinating a rebase on a shared branch is very tough, especially when not everyone in the team is a Git master. When there is a problem, it’s usually very complicated. So in that situation I generally avoid it. Otherwise I see no problem with it.

@JasonNickel yeah i always assume everyone agrees that you should basically never rebase a shared branch

@codonell @monsieuricon @b0rk I also always use "git rebase --interactive --autostash --autosquash".
--interactive is the same as -i, essential.
--autosquash will automatically move fixup/squash created with "git commit --fixup=<hash>" after the commit it fixes up AND change the type of commit "picking" to fixup/squash. rebase will then do the proper thing. This doesn't work if you have 2+ commits with the same title.
--autostash stashes your current changes and pop the stash after the rebase.

@b0rk
It can be painful to review feature branches that are rebased during development (often for good reasons). My local branch gets out of sync with the remote, rebased branch so I can't just pull in the most recent work on top of what I had from before. I usually just delete the local branch and re-fetch and checkout it. I don't know if there's a more elegant way.

@osma @b0rk git checkout local/branch; git fetch origin; git reset --hard origin/branch; should do it, provided your remote is called origin.
This basically moves the branch to point to the last commit of branch branch in origin remote.
I usually like to create v2, v3 branches from rebases, so I can then check the differences between versions. And you can then thank git range-diff which allows you to do a commit diff between two branches with different commit base.

@b0rk it cause me problems because i know i can use it 😅 i'm not afraid of doing it, because i think i understand it pretty well, so i am prone to make a mess i'll have to fix later, because i can fix it… until it becomes too much of a mess that moving commits around and splitting changes to make a nice history is just not possible anymore, no matter how good i'm with rebase… 😅

@tshirtman i hadn’t thought about that thanks!

@nf3xn @b0rk I agree with the main development branch (or any that can be used by any developer other than yourself). But.. do you have the same opinion on feature branches? How do you fix some feedback you received on one of the first commits in said branch? Just a fixup commit as last commit of the branch?

@DerMolly thanks, this explains really clearly something i’ve been struggling to understand (i’ve been thinking about rebasing short lived branches but with an older branch I can see how it can get MUCH harder)

@b0rk rebase with the wrong differencing algorithm for the content type you're using turns into an endless cascade of manual merges.

(I think the goal of rebase is to identify conflict locally, before you try to contribute that code to a project repository.)

@graydon wait what are the right algorithms? does git give you options?

@b0rk
Rebasing is a pain with poetry.lock (and other generated, non-human-diffable files).

I think rebases can be really helpful on merge/PR branches, to be certain of the diff against HEAD. But you do have to skip merge conflicts on things like poetry.lock and then remember to regenerate them before pushing the rebased branch.

@BigBeardyGeek ooh I hadn’t thought of that

@b0rk it's easy to mess it up. i've seen rebases where i mashed togethers commits that shouldn't have been mashed, and lost commits. it can also be harder to resolve conflicts if multiple commits edit the same section of the file that's in conflicts, where a merge might need to resolve that conflict only once

@squid @b0rk what issue is that? i actually find that squash and rebase work pretty well! i use rebase.autosquash=true to great effect

@Anarcat @b0rk I mostly just wish that rebase and squash played a bit nicer together. I get why the issue exists, just wish it didn't

@b0rk I try to stop rebasing once people are reviewing the code, this is when hashes should stay intact.

@hq1 @b0rk you can still push additional commits (git commit --fixup if appropriate) and then rebase once you’ve all agreed it’s ready to be rebased.

@b0rk @JasonNickel Everybody except me. If two or three people collaborate on a topic branch, fixups and a final rebase or two before the merge is fine. Sure, don't rebase a master/main/trunk or even a long-lived dev branch, but topic branches, even if shared, are fair game.

@ellotheth these are all SO good thank you!

@b0rk I dunno why but at least two times in my career (15y) I ended up in the infinite loop of rebase. I don't recall how that happened but I was fixing conflicts and after a while, I just realized that I was following my footprints. Never-ending the conflict and passing through the same conflicts over and over.

@pastorinni great example, what a nightmare

@chadgeidel @nf3xn if you're going to have an argument about this can you leave me out of it? thanks. I’m just interested in hearing about people's experiences.

@jani dude can we not tell people they're doing programming wrong on here thanks, I asked for people's experiences and I don't want to scare folks off because people are going to come in and make unsolicited comments about their workflow

@b0rk rebasing, it's really easy to lose hunks of changes. it's also really hard to share a branch you're rebasing because the ids change (this also applies to github working out if you merged it).

@lenary can you say a little more? i'm trying to understand how people lose their work with rebase in practice

@b0rk I work on a product (cloud development environment) that does a lot of automatic syncing/pulling which can be chaotic when you use a rebase workflow. Any tool that tries to be helpful in the background usually assumes that you want to merge branches.

@ajf thanks!

@b0rk For me the cost of rebases is the fact that you basically have to have two completely separate mental models of what a 'commit' is.

Normally, git treats commits as the state of the repository and its whole history at some point, while with rebase and cherry-pick and the like, git treats commits and branches as basically a set of patches. (1/2)

@b0rk This patch-based model allows you to very easily rework and combine commits in a feature branch (especially with interactive rebasing!) to create a nice and easily reviewable history, which is great, but comes at the cost of making it hella easy to shoot yourself in the foot by failing to correctly merge, re-merge, split, or combine commits.

2/2

@b0rk As for bad experiences I've had with it: Especially in my early days with git, I've often had weird situations where I left an unfinished rebase with something like 'git reset --hard' and forgot to actually abort the rebase, making some things behave weirdly until I noticed a while later

@yrlf that's a great example thanks!

@b0rk i avoid rebasing on shared branches after I've pushed.

If you do rebase and you force-push your changes to the branch you're over-writing the ref's (the history) on the remote host to match yours.

Other folks on the team have their own copy of the history that is no longer valid and when they go to push their changes they will get scary looking errors (unless you were kind and notified them of what you did).

I use rebase a lot, like voraciously, on my local repository. I got into the habit of making very small, incremental changes and I'm not always keen on formatting messages properly for others to view, etc. However once I'm ready to push I rebase my local history to look good for the public before pushing the final result.

@b0rk maybe there should be a rhyme like...

rebase before push
protect your tush
rebase and force push
with ire you will brush

I dunno... i'm bad at poems I guess. 🤣​

@agentultra totally! that's why I didn't ask how people use rebase **successfully** (I use rebase the exact same as way you! it works great!), I'm interested in what goes **wrong** for you :).

do you have any examples of things that have gone wrong with rebase

@b0rk a papercut: if you rebase your working copy to keep a clean history for a branch, but the underlying project uses merges, the result can be ugly. If you do rebase -i HEAD~4 and the fourth commit back is a merge, you can see dozens of commits in the interactive editor.

@hyperpape thanks I didn't know that!

@b0rk so when I was doing a rebase as a newbie it was pretty easy for me to massively fuck up my local tree. Sometimes I messed up so bad that I had to delete my local code and re clone it.

I think git rebase is like a circular saw. It's powerful and when used properly does a good job but when improperly used it has pretty serious consequences.

@ladytel thank you!! do you have any idea why things got messed up in a rebase? i've heard this a lot of times and i'm trying to understand a bit better what goes wrong exactly

@spudlyo great example thank you!

@monsieuricon @b0rk

git config --global pull.rebase true

When developing, rebase your branch to keep up with upstream.

When maintaining, merge to accept the patches.

IMO it's always about from which point of view you want solve conflicts.

@b0rk in my experience what normally happens whenever someone "rewrites history" (aka. updates a published ref in a non-fast-forwardable way) means that every copy of the old ref in someone's clone could be resurrected sometime in the future when they do an absent-minded "git pull" on that branch. I normally teach my colleagues that the "pretty history" is not worth the trouble once an intermediary commit has been published.

@rpetre do people really rewrite history on shared branches? i’ve never seen that and i’m confused about why anyone would do that

In a sense, it's similar to "there's no rollback in production": once someone else experienced the $BAD_THING, it's better to realize that you can only "fix forward" aka. also deal with the consequences of 3rd party state being fubared.

@b0rk it is quite popular on pull requests.

You make a branch, push it to the server and open pull/merge request. Then have a fix for it. You rebase and force push. Repeat after each review.

Also there are development branches shared with people. Feature is developed until being ready for upstreaming. With rebasing in meantime.

@b0rk I used it on a small project & enjoyed the purity, but the second I’m on a project with any kind of dev velocity I find it super irritating; everyone constantly has to rebase (and potentially invalidate their code review, depending on how you’ve set that up); the conflicts can be confusing to resolve (somehow more so than with merge imho). I hit my frustration ceiling with it fast.

(I’m married to @nonnihil , elsewhere in replies; obviously I am the Chaos Muppet of this relationship 🤣)

@thatandromeda @nonnihil thank you! i agree that rebase conflicts are somehow worse than merge conflicts even though it feels like it should be the same thing, maybe because there can be more than one

(i am also the chaos muppet of my relationship :))

@b0rk

Oh god, I'm strongly on team "merge-only", and part of it is the "resolve the same conflict for every commit on my branch" is only part of it.

It breaks the entire expectation of multiple people being able to cooperatively work on a branch, the requirement that things get force pushed (mucking up everyone's histories) is just stressful.

I used github's automated rebase today and had to delete my local branch and switch to the remote again to fix it. Had I just merged, it would have been as simple as reset and pull.

And I'm philosophically opposed to squash rebase but don't have space in this post.

@pathunstrom thanks, the idea of people cooperatively working on a branch is so interesting to me!! i didn't realize people did that somehow, and I can see how it wouldn't work well with rebase

@pathunstrom @b0rk Why is git the almost universal standard these days? Like, these shenanigans in the middle of your coding workflow... Ugh! That sounds terrible! It's confusing to the point of being dangerous and painful. Why do we all collectively put up with it?

@chadgeidel thanks so much!

@b0rk Early on in my git journey I completely wiped out 2 weeks worth of work by misunderstanding what the rebase was doing. It's frankly ridiculous that the tool doesn't at least automatically make a backup branch before embarking on irreversible changes. The community response seems to be "you should have made your own backup." But we write code to protect users from themselves ALL THE TIME. Just the first approximation of safety would be absolutely trivial. (And I *like* rebasing.)

@cammerman yeah I agree it should be much easier to undo a rebase. I think git's attitude towards this (which I don't agree with) is "well, technically all the information about where your branch was before is in it `git reflog BRANCH`, so you don't need a backup branch”. But the reflog is really hard to use and I think from a newcomer's perspective it might as well not exist.

@b0rk Sure.

- One weird thing is trying to rebase a commit from its original place *past* its own revert, so you can reapply it (as is common in projects that prefer reverting to fixing forwards). The rebase will by default drop the original commit when it sees the revert, and you're left with a branch that doesn't contain the branch itself. Usually in this case, you're starting with a "branch" pointing at the merged commit on main, and wanting to `git rebase --onto` to move the whole branch to being based of main post-revert, so you can fix up the patch.

- Another case is when i have a branch I did the work on, and i merge it with e.g. github using "merge and rebase". Likely the sha for the rebased change on `main` is different to the one on my branch, so i don't get any help from git telling me the branch's contents are on main, and so i have to manually work out which of my local branches are merged or not, and git's "git branch -d" doesn't work, so you have to use "git branch -D" which, if you confuse branch names, means you lose a branch that's not actually been merged into main.

- The other case that's difficult is when multiple people are working on a shared branch, and person A rebases. Person B now needs to move their changes from their branch, onto the rebased branch, which frequently results in rebase/merge conflicts, which are just difficult in general, and make B prone to losing half their changes if the rebases are frequent.

- Another thing is that I find `git config rerere.enabled true` untrustworthy in some conflicts from rebases, so it can save an old resolution and use it whereas you actually wanted to do a new resolution, with new contents. I don't know how git rerere works out it's the "same conflict", but it's fuzzy and not always correct.

@lenary these are all SO helpful thank you! i'm learning from a few different people that people do rebases on shared branches, which I'm still a bit confused about (why would you do a rebase on a shared branch?) but it's very easy for me to see how that could go wrong

@jani thanks so much! i'm trying to improve the vibe in these replies a bit so I really appreciate it

@b0rk I love rebasing, but one thing I find uncomfortable and error-prone is splitting commits in interactive rebase (plug: my current recipe is described here: https://github.com/kimgr/git-rewrite-guide#split-a-commit)

@kimgr thanks that's a great point -- it's so hard to do!

@b0rk ah no, just force pushing changes I rebased when I didn’t know how it worked and tried to follow the “I always rebase to keep my history clean” advice but had no idea what I was doing. Never did figure that out. But I did ruin a good number of repos that way; losing changes or causing hard merge conflicts.

@agentultra those are great examples, thanks!

@b0rk Outside of git submodules, I find git rebase to be the most confusing part of git. My experiences have always come down to some flavor of not quite understanding the state of a rebase in progress and either propagating incomplete merge conflict resolutions to future commits or otherwise adding junk to the final state of my branch.

rebase -i and editor support have helped a lot, but it's still always tricky to parse out the state of a rebase.

@xgranade thank you!

@b0rk replies to your git post sreally underline how people really use git differently than I do. (ie a particular flavor of Etsy style)

@stormsweeper yeah it's so interesting! which differences stood out to you? i was surprised to hear that people collaborate on branches

@stormsweeper @b0rk I was thinking the exact same thing. Every where I've worked people have different methods for using Git. Hey look. I didn't even say GitOps! Dammit.

@b0rk some people have made a habit of using rebase instead of merge. And when they try to do that on a PR, they have to force push.

@kboyd @b0rk I do that sometimes #sorrynotsorry #idontknowanybetter

@kf @b0rk I think I went a few years without rebasing, until I had to do it twice last month. On PRs that were themselves a few years old. (it was a way better experience than merge for bringing things up-to-date)

@kboyd @kf yeah I do that too, to me a PR isn't really a "shared branch" in the same way (nobody's going to `git pull` from it for example)

@b0rk I see it from junior developers pretty often, actually! Folks who have not yet developed the mental model for the “distributed” and “branch” parts of it will sometimes hit a situation where they have rebased (and shouldn’t!) because of other guidance, and cannot push, look up the error Git reports, find an answer suggesting to force push, and do so. I blame this primarily on the inherent trickiness of the “distributed” part of DVCS plus the affordances of Git as a tool making that easy.

@chriskrycho that makes a lot of sense thank you!

@b0rk
I've seen it last week; and I'm partly responsible for the situation that caused it.

(Private) project decided, a few months ago, to enforce some rules on commit messages for commits in the main branch. Other branches are considered more or less private or playground, so not enforced there, but as a check that PRs must pass to be merged.

A specific branch, dedicated to some feature, common to two people, started before the rules were set. Last week they wanted to merge it.

@shaib thanks that's a great example!

@b0rk all the time. When working with others I communicate about it and ask if anything is outstanding in their local. If there is we coordinate it then someone will rebase and we fetch it.
I don’t understand why others have problems with it but from my perspective it seems like a personal communication problem.
Also we always —force-with-lease

@saturnflyer thanks! I can see how it would work that way

@b0rk not sure how common it is, I've seen it happen a few times and I've learned to make sure to impart spooky stories about it whenever I help someone with a rebase. From what I can tell, it can feel a bit like a magic wand that can fix anything so eventually someone abuses it. I remember a quote from Linus about rebase being able to do any git operation, but I can't find it, instead I found this lkml thread that has several points that could fit in this context: https://yarchive.net/comp/linux/git_rebase.html

@rpetre thanks, this LKML thread is really interesting

@b0rk you've never seen it because you worked with good developers.

Early in my career, I've seen horrible things... People destroying entire repos, force-pushing (using graphical tools cause they didn't understand the command line) and leaving on holidays for 2 weeks, leaving the rest of the team to struggle for 2-3 days to understand what had happened and get everything back on track...

@pieq thanks! I can definitely see that people who don't understand git would make a lot more mistakes

@pieq @b0rk 🧵our stance at meta is

* always rebase on your private branch
* private branch stays on local machine or gets pushed to our code review tool, not to the shared repo
* one commit == one PR, but there’s ui in code review tool for making PRs that depend on other PRs
* you can pull down someone’s pending changes from code review tool. It does magic to make a copy of that branch on another users computer

@pieq @b0rk

* there’s no concept of pushing to someone else’s branch. One commit is one PR, remember? Just make your own PR stacked upon theirs
* when you land a PR, the CI does the magic to rebase it onto master and push. So nobody can directly push to master except the CI.

I love the stacked workflow. It frustrates me when I use systems that don’t support it.

@b0rk Is there going to be another thread to make the case *for* rebase? 😆​

I used to think rebasing was scary because I didn't understand it, but now I always rebase into my feature branches, and only merge when going back up into main.

@xenomachina maybe, though I'm a big rebase believer already so I don't have as many questions about it :)

@b0rk there are two reasons off the top of my head to rebase a shared branch. they're roughly analogous to why you'd do merges with a merge-y history, but for when you want a linear history:

- to get recent changes from main, without a merge commit (because merge commits are really hard to rebase, and you will rebase eventually, when "merging")
- just prior to "merging" you rebase your branch so the merge can be a fast-forward
- another reason is just to tidy up WIP commits in a long running branch, including fixups and typos in messages (all of which change the sha, which is what causes the difficulties)

hopefully it's clear why linear histories are good? (for me: bisecting, but i also like how tidy they are)

@lenary I think what I've learned from this discussion is mainly that I've never done development on a shared branch (the only shared branch I've really used is `main` and I would never rebase and force push main)

@b0rk had a setup where different environments are based on certain branches. Merges seem to be easier to make sure changes are in multiple environments. And the rebases would cause "duplicated" commits. So, we'd just rebase/squash within our own branches, but not after it merges. Don't know if there's a better pattern if you do the environment branch setup.

@ciel that makes a lot of sense thanks

@b0rk On further reading it’s possible I use rebase _all the time_ - I love rewriting my messy work-in-progress history to something that tells a clean story for review - but I do that through a GUI because I need it to literally see what I’m doing. Maybe that’s rebase under the hood? Very different use case from alternative-to-post-PR-merge, though.

@thatandromeda ooh what GUI do you use for it?

@b0rk gitup! It is the actual best.

@thatandromeda ooh yes i’ve been looking at it and i’ve been SO impressed (even though I don’t use GUIs for git), love hearing about how people are using it

@b0rk The usual problem with rebasing that I run into is that if a series of commits touches the same area of code, and then I decide to rebase them onto main which introduces a conflict, I will have to resolve the same conflict separately for every commit. Whereas if I merged main instead, I could have resolved it only once.

@mislav @b0rk That's solved by `git rerere` https://git-scm.com/book/en/v2/Git-Tools-Rerere unless I am misunderstanding the problem stated.

@manu_j do you find rerere easy to use? I've never tried it

@b0rk Yeah. You don't need to do anything other than set it once. Then everything happens automatically.

@manu_j thanks!

@b0rk Disclaimer: I tend to avoid merge commits for all the usual reasons and do a lot of rebasing.
I think the main problem is really poor tooling more than anything else. Part of it is that ‘rebase’ is really one operation for achieving a bunch of different subtly different goals, e.g.:
1. A mass cherry-pick for advancing a feature branch past upstream progress.
2. Attempting to reconcile different variants of a feature branch.
3. Actively rewriting history to tidy up commits, etc

@b0rk The “cost” is (a) training the team to use it effectively despite the horrendous UX (b) dealing with the fallout when it nevertheless goes wrong. (I should mention that merge isn’t much better in this regard; our intern got 2 feature branches into a horrible knot of merge commits some months ago.)

The UX for when things go wrong is far from ideal, and to some extent could be improved by behaving differently depending on what you’re using the rebase for.

@pmdj thanks, I've been thinking along the same lines ("rebase actually does 3 totally different things”) and this framing is really good

@b0rk would a good experience going the opposite way (TCR not rebased) be helpful to you to discover not so obvious costs?

@xdetant what’s TCR?