How Tridge reverse engineered BitKeeper [LWN.net]

Andrew Tridgell delivered the first linux.conf.au keynote on Thursday morning. The bulk of the talk covered software engineering techniques and how the free software community is taking a leading role in adopting those techniques. It was a good talk, and your editor will attempt to write it up later on.

At the end, however, Tridge touched on his role in the separation of the kernel project and BitKeeper. He couldn't talk about much, and he did not announce the release of his BitKeeper client. But he noted that there has been quite a bit of confusion and misinformation regarding what he actually did. It was not, he says, an act of wizardly reverse engineering. Getting a handle on the BitKeeper network protocol turned out to be rather easier than that.

He started by noting that a BitKeeper repository has an identifier like bk://thunk.org:5000/. So, he asked, what happens if you connect to the BitKeeper server port using telnet? A quick demonstration sufficed:

    telnet thunk.org 5000
    Trying 69.25.196.29...
    Connected to thunk.org.
    Escape character is '^]'.

Once connected, why not type a command at it?

    help
    ? - print this help
    abort - abort resolve
    check - check repository
    clone - clone the current repository
    help - print this help
    httpget - http get command
    [...]

Tridge noted that this sort of output made the "reverse engineering" process rather easier. What, he wondered, was the help command there for? Did the BitKeeper client occasionally get confused and have to ask for guidance?

Anyway, given that output, Tridge concluded that perhaps the clone command could be utilized to obtain a clone of a repository. Sure enough, it returned a large volume of output. Even better, that output was a simple series of SCCS files. At that point, the "reverse engineering" task is essentially complete. There was not a whole lot to it.

Now we know about the work which brought about an end to the BitKeeper era.

(Log in to post comments)

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 13:29 UTC (Thu) by nhasan (guest, #1699) [Link]

I think Larry should hide behind the good old DMCA. Just add rudimentary encryption, ROT13 would do, and his job is done.

Seriously though, why is Tridge reverse engineering the CIFS wire protocol OK and not Bitkeeper?

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 13:50 UTC (Thu) by hppnq (guest, #14462) [Link]

Of course not. Telnet to port 80 at your favourite site and GET /whatever_url_seems_right and then draw the conclusion that, because you get HTML, the webserver simply spits out (static) HTML.

That would be a stupid conclusion for most sites/webpages.

So, what interests me, is why would Tridge want to share this with us?!

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 14:19 UTC (Thu) by vmole (guest, #111) [Link]

So, what interests me, is why would Tridge want to share this with us?

Perhaps because he's tired of being accused of doing something wrong? In particular, "How could Tridge possibly investigate the Bitkeeper protocol w/o violating the BK license?" Well, here's how you do it.

How Tridge reverse engineered BitKeeper

Posted Apr 22, 2005 1:34 UTC (Fri) by akumria (subscriber, #7773) [Link]

Andrew, during his talk, said (paraphrased) "People keep believeing I'm a reverse engineering wizard. I'm not. Let me show you the process for BitKeeper"

All the commands Tridge subsequently ran, were shouted out by the audience. The talk was recorded but I am not sure if/where it is available though.

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 14:52 UTC (Thu) by jamesh (guest, #1159) [Link]

It depends on what you are trying to clone. It would help you interoperate with BK repositories, but I'd guess there is still a fair bit of smarts in the client (merge algorithms, etc).

So it is probably enough to work out how to mirror a bitkeeper repo for use with some other SCM tool, but not enough to make a clone of the bitkeeper tool.

How Tridge reverse engineered BitKeeper

Posted Apr 22, 2005 1:42 UTC (Fri) by akumria (subscriber, #7773) [Link]

Might be. I'm not sure. Why not see for yourself?

Tridge released his code this morning.

http://sourceforge.net/projects/sourcepuller/

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 20:31 UTC (Thu) by vonbrand (guest, #4458) [Link]

This report just doesn't match the claims that Linus tried to disuade Tridge from working on bk. If this is truly all Tridge did, there simply was no time to try to convince anybody in between.

In any case, Larry McVoy specifically asked for no reverse engineering. If he was right or wrong, if it was or not legal to ask for it, etc. just doesn't matter to me. If somebody wants her wishes (as set forth via GPL, BSD, or whatever) to be followed, she should do the courtesy to reciprocate.

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 22:28 UTC (Thu) by dvdeug (subscriber, #10998) [Link]

There's reasons why BSD and GPL are legal licenses, and not wishes. I want to be independently wealthy; Larry wanted no one to look at his program. I don't see why people should jump to fulfill either wish.

How Tridge reverse engineered BitKeeper

Posted Apr 28, 2005 6:29 UTC (Thu) by bignose (subscriber, #40) [Link]

> Larry McVoy specifically asked for no reverse engineering.

As do Microsoft.

> If somebody wants her wishes (as set forth via GPL, BSD, or whatever) to be followed, she should do the courtesy to reciprocate.

In both cases (the SMB protocols, the Bitkeeper protocol), Tridge did not use programs from the vendor (Microsoft, Bitmover) to connect to their services. He used programs under terms that he presumably *does* agree with.

Users of Samba should and must follow the wishes of Tridge (and its other authors), as set forth in the GPL. That has no hold, moral or legal, on anyone who simply connects their own client program to a Samba service.

How Tridge reverse engineered BitKeeper

Posted Apr 21, 2005 7:35 UTC (Thu) by Duncan (guest, #6647) [Link]

Why was the "help" command there, indeed. Could it be from an earlier era
when Larry wasn't so paranoid? Perhaps it too is to be removed from
upcoming versions (especially now, eh?), now that the unfreedomware but
still zerocostware version is being removed.

As others have observed, the continuously more enslaved era of BK did
allow the kernel to advance faster, over a shorter period, than it would
have otherwise, particularly with Linus headed for burnout. However, as
they say, all good things come to an end, and regardless of how it ended,
the "good" of this relationship was already ending, with an ever more
draconian license to the ever more slaveware that BK was becoming.
Whatever his reasons, I'm glad Tridge's actions in combination with
Larry's reactions forced the issue now. There's never a good time for
such a forced change, and now, with 2.6 fairly stable and development set
to continue on at a brisk but steady pace, this little detour is less
disruptive than it would have been since mid 2.5, and likely less
disruptive now than it would have been had the current development model
been allowed to continue to get more dependent on BK than it already was.

While it may not have looked like it in the first few hours/days, and
despite Linus' ravings as someone obviously too close to the action to
have a sane perspective (much as it hurts me to say this due to his famed
ability to step back and view the action from a neutral perspective in
other cases), I expect a year from now Linux will be the stronger for it.
BK may be as well, but as it's a proprietary product, that's nothing I'm
concerned with. It can go its own way, and I believe we should let it do
just that, breaking the increasingly unhealthy if symbiotic dependence we
had on it.

Still, very interesting to see just what sort of "reverse engineering"
Larry was attacking, here. Interesting indeed!

Duncan

Simultaneous reverse engineering by the hundreds

Posted Apr 21, 2005 14:07 UTC (Thu) by hppnq (guest, #14462) [Link]

Am I missing something or are you all thinking that Larry McVoy thought that using port 5000 and the extremely cunning bk:// protocol would be enough legal and practical protection against reverse engineering BitKeeper?! And that Tridge argued with OSDL and Larry about *this* kind of reverse engineering?

Come on. ;-)

Simultaneous reverse engineering by the hundreds

Posted Apr 21, 2005 18:06 UTC (Thu) by flewellyn (subscriber, #5047) [Link]

The evidence would seem to point to it being the case, so yes.

Given the ease with which this feat was accomplished, can it even be
called "reverse engineering"?

Simultaneous reverse engineering by the hundreds

Posted Apr 22, 2005 11:31 UTC (Fri) by hppnq (guest, #14462) [Link]

The relation between BitKeeper and SCCS has been known for ages. Using telnet instead of another program to connect to a remote port is standard practice, nothing fancy about that.

So if Tridge only wanted to show what he has been accused of, BitMover must be crazy. If the reverse engineering accusation goes a little further than telnetting to the BitKeeper port, Tridge must be crazy.

Strange in any case. Or I am crazy. ;-)

Software engineering techniques

Posted Apr 21, 2005 15:35 UTC (Thu) by richardr (guest, #14799) [Link]

As would I.

Richard.

Software engineering techniques

Posted Apr 21, 2005 22:34 UTC (Thu) by corbet (editor, #1) [Link]

I have the notes, and I do plan to write up the session. There just wasn't time...had to do my talk right after Tridge did his, and by then LWN was already a little late...

Understood

Posted Apr 22, 2005 15:31 UTC (Fri) by jvotaw (subscriber, #3678) [Link]

If you have time, it would be great. I've heard plenty of buzzwords in the corporate world and am interested in what open source people have to say.

Thanks, as always, for all of your hard work,

-Joel