sábado, 28 de marzo de 2009

One DVCM to rule them all

Hi!

Given the recent news that GNOME development will be moving to git soon, I decided to take a look at three DVCMs to see how they matched each other. Not that I made exhaustive tests. I just thought of a series of operations to perform on a small and a big project to see how they matched against each other.

I compared git, mercurial and bazaar (the one I use).

The small project I used was a project of mine (some thousand lines of code.... not too big). I used a sequence of ten revisions taken from the project and stuffed them in the three VCMs.

The big project was linux (the kernel). Given the time that some operations took (plus the room that it took on my already mostly filled up box) I only tried with revisions 2.6.27 (327 MB) and 2.8.28.8 (348 MB).

I measured performance both in time taken for the operations and room taken by the repository. I'm using my dated box (4 years old?) on jaunty using the repository packages. Verbigracia: git 1.6.0.4, hg 1.1.2 and bzr 1.13

Here are the results:

Time Performance
Small Project
On the small project the absolute winner was git. Second was mercurial and third bazaar. Git made most of the operations 3 to 10 times faster than mercurial and the later did them mostly three times faster than bazaar. Of course, as it's a smal project, what git could do in the blink of an eye, bazaar could do it in a longer blink of an eye. The slower operation on all three VCMs was a revert after having deleted the whole project. It took bazaar 1,72 seconds on the biggest revision (the 10th), mercurial made it in 0,56 and git took 0,13 (on the 8th revision... I was already bored with seeing git kicking a55es).

Big project
Here I expected git to mop the floor with the other VCMs. Given Linus' dislike for slowness (at least on VCMs subjects) and that at least bazaar recommends not to use it for big projects, I expected to see git go faster than Ussain Bolt yet here I saw mixed results amazingly having bazaar (the slowest of them all on a small project) be the fastest sometimes (not that I did things people would normally do on a project.... I don't think you will work with a 20 MB patch between one revision an another, anyway).

On the first add (a 300+MB add) results were veeeeery strange. Here git dragged itself to let the others pass over it. Mercurial took ~6 seconds, Bazaar made it in ~41 seconds, but git.... well, it made it in ~106 seconds. That shattered my expectations... and that was for starters.

Status after an add: On the first revision, bazaar and git were very close around 3-4 seconds... bazaar made it at over 10 seconds.... but on the add of the second revision we had very different results: ~3 seconds for mercurial, ~14 seconds for bazaar and git made it at ~191 seconds.

For the first commit, git won easily. It took git ~55 seconds, Mercurial did it in ~131 seconds and bazaar arrived when the party was already over at ~203 seconds (mercurial was already drunk and git was on the way to the hospital)... however on the second commit (remember, it's over a 20 MBs difference), they were all much closer. Mercurial arrived last with ~168 seconds, git second with ~144 seconds and bazaar (oh, my!) first with ~130 seconds. Now I didn't expect that.

After the commit, (again) I expected git to fly when asked for a status. But it didn't deliver. Both times mercurial was first, bazaar was second with about double the time and git was third with about 5 times the time for mercurial. By the way, the status after the first commit was about 3 times slower than the second on all VCMs.

Now, when I removed all the content of the project (rm * -dfR) and asked for a status, we went back to normal: git first, mercurial second bazaar third. For both revisions, bazaar stayed around 8-9 seconds. Mercurial made it in ~8 seconds for the first and ~4 seconds for the second. Git made it in ~2 for the first and ~4 for the second.

Then I tried to revert (in bazaar and mercurial, reset --hard HEAD for git) after having removed everything. Here the result were strange again. Bazaar made it first with 213 and ~212. Mercurial and git exchanged places between the two revisions. First revision, mercurial made it in ~356 and git made it in ~435. Second revision, git made it in ~455 and mercurial in ~459.

Then I tried to go to 2.6.27. Mercurial took ~88 seconds and bazaar was waaaaaaaaaay behind at ~507. When i revert back to the last revision, mercurial made it in ~186 and bazaar made it in ~393. When I tried it in git (with a checkout) when I reverted to the first revision, I destroyed (I think) the second revision, so I decided to not include the time it took to revert/revert.

So, I thought that git would be the clear winner and the fact is that I got mixed results. Perhaps people can join in and give me some insights about why it was like that.

Repository Size Performance
Here there were no mixed results. Both for a small project and a big project, bazaar was the clear winner. Mercurial made it second and git arrived last.

Bazaar's repository size for the small project was around 10-40% smaller than mercurial's. Also bazaar was around a third the size of git's.

On the big project, here are the sizes:
Bazaar: 87152k for the first revision and 99456k for the second.
Mercurial: 148844k and 166122k
Git: 158912k and 228544k

Bottomline
Well.... I would have loved to give you a clear winner, but the fact is that there wasn't. For a small project you already know who the best is in terms of user time and repository size... but for a big project, it's a little more blury. I urge you to jump into the comments area to give me your thoughts on it.

As usual, keep in mind that other than performance, there are other differences between all of them in terms of features. It's not just about performance to decide what DVCM to use for a project.

Disclaimer
I could have made a mistake when working with mercurial or git, as I didn't know how to use them before writing this article (and I still don't). So if you think I could try to do something a little different, then go ahead, mail me or post a comment. Perhaps we could create a test suite or something to compare their performance.

miércoles, 25 de marzo de 2009

Microsoft uses git for version control

Not only has GNOME decided to switch to using git. We have learned that development at Microsoft is based on git too.

Development for Windows 7 is handled on a modified (in-house) version of git that allows it to run natively on Windows (the blessings of the GPL, I believe they must think at Redmond). Why reinvent the wheel when one of the best tools for the job is out there for free?

According to our sources, at first, some developers introduced git behind the backs of their managers by using it on top of cygwin. As they felt more comfortable with the tool and presented their benchmarks to management, it was obvious that git allowed development to be carried out in a much more efficient fashion. Then they started the process of making git run natively on windows without needing an intermediate layer, which allowed it to run even faster. It has been reported that the patches to git will be sent upstream once Windows 7 reaches RTM.

This may sound as shocking news for some people, but over the last couple of years Microsoft has been trying to get cozier with open source... and this is yet another example of it. Unfortunately Microsoft's Horacio Gutierrez wasn't available for comments, nor Lawrence Crumpton.

Having said that, did you notice that it's April 1st? It's the only way something that far fetched could ever be published.

My opinion: I hope they are using RCS... or even better: no source control at all. :-)

"Get the facts" on browsers

When I was heading to download IE8 to test its JS performance, I saw this link about "Getting the facts" on web browsers. Having the knowledge of what "Get the facts" meant back in time when that was devoted to trying to put GNU/Linux in bad light against Windows-et-all, I just couldn't help myself and went in there to see what was showing up.

They have this video where they show how IE8 renders "index" pages of popular sites. Apparently IE8 kicks ass on that (as a matter of fact the differences between rendering times of the browsers they compare is very little, sometimes the blink of an eye). And so they have the guts to affirm that IE8 is one of the fastest browsers on earth. Is that really? Is rendering times for pages the real measure for browser speed? Come on! As applications are more and more "functional" and so becoming more Javascript intensive (yes, not VBScript intensive, but Javascript... thanks heaven) then I think Javascript performance is just as important.. or even more so that rendering times (specially if the difference in terms of rendering is so close between the browsers they showcase).

And how does IE8 do on that front (on my box)?
V8: 34.4
SunSpider: 9164.4

And, as you can see here, that's a looooooooong shot from being fast. So Microsoft, why don't you try to get the facts right at least once?

PS Were they saving that video for April Fool's day? I guess I just blew it.

jueves, 19 de marzo de 2009

IBM(+Sun) Vs Microsoft

Well well.... "History repeats itself" is something we hear sometimes.

So IBM appears to be headed to buy Sun Microsystems. That's quite a move.... I have seen comments from people noticing how Solaris/OpenSolaris will affect/be affected by the merge... but I haven't seen people taking these other three things in consideration:

- Java
- MySQL
- OpenOffice

Lotus Symphony is based on OpenOffice so I guess it will be easily assimilated inside IBM. How will that affect OpenOffice development? It's being criticized before for being too Sun-centric. How will this affect the MS Office / OpenOffice saga?

I don't know how MySQL will play out inside IBM. Now IBM would be in charge of DB2, Informix (bought in 2001) and MySQL. Will that be enough to wipe SQL Server out of this world?

But Java! Java vs .Net is old news. And Java's development (as OpenOffice) had being criticized for being too Sun-centric. But now Java being pushed forward by IBM (in house).... that will be interesting to see.

Are these things gonna take the IBM/Microsoft battle into another chapter?

Sidenote
I wonder what Darl McBride would have to say about this. :-)

sábado, 14 de marzo de 2009

Give a presentation? Use a LiveCD!

Hi!

Yesterday I had to give a presentation on the way (or say, my way) to develop a small (so far) web system. It's built on JSP and I'm working on Eclipse (no fancy thing... just yet). So I wanted to go there and show my workflow in terms of branches (plain application, application+design, etc) plus I would have liked to control my home computer remotely (through ssh remotely... if possible).

On top of eclipse (and java), I would have had to use bzr and qbzr... now... I wasn't going to be able to carry a computer with me, so I was going to get a box from the company I was doing the presentation for. Well.... what are my options? Expect to get a Güindous Computer ad start installing things on a computer that's not mine. That didn't sound cool. Another was to get a GNU/Linux box (perhaps) and start installing stuff (not cool either). But how about taking a LiveCD with me and carry along all things I'd need for the presentation (in case there's no internet access)? Let's give it a try.

I use a Kubuntu Intrepid LiveCD (but the procedure will wok like that on any debian-based distro). I have one standalone installer of the JRE from sun and Eclipse in a compressed format, so that side of the story is covered. But then bzr/qbzr? I'd have to get them from packages with apt... but how do I know what packages I will need? Get your notebook.

Home Procedure
Start the LiveCD, wait for it to finish starting. Then, add the stable repositories of bzr and qbzr in sources.list (that's because the normal repos of ubuntu intrepid have very dated packages.. at least of qbzr). So far, so good. Now, the normal procedure to install bzr/qbzr:

$ sudo apt-get update
$ sudo apt-get install bzr qbzr

By now, you have both packages installed in your LiveCD session.... but you want to be able to install them later on (like, at the company without an internet connection, right?). So, what can you do? Well.... let me tell you a little secret: If you didn't clean after you installed the packages, you can grab all those beautiful .deb packages (bzr, qbzr and their dependencies) from this location: /var/cache/apt/archives/

So, hook up a pen drive, mount it and copy those .deb packages on it:

cp /var/cache/apt/archives/*.deb /somewhere/

Now head for your presentation.

Company Procedure (or home test.... just in case)
You arrive at the company and are presented with the computer you will give your presentation on. So let's get our hands dirty right away. Start your LiveCD, hook up your pen drive and mount it. Then, install all those .deb packages:

sudo dpkg -i /somewhere/*.deb

And you are done. bzr and qbzr are installed. You had the jre and eclipse files in the pendrive too.. two branches of the project too, so you do the installation procedure and can start my presentation right away. Finish your presentation, turn of the box, give it back.... nothing happened. End of Story.

A little rant.... back in time...
I learned about this trick some time ago. About 3 years ago, I was going to a Cisco Accademy course (CCNA preparation) and I remember that when we reached the DHCP part, our teacher (by the way, and old friend of mine from my going-to-the-youth-orchestra days, Juan Jakymec) told us that one of our fellow students was going to have one of the boxes converted into a Güindous Server so that we could see how it worked. I volunteered to have GNU/Linux used as a DHCP server too.

Comes the day.. the other guy had showed up at the classroom who knows when in order to install Güindous Server on the computer. So, I'm told to go first, I use this trick that I described before to install the DHCP server, try to start it but it fails.... so I dig a little on the logs (or perhaps running it as a normal process to see its output... can't remember). I get the error cause of the IP addresses that the DHCP service is told to provide and the IP address of the network interface. So I change the configuration of dhcpd, start it.... we're ready to go. I even start tcpdump so that we can see what's going on between the server and the clients. All lovely. Less than 10 minutes after I was given the chance to do the show, I'm heading to my chair with a big ear-to-ear smile on my face.

Time for the Güindous Server to show off.... well... let's just say that it wasn't pretty. You know how these things work in güindous when something fails. You get a small window telling you that something failed... no details. So he starts going (guessing, basically) here and there to see what happened... tries and retries.... well... some minutes go by and then he is able to start the service... no tcpdump, though. Bottom line:

Güindous = esoterism, GNU/Linux = determinism.

Come on.... think about it... I was able to do the whole thing in 10 minutes.... I left the computer as it was before I had a finger on it.... the other guy? he had to install a whole operating system beforehand in order to see a dhcp service working. Does it even make sense at all?

jueves, 12 de marzo de 2009

Jaunty: Apt is broken? Move to another country!

Hi!

You might remember that I had switched to Jaunty to give it a test drive. Well, since yesterday I was having this problem with apt. It was throwing a segmentation fault whenever I tried to work with its repos (upgrade would seg fault, and dist-upgrade, and the same for install). In other words.... a broken apt = an un-upgradable system. Not good. That's not something I wanted to face at all. Having X not start once in a while is something... but having a broken package manager is a whole different story.

After submitting the bug to launchpad and spending a little time at #ubuntu+1 one guy there (Thanks, IntuitiveNipple) noticed that one of the files that apt handles appeared to be broken (/var/cache/apt/pkgcache.bin). I removed the file, but the file stayed there on the FS (go figure!). After some attempts at it, I was able to remove the file. But then I tried to update again... and BOOM! SEGFAULT. It's not going to give itself away so easily.

So.. after about 24 hours I'm stuck with no clue of what to do. I even had the crazy thought of reinstalling kubuntu from scratch (hell, I hated that thought.... made me think of my long forgotten days of formatting every 6 months to reinstall güindous). But I did a simple test instead: Switched my repos to use the venezuelan repositories instead of the colombian ones:

$ sudo su
# cd /etc/apt/
# sed 's/co\./ve\./' sources.list > sources.list.ve
# mv sources.list sources.list.co
# mv sources.list.ve sources.list
# exit
$

And voilà, apt is back on track. So the problem was the repos for colombia after all. Hope they solve it soon.... and why didn't apt complain about a discrepancy or something instead of exploding? Weird.

viernes, 6 de marzo de 2009

Oops! Removed all kernels!

Hi!

My last post on this blog was about how to switch from intrepid to jaunty without having much room on hard drive. One accident I had while doing this was that I removed all my linux kernels and so I ended up with a box that could do mem tests all day long or boot into windows (god forbid). So, here's how I got it back. I did it on kubuntu jaunty, but it could work almost the same if you tried on debian or any other distribution that has a package manager (or even without it, though the steps will change, of course).

First thing, I booted the box with a Kubuntu LiveCD. After having booted, I mounted the root partition of my busted GNU/Linux:

$ sudo su
# mkdir /mnt/tmp
# mount /dev/sda5 /mnt/tmp

After having mounted, let's get into a chroot cage so I can manage packages from that mounted partition as if it were root:

# chroot /mnt/tmp

Well... it's basically like being on a terminal on the GNU/Linux that has no kernel. I'm at the root of that box, I have a working internet link..... so, let's just install the missing kernel package:

# apt-get update

# apt-get install linux-image-generic

We let it do its stuff and reboot and tadaaaaaaaaaaaaaaaa, we're back on track. See? It wasn't that difficult, was it?

PS If you have separate boot partitions or perhaps a separate /usr, perhaps you'll have to do more mounting before you do the chroot, but the concept remains the same.

PS2 One reader left a comment where he said that he hated the terminal and wanted a GUI based application to get it done. I was going to post a picture of such an application: konsole. :-D I don't know if such application really exists. I think the situation I described is 1 fairly uncommon, 2 rather easy to fix (if you know where to look). But if you feel really lost, then email me. I'm always available to help. As a matter of fact, back in december I got a box of 9 bottles of italian wine for helping an italian guy recover some data from a broken RAID5 (I wish all people thanked that way... thanks, Marco!).

miércoles, 4 de marzo de 2009

From intrepid to jaunty.... with no room left

Hi!

Today I decided to move my desktop to jaunty. It's still an alpha but I'm not afraid of having to face a non-working Xorg once in a while and having to use the VTs instead. Given the speed improvements I've read about... the time is now... there's just a tiny little problem. I have no room left to do the dist-upgrade:

$ df
S.ficheros Bloques de 1K Usado Dispon Uso% Montado en
/dev/sda5 5162796 4783244 117296 98% /
tmpfs 508248 0 508248 0% /lib/init/rw
varrun 508248 228 508020 1% /var/run
varlock 508248 4 508244 1% /var/lock
udev 508248 2824 505424 1% /dev
tmpfs 508248 12 508236 1% /dev/shm
lrm 508248 2004 506244 1% /lib/modules/2.6.27-11-generic/volatile
/dev/sda8 6190664 5793332 82864 99% /home

Anyway... let's just try, ok? This is the second time i have to do this process. Why? Cause it's my wife's computer and I have a little space set up for GNU/Linux on it (and if people don't start using www.cdfleamarket.com, I'll have to repeat the process for Kinky Kangaroo... I mean, Karmic Koala as well.... so why don't you give me a hand and use the site? It's free anyway!).

First, let's set up apt to use the jaunty repositories instead:
$ sudo su
# cd /etc/apt
# sed 's/intrepid/jaunty/' sources.list > sources.list.jaunty
# rm sources.list
# ln -s sources.list.jaunty sources.list
# exit

I like to keep the different versions of sources.list so I have a version of each ubuntu release sin edgy.

Now, let's try to dist-upgrade to see what happens:

$ sudo apt-get update
$ sudo apt-get dist-upgrade
1010 upgraded, 98 newly installed, 7 to remove and 1 not upgraded.
Need to get 1073MB of archives.
After this operation, 639MB of additional disk space will be used.
E: You don't have enough free space in /var/cache/apt/archives/

Well.... that's a long no.

Ok... here's the deal: As long as you keep the package ubuntu-standard installed, you can remove basically everything else and the system will keep on working. So let's remove all this desktop packages (a lot of them at least) so that we have a system basic enough that we could dist-upgrade.

You have to know how to use a VT before you proceed, cause you won't be able to use the graphical environment for a while.

If you are ready, just close your graphical session and head to one of the VTs.

I use kubuntu, so the first thing is to remove the kubuntu-desktop package so that I get "permission" to automagically remove all the other packages that depend on it (if you use Ubuntu instead, then it's ubuntu-desktop):

$ sudo apt-get remove --purge kubuntu-desktop
roughly 50 Ks freed. :-S
$ sudo apt-get autoremove --purge
Nothing :'(

Oh well.... I didn't expect that. Seems like kubuntu-desktop has no dependencies... that could be because I use the repositories for KDE 4.2. So let's try with something a little more straight-forward:

$ sudo apt-get remove --purge kde'*' openoffice'*' xfce'*' firefox'*'
168 packages to be removed
771 MBs to be freed
3 Mbs of data to install right away

Much better now.

$ sudo apt-get autoremove --purge
another 153 packages to eliminate
235 Mbs to be freed

Now I have about 1Gb of space, but apt-get update still needs to download
718 MBs of data plus take another 530 MBs. I have to keep on cutting stuff out.

Let's go with some other things that take a lot of room till I get enough free space:
$sudo apt-get autoremove texlive'*' lilypond'*' --purge
I didn't know I could ask autoremove directly to remove packages and their unnecessary dependencies. You never know when knowledge will hit at your screen.
200+ MBs freed
Now the dist-upgrade would barely fit in there, but let's make a little more room just in case:

$ sudo apt-get autoremove xorg --purge
About 1.4 GBs are free by now.

There are still a lot of packages that are not basic still installed... but I won't care about them now.

Let's try to dist-upgrade to see how it goes now.
sudo apt-get dist-upgrade

Now it's a 500 MBs download plus 357 Mbs to take for installation. I'll let it do it like that, then I'll clean and then I will install kubuntu-desktop so I get the environment back.

I'll use lynx to read the latest news on another VT while the download takes place to see what I'll do next.

After a text round of www.linuxtoday.com, www.osnews.com, www.eltiempo.com, www.noticias24.com, www.talcualdigital.com, prelude of the Suite Antillaise of Francis Kleynjans (that'd be me playing guitar), Aire de Zamba and a little more guitar fiddling, it's downloaded. Now I have to see if the upgrading process asks me something (some more guitar fiddling).

dist-upgrade is done. Let's run it one more time just in case. Ran it. Nothing is left to do. So let's reboot so I can uninstall this kernel I'm running from intrepid and continue the upgrading process.

sudo reboot

Now... booting was FAST! At least to the console login. Let's see if this wonder can keep that way after I install the desktop ;-).

Let's remove the old kernel then:

ls /boot/vmlinuz*
There I see that I have a 2.6.27-x kernel.

Let's remove the 2.6.27-whatever:

sudo apt-get autoremove --purge linux-image-2.6.27'*'

Some more packages went away along with it. A total of over 280 Mbs are freed.

Sidenote: I just discovered (next morning) that the new kernel went away with it, so perhaps it's a good idea to remove the linux-image-2.6.27-x-generic instead (providing the exact number in x instead... so watch out!). Now... how do I install a new kernel on a non booting system? I'll be back!

Now, let's remove all the .deb files I've downloaded so far that are already installed so I free up that space as well (that won't uninstall the packages... just remove the .deb files that were used to install those packages). Before I clean, I have about 800 Mbs free.

sudo apt-get clean

Now, I have about 1.3 GBs free.

Next step? Let's bring back the desktop, shall we?

sudo apt-get install kubuntu-desktop

257 Mbs to be downloaded, 825 MBs to be used on top of that. So be it.

Here, I just hit my first problem. The network is down. Why? I just don't know right now. ifconfig -a shows me eth1 instead of the usual eth0, tried reconfiguring the interface using /etc/network/interfaces but still won't come up. So I take it to the next lower level:

sudo dhclient eth1

I get the configuration from my ISP. So let's go on installing so I can focus on this problem later (and I won't publish the solution in this post.. perhaps in another post tomorrow or so).

Again:
sudo apt-get install kubuntu-desktop

It does work now. Install finishes and now let's go back to graphics mode.

sudo /etc/init.d/kdm restart

And we're back in track. Now I have to install the packages that I use that were uninstalled in the process... but I'll do it as I need them.

Now, see if you can do that from Vista to 7. :-)

PS: I was able to reinstall the new kernel after having removed it. The keywords to fix it are: LiveCD + chroot.

lunes, 2 de marzo de 2009

Bill Gates suffers from digital phobia

Yep... the same guy who is credited with creating the "desktop computer" (or so they say) must be suffering from phobia to anything electronic.

How can we tell? Well... I inferred that after knowing that Bill Gates has banned iPods and iPhones at home... in the same page where we find that Steve Ballmer banned using google at home. We already know that Bill Gates hates GPL. But then, if everything that is against Microsoft || Windows won't be used by these two (and their families), I can't help but wonder... will they really try to escape using GNU/Linux? Come on... it's like escaping big brother. It's everywhere... even if you can't see it. Does Steve Ballmer use (did Bill Gates use) Microsoft's Wireless? If he does, shame on him. Do they ban sites powered by anything but IIS? If that's the case, then poor guys.... they can only use about 30% of the web.

So.... are Bill Gates and Steve Ballmer's lives going to be increasingly restricted in terms of access to digital resources as time goes by and GNU/Linux makes more and more inroads into our daily lives in unnoticeable ways? I wish they were. It'll be cool to have them both (specially Steve) recluded in one of those monasteries that time has basically forgotten about.