Portato moving to git — Part I

So — as announced in my previous post, I started moving to git. You can find the current git repository at Github: http://github.com/Necoro/portato

Note: This move is not yet “official”. I have to test first, if I get completely used with git. And then I have to find a hosting platform (github was chosen, because I have an account there already) :). Or perhaps, I am going to host it myself … dunno.

Bzr vs Git

I had to deal with git yesterday, as I worked on ktsuss. And as always: I couldn’t make it to do what I wanted it to do ;). Thus I head to read the docs … and believe it or not: I’ve been always a contra-Git-guy, but when you get rid of the “it must be similar to all the other SCMs”-thought, git’s concepts begin to make sense. So I spent even more time with reading docs and playing around. And currently it seems, like I am going to replace bzr by git sometime in the future.

Reasons:

• way faster
• branches inside the one repository file — no more ‘gnah — the branch is still on my laptop’
• allows to finally get the currently split svn history back in (though using ugly hacks … but hey — git is all about ugliness :P)
• git is better supported by most tools — bzr plays only a small role (unfortunately)

I just need to find another bugtracker then, as I planned to use the launchpad one, which does not make sense anymore.

A last remark: I still think, that bzr is a GREAT scm :).

Probably move back to sf.net

Quite some time ago, I moved the main Portato hosting from Sourceforge to Origo. The reason was mainly, that I wanted some nice wiki-stuff w/o having to deal myself with hosting and configuring it. Also SF.net is … well … not really fast :).

But Origo has many other disadvantages, which I ignored so far:

• Does not support bzr — and does not allow to make the “Development” link point to somewhere useful.
• Uploading system sucks hard. Adds a random integer to each download URL, which makes it necessary to change the ebuild for each release.
• Flat namespace for attachments… Yes! … No comment -.-
• Bug tracker is ok for — but has real issues with attachments -.- … And it being built as somehow part of the wiki is not a good idea either.
• No possibility to put extra files somewhere (docs, …).

Additionally I noted, that sf.net now has some real nice features with its “Hosted Apps” — so it is possible for example to add a trac-instance — just with one click :D. So I’m now considering to move back to sf.net. I just do not know yet, whether I try to use trac and it’s bug tracker — or if I’m going to use the one in Launchpad. And where to host the bzr-branches? Also on sf.net — or leaving them on LP?

Comments are welcome :)

Portato 0.13 released

So — there it is: the new Portato release :)

Changelog is here.

Feel free to use it — it still waits for inclusion in the portage tree, but this will happen in the next future, I’m sure.

Portato’s new C-Modules

As I’ve written yesterday I created an eix cache parser. But … it was overengineered: It took like ages and was way slower than the normal portage way:


$ python -m timeit "import test; test.eix()"
10 loops, best of 3: 3.18 sec per loop
$ python -m timeit "import test; test.portage()"
10 loops, best of 3: 280 msec per loop


Thus I had to improve it: The first thing was to re-implement it using Cython. And step after step I ripped out features (in the original approach, the parser was lazy, i.e. only reading data if he needed to do so) and trimmed it down to what I really needed (opposed to being complete). Oh – and some C-level improvements :) (like using | instead of +).

Well: This is where I ended:

$ python -m timeit "import test; test.eix()"
10 loops, best of 3: 56 msec per loop


Now it is usable :)

Well — and while testing the new eix parser, I noticed, that Portato segfaults sometimes during shutdown. I thought the shm module the culprit and thus replaced the external dependency by a self-written module (again in Cython). And this also now uses message queues instead of shared memory — which is more intuitive and easier (no stupid locks and stuff). Seems to work up to now :)

Result: In the future, portato ships with C-Modules :) (hence the title

EixReader for Portato

Yay :) … I’ve finished the EixReader for Portato. It is now able to parse eix-files.

Now I only need to write a DB-Backend that makes use of it. And then some operations will be much faster — especially during startup, where the internal database is filled.

(Perhaps I am going to write a complete eix-Backend … thus not using portage for most operations… We’ll see)

Branch is here.

Portato 0.13 ahead

(Long time no post :P)

Well – Portato 0.13 is lurking at the horizon. After half a year it is well about time for a new release. Unfortunately this one does not bring lots of new features (except some enhancements in the plugin system), but more or less only bugfixes.

I have currently three branches where I work on to implement new features (supporting use dependencies, supporting the eix database, reducing the memory footprint) – but all of them are quite complex (especially the first one), and I do not know when they are going to hit trunk …

Anyway: It would be great, if you could test the current beta (it’s in the overlay) – and perhaps send the one or other bugreport.

Oh – and I’d like to thank all the translators — and encourage everyone to translate into their favored language too :). (Currently we have: Turkish, Catalan, Spanish, Portugese, German, Polish). So – if you are interested, have a look at the Translating Guide.

My own online feed reader

I use different computers to read my RSS feeds (at least one PC and two laptops …) – sometimes I want to read them also if I’m surfing with someone other’s computer. Thus a desktop reader like akregator or liferea, or a browser plugin is definitely not what I need. Additionally, I have different feeds (e.g. Spiegel Online) which only put 20 entries in their feed, but produce about 100 entries per day. Hence, if you use a desktop reader which is not connected 24 hours a day (none of my computers is), you’ll definitely miss entries.

Result: Something web based, running all day long and being accessible from everywhere. There are lots of sites offering this service: Bloglines and Google Reader being the best known ones – or Alesti, which I used to use. But Alesti often had technical issues and lacked several features (“searching” being the most important one). And I was not able to find a good replacement (I try to not use Google services at all). So for quite some time now, I looked for something, which I can put on my own webspace.

And finally — someone pointed me to Tiny Tiny RSS. Et voilà: I managed to install and configure it – and it runs :). I had several issues to make it update the feeds though: This is managed by a cron job, but – as I own a small old-school webspace only – I can’t start my own services or mess around with cron directly, but have to rely on what Confixx offers. But well … it works now, and I am happy.

So … the title of this post could have also been: Goodbye Alesti

Hope I can stick with my current solution :).

Firefox Directory in RAM

So I’ve read Nikos’ post about putting the mozilla stuff into RAM to speed up Firefox. But the script he posted is not really suited for Gentoo. So I thought to use it and pour it into a Gentoo-Init-Script. Using it results in quite a good speed-up – especially as FF used to block when another process makes heavy use of the HDD.

You can find the script and its config file in a git repository at github. Just branch (or download directly) and put the files in the correct directories.

Note: Works only with OpenRC and not with the old baselayout-1.

Note: If you computer does not shutdown properly, all changes you made during your browsing session are lost.

Note: Only use this script, if you have sufficient free RAM, as firefox profiles can be quite large (>100MB). One point to reduce the size is to delete the urlclassifier3.sqlite (which can be about 50MB) and is only used for the phishing protection (i.e. ask google for each site you are visiting :/). See for example this post about the details (or use $SEARCH_ENGINE).

A further idea would be to have an inotify-task, which looks for the lock file in the firefox dir. If it is removed (i.e. firefox is closed), it could to a sync.

/edit[2011-03-04]: Moved from bzr to git.

Pumping Lemma

Lemma Lemma Lemma …

(hi prof schwinn :))