Necoro’s Blog

As I’ve written yesterday I created an eix cache parser. But … it was overengineered: It took like ages and was way slower than the normal portage way:


$ python -m timeit "import test; test.eix()"
10 loops, best of 3: 3.18 sec per loop
$ python -m timeit "import test; test.portage()"
10 loops, best of 3: 280 msec per loop


Thus I had to improve it: The first thing was to re-implement it using Cython. And step after step I ripped out features (in the original approach, the parser was lazy, i.e. only reading data if he needed to do so) and trimmed it down to what I really needed (opposed to being complete). Oh – and some C-level improvements :) (like using | instead of +).

Well: This is where I ended:

$ python -m timeit "import test; test.eix()"
10 loops, best of 3: 56 msec per loop


Now it is usable :)

Well — and while testing the new eix parser, I noticed, that Portato segfaults sometimes during shutdown. I thought the shm module the culprit and thus replaced the external dependency by a self-written module (again in Cython). And this also now uses message queues instead of shared memory — which is more intuitive and easier (no stupid locks and stuff). Seems to work up to now :)

Result: In the future, portato ships with C-Modules :) (hence the title