Necoro’s Blog

Blog? Blog? There is no blog…

Posts Tagged ‘Cython

Portato’s new C-Modules

leave a comment »

As I’ve written yesterday I created an eix cache parser. But … it was overengineered: It took like ages and was way slower than the normal portage way:


$ python -m timeit "import test; test.eix()"
10 loops, best of 3: 3.18 sec per loop
$ python -m timeit "import test; test.portage()"
10 loops, best of 3: 280 msec per loop

Thus I had to improve it: The first thing was to re-implement it using Cython. And step after step I ripped out features (in the original approach, the parser was lazy, i.e. only reading data if he needed to do so) and trimmed it down to what I really needed (opposed to being complete). Oh – and some C-level improvements :) (like using | instead of +).

Well: This is where I ended:

$ python -m timeit "import test; test.eix()"
10 loops, best of 3: 56 msec per loop

Now it is usable :)

Well — and while testing the new eix parser, I noticed, that Portato segfaults sometimes during shutdown. I thought the shm module the culprit and thus replaced the external dependency by a self-written module (again in Cython). And this also now uses message queues instead of shared memory — which is more intuitive and easier (no stupid locks and stuff). Seems to work up to now :)

Result: In the future, portato ships with C-Modules :) (hence the title

Written by Necoro

August 15, 2009 at 13:38

Posted in Portato, Python

Tagged with , , , , ,