I consider myself mainly a Python developer these days, but coming from a history of C++ development, I’ve been really interested in seeing how Rust pans out. Could it be an ideal language to use when I need the performance that Python just can’t provide? Can I happily replace C or C++ (using boost) with Rust?
I set out to see the current state of binding Python to Rust.
Let’s begin with a fibonacci sequence calculator. It’s pretty basic in Python:
C has a lot of boiler plate where I could just use the cffi module. But
using the actual API is worth it in case you want to do any sort of error
handling in the C side. For example, if you pass a wrong type through cffi,
you basically just segfault. You can wrap your C code with error handling in
Python. But if the intent is to make a function you call in a loop, you want the
error checking to also be fast.
If I’m honest, I always feel like I’m putting in a lot of bugs when I use the C API. Especially when I compare it with the Cython output.
Here’s what the equivalent looks like in Boost.Python. Boost.Python is a really nice library for wrapping C++ to call from Python. Except if you manage to make a mistake you get all the associated C++ compilation errors some of us love to hate. It’s allegedly gotten better, but in making this example, I still found some error explosions which take some searching around on the internet to figure out.
Sadly Boost.Python only seems to support Python 2. I know a lot of people using Python 2 who have no interest in migrating to Python 3, but if you end up wrapping a lot of your code in Boost.Python, you’re really sealing your fate. I don’t recommend it.
Cython is a fantastic tool for writing Python bindings. It’s a dialect of Python
that looks so much like Python that you can almost copy paste your code into a
pyx module, configure setuptools to find it, and you’re running at a much
faster speed. If you take a look at the Cython code, set the types on the
variables, turn off array bounds checking, and other tweaks, it competes with
hand written C. It’s really awesome.
The best library I’ve found for writing modules in Rust is
rust-cpython. Using it took some
trial and error since it’s so new and I wouldn’t call myself a Rustacean just
yet. But it was actually a very pleasant experience. With a little manipulation
of the Cargo.toml file (used to define a reproducable build in Rust), you can
build against Python 2 or 3. None of the other wrappers, aside from
Cython can do this.
Finally, here’s how the Fibonacci number generator looks in Rust:
I ran some basic tests so see how the performance fares, and it turns out that it’s pretty competetive. I wouldn’t put too much stock into the actual numbers aside from the fact that they’re within an order of magnitude of each other. I used GCC for C and Rust is built on LLVM.
The test is a really noddy one. It’s just using ‘timeit’ on the 91st fibonacci number from the IPython shell:
timeit fib(90)
| Python Version | Implementing language | timeit result |
|---|---|---|
| Python 3.4.2 | Python | 100000 loops, best of 3: 7.05 µs per loop |
| Python 3.4.2 | C | 1000000 loops, best of 3: 288 ns per loop |
| Python 3.4.2 | Rust | 1000000 loops, best of 3: 229 ns per loop |
| Python 3.4.2 | Cython | 10000000 loops, best of 3: 192 ns per loop |
| Python 2.7.10 | Python | 100000 loops, best of 3: 6.11 µs per loop |
| Python 2.7.10 | C++ | 1000000 loops, best of 3: 219 ns per loop |
| Python 2.7.10 | Rust | 1000000 loops, best of 3: 177 ns per loop |
| Python 2.7.10 | Cython | 10000000 loops, best of 3: 171 ns per loop |
As it’s a microbenchmark, the only thing I would say with any amount of confidence is that the Rust API wrapper probably isn’t getting in the way too much.
Some Rustaceans/Pythonistas are working on this infrastructure and it’s pretty
exciting. Other pieces of work being put into place are integration with
Python’s setuptools so you can
run python setup.py install and it will be able to build the Python modules
just like you would expect.
lib. This means you need to import
the module as import libXYZ which isn’t really what you want.I haven’t tested managing objects on the Rust side, but this is apparently a
place where Python can potentially get some performance improvements. For
example, collections.OrderedDict was implemented in raw Python until Python
3.5. Of course, the ability to use
RAII
on the Rust side should make writing containers in Rust a lot easier than in C.
Rust looks like a really great language to implement Python modules. It’s not there yet if you want to integrate with the numeric/stats/machine learning libraries that Python offers. But with some love and more community effort, I could see a community building in this direction. I think it’s a very promising start and I hope it continues.