Announcing Tino: a One-of-a-Kind, Stupid Fast, Python API built on Redis Protocol, MessagePack and Uvicorn
Last week I wrote about the problems with HTTP and how replacing it with RESP, the Redis Serialization Protocol, could eliminate some of those problems. After some more hacking, I am very proud to announce the end result of those efforts with Tino a very special and unique way to build APIs. Redis’ efficient protocol makes Tino over 10–20x faster in some cases compared with FastAPI, the fastest HTTP framework in Python.
Why not use ProtocolBuffers, Thrift, etc.
Both Redis and MsgPack are extremely efficient and widely available across languages. If a language doesn’t have it, its pretty simple to write either by hand. Other protocols are complex and don’t have wide support across languages or you have generate a bunch of code just to call one method.
How to Use It
Tino’s methodology follows closely to FastAPI. Type annotations are REQUIRED because the framework does a lot of work for you:
- Serialize arguments
- Automatic validation and structured error responses
- De-serialize the response
- Automatically build a client
To use, import the app and call the client function:
Because the protocol is Redis, you can also call the command from any Redis client that accepts custom commands in an environment that also has Msgpack available.
Pydantic Models
In the above example, NumberInput inherits from pydantic.BaseModel to automatically provide conversion from the raw dict that Msgpack parser produces to a rich python object. In addition, they also parse and validate the data structure giving back structured errors when there is a problem. You can read more about why FastAPI chose Pydantic here.
Uvicorn
The fastest HTTP benchmarks for frameworks are typically done with Uvicorn. While Uvicorn markets itself as an ASGI and HTTP server, its also secretly a powerful way to build load-balanced workers for TCP connections.
Here we take the connection handler we built in the last post, and pass it in to Uvicorn to create a server that talks Redis Protocol and can spread the work across cores of your machine.
Benchmarks
Because Redis Protocol is simpler than HTTP and MsgPack is simpler than JSON so it follows that Tino should be faster than FastAPI, but how much faster?
For this initial benchmark, I took a ~400 emoji unicode string (1234 bytes encoded in json) and implemented a simple echo client in both. Because Tino is both a server and a client, I thought it only fair to compare the performance of the whole round trip. Each benchmark except the last uses 6 worker processes.
For the Python client, I chose httpx because it is the most advanced async http client. However, I ran into problems running it concurrently. While Tino’s client easily scaled to hundreds of thousands of current tasks, httpx fell flat and could not handle concurrency very well, (like 100x worse performance). So for concurrency, I spun up 100 different python processes with 1 connection each. This allowed httpx to finish in a reasonable time. However that is a pretty substantial asterisks next to the results.
Because of httpx’s bad results, I also used ab to give FastAPI the best possible fight against Tino, Tino still was substantially faster. However, these results don’t reflect reality for HTTP since you still need to add the overhead of encoding the data structure before sending it.
As the structures get more complex, I would expect Tino to be better and better because of MsgPack’s superiorly efficient serialization format.
The Future
Better security is needed to prevent clients from maliciously sending large messages. Because of this, Tino should only be used behind a firewall.
Iterators would also be great to implement as a way to efficiently send down large streams of data.
The Future Far Far From Now
Because both RESP and MsgPack have binary formats that are very simple and easy to work with, it would be possible to generate a parser for both the protocol AND the MsgPack arguments. This would enable nearly perfect de-serialization and serialization of memory structures and produce the fastest API possible.
