Implementing gRPC In Python: A Step-by-step Guide

In the last few years, we saw a great shift in technology, where projects are moving towards “microservice architecture” vs the old “monolithic architecture”. This approach has done wonders for us.

As we say, “smaller things are much easier to handle”, so here we have microservices that can be handled conveniently. We need to interact among different microservices. I handled it using the HTTP API call, which seems great and it worked for me.

But is this the perfect way to do things?

The answer is a resounding, “no,” because we compromised both speed and efficiency here.

Then came in the picture, the gRPC framework, that has been a game-changer.

Quoting the official documentation” gRPC or Google Remote Procedure Call is a modern open-source high-performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication.”

Credit: gRPC

RPC or remote procedure calls are the messages that the server sends to the remote system to get the task(or subroutines) done.

Google’s RPC is designed to facilitate smooth and efficient communication between the services. It can be utilized in different ways, such as:

• Efficiently connecting polyglot services in microservices style architecture
• Connecting mobile devices, browser clients to backend services
• Generating efficient client libraries

- HTTP/2 based transport — It uses HTTP/2 protocol instead of HTTP 1.1. HTTP/2 protocol provides multiple benefits over the latter. One major benefit is multiple bidirectional streams that can be created and sent over TCP connections parallelly, making it swift.

- Auth, tracing, load balancing and health checking — gRPC provides all these features, making it a secure and reliable option to choose.

- Language independent communication- Two services may be written in different languages, say Python and Golang. gRPC ensures smooth communication between them.

- Use of Protocol Buffers — gRPC uses protocol buffers for defining the type of data (also called Interface Definition Language (IDL)) to be sent between the gRPC client and the gRPC server. It also uses it as the message interchange format.

Let’s dig a little more into what are Protocol Buffers.

Protocol Buffers

Protocol Buffers like XML, are an efficient and automated mechanism for serializing structured data. They provide a way to define the structure of data to be transmitted. Google says that protocol buffers are better than XML, as they are:

• simpler
• three to ten times smaller
• 20 to 100 times faster
• less ambiguous
• generates data access classes that make it easier to use them programmatically

Protobuf are defined in .proto files. It is easy to define them.

Types of gRPC implementation

1. Unary RPCs:- This is a simple gRPC which works like a normal function call. It sends a single request declared in the .proto file to the server and gets back a single response from the server.

2. Server streaming RPCs:- The client sends a message declared in the .proto file to the server and gets back a stream of message sequence to read. The client reads from that stream of messages until there are no messages.

3. Client streaming RPCs:- The client writes a message sequence using a write stream and sends the same to the server. After all the messages are sent to the server, the client waits for the server to read all the messages and return a response.

4. Bidirectional streaming RPCs:- Both gRPC client and the gRPC server use a read-write stream to send a message sequence. Both operate independently, so gRPC clients and gRPC servers can write and read in any order they like, i.e. the server can read a message then write a message alternatively, wait to receive all messages then write its responses, or perform reads and writes in any other combination.

**gRPC guarantees the ordering of messages within an individual RPC call. In the case of Bidirectional streaming, the order of messages is preserved in each stream.

Implementing gRPC in Python

Currently, gRPC provides support for many languages like Golang, C++, Java, etc. I will be focussing on its implementation using Python.

This will install all the required dependencies to implement gRPC.

Unary gRPC

For implementing gRPC services, we need to define three files:-

• Proto file — Proto file comprises the declaration of the service that is used to generate stubs (<package_name>_pb2.py and <package_name>_pb2_grpc.py). These are used by the gRPC client and the gRPC server.
• gRPC client — The client makes a gRPC call to the server to get the response as per the proto file.‍
• gRPC Server — The server is responsible for serving requests to the client.

In the above code, we have declared a service named Unary. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a MessageResponse. Below the service declaration, I have declared Message and Message Response.

Once we are done with the creation of the .proto file, we need to generate the stubs. For that, we will execute the below command:-

Two files are generated named unary_pb2.py and unary_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and the client.

Implementing the Server

In the gRPC server file, there is a GetServerResponse() method which takes `Message` from the client and returns a `MessageResponse` as defined in the proto file.

server() function is called from the main function, and makes sure that the server is listening to all the time. We will run the unary_server to start the server

Implementing the Client

In the __init__func. we have initialized the stub using ` self.stub = pb2_grpc.UnaryStub(self.channel) ‘ And we have a get_url function which calls to server using the above-initialized stub

This completes the implementation of Unary gRPC service.

message: “Hello Server you there?” message: “Hello I am up and running. Received ‘Hello Server you there?’ message from you” received: true

Run -> python3 unary_client.py

Bidirectional Implementation

In the above code, we have declared a service named Bidirectional. It consists of a collection of services. For now, I have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a Message. Below the service declaration, I have declared Message.

Once we are done with the creation of the .proto file, we need to generate the stubs. To generate the stub, we need the execute the below command:-

Two files are generated named bidirectional_pb2.py and bidirectional_pb2_grpc.py. Using these two stub files, we will implement the gRPC server and client.

Implementing the Server

In the gRPC server file, there is a GetServerResponse() method which takes a stream of `Message` from the client and returns a stream of `Message` independent of each other. server() function is called from the main function and makes sure that the server is listening to all the time.

We will run the bidirectional_server to start the server

Implementing the Client

In the run() function. we have initialised the stub using ` stub = bidirectional_pb2_grpc. BidirectionalStub(channel)’

And we have a send_message function to which the stub is passed and it makes multiple calls to the server and receives the results from the server simultaneously.

This completes the implementation of Bidirectional gRPC service.

Hello Server Sending you the First message Hello Server Sending you the Second message Hello Server Sending you the Third message Hello Server Sending you the Fourth message Hello Server Sending you the Fifth message Hello from the server received your First message Hello from the server received your Second message Hello from the server received your Third message Hello from the server received your Fourth message

Run -> python3 bidirectional_client.py

Hello from the server received your Fifth message ‍

For code reference, please visit here.

Conclusion‍

gRPC is an emerging RPC framework that makes communication between microservices smooth and efficient. I believe gRPC is currently confined to inter microservice but has many other utilities that we will see in the coming years. To know more about modern data communication solutions, check out this blog.

Originally published at https://www.velotio.com.