Tutorial – Writing your first gRPC service in Python

In my last blog post(here) I explained the advantages of moving your micro services to gRPC, and I felt that a tutorial for writing a service in gRPC was due for the same. This tutorial is going to be in Python(since it’s the most easiest to follow), however gRPC is language independent.

We’re going to write a simple service, which returns the Hex digest of any text you send it.

For this tutorial I am going to use python 3.6(and this is only for Mac and Linux, but I am sure the only thing different for windows will be the setup process)

Setup and install virtual env with python3 on your system. This is a great article for setting it up if you’re are doing it for the first time.

Starting with grpcio

Assuming you have successfully installed virtual env on your system along with python, you can start by running the following commands:

mkdir grpc_tutorial 
cd grpc_tutorial
virtualenv -p python3 venv
source venv/bin/activate

Next you need to install grpcio in your virtual environment to start working with it:

pip install grpcio grpcio-tools

At this point your virtual environment has all the relevant libraries to starting your grpc service.

A gRPC service has 3 parts to it:

1. Proto File: Contains all the service definitions for the current package. This will be used for generating stubs to be used by the gRPC server and client.
2. gRPC Server: This is used for servicing incoming requests (kinda like an HTTP server)
3. gRPC Client: This is distributed out to other people so that they can access our server using it. Essentially this makes the gRPC call as simple as calling a native function in the same code base itself.

Writing the Proto File

Let start by writing the proto file for the service. Create a file called digestor.proto and copy the text below(explanation follows):

syntax = "proto3";

// You can ignore these for now
//option java_multiple_files = true;
//option java_package = "example-digestor.resource.grpc.digestor";
//option java_outer_classname = "DigestorProto";
//option objc_class_prefix = "DIGEST";

package digestor;

service Digestor{
 rpc GetDigestor(DigestMessage) returns (DigestedMessage) {}
}

message DigestMessage{
 string ToDigest = 1;
}

message DigestedMessage{
 string Digested = 1;
 bool WasDigested = 2;
}

The first line specifies the type/dialect of the proto definition:

syntax = "proto3"

The next couple of lines are commented out. They are package definitions for Java(Please don’t ever expect me to write a tutorial for that ;-P ) incase we needed stubs to be generated for java as well(since gRPC offers polyglot implementations).

The last line declares the name of the gRPC package.

// You can ignore these for now
//option java_multiple_files = true;
//option java_package = "example-digestor.resource.grpc.digestor";
//option java_outer_classname = "DigestorProto";
//option objc_class_prefix = "DIGEST";

package digestor;

The block of config declares the gRPC services. This essentially states that there’s a collection of services/service called Digestor, which currently has a single service called GetDigest. This takes as Input a message type DigestMessage and return a message type DigestedMessage. Definitions for both these message types is declared below the service declaration. 

service Digestor{
    rpc GetDigest(DigestMessage) returns (DigestedMessage) {};
}
message DigestMessage{
 string ToDigest=1;
}

message DigestedMessage{
 string Digested=1;
 bool WasDigested=2;
}

 

Generating the Stubs

Run the following command to generate the gRPC stubs from the proto file:

python -m grpc_tools.protoc --proto_path=. ./digestor.proto --python_out=. --grpc_python_out=.

You should be to see 2 files called digestor_pb2.py and digestor_pb2_grpc.py generated. These are the two stub files using which we can write out gRPC server and gRPC client.

Writing the gRPC Server

We can now start writing the gRPC server. Create a file called digester_server.py and save the code below in it. Follow the comments in the code for the explanations.

import grpc
import time
import hashlib
import digestor_pb2
import digestor_pb2_grpc
from concurrent import futures

class DigestorServicer(digestor_pb2_grpc.DigestorServicer):
    """
    gRPC server for Digestor Service
    """
    def __init__(self, *args, **kwargs):
        self.server_port = 46001

    def GetDigestor(self, request, context):
        """
        Implementation of the rpc GetDigest declared in the proto
        file above.
        """
        # get the string from the incoming request
        to_be_digested = request.ToDigest

        # digest and get the string representation
        # from the digestor
        hasher = hashlib.sha256()
        hasher.update(to_be_digested.encode())
        digested = hasher.hexdigest()

        # print the output here
        print(digested)

        result = {'Digested': digested, 'WasDigested': True}

        return digestor_pb2.DigestedMessage(**result)

    def start_server(self):
        """
        Function which actually starts the gRPC server, and preps
        it for serving incoming connections
        """
        # declare a server object with desired number
        # of thread pool workers.
        digestor_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

        # This line can be ignored
        digestor_pb2_grpc.add_DigestorServicer_to_server(DigestorServicer(),digestor_server)

        # bind the server to the port defined above
        digestor_server.add_insecure_port('[::]:{}'.format(self.server_port))

        # start the server
        digestor_server.start()
        print ('Digestor Server running ...')

        try:
            # need an infinite loop since the above
            # code is non blocking, and if I don't do this
            # the program will exit
            while True:
                time.sleep(60*60*60)
        except KeyboardInterrupt:
            digestor_server.stop(0)
            print('Digestor Server Stopped ...')

curr_server = DigestorServicer()
curr_server.start_server()

Run the above python file to start a gRPC server.

Writing the gRPC Client

The final step in writing our little gRPC service is to write a client which we will use to access to the rpc running on the server. Save the below file as digestor_client.py

import grpc
import digestor_pb2
import digestor_pb2_grpc

class DigestorClient(object):
    """
    Client for accessing the gRPC functionality
    """

    def __init__(self):
        # configure the host and the
        # the port to which the client should connect
        # to.
        self.host = 'localhost'
        self.server_port = 46001

        # instantiate a communication channel
        self.channel = grpc.insecure_channel(
                        '{}:{}'.format(self.host, self.server_port))

        # bind the client to the server channel
        self.stub = digestor_pb2_grpc.DigestorStub(self.channel)

    def get_digest(self, message):
        """
        Client function to call the rpc for GetDigest
        """
        to_digest_message =digestor_pb2.DigestMessage(ToDigest=message)
        return self.stub.GetDigestor(to_digest_message)

 

This completes our Simple gRPC service.

Testing it

With the server running in another console(but in the same virtualenv), fire up the python interpreter, and write the following:

from digestor_client import DigestorClient
curr_client = DigestorClient()

Using the above client instance you can call the gRPC without doing any sort of network handling. Just call the get_digest function on the client object to invoke the gRPC. For example:

currs.get_digest('Random12312312ascsadvsascdaasdcsadcsds')

prints out the output

Digested: "b7ef49c5a735a883b137fe54d734d96a16ce66ec9f1768f7c81c555d1b54336d"
WasDigested: true

 

Where Now?

This is just the beginning. You can implement whatever type of services you want on top of this. A topic to explore further would be streaming in gRPC(check it out here).

You can find all the code for this example here.

Using gRPC in production? Tell me all about it!!!