Go RPC With gRPC

Cross-platform, cross-language, simply.

6 October 2016

Daved

RPC - Remote Procedure Call

2

What's old is new again.

RPC: Simplified-for-caller remote subroutine execution.

1960s - Materialization

1970s - Formalization

1980s - Stablization

1990s - Popularization

2000s - Depopularization

2010s - Repopularization

3

Project Resources

4

Interacting with the project.

Home

Get

go get -u github.com/daved/grpcbasic0/...  # "-u" forces update

Run

# run both apps in the background
grpcbasic0 & grpcbasic0http &

Kill

# send SIGTERM to both apps
pkill grpcbasic0
5

RPC Server Example

6

Server Project Stages

- Define Expectations
- Generate Code
- Implement Logic

7

Final Directory Tree

grpcbasic0
├── cmd
│   ├── grpcbasic0
│   │   ├── main.go
│   │   └── rpc.go
│   └── grpcbasic0http
│       └── main.go
├── pb
│   ├── gogen.go
│   ├── grpcbasic0.pb.go
│   ├── grpcbasic0.pb.gw.go
│   └── grpcbasic0.swagger.json
└── proto
    └── grpcbasic0.proto
8

Define Expectations (Stage 1/3)

9

Generate Code (Stage 2/3)

10

Implement Logic (Stage 3/3)

11

Using The IDL

12

Start Proto File

syntax = "proto3";

// User Service
//
// User Service API provides create, read, and read (many) access to a list of 
// users.
package pb;

import "google/api/annotations.proto";
"proto/grpcbasic0.proto" (1/3)
13

Define Messages

message UserResp {
    int64 id = 1;
    string name = 2;
    int64 age = 3;
    string fortune = 4;
}
message NewUserReq {
    string name = 1;
    int64 age = 2;
}
message GetUserReq {
    int64 id = 1;
}
message UsersResp {
    repeated UserResp users = 1;
}
message GetUsersReq {
    int64 start = 1;
    int64 count = 2;
    bool desc = 3;
}
"proto/grpcbasic0.proto" (2/3)
14

Define Services And Related HTTP Paths

service UserService {
    rpc NewUser(NewUserReq) returns (UserResp) {
        option(google.api.http) = {
            post: "/v1/user"
            body: "*"
        };
    }

    rpc GetUser(GetUserReq) returns (UserResp) {
        option(google.api.http) = {
            get: "/v1/user/{id}"
        };
    }

    rpc GetUsers(GetUsersReq) returns (UsersResp) {
        option(google.api.http) = {
            get: "/v1/users"
        };
    }
}
"proto/grpcbasic0.proto" (3/3)
15

Generate gRPC Server, Go code, and OpenAPI Definition

package pb

//go:generate protoc -I../proto -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis --go_out=plugins=grpc:. ../proto/grpcbasic0.proto
//go:generate protoc -I../proto -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis --grpc-gateway_out=logtostderr=true:. ../proto/grpcbasic0.proto
//go:generate protoc -I../proto -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis --swagger_out=logtostderr=true:. ../proto/grpcbasic0.proto
"pb/gogen.go"

Now these commands can be run using `go generate`.

But what do the go generate declarations contain? From a typical command line:

protoc -I../proto -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
--go_out=plugins=grpc:. ../proto/grpcbasic0.proto

protoc -I../proto -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
--grpc-gateway_out=logtostderr=true:. ../proto/grpcbasic0.proto

protoc -I../proto -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
--swagger_out=logtostderr=true:. ../proto/grpcbasic0.proto
16

RPC Server Setup

17

Implement Service Interface (1/2)

package main

import (
    "github.com/daved/grpcbasic0/pb"
)

type UserService struct {
    UsersResp *pb.UsersResp
}

func NewUserService() *UserService {
    return &UserService{
        UsersResp: &pb.UsersResp{
            Users: []*pb.UserResp{
                {Id: 1, Name: "Alice", Age: 21, Fortune: fortunes.get(1)},
                {Id: 2, Name: "Bob", Age: 30, Fortune: fortunes.get(2)},
                {Id: 3, Name: "Charlie", Age: 25, Fortune: fortunes.get(3)},
                {Id: 4, Name: "Dana", Age: 18, Fortune: fortunes.get(4)},
            },
        },
    }
}
"cmd/grpcbasic0/rpc.go" (1/2)
18

Implement Service Interface (2/2)

func (s *UserService) NewUser(ctx context.Context, req *pb.NewUserReq) (*pb.UserResp, error) {
    // ...
}

func (s *UserService) GetUser(ctx context.Context, req *pb.GetUserReq) (*pb.UserResp, error) {
    // ...
}

// GetUsers ...
func (s *UserService) GetUsers(ctx context.Context, req *pb.GetUsersReq) (*pb.UsersResp, error) {
    // ...
}
"cmd/grpcbasic0/rpc.go" (2/2)
19

Implement gRPC Server

import (
    // ...
    "github.com/daved/grpcbasic0/pb"
    "google.golang.org/grpc"
)

In main():

    l, err := listener(port)
    if err != nil {
        fmt.Fprintf(os.Stderr, "cannot get listener: %s\n", err)
        os.Exit(1)
    }

    srvr, svc := grpc.NewServer(), NewUserService()
    pb.RegisterUserServiceServer(srvr, svc)
    if err = srvr.Serve(l); err != nil {
        fmt.Fprintf(os.Stderr, "rpc server error: %s\n", err)
        os.Exit(1)
    }
"cmd/grpcbasic0/main.go"
20

Intermission

21

A Minor Reflection

In a pinch, this is now enough for us to deploy our microservice.

However, what about ...

22

HTTP API Server Setup

23

Implement Reverse-proxy Server (1/2)

package main

import (
    // ...

    "github.com/daved/grpcbasic0/pb"
    "github.com/grpc-ecosystem/grpc-gateway/runtime"
    "google.golang.org/grpc"
)
"cmd/grpcbasic0http/main.go" (1/2)
24

Implement Reverse-proxy Server (2/2)

In main():

    ctx := context.Background()
    m := runtime.NewServeMux()
    opts := []grpc.DialOption{grpc.WithInsecure()}

    err := pb.RegisterUserServiceHandlerFromEndpoint(ctx, m, rpcAddr, opts)
    if err != nil {
        fmt.Fprintf(os.Stderr, "cannot register service handler: %s\n", err)
        os.Exit(1)
    }

    // custom routes first, and cors handling on all requests
    h := cors(preMuxRouter(m))

    if err = http.ListenAndServe(port, h); err != nil {
        fmt.Fprintf(os.Stderr, "http server error: %s\n", err)
        os.Exit(1)
    }
"cmd/grpcbasic0http/main.go" (2/2)
25

Extras

26

Interact via SwaggerUI

Get

go get -u github.com/codemodus/swagui/cmd/swagui

Run

swagui

Visit

http://localhost:2288/?url=http://localhost:3343/v1/swagger.json
:2288 is the swagui default port, and :3343 is the HTTP API default port.
To note, the RPC Server default port is :3323
27

Python Client Example

Install Dependencies and Generate Structures/Stub

python -m pip install grpcio grpcio-tools googleapis-common-protos
python -m grpc_tools.protoc -I../../proto \
    -I$GOPATH/src/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis \
    --python_out=. --grpc_python_out=. ../../proto/grpcbasic0.proto

Code

#!/usr/bin/env python

import sys, grpc, grpcbasic0_pb2, grpcbasic0_pb2_grpc

if __name__ == '__main__':
    strt = int((sys.argv[1:2]+[0])[0])
    chan = grpc.insecure_channel('localhost:3323')
    stub = grpcbasic0_pb2_grpc.UserServiceStub(chan)
    resp = stub.GetUsers(grpcbasic0_pb2.GetUsersReq(start=strt, count=2))
    tmpl = "ID: {}, Name: {}, Fortune: {}"
    for u in resp.users:
        print(tmpl.format(u.id, u.name, u.fortune))
28

Project Dependencies

29

30

Thank you

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)