gRPC and GPB for Networking Engineers

GPB and gRPC testing. Based on the protobuf examples and Pluralsight training.

Table of contents

• gmessaging
  • Code Examples
  • Compiling your protocol buffers
  • Understanding GPB encoding
  • Understanding Go proto code
  • Compiling the code
  • Running some examples
  • Generating Server Certificate and Private Key
  • Links

Code Examples

• add_router.go takes a static router entry and adds it to routers.data. Example:

	routers := &pb.Routers{}
	router := &pb.Router{}

	router.IP = []byte("2001:db8::123:44:4")
	router.Hostname = "router4.cisco.com"

	routers.Router = append(routers.Router, router)

If we inspect routers.data.

$ hexdump -c routers.data
0000000  \n   &  \n 020   r   o   u   t   e   r   .   c   i   s   c   o
0000010   .   c   o   m 022 022   2   0   0   1   :   d   b   8   :   :
0000020   1   2   3   :   1   2   :   1  \n   '  \n 021   r   o   u   t
0000030   e   r   2   .   c   i   s   c   o   .   c   o   m 022 022   2
0000040   0   0   1   :   d   b   8   :   :   1   2   3   :   1   2   :
0000050   2  \n   '  \n 021   r   o   u   t   e   r   3   .   c   i   s
0000060   c   o   .   c   o   m 022 022   2   0   0   1   :   d   b   8
0000070   :   :   1   2   3   :   3   3   :   3  \n   '  \n 021   r   o
0000080   u   t   e   r   4   .   c   i   s   c   o   .   c   o   m 022
0000090 022   2   0   0   1   :   d   b   8   :   :   1   2   3   :   4
00000a0   4   :   4
00000a3

$ cat routers.data | protoc --decode_raw
1 {
  1: "router.cisco.com"
  2: "2001:db8::123:12:1"
}
1 {
  1: "router2.cisco.com"
  2: "2001:db8::123:12:2"
}
1 {
  1: "router3.cisco.com"
  2: "2001:db8::123:33:3"
}
1 {
  1: "router4.cisco.com"
  2: "2001:db8::123:44:4"
}

• list_router.go reads routers.data and prints it out.

in, err := ioutil.ReadFile(fname)
if err != nil {
	log.Fatalln("Error reading file:", err)
}
routers := &pb.Routers{}
if err := proto.Unmarshal(in, routers); err != nil {
	log.Fatalln("Failed to parse the routers file:", err)
}

• data.go assigns values to different instances of our Routers struct. Example:

var router = []*pb.Router{
	&pb.Router{
		Hostname: "router1.cisco.com",
		IP:       []byte("2001:db8::111:11:1"),
	},
}

routers := pb.Routers{router}

• server.go creates a Server that implements the DeviceServiceServer interface.

type server struct{}

func (s *server) GetByHostname(ctx context.Context,
	in *pb.GetByHostnameRequest) (*pb.Router, error) {
	return nil, nil
}
...

• client.go creates a Client that creates a new DeviceServiceClient type.

conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
	log.Fatalf("did not connect: %v", err)
}
defer conn.Close()

client := pb.NewDeviceServiceClient(conn)
...

Compiling your protocol buffers

• protoc --go_out=gproto devices.proto only defines the GPB part, to read and write as demonstrated in list_routers.go and add_router.go.
• protoc --go_out=plugins=grpc:gproto devices.proto adds the RPC services. It creates gproto/devices.pb.go. You need this one to run the client and server below.

Understanding GPB encoding

Let's print out the GPB encoded slice of bytes

out, err := proto.Marshal(routers)
if err != nil {
	log.Fatalln("Failed to encode routers:", err)
}
fmt.Printf("%X", out)

After grouping the output for convenience, we get something like:

0A 26 0A 10 72 6F 75 74 65 72 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 31 32 3A 31
0A 27 0A 11 72 6F 75 74 65 72 32 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 31 32 3A 32
0A 27 0A 11 72 6F 75 74 65 72 33 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 33 33 3A 33
0A 27 0A 11 72 6F 75 74 65 72 34 2E 63 69 73 63 6F 2E 63 6F 6D
12 12 32 30 30 31 3A 64 62 38 3A 3A 31 32 33 3A 34 34 3A 34

Considering the definitions on the proto file (devices.proto)

message Router {
  string hostname = 1;
  bytes IP = 2; 
}

message Routers {
  repeated Router router = 1;
}

Protobuf uses Varint to serialize integers. The last three bits of the number store the wire type. Having this in mind and how to convert Hex to ASCII, the first 40 bytes (or two rows from the output) translate to:

Hex  Description
0a  tag: router(1), field encoding: LENGTH_DELIMITED(2)
26  "router".length(): 38
0a  tag: hostname(1), field encoding: LENGTH_DELIMITED(2)
10  "hostname".length(): 16 
72 'r'
6F 'o'
75 'u'
74 't'
65 'e'
72 'r'
2E '.'
63 'c'
69 'i'
73 's'
63 'c'
6F 'o'
2E '.'
63 'c'
6F 'o'
6D 'm'
12 tag: IP(2), field encoding: LENGTH_DELIMITED(2)
12 "IP".length(): 18
32 '2'
30 '0'
30 '0'
31 '1'
...
31 '1'

Its equivalent in JSON would be something like this (routers.json):

{
  "Router": [
    {
      "Hostname": "router.cisco.com",
      "IP": "2001:db8::123:12:1"
    }
  ]
}

Understanding Go proto code

Marshal takes the protocol buffer and encodes it into the wire format, returning the data.

func Marshal(pb Message) ([]byte, error)

Unmarshal parses the protocol buffer representation in buf and places the decoded result in pb

func Unmarshal(buf []byte, pb Message) error

Message is implemented by generated protocol buffer messages.

type Message interface {
    Reset()
    String() string
    ProtoMessage()
}

In our example generated code devices.pb.go, Router and Routers structs are defined

type Router struct {
	Hostname string `protobuf:"bytes,1,opt,name=hostname" json:"hostname,omitempty"`
	IP       []byte `protobuf:"bytes,2,opt,name=IP,proto3" json:"IP,omitempty"`
}

type Routers struct {
	Router []*Router `protobuf:"bytes,1,rep,name=router" json:"router,omitempty"`
}

Both implement the Message interface

func (m *Router) Reset()                    { *m = Router{} }
func (m *Router) String() string            { return proto.CompactTextString(m) }
func (*Router) ProtoMessage()               {}

func (m *Routers) Reset()                    { *m = Routers{} }
func (m *Routers) String() string            { return proto.CompactTextString(m) }
func (*Routers) ProtoMessage()               {}

Compiling the code

• gRPC client: go build -o client gclient/main.go
• gRPC server: go build -o server gserver/*.go

Running some examples

• Examples are pretty static for now. The client just executes a method based on the arguments the command line provides.

switch *option {
case 1:
	SendMetadata(client)
case 2:
	GetByHostname(client)
case 3:
	GetAll(client)
case 4:
	Save(client)
case 5:
	SaveAll(client)
}

• SaveAll looks like this, the client prints the devices it wants to add and the server prints the new complete list.

$ ./client -o 5
hostname:"router8.cisco.com" IP:"2001:db8::888:88:8" 
hostname:"router9.cisco.com" IP:"2001:db8::999:99:9" 

$ ./server
2017/04/29 20:27:35 Starting server on port :50051
hostname:"router1.cisco.com" IP:"2001:db8::111:11:1" 
hostname:"router2.cisco.com" IP:"2001:db8::222:22:2" 
hostname:"router3.cisco.com" IP:"2001:db8::333:33:3" 
hostname:"router8.cisco.com" IP:"2001:db8::888:88:8" 
hostname:"router9.cisco.com" IP:"2001:db8::999:99:9"

Generating Server Certificate and Private Key

This is optional in order to generate secure connections. We create a new private key 'key.pem' and a server certificate 'cert.pem'

$ openssl req -new -x509 -nodes -subj '/C=US/CN=localhost' \
                  -addext "subjectAltName = DNS:localhost" \
                  -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365

Links

• Sublime Protobuf Syntax Hightlighting
• proto3 Language Guide
• Protocol Buffer Basics: Go
• Using protocol buffers with Go
• gRPC Basics - Go
• Using Go to generate Certs and Private Keys
• Authentication in gRPC
• Use cases for gRPC in network management
• gRPC Network Management Interface (gNMI)
• Network Management Datastore Architecture