Blogs

OpenConfig and gRPC Junos Telemetry Interface

By mwiget posted 11-29-2017 15:40

  

Starting in Junos OS Release 16.1R3 one can request streaming telemetry data using the gRPC framework with vendor-neutral data models based on OpenConfig. Great, but what configuration is required and is there a simple client available to test and troubleshoot it? I recently faced this question myself and found some great resources, which I documented in this post. For an in-depth discusssion about the various streaming telemetry option from Juniper devices, read thru the 3 parts on Tech Mocha, starting with USING OPENNTI AS A COLLECTOR FOR STREAMING TELEMETRY FROM JUNIPER DEVICES: PART 1.

Ingredients

Juniper vMX

You need a Junos network device to test against. I’m using the latest Junos vMX running 17.3R1 on KVM. Any MX, PTX or QFX with the same version should work equally well.


My lab vMX device uses the fxp0 IP address 172.17.0.4, which will be used to connect the JTI client to.
Before we can use any telemetry client, we must install two software packages on top of Junos:

 

  • Junos Openconfig specifies an RPC model to enable the Junos Telemetry Interface plus configuration models based on OpenConfig
  • Network Agent Software provides a framework to support OpenConfig and gRPC for the Junos Telemetry Interface. It functions as a gRPC server that terminates the OpenConfig remote procedure call (RPC) interface and streams the telemetry data according to the OpenConfig specification.

The latest versions downloaded are:

junos-openconfig-x86-32-0.0.0.8.tgz
network-agent-x86-32-17.3R1.10-C1.tgz

Install both packages on Junos with the following commands:

request system software add junos-openconfig-x86-32-0.0.0.8.tgz no-validate
request system software add network-agent-x86-32-17.3R1.10-C1.tgz no-validate

The CLI will ask for permition to relaunch its Daemon, which shall be granted. There is no need to restart Junos.

Junos Configuration

We need to enable gRPC and allow clients to connect from any source via

set system services extension-service request-response grpc clear-text port 50051
set system services extension-service request-response grpc skip-authentication

This opens up TCP port 50051 on the vMX and accepts gRPC requests in clear-text. Server-based SSL authentication is supported, but not used in this exercise. Earlier version of this blog post had notifications enabled, which is unrelated to gRPC and hence not needed.

JTI client

Install language Go and required libraries, then download and compile the client from github:

sudo apt install golang-go
go get github.com/golang/protobuf/proto
go get github.com/gorilla/mux
go get github.com/influxdata/influxdb/client/v2
go get github.com/prometheus/client_golang/prometheus/promhttp
go get github.com/spf13/pflag
go get golang.org/x/net/context
go get google.golang.org/grpc
go get github.com/nileshsimaria/jtimon/telemetry
go get github.com/nileshsimaria/jtimon/authentication

Clone and compile the client:

git clone https://github.com/nileshsimaria/jtimon.git
cd jtimon
go build

The build process takes no time at all. Display the tools help desk with:

$ ./jtimon -h
Usage of ./jtimon:
      --compression string   Enable HTTP/2 compression (gzip, deflate)
      --config string        Config file name
      --csv-stats            Capture size of each telemetry packet
      --drop-check           Check for packet drops
      --gtrace               Collect GRPC traces
      --latency-check        Check for latency
      --log string           Log file name
      --max-kv uint          Max kv
      --max-run int          Max run time in seconds
      --prefix-check         Report missing __prefix__ in telemetry packet
      --print                Print Telemetry data
      --prometheus           Stats for prometheus monitoring system
      --sleep int            Sleep after each read (ms)
      --stats int            Print collected stats periodically
      --time-diff            Time Diff for sensor analysis using InfluxDB
pflag: help requested

Action!

All the pieces are now in place and ready to have the JTI client connect to Junos and collect data.

Copy the sample config file sample-config/1.json to a filename of your choice and adjust the username/password and sensors to match your device. My example config looks like this:

$ cat wdm.json
{
    "host": "172.17.0.4",
    "port": 50051,
    "user": "mwiget",
    "password": "hb9rwm",
    "cid": "my-client-id",
    "grpc" : {
        "ws" : 524288
    },
    "paths": [{
        "path": "/interfaces",
        "freq": 2000
    }, {
        "path": "/junos/system/linecard/cpu/memory/",
        "freq": 1000
    }, {
        "path": "/bgp",
        "freq": 10000
    }, {
        "path": "/components",
        "freq": 10000
    }]
}

Launch the client with the config file argument and –print:

$ ./jtimon --config wdm.json --stats 10 --drop-check --print
2017/11/28 09:46:44 LoginCheck failed

Well, if you get the LoginCheck failed message as I did initially, you got the username or password wrong, but got proof of a working gRPC connection between the client and your device. Once I enabled the lab account on Junos, the client succeeds and prints out all requested telemetry data:

$ ./jtimon --config wdm.json --print
gRPC headers from Junos:
  init-response: [response { subscription_id: 1 } path_list { path: "/interfaces/" sample_frequency: 2000 } path_list { path: "/junos/system/linecard/cpu/memory/" sample_frequency: 1000 } path_list { path: "/bgp/" sample_frequency: 10000 } path_list { path: "/components/" sample_frequency: 10000 } ]
  grpc-accept-encoding: [identity,deflate,gzip]
2017/11/28 09:50:43 system_id: wdm
2017/11/28 09:50:43 component_id: 0
2017/11/28 09:50:43 sub_component_id: 0
2017/11/28 09:50:43 path: sensor_1000_1_1:/junos/system/linecard/interface/:/junos/system/linecard/interface/:PFE
2017/11/28 09:50:43 sequence_number: 1
2017/11/28 09:50:43 timestamp: 1511859043721
2017/11/28 09:50:43 sync_response: %!d(bool=false)
2017/11/28 09:50:43   key: __timestamp__
2017/11/28 09:50:43   uint_value: 1511859043960
2017/11/28 09:50:43   key: __prefix__
2017/11/28 09:50:43   str_value: /interfaces/interface[name='xe-0/0/0']/
2017/11/28 09:50:43   key: init_time
2017/11/28 09:50:43   uint_value: 1511854410
2017/11/28 09:50:43   key: state/parent_ae_name
2017/11/28 09:50:43   str_value:
2017/11/28 09:50:43   key: state/high-speed
2017/11/28 09:50:43   int_value: 10000
2017/11/28 09:50:43   key: state/oper-status
2017/11/28 09:50:43   str_value: UP
2017/11/28 09:50:43   key: state/counters/in-pkts
2017/11/28 09:50:43   uint_value: 163
2017/11/28 09:50:43   key: state/counters/in-octets
2017/11/28 09:50:43   uint_value: 48226
2017/11/28 09:50:43   key: state/counters/in-unicast-pkts
2017/11/28 09:50:43   uint_value: 163
2017/11/28 09:50:43   key: state/counters/out-pkts
2017/11/28 09:50:43   uint_value: 166
2017/11/28 09:50:43   key: state/counters/out-octets
2017/11/28 09:50:43   uint_value: 49800
2017/11/28 09:50:43   key: state/counters/carrier-transitions
2017/11/28 09:50:43   uint_value: 1
2017/11/28 09:50:43   key: __prefix__
2017/11/28 09:50:43   str_value: /interfaces/interface[name='xe-0/0/1']/
2017/11/28 09:50:43   key: init_time
2017/11/28 09:50:43   uint_value: 1511854410
2017/11/28 09:50:43   key: state/parent_ae_name
...

Pretty cool! The shown output also answers a question I had and never dared to ask: Do I need an actual proto file to understand the content of the streamed data? Turns out the answer is ‘no’ and here is why: The output shows each value with a timestamp plus key and value pair. The key is text and self explenatory: e.g. ‘/interfaces/interface[name=’xe–0/0/1’]/’, which clearly describes the source of the data and ‘state/counters/out-octets’ its nature.

Similar, the CPU sensors reports, amongst other data:

2017/11/28 09:50:48   key: /components/component[name='FPC0:CPU0']/properties/property[name='mem-util-kernel-inline-ka-bytes-allocated']/value
2017/11/28 09:50:48   uint_value: 56

Again, no additional explanation needed !

The JTI tool has a lot more to offer, e.g. to report how much data is received every 10 second plus a summary once the client is terminated (using Ctrl-C), use:

$ ./jtimon --config wdm.json --stats 10 --drop-check
gRPC headers from Junos:
  init-response: [response { subscription_id: 1 } path_list { path: "/interfaces/" sample_frequency: 2000 } path_list { path: "/junos/system/linecard/cpu/memory/" sample_frequency: 1000 } path_list { path: "/bgp/" sample_frequency: 10000 } path_list { path: "/components/" sample_frequency: 10000 } ]
  grpc-accept-encoding: [identity,deflate,gzip]
+------------------------------+--------------------+--------------------+--------------------+--------------------+
|         Timestamp            |        KV          |      Packets       |       Bytes        |     Bytes(wire)    |
+------------------------------+--------------------+--------------------+--------------------+--------------------+
| Tue Nov 28 09:56:40 CET 2017 |               4525 |                 20 |             189338 |             189338 |
| Tue Nov 28 09:56:50 CET 2017 |              11818 |                 49 |             481065 |             481065 |
| Tue Nov 28 09:57:00 CET 2017 |              19285 |                 80 |             793379 |             793379 |
| Tue Nov 28 09:57:10 CET 2017 |              26816 |                112 |            1108421 |            1108421 |
^C
 Drops Distribution
+----+-----+-------+----------+-------------------------------------------------------------------------------------------------------------------------+
| CID |SCID| Drops | Received | Sensor Path                                                                                                             |
+----+-----+-------+----------+-------------------------------------------------------------------------------------------------------------------------+
|65535|   0|      0|       16 | sensor_1003:/components/:/components/:chassisd                                                                          |
|65535|   0|      0|       36 | sensor_1000_4_1:/interfaces/:/interfaces/:mib2d                                                                         |
|65535|   0|      0|        9 | sensor_1000_6_1:/interfaces/:/interfaces/:xmlproxyd                                                                     |
|    0|   0|      0|       17 | sensor_1001:/junos/system/linecard/cpu/memory/:/junos/system/linecard/cpu/memory/:PFE                                   |
|    0|   0|      0|       17 | sensor_1000_1_1:/junos/system/linecard/interface/:/junos/system/linecard/interface/:PFE                                 |
|    0|   0|      0|       17 | sensor_1000_1_2:/junos/system/linecard/interface/logical/usage/:/junos/system/linecard/interface/logical/usage/:PFE     |
+----+-----+-------+----------+-------------------------------------------------------------------------------------------------------------------------+

Collector Stats (Run time : 40.257567985s)
112          : in-packets
26816        : data points (KV pairs)
415          : in-header wirelength (bytes)
1108421      : in-payload length (bytes)
1108421      : in-payload wirelength (bytes)
27710        : throughput (bytes per seconds)
0            : total packet drops

While the client is running, you can check the sensors on Junos:

lab@wdm> show agent sensors

Sensor Information :

    Name                                    : __default_fabric_sensor__
    Resource                                : /junos/system/linecard/fabric/
    Version                                 : 1.0
    Sensor-id                               : 129585707
    Subscription-ID                         : 562952230490667
    Parent-Sensor-Name                      : Not applicable
    Component(s)                            : PFE

    Server Information :

        Name                                : __default__snmp_server__
        Scope-id                            : 0
        Remote-Address                      : 0.0.0.0
        Remote-port                         : 0
        Transport-protocol                  : UDP

    Profile Information :

        Name                                : __default_snmp_export_profile__
        Reporting-interval                  : 30
        Payload-size                        : 5000
        Address                             : 0.0.0.0
        Port                                : 0
        Timestamp                           : 0
        Format                              : GPB
        DSCP                                : 0
        Forwarding-class                    : 0
        Loss-priority                       : low

Sensor Information :

    Name                                    : sensor_1000
    Resource                                : /interfaces/
    Version                                 : 1.0
    Sensor-id                               : 2657203
    Subscription-ID                         : 1000
    Parent-Sensor-Name                      : Not applicable
    Component(s)                            : PFE,PFE,mib2d,xmlproxyd

    Profile Information :

        Name                                : export_1000
        Reporting-interval                  : 2
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1000_1_1
    Resource                                : /junos/system/linecard/interface/
    Version                                 : 1.1
    Sensor-id                               : 186469721
    Subscription-ID                         : 1000
    Parent-Sensor-Name                      : sensor_1000
    Component(s)                            : PFE

    Profile Information :

        Name                                : export_1000
        Reporting-interval                  : 2
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1000_1_2
    Resource                                : /junos/system/linecard/interface/logical/usage/
    Version                                 : 1.1
    Sensor-id                               : 186469722
    Subscription-ID                         : 1000
    Parent-Sensor-Name                      : sensor_1000
    Component(s)                            : PFE

    Profile Information :

        Name                                : export_1000
        Reporting-interval                  : 2
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1000_4_1
    Resource                                : /interfaces/
    Version                                 : 1.0
    Sensor-id                               : 186472793
    Subscription-ID                         : 1000
    Parent-Sensor-Name                      : sensor_1000
    Component(s)                            : mib2d

    Profile Information :

        Name                                : export_1000
        Reporting-interval                  : 2
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1000_6_1
    Resource                                : /interfaces/
    Version                                 : 1.0
    Sensor-id                               : 186470745
    Subscription-ID                         : 1000
    Parent-Sensor-Name                      : sensor_1000
    Component(s)                            : xmlproxyd

    Profile Information :

        Name                                : export_1000
        Reporting-interval                  : 2
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1001
    Resource                                : /junos/system/linecard/cpu/memory/
    Version                                 : 1.0
    Sensor-id                               : 2657202
    Subscription-ID                         : 1001
    Parent-Sensor-Name                      : Not applicable
    Component(s)                            : PFE

    Profile Information :

        Name                                : export_1001
        Reporting-interval                  : 1
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1002
    Resource                                : /bgp/
    Version                                 : 1.0
    Sensor-id                               : 2657201
    Subscription-ID                         : 1002
    Parent-Sensor-Name                      : Not applicable
    Component(s)                            : rpd

    Profile Information :

        Name                                : export_1002
        Reporting-interval                  : 10
        Payload-size                        : 5000
        Format                              : GPB

Sensor Information :

    Name                                    : sensor_1003
    Resource                                : /components/
    Version                                 : 1.0
    Sensor-id                               : 2657200
    Subscription-ID                         : 1003
    Parent-Sensor-Name                      : Not applicable
    Component(s)                            : chassisd

    Profile Information :

        Name                                : export_1003
        Reporting-interval                  : 10
        Payload-size                        : 5000
        Format                              : GPB

Summary

The JTI client tool is a nifty open-source too to analyze, troubleshoot and learn about the streaming telemetry capabilities via gRPC. For more information, check out the following resources:


#How-To