Network Management
Network Management

FAQ: Junos Space Security Director Logging API reference

by Juniper Employee on ‎06-13-2016 07:05 AM - edited on ‎10-06-2017 03:21 PM by Administrator Administrator (3,438 Views)

Overview

Junos Space Security Director, when used with Log Collector, provides the additional capabilities of analytics based on system log messages (syslogs). This document describes the REST end points exposed for querying information from logs as aggregated data and searches on logs.

All API access is over HTTPS and accessed from the [[:spaceserver]]/api/juniper/ecm/log-scoop domain. All data is sent and received in JSON form. All the APIs are HTTP POST, since the query size is big. They do not create or modify any data in system, but just fetch the queried result.

Authentication

The API needs to be authenticated using the Junos Space Platform user's credentials. The RBAC access is applied as per the user's permissions.

Parameters

Filter is an optional parameter. The request parameter needs to be attached with each POST request in JSON form. The header of the request should have the accept and content-type information.

 

Content-Type of ‘application/json’:

Accept 'appliction/json'

Time Handling

All timestamps are consumed and returned in ISO 8601 format: 

YYYY-MM-DDTHH:MM:SSZ

Examples:

"2007-04-05T14:30Z" or "2007-04-05T12:30-02:00" 

Time Interval 

The following three formats are supported. 

  • <start>/<end>
  • <start></duration>
  • <duration></end>

Examples: 

Start and end, such as "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z"
Start and duration, such as "2007-03-01T13:00:00Z/P1Y2M10DT2H30M"
Duration and end, such as "P1Y2M10DT2H30M/2008-05-11T15:30:00Z"

Root Endpoints

The following lists the available end points, followed by an example request and response of each type.

  • Aggregation Query: /api/juniper/ecm/log-scoop/aggregate
  • Time Line Query: /api/juniper/ecm/log-scoop/time-aggregate
  • Non Aggregation Query: /api/juniper/ecm/log-scoop/logs 

Aggregated Query

The API is useful in fetching the information based on count or sum of a given attribute. An example of this is the top source address and their counts with a given filter criteria.

End Point: /api/juniper/ecm/log-scoop/aggregate

HTTP Method: POST

Request Structure: 

{  
   "request":{  
      "aggregation":"COUNT",
      "aggregation-attributes":"source-address",
      "time-interval":"P1M/2015-07-02T00:30:15+05:00",
      "size":"5",
      "order":"ascending",
      "filters": {
      "and": [
        {
          "or": [
            {
              "filter": {
                "key": "username",
                "operator": "EQUALS",
                "value": "root"
              }
            },
            {
              "filter": {
                "key": "source-address",
                "operator": "EQUALS",
                "value": "1.1.1.1"
              }
            }
          ]
        }
      ]
    }
   }
}

 

Response:

 {
  "response": {
    "header": {
      "total-count": 5,
      "message": "Successfully data retrieved",
      "response-code": 200
    },
    "result": [
      {
        "key": "12.12.12.12",
        "value": 304134
      },
      {
        "key": "15.15.15.15",
        "value": 202532
      },
      {
        "key": "11.11.11.11",
        "value": 202009
      },
      {
        "key": "18.18.19.19",
        "value": 101008
      },
      {
        "key": "21.21.21.21",
        "value": 100955
      }
    ]
  }
}

Multiple Aggregation Query

This option is useful in querying multipoint aggregation. An example is -  which attack happened, how many times, and from which place to which place.

End Point: /api/juniper/ecm/log-scoop/aggregate

HTTP Method: POST

Request Structure:

{      
"request": {
"aggregation":"COUNT", "aggregation-attributes": ["attack-name", "src-geo.country-code2","dst-geo.country-code2"], "time-interval":"P1D/2015-07-22T20:30:15+05:30",
"size":"2", "order":"ascending" } }

 

Response Structure:

{
"response": {
"header": {
"total-count": 3,
"message": "Result count mismatch.",
"response-code": 201,
"key": "aggregation-attribute",
"value": ["attack-name", "src-geo.country-code2", "dst-geo.country-code2" ]
},
"result": [ {
"keys": [ "TCP:C2S:AMBIG:C2S-SYN-DATA",
"US",
"US" ],
"value": 716
},
{
"keys": [ "TCP:C2S:AMBIG:C2S-SYN-DATA",
"US",
"SY" ],
"value": 11
},
{
"keys": [ "APP:CITRIX:PROVISIONINGSERV-UF",
"SY",
"US" ],
"value": 267
}
]
}
}

Time Line Query

This query is similar to the aggregate query, with the addition of time series information on the data.

End Point: /api/juniper/ecm/log-scoop/time-aggregate

HTTP Method: POST

Request Structure

{  
   "request":{  
      "aggregation":"COUNT",
      "aggregation-attributes":"source-address",
      "time-interval":"P1M/2015-07-01T00:30:15+05:00",
      "size":"2",
      "order":"ascending",
      "slots":"10",
      "filters": {
      "and": [
        {
          "or": [
            {
              "filter": {
                "key": "username",
                "operator": "EQUALS",
                "value": "root"
              }
            },
            {
              "filter": {
                "key": "source-address",
                "operator": "EQUALS",
                "value": "1.1.1.1"
              }
            }
          ]
        }
      ]
    }
   }
}

 

 Response:

{

  "response": {
    "header": {
      "total-count": 10,
      "message": "Successfully data retrieved",
      "response-code": 200,
      "key": "time-series",
      "value": [
        "2015-06-01T01:00:15-2015-06-04T01:00:15",
        "2015-06-04T01:00:15-2015-06-07T01:00:15",
        "2015-06-07T01:00:15-2015-06-10T01:00:15",
        "2015-06-10T01:00:15-2015-06-13T01:00:15",
        "2015-06-13T01:00:15-2015-06-16T01:00:15",
        "2015-06-16T01:00:15-2015-06-19T01:00:15",
        "2015-06-19T01:00:15-2015-06-22T01:00:15",
        "2015-06-22T01:00:15-2015-06-25T01:00:15",
        "2015-06-25T01:00:15-2015-06-28T01:00:15",
        "2015-06-28T01:00:15-2015-07-01T01:00:15"
      ]
    },
    "result": [
      {
        "time-value": [
          {
            "key": "12.12.12.12",
            "value": 304134
          },
          {
            "key": "15.15.15.15",
            "value": 202532
          }
        ]
      },
      ""
    ]
  }
}

  

Search on Event Query

This API can be used to fetch the logs in JSON form, with a given search criteria.

End Point: /api/juniper/ecm/log-scoop/logs 

HTTP Method: POST

Request Structure:

{
  "request": {
    "time-interval": "P10D/2015-07-01T05:30:15+05:30",
    "size": "2",
    "order": "ascending",
    "slots": "10",
                "resolve-addresses": true,
    "filters": {
      "and": [
        {
          "or": [
            {
              "filter": {
                "key": "username",
                "operator": "EQUALS",
                "value": "root"
              }
            },
            {
              "filter": {
                "key": "source-address",
                "operator": "EQUALS",
                "value": "1.1.1.1"
              }
            }
          ]
        }
      ]
    }
  }
}

 

Response:

{
  "response": {
    "header": {
      "total-count": 2,
      "message": "Successfully data retried",
      "response-code": 200
    },
    "result": [
      {
        "index": "logstash-2015.06.26-13",
        "host": "10.207.99.44",
        "id": "AU4wDbd51mDx7k7neYEc",
        "timestamp": "2015-06-26T13:31:42.111Z",
        "priority": 0,
        "severity": 0,
        "facility": 0,
        "facility-label": "kernel",
        "severity-label": "Emergency",
        "username": "root]",
        "country-code": "US"
      },
      {
        "index": "logstash-2015.06.26-13",
        "host": "10.207.99.44",
        "id": "AU4wDbjQ1mDx7k7neZFk",
        "timestamp": "2015-06-26T13:31:42.110Z",
        "priority": 0,
        "severity": 0,
        "facility": 0,
        "facility-label": "kernel",
        "severity-label": "Emergency",
        "username": "root]",
        "country-code": "US",
        "nat-destination-address-host-name": "172.19.51.235",
        "source-address-host-name": "1.1.1.1",
        "destination-address-host-name": "172.19.51.235",
        "nat-source-address-host-name": "172.19.51.235"
      }
    ]
  }
}

Finding Attributes to Query

For Data Coming in the Log

You can pick the attribute names from the structured logs themselves. For example, in the following log:

1 2013-03-05T09:28:12.945 junosdut RT_FLOW - RT_FLOW_SESSION_CREATE [junos@2636.1 .1.1.2.36 source-address="1.1.1.1" source-port="0" destination-address="2.1.1.1" destination-port="46613" service-name="icmp" nat-source-address="1.1.1.1" nat-sou rce-port="0" nat-destination-address="2.1.1.1" nat-destination-port="46613" src-n at-rule-name="None" dst-nat-rule-name="None" protocol-id="1" policy-name="3" source-zone-name="AS_AV_UF_Zone1.90" destination-zone-name="AS_AV_UF_Zone1.91" session-id-32=" 31117" username="N/A" roles="N/A" packet-incoming-interface="fe-0/0/2.0"

The key value pairs appear as:

destination-address="2.1.1.1" destination-port="46613"

 These keys can form the query attributes.

 

For Data Augmented by Log Collector

There are a few attributes that are not part of logs directly, but are added by  the log collection system. These are listed below:

  • "host" -- IP address using which device is sending logs
  • "src-geo.country-name"  --  Country Name of origin of traffic
  • "src-geo.country-code2" -- two letter contry code
  • "src-geo.country-latitude"
  • "src-geo.country-longitude"
  • "dst-geo.country-name" -- Country Name of destination of the traffic
  • "dst-geo.country-code2" -- two letter country code
  • "dst-geo.country-latitude"
  • "dst-geo.country-longitude"
'
Comments
Dec 14, 2017
ACAS-IPC

Thank you much needed article.  If connecting to the API on a Space platform with a seperate log connector and I am getting the following what does it mean.  Does this mean my request is no good:

 

{
    "request":{
        "aggregation":"COUNT",
        "aggregation-attributes":"source-address",
        "time-interval":"2017-12-14T12:00:00Z/2017-12-14T12:15:00Z",
        "size":"50",
        "resolve-addresses":false,
        "case-sensitive":false,
        "sort":"timestamp",
        "order":"descending",
        "filters": {
            "and":[
                {
                    "filter": {
                        "key":"event-type",
                        "operator":"EQUALS",
                        "value":"RT_FLOW_SESSION_DENY"
                        }
                }
            ]
        }
    }
}

{
    "response": {
        "header": {
            "key": "aggregation-attributes",
            "value": [
                "source-address"
            ],
            "result-count": 0,
            "message": "Unable to connect the Log Collector",
            "response-code": 405
        },
        "result": []
    }
}

Aug 2, 2019
Juniper Employee

Hi,

 

We are getting the same error when the LC is on a different VM (not integrated): "Unable to connect the Log Collector"

 

Any clue to solve it?

 

Thanks!

Br