Routing

last person joined: 3 days ago 

Ask questions and share experiences about ACX Series, CTP Series, MX Series, PTX Series, SSR Series, JRR Series, and all things routing, including portfolios and protocols.
  • 1.  DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-11-2018 17:22

    We are getting a *lot* of these in the logs on a Juniper MX960 and an MX480 that both have interfaces on a lot of networks:

     

    Sep 11 21:24:18 r10b jddosd[39401]: DDOS_PROTOCOL_VIOLATION_CLEAR: INFO: Host-bound traffic for protocol/exception NDPv6:invalid-hop-limit has returned to normal. Its allowed bandwith was exceeded at fpc 0 for 3298 times, from 2018-09-11 21:15:32 UTC to 2018-09-11 21:19:18 UTC
    Sep 11 21:36:03 r10b jddosd[39401]: DDOS_PROTOCOL_VIOLATION_SET: Warning: Host-bound traffic for protocol/exception NDPv6:invalid-hop-limit exceeded its allowed bandwidth at fpc 0 for 3299 times, started at 2018-09-11 21:36:03 UTC
    Sep 11 21:44:29 r10b jddosd[39401]: DDOS_PROTOCOL_VIOLATION_CLEAR: INFO: Host-bound traffic for protocol/exception NDPv6:invalid-hop-limit has returned to normal. Its allowed bandwith was exceeded at fpc 0 for 3299 times, from 2018-09-11 21:36:03 UTC to 2018-09-11 21:39:28 UTC
    Sep 11 21:56:19 r10b jddosd[39401]: DDOS_PROTOCOL_VIOLATION_SET: Warning: Host-bound traffic for protocol/exception NDPv6:invalid-hop-limit exceeded its allowed bandwidth at fpc 0 for 3300 times, started at 2018-09-11 21:56:18 UTC

     

    How do I identify where this is coming from?  "FPC 0" doesn't give me much to go on; FPC 0 is an MPC5EQ-40G10G with 15 active ports on it, and I'm pretty sure they all have IPv6 on them.

     

     

    r10b> show ddos-protection protocols ndpv6 invalid-hop-limit
    Currently tracked flows: 0, Total detected flows: 0
    * = User configured value
    Protocol Group: NDPv6
    
      Packet type: invalid-hop-limit (NDPv6 with invalid hop limit)
        Individual policer configuration:
          Bandwidth:        0 pps
          Burst:            0 packets
          Priority:         Low
          Recover time:     300 seconds
          Enabled:          Yes
          Bypass aggregate: No
        Flow detection configuration:
          Detection mode: Automatic  Detect time:  3 seconds
          Log flows:      Yes        Recover time: 60 seconds
          Timeout flows:  No         Timeout time: 300 seconds
          Flow aggregation level configuration:
            Aggregation level   Detection mode  Control mode  Flow rate
            Subscriber          Automatic       Drop          10 pps
            Logical interface   Automatic       Drop          10 pps
            Physical interface  Automatic       Drop          0  pps
        System-wide information:
          Bandwidth is no longer being violated
    	No. of FPCs that have received excess traffic: 1
            Last violation started at: 2018-09-11 23:58:57 UTC
            Last violation ended at:   2018-09-12 00:02:52 UTC
            Duration of last violation: 00:03:55 Number of violations: 3306
          Received:  5302                Arrival rate:     0 pps
          Dropped:   5302                Max arrival rate: 0 pps
        Routing Engine information:
          Bandwidth: 0 pps, Burst: 0 packets, enabled
          Policer is never violated
          Received:  0                   Arrival rate:     0 pps
          Dropped:   0                   Max arrival rate: 0 pps
            Dropped by aggregate policer: 0
        FPC slot 0 information:
          Bandwidth: 100% (0 pps), Burst: 100% (0 packets), enabled
          Policer is no longer being violated
            Last violation started at: 2018-09-11 23:58:57 UTC
            Last violation ended at:   2018-09-12 00:02:52 UTC
            Duration of last violation: 00:03:55 Number of violations: 3306
          Received:  5302                Arrival rate:     0 pps
          Dropped:   5302                Max arrival rate: 0 pps
            Dropped by this policer:      5302
            Dropped by aggregate policer: 0
            Dropped by flow suppression:  0
    

     

    That doesn't really give me a lot to go on.  This seems unlikely to be an attack, more likely some idiot has something somewhere configured improperly, and there's a pretty good chance the idiot is me.  But starting from here, how do I track this down?

     

    Thanks for any suggestions!

     



  • 2.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-11-2018 18:25

    Hi , try to configure flow-detection, 

     

    set system ddos-protection global flow-detection

     

    And after next occurence check with "show ddos-protection protocols flow-detection"



  • 3.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-11-2018 20:14

    Setting flow detection doesn't seem to make any difference,  possibly because NDPv6 packets are not flow-oriented?

     

    r10b> show configuration system ddos-protection
    global {
    flow-detection;
    }

    {master}
    r10b> show ddos-protection protocols flow-detection Packet types: 225, Modified: 0 * = User configured value [...] Protocol Group: NDPv6 [...] Packet type: invalid-hop-limit Flow detection configuration: Detection mode: Automatic Detect time: 3 seconds Log flows: Yes Recover time: 60 seconds Timeout flows: No Timeout time: 300 seconds Flow aggregation level configuration: Aggregation level Detection mode Control mode Flow rate Subscriber Automatic Drop 10 pps Logical interface Automatic Drop 10 pps Physical interface Automatic Drop 0 pps
    [...]

    {master}
    r10b> show ddos-protection protocols culprit-flows
    Currently tracked flows: 0, Total detected flows: 0
    r10b> show ddos-protection protocols violations
    Packet types: 225, Currently violated: 1

    Protocol Packet Bandwidth Arrival Peak Policer bandwidth
    group type (pps) rate(pps) rate(pps) violation detected at
    ndpv6 inval-hop 0 0 0 2018-09-12 03:03:19 UTC
    Detected on: FPC-0

    {master}
    r10b> ...otocols statistics terse | match inval-hop
    ndpv6 inval-hop 5338 5338 0 3315 viol

    (The last command shown was "show ddos-protection protocols statistics terse | match inval-hop.")

     

     



  • 4.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-11-2018 18:28

    Hi David,

     

    Please check below output and check the protocol statistics:

     

    show ddos-protection protocols statistics terse

     

    Also refer below links for more details:

     

    https://www.juniper.net/documentation/en_US/junos/topics/concept/subscriber-management-ddos-protection.html

     

    https://www.juniper.net/documentation/en_US/junos/topics/reference/command-summary/show-ddos-protocols-culprit-flows.html

     

    Please let me know if these are helpful to answere your query.

     

     



  • 5.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-11-2018 21:12

    next-header header-type Match the first 8-bit Next Header field in the packet. Support for the next-header firewall match condition is available in Junos OS Release 13.3R6 and later. For IPv6, we recommend that you use the payload-protocol term rather than the next-header term when configuring a firewall filter with match conditions. Although either can be used, payload-protocol provides the more reliable match condition because it uses the actual payload protocol to find a match, whereas next-header simply takes whatever appears in the first header following the IPv6 header, which may or may not be the actual protocol. In addition, if next-header is used with IPv6, the accelerated filter block lookup process is bypassed and the standard filter used instead. Match the first 8-bit Next Header field in the packet. In place of the numeric value, you can specify one of the following text synonyms (the field values are also listed): ah (51), dstops (60), egp (8), esp (50), fragment (44), gre (47), hop-by-hop (0), icmp (1), icmp6 (58), icmpv6 (58), igmp (2), ipip (4), ipv6 (41), mobility (135), no-next-header (59), ospf (89), pim (103), routing (43), rsvp (46), sctp (132), tcp (6), udp (17), or vrrp (112). And for the payload-protocol: payload-protocol protocol-type Match the payload protocol type. In place of the protocol-type numeric value, you can specify one of the following text synonyms (the field values are also listed): specify one or a set of of the following: ah (51), dstopts (60), egp (8), esp (50), fragment (44), gre (47), hop-by-hop (0), icmp (1), icmp6 (58, igmp (2), ipip (4), ipv6 (41), no-next-header, ospf (89), pim (103), routing, rsvp (46), sctp (132), tcp (6), udp (17), or vrrp (112). You can also use the payload-protocol condition to match an extension header type that the Juniper Networks firmware cannot interpret. You can specify a range of extension header values within square brackets. When the firmware finds the first extension header type that it cannot interpret in a packet, the payload-protocol value is set to that extension header type. The firewall filter only examines the first extension header type that the firmware cannot interpret in the packet. Note: This match condition is only supported on MPCs on MX Series Routers. Initialize new firewall filters that include this condition by walking the corresponding SNMP MIB. Regarding that last note of initialize the filter with the snmp walk: Note: For MX Series routers with MPCs, you need to initialize certain new firewall filters by walking the corresponding SNMP MIB, for example, show snmp mib walk name ascii. This forces Junos to learn the filter counters and ensure that the filter statistics are displayed. This guidance applies to all enhanced mode firewall filters, filters with flexible conditions, and filters with the certain terminating actions. See those topics, listed under Related Documentation, for details. That would be something like this: >show snmp mib walk jnxFirewallCounterTable ascii There are similar OID: https://apps.juniper.net/mib-explorer/search.jsp#object=jnxFirewallCounterTable&product=Junos%20OS&release=15.1R7 But the idea is to use payload-protocol instead of next-header to confirm if the packets are hitting the filter correctly



  • 6.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit
    Best Answer

    Posted 09-11-2018 21:48

    Hi David,

     

    Please configure below it will allow the flow detection:

     

    set system ddos-protection protocols ndpv6 invalid-hop-limit flow-level-detection physical-interface on

    set system ddos-protection protocols ndpv6 invalid-hop-limit flow-level-detection logical-interface on

     

    then check the culprit flows 

    show ddos-protection protocols pppoe culprit-flows

     



  • 7.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-12-2018 11:47

    Yes, thanks, this:

     

    set system ddos-protection protocols ndpv6 invalid-hop-limit flow-level-detection physical-interface on
    
    set system ddos-protection protocols ndpv6 invalid-hop-limit flow-level-detection logical-interface on
    
    

    did appear to be the magic incantation.

    After that, the culprit-flows and violation statistics weren't really any different:

    {master}
    r10b> show ddos-protection protocols ndpv6 culprit-flows
    Currently tracked flows: 0, Total detected flows: 2
    
    {master}
    r10b> show ddos-protection protocols violations
    Packet types: 225, Currently violated: 0
    

    But the logs did show more detailed information, including the source interface!

    Sep 12 18:24:41  r10b jddosd[39401]: DDOS_SCFD_FLOW_FOUND: A new flow of protocol NDPv6:invalid-hop-limit on irb.10 with source addr -- -- -- is found at 2018-09-12 18:24:38 UTC
    Sep 12 18:24:41  r10b jddosd[39401]: DDOS_SCFD_FLOW_FOUND: A new flow of protocol NDPv6:invalid-hop-limit on xe-0/0/2 with source addr -- -- -- is found at 2018-09-12 18:24:38 UTC
    Sep 12 18:25:41  r10b jddosd[39401]: DDOS_SCFD_FLOW_RETURN_NORMAL: A flow of protocol NDPv6:invalid-hop-limit on irb.10 with source addr -- -- -- returned normal and is removed from monitoring. Found at 2018-09-12 18:24:38 UTC, last observed at 2018-09-12 18:24:40 UTC
    Sep 12 18:25:41  r10b jddosd[39401]: DDOS_SCFD_FLOW_RETURN_NORMAL: A flow of protocol NDPv6:invalid-hop-limit on xe-0/0/2 with source addr -- -- -- returned normal and is removed from monitoring. Found at 2018-09-12 18:24:38 UTC, last observed at 2018-09-12 18:24:40 UTC
    

    That's a huge help!

    That interface is to a public exchange point LAN, so our next step would be to identify which other participant is the culprit, probably by finding out the hardware MAC address. 

    Any idea why the source address is reporting as -- -- -- ?

    Or would I be better off starting to learn how packet sampling and the DDOS protection feature interact? 🙂

     



  • 8.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 09-12-2018 19:09

    Hi David,

     

    In this type of traffic source ip/destination ip is not printed, hence the source address is reporting as "----".

    you may filter the traffic on interfaces mentioned in logs to get more details

     

    Refer below KB for configuring ddos-protection flow detection for future referance:

    https://kb.juniper.net/InfoCenter/index?page=content&id=KB29408

     

    Please refer velow link for more detail on "DDOS_SCFD_FLOW" logs.

    https://www.juniper.net/documentation/en_US/junos/topics/concept/subscriber-management-scfd-overview.html

     

    ******

    If my reply is helpful, please mark solution as accepted.

     



  • 9.  RE: DDOS protocol violation: NDPv6:invalid-hop-limit

    Posted 11-07-2018 09:31

    I'm having this problem as well. For example:

    Nov  7 08:01:02  routera jddosd[18562]: DDOS_PROTOCOL_VIOLATION_CLEAR: INFO: Host-bound traffic for protocol/exception NDPv6:invalid-hop-limit has returned to normal. Its allowed bandwith was exceeded at fpc 5 for 133 times, from 2018-11-07 07:52:12 MST to 2018-11-07 07:56:02 MST

    I get a lot of these. But only on the VRRP master, none on the VRRP backup. Does anyone know why?

    My question is: How do I find these actual packets? I used the link above to log traffic (changing icmp to icmpv6), but it doesn't show me what I want to see. I think what I want to see is an NDPv6 packets with a hop limit of less than 255 right?

    If so, I did a packet trace like this:

    monitor traffic interface ae3 matching icmpv6 write-file /var/tmp/icmpv6.pcap

    Then I opened it in wireshark and applied this filter:

    !ipv6.hlim==255 && icmpv6.type==135

    And I get nothing. There are plenty of NDPv6 packets in the trace. Is it possible that the router itself is decrementing the ttl and then the policer reports it? This would be a bug I guess.

    I opened a Juniper case and the solution was to just filter out stuff without a ttl of 255:

    https://kb.juniper.net/InfoCenter/index?page=content&id=JSA10749

    I'd rather see the actual culprit with my own eyes. Can someone help?