Ethernet Switching
Ethernet Switching

Virtual Chassis Links - NO MONITORING AVAILABLE FROM JUNOS

[ Edited ]
07.07.14   |  
‎07-07-2014 12:33 PM

Hi All,

 

We have ~300 Juniper EX series switches. They are configured as ~110 'extended' virtual chassis, dispersed throughout our campus. Subsequent to purchase and installation, we gradually came to realize that Juniper either forgot or chose not to implement SNMP in Junos for virtual chassis interfaces. That means network monitoring tools, including Juniper's own SPACE, have no visibility into, or way to gather stats and link status on most of our network topology. That includes our core switch - a VC consisting of 4 EX8208's and 2 XRE200's divided between two data centers on our campus.

 

As we became aware of this hole in the product and the exposure it gives us, we engaged our Juniper sales and engineering reps. I've searched here in the forums for others in the same predicament and haven't found a solution. You can rig things up to be notified when a VC member disappears for whatever reason, but we want and expect that we should be able to monitor VC links to know proactively about traffic and error conditions. We need to be in front of problems in these areas, not hearing from users about connectivity issues and then having to hunt through switches interactively with Junos CLI commands to look for link/utilization/error issues on virtual chassis interfaces.

We are being as persistent and patient as we can with Juniper. They have acknowledged the issue and have an open 'enhancement request' (ER) for it - ER-057120. They've said that we are not the only site concerned about this matter and they're planning on letting us know as soon as they can when they have established a timeline for addressing it.

 

I'm posting to look for anyone else out there in the same situation. If so, what are you doing about it? Would you like to try working together to get this as much attention from Juniper as possible? Got any other ideas or suggestions? I'd love to hear them Smiley Happy

 

-John Jackson
jpjackso@smith.edu
Network Administrator
Smith College
Northampton, MA

 

19 REPLIES
Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 01:13 PM

Is this helping you?

 

http://kb.juniper.net/InfoCenter/index?page=content&id=KB27711

 

http://www.juniper.net/techpubs/en_US/junos12.3/topics/reference/mibs/mib-jnx-virtualchassis.txt

 

 

 

=====

If this worked for you please flag my post as an "Accepted Solution" so others can benefit. A kudo would be cool if you think I earned it.

Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 01:33 PM

Hi Parau,

 

     Thank you for your reply.  Unfortunately neither of those pieces of information is of any help.  KB27711 is only a partial hack that a well intentioned Juniper engineer created but it is not complete.  It will only get you visibility of the VC interfaces in the Master chassis member of a VC.  It has no visibility into the VCP's of any of the other chassis members.  Also, it is not certain that actually using that slax script would be ok.  In order to use it, it needs to be configured to run at the Unix level as a cron job in the switch, and I don't know if it would be a problem for CPU utilization if it was configured to run at 1 or 5 minute intervals.  But that isn't so much an issue since it doesn't provide visibility for all the VCP's in the VC in the first place.  The second link you provided just gives MIB definitions for the VCP interfaces that the slax from KB27711 would establish, I believe.

 

Is there anything you can do to look into how urgently or not Juniper is giving attention/priority to ER-057120?

Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 04:09 PM

Hello John,

"... KB27711 is only a partial hack that a well intentioned Juniper engineer created but it is not complete.  It will only get you visibility of the VC interfaces in the Master chassis member of a VC.  It has no visibility into the VCP's of any of the other chassis members..."

 

If you add all-members, you get info on the members of the VC.

 

I think you can get something working one way or the other. You could also post your query to the Automation forum as you could definitely find a scripting guru who could offer more help. I noticed that the interface type for vcp use Trunk, so I am thinking you can pull the data from trunk ports up/down. Or using  jnxVirtualChassisPortOperStatus looking for Interger =2 which is vcp down. If you poll for jnxVirtualChassisPortAdminStatus, that would tell you if an admin manually disabled a vcp.
If I get some time later this week, I will test to see if using the messages logfile as a temporary workround can be implemented. Or better yet create a specific logfile to only log vcp changes and monitor this log file - you could even send this logfile to a syslog server.
>show virtual-chassis vc-port statistics all-members
>show virtual-chassis vc-port all-members


Interface Type Trunk Status Speed Neighbor
or ID (mbps) ID Interface
PIC / Port
vcp-0 Dedicated 2 Up 32000 4 vcp-1
vcp-1 Dedicated 1 Down 32000

show snmp mib walk jnxUtilCounter64Value ascii | match 4743
jnxUtilCounter64Value."4743-fpc4-internal-0/24-bps-in" = 0
jnxUtilCounter64Value."4743-fpc4-internal-0/24-bps-out" = 0
jnxUtilCounter64Value."4743-fpc4-internal-0/24-bytes-in" = 0

jnxVirtualChassisPortAdminStatus OBJECT-TYPE
      SYNTAX     INTEGER {
                  up(1),
                  down(2),
                  unknown(3)
                }
      MAX-ACCESS    read-only
      STATUS        current
      DESCRIPTION
           "Indicates the actual admin status of this
             vccp port, which is typically but not limited to, a
             function of the state of individual segments of
             this port."
      ::= { jnxVirtualChassisPortEntry 3 }

   jnxVirtualChassisPortOperStatus OBJECT-TYPE
      SYNTAX     INTEGER {
                  up(1),
                  down(2),
                  unknown(3)
                }

jnxVccpPortUp NOTIFICATION-TYPE
    OBJECTS { jnxVirtualChassisPortAdminStatus,
              jnxVirtualChassisPortOperStatus,
              jnxVirtualChassisMemberModel,
              jnxVirtualChassisMemberLocation


show snmp mib walk jnxVirtualChassisPortOperStatus
jnxVirtualChassisPortOperStatus.0.11.118.99.112.45.48.46.51.50.55.54.56 = 2
jnxVirtualChassisPortOperStatus.0.11.118.99.112.45.49.46.51.50.55.54.56 = 2

[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

[ Edited ]
07.07.14   |  
‎07-07-2014 08:11 PM

Hi Lyndidon,

 

     Where are you talking about doing this:

 

     "If you add all-members, you get info on the members of the VC."

 

Do you mean, when you execute the ex-vcp-to-mib.slax script on the VC?  Are you referring to this bunch of info, at the top of the script?

/* ------------------------------------------------------------------------------------------------------------------------------------------------
--- Author : Damien GARROS / dgarros@juniper.net
--- Version : v1.1
--- Last Modified : 2012-April-02
--- Tested on EX4200-VC & EX8200-VC
--- Description ---

        This script will gather several information about VCP links and populate Juniper Mibs entries with them :
        Information gather per VCP link
               - Input/Output Bytes per second  Counter 64
               - CRC Errors                     Counter 64
               - Input/Output Bytes             Counter 64
        
        A VCP link not available for one hour will be automatically removed from Mib 
               
        By default, only vcp-* links will be gather, if you want to get all VCP links including "internal" you must add "ports all" option 
                op ex-vcp-to-mib ports all
               
        To check if Mib is correctly populated, and to get OID for each entry, we can use these commands from CLI : 
                root@EX> show snmp mib walk jnxUtilCounter64Value
                root@EX> show snmp mib walk jnxUtilCounter64Value ascii | match 4743
        
        As a reminder 
                jnxUtilCounter32Value           1.3.6.1.4.1.2636.3.47.1.1.1.1.2
                jnxUtilCounter64Value           1.3.6.1.4.1.2636.3.47.1.1.2.1.2
                jnxUtilIntegerValue             1.3.6.1.4.1.2636.3.47.1.1.3.1.2
                jnxUtilUintValue                1.3.6.1.4.1.2636.3.47.1.1.4.1.2
                jnxUtilStringValue              1.3.6.1.4.1.2636.3.47.1.1.5.1.2

       for-each( $vcport-stat-ext-output/multi-routing-engine-item/virtual-chassis-port-statistics-information/port-list/statistics ) {
                
                
--- Installation guide ---           
        This script can run as "OP" or "Events" script
        
        To execute it as "op script"
                Copy it on /var/db/scripts/op on each RE
                Add "set system scripts op file ex-vcp-to-mib.slax"
                
                Then, execute it from cli with 
                        op ex-vcp-to-mib <option>
                        
        To execute it as "Event script"
                 Copy it on /var/db/scripts/event on each RE
                 
                 Declare it into the configuration
                        set event-options policy VCP-TO-MIB events <event-name>
                        set event-options policy VCP-TO-MIB then event-script "ex-vcp-to-mib <option>"
                        
                 if you want to execute it periodically, you can create an event like this:
                        set event-options generate-event 2MIN time-interval 120
                 
                 For the first execution as Event script, it is recommanded to activate debug to syslog
                        set event-options policy VCP-TO-MIB then event-script "ex-vcp-to-mib debug syslog"
                 Then you can monitor log file to confirm if script is running as expected.

 

 

 

-John

 

Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 09:17 PM

No, i was referring to this:

>show virtual-chassis vc-port statistics all-members
>show virtual-chassis vc-port all-members

 

However looking at the script, there is this detail which I have no way of knowing what to do with it so it would be automatically work when the script is executed,

"By default, only vcp-* links will be gather, if you want to get all VCP links including "internal" you must add "ports all" option op ex-vcp-to-mib ports all"

I am just brainstorming. If the above two commands show information about all the VCP in all members, and the VC MIB reports information on all VCP links, I think there has to be a way to poll for this data.

The other option I was thinking about is to disable a VCP and then unplug a cable and see what is logged in the chassid and messages log files. Then create a specific log file which will only look for those events and log them and a syslog server will send email or text to alert that port is down and the reason.

[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 09:28 PM

What I really need is to be able to gather stats of bits per second, errors incremented, and interface up/down status, on the VCP interfaces.  I'm looking for other customers that have the same situation - I can't believe Juniper's installed base in this area is so small that no one else has discovered this gap.  In fact, I know we're not the only ones because the SE has told us there are a number of other customers who are associated with the ER.  But, even though the SE and his next higher up are saying things that sound promising in terms of Juniper acknowledging the matter, and in terms of them sounding like they're interested in addressing it, I also have the impression that this is something that we can't count on getting fixed until it really has happened.  It seems like if we do not advocate for it as strongly as we can, there's plenty of chance for it to remain unresolved.  We just can't walk away and close our eyes and hope we don't get burned by the problem.  Whether it be a workaround or a proper fix, we need to get traffic and error counts for all these links (hundreds of them), and we need to get real time notice of up/down events.  It seems like there's more hope for the latter from workarounds, but not much hope at all, practically, for the former.  But we can't quit, so we won't Smiley Happy

Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 09:46 PM
I totally agree. i have advocated for a special forum for "Feature Requests" which would then be closely monitored by the developers and code writers.
I say keep on them till they fix it.
[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 09:51 PM

Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

[ Edited ]
07.07.14   |  
‎07-07-2014 09:59 PM

I really do want to stress something about this particular situation though.  I wouldn't agree that this is a 'feature request'.  I would say very strongly that this is a basic, essential feature that was somehow forgotten or otherwise allowed to slip, and it never should have.  This is like Juniper having built and sold us a car that is completely missing an oil pressure gauge or even an 'idiot' light for oil pressure.  That is something that should be included, without any uncertainty, in even the most basic vehicle.  Likewise, selling us an architecture that doesn't include the capability to allow for SNMP monitoring and stats gathering from a large portion of the interconnecting links in the topology was someone's blunder somewhere.  So, I would call this an emergency feature repair, or something along those lines.  Yes?

 

-John

 

Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE

07.07.14   |  
‎07-07-2014 10:37 PM

Haha:Smiley Happy I like it! I guess, until they find a solution we are waiting for the explanation then?

[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]
Ethernet Switching

Extended VC Links - NO MONITORING AVAILABLE

07.10.14   |  
‎07-10-2014 10:31 AM

Hi,

 

     So, are there any of you reading this thread that use 'extended' virtual chassis links, to form VC's between physical chassis that are not right next to each other, by using regular fiber or copper ports in VC mode?

 

-John

 

Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

07.14.14   |  
‎07-14-2014 11:40 AM

Yikes.  When Juniper proposed this design to us they made it sound like it was something a lot of customers were doing..

Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

07.14.14   |  
‎07-14-2014 02:08 PM

Yes, I have created a virtual chassis with extended vc links.

Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

07.14.14   |  
‎07-14-2014 02:10 PM

Great!!!  Have you figured out a way to get networking monitoring stats for the VCP interfaces for those links?  You know, like bps in and out, and errors incremented, plus link up/down status and events?

 

-John

Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

07.14.14   |  
‎07-14-2014 02:31 PM

No, it's a temporarly network put up for a show every year, we initially tried and couldn't see how to get it done, there have been some JunOS upgrades since so perhaps we'll try again, but I'm afraid that I don't have the answer.

Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

[ Edited ]
07.14.14   |  
‎07-14-2014 02:40 PM

I have your answer.  There still is no way to get it done, and there isn't expected to be, for at least the rest of this year, and perhaps at least until half way into next year.  I have been trying for months to get Juniper to say they are going to work to resolve this sooner but so far I haven't gotten any committments.  To be fair, I have gotten sympathetic feedback, but I am sitting here with our entire network riding these invisible links and sympathy isn't enough.  I don't have anything against Juniper, it's just a matter of business.  There's no way I can responsibly just say, "Ok, well whenever you can get that to us, that'll just have to do."  If we have link problems on these paths that we have no reasonable way of getting informed about, other than by impact on user activity, we're the ones who are going to have to answer for it.  It's not a situation any customer wants to be left in by their vendor.

 

-John

 

Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

08.08.17   |  
‎08-08-2017 02:04 AM

Hi John,

 

Have you finally found solution for that problem?

I have EX4200 VC and would like to monitor SFP+ interfaces utilisation...

 

Regards,

Kacper

Highlighted
Ethernet Switching

Re: Virtual Chassis Links - NO MONITORING AVAILABLE FROM JUNOS

08.08.17   |  
‎08-08-2017 08:09 AM
Ethernet Switching

Re: Extended VC Links - NO MONITORING AVAILABLE

08.08.17   |  
‎08-08-2017 10:41 AM

Could these help?

 

show virtual-chassis vc-port statistics extensive

show virtual-chassis vc-port diagnostics optics
show virtual-chassis vc-port diagnostics optics (interface-name)
show virtual-chassis vc-port diagnostics optics local
show virtual-chassis vc-port diagnostics optics (member member-id)

[KUDOS PLEASE! If you think I earned it!
If this solution worked for you please flag my post as an "Accepted Solution" so others can benefit..]