Switching

last person joined: 2 days ago 

Ask questions and share experiences about EX and QFX portfolios and all switching solutions across your data center, campus, and branch locations.
  • 1.  ex2300 zombi state where 2 48 port switch in virtual chassis

     
    Posted 02-06-2020 16:05

    Hi,

     

    I also got hit by ex2300 zombi state where 2 48 port switch in virtual chassis, Only power cycle reboot solved the issue I am still in Junos: 18.2R3-S2.9

    fpc0:
    --------------------------------------------------------------------------
    Hostname: kannurcore
    Model: ex2300-48t
    Junos: 18.2R3-S2.9
    JUNOS OS Kernel 32-bit [20191022.14c2ad5_builder_stable_11]
    JUNOS OS libs [20191022.14c2ad5_builder_stable_11]
    JUNOS OS runtime [20191022.14c2ad5_builder_stable_11]
    JUNOS OS time zone information [20191022.14c2ad5_builder_stable_11]
    JUNOS py extensions [20191115.190104_builder_junos_182_r3_s2]
    JUNOS py base [20191115.190104_builder_junos_182_r3_s2]
    JUNOS OS crypto [20191022.14c2ad5_builder_stable_11]
    JUNOS network stack and utilities [20191115.190104_builder_junos_182_r3_s2]
    JUNOS libs [20191115.190104_builder_junos_182_r3_s2]
    JUNOS runtime [20191115.190104_builder_junos_182_r3_s2]
    JUNOS Web Management Platform Package [20191115.190104_builder_junos_182_r3_s2]
    JUNOS ex libs [20191115.190104_builder_junos_182_r3_s2]
    JUNOS ex runtime [20191115.190104_builder_junos_182_r3_s2]
    JUNOS ex platform support [20191115.190104_builder_junos_182_r3_s2]
    JUNOS dcp network modules [20191115.190104_builder_junos_182_r3_s2]
    JUNOS modules [20191115.190104_builder_junos_182_r3_s2]
    JUNOS ex modules [20191115.190104_builder_junos_182_r3_s2]
    JUNOS ex Data Plane Crypto Support [20191115.190104_builder_junos_182_r3_s2]
    JUNOS daemons [20191115.190104_builder_junos_182_r3_s2]
    JUNOS SDN Software Suite [20191115.190104_builder_junos_182_r3_s2]
    JUNOS Extension Toolkit [20191115.190104_builder_junos_182_r3_s2]
    JUNOS Phone-home [20191115.190104_builder_junos_182_r3_s2]
    JUNOS Packet Forwarding Engine Support (EX34XX) [20191115.190104_builder_junos_182_r3_s2]
    JUNOS jdocs ex [20191115.190104_builder_junos_182_r3_s2]
    JUNOS jail runtime [20191022.14c2ad5_builder_stable_11]
    JUNOS FIPS mode utilities [20191115.190104_builder_junos_182_r3_s2]

    fpc1:
    --------------------------------------------------------------------------
    Hostname: kannurcore
    Model: ex2300-48t
    Junos: 18.2R3.4
    JUNOS OS Kernel 32-bit [20190605.30b921f_builder_stable_11]
    JUNOS OS libs [20190605.30b921f_builder_stable_11]
    JUNOS OS runtime [20190605.30b921f_builder_stable_11]
    JUNOS OS time zone information [20190605.30b921f_builder_stable_11]
    JUNOS py extensions [20190620.215446_builder_junos_182_r3]
    JUNOS py base [20190620.215446_builder_junos_182_r3]
    JUNOS OS crypto [20190605.30b921f_builder_stable_11]
    JUNOS network stack and utilities [20190620.215446_builder_junos_182_r3]
    JUNOS libs [20190620.215446_builder_junos_182_r3]
    JUNOS runtime [20190620.215446_builder_junos_182_r3]
    JUNOS Web Management Platform Package [20190620.215446_builder_junos_182_r3]
    JUNOS Web Management Application package [18.2A1]
    JUNOS ex libs [20190620.215446_builder_junos_182_r3]
    JUNOS ex runtime [20190620.215446_builder_junos_182_r3]
    JUNOS ex platform support [20190620.215446_builder_junos_182_r3]
    JUNOS dcp network modules [20190620.215446_builder_junos_182_r3]
    JUNOS modules [20190620.215446_builder_junos_182_r3]
    JUNOS ex modules [20190620.215446_builder_junos_182_r3]
    JUNOS ex Data Plane Crypto Support [20190620.215446_builder_junos_182_r3]
    JUNOS daemons [20190620.215446_builder_junos_182_r3]
    JUNOS SDN Software Suite [20190620.215446_builder_junos_182_r3]
    JUNOS Extension Toolkit [20190620.215446_builder_junos_182_r3]
    JUNOS Phone-home [20190620.215446_builder_junos_182_r3]
    JUNOS Packet Forwarding Engine Support (EX34XX) [20190620.215446_builder_junos_182_r3]
    JUNOS jdocs ex [20190620.215446_builder_junos_182_r3]
    JUNOS jail runtime [20190605.30b921f_builder_stable_11]
    JUNOS FIPS mode utilities [20190620.215446_builder_junos_182_r3]

     

     

    and the curious thing is one switch automatically rolled back to problematic version Junos: 18.2R3.4 can you any suggest me the 15.1 software train where this issue is not there, I can't take the risk I narrowly escaped from production outage.

     

     

    What is the alternate command to make the backup partition also in 18.2R3-S2.9 to avoid rollbacks?

    request system snapshot slice alternate is not working any more in new Junos.



  • 2.  RE: ex2300 zombi state where 2 48 port switch in virtual chassis

     
    Posted 02-06-2020 20:11

    Hi

     

    It looks like the software upgrade for fpc1 failed rather than the rollback.

    for 15.1 the recommended version would be Junos 15.1X53-D591 however without investigating this properly we cannot say this issue will not be there on 15.1.

    As far as system snapshot is concerned are you seeing any error while doing the "request system snapshot slice alternate member all" ?

     

    Thanks

     



  • 3.  RE: ex2300 zombi state where 2 48 port switch in virtual chassis

     
    Posted 02-06-2020 21:30

    Hi,

     

    root@kannurcore> request system snapshot sl
    ^
    syntax error.
    root@kannurcore> request system snapshot sl
    ^
    syntax error.
    root@kannurcore> request system snapshot ?
    Possible completions:
    <[Enter]> Execute this command
    all-members Archive data and executable areas of all virtual chassis members
    delete Delete an existing snapshot
    load Load a given snapshot and activate it upon reboot
    local Archive data and executable areas of local virtual chassis member
    media Media to snapshot to
    member Archive data and executable areas of specific virtual chassis member
    partition Partition the media
    recovery Create a recovery snapshot
    | Pipe through a command

     

     

    There is no option in these version of junos.

     

    I made sure that both had same os was on both switch



  • 4.  RE: ex2300 zombi state where 2 48 port switch in virtual chassis

     
    Posted 02-06-2020 23:00

    For the snapshot i believe there has been changes in the New FreeBSD version.

    You can simply try 

     

    >request system snapshot

     

    {master:1}
    root@EX3400> request system snapshot
    fpc1:
    --------------------------------------------------------------------------
    NOTICE: Snapshot snap.20170223.125901 created successfully
    
    {master:1}
    root@EX3400> file list /packages/sets/
    
    /packages/sets/:
    active/
    optional/
    previous/
    snap.20170223.125901/

    Now for more details you can refer to the following

     

     

    https://kb.juniper.net/InfoCenter/index?page=content&id=KB32170&actp=METADATA

     

    Hope this helps.



  • 5.  RE: ex2300 zombi state where 2 48 port switch in virtual chassis

     
    Posted 02-07-2020 08:34

    Hi 

     

    Thank you for the post, I have gone through earlier, but in legacy Junose there was resilient Juno's partition that is main and backup. In the event of the main Junos gets corrupt it will automatically load from backup Junos. when we install new Junos version the existing Junos version gets replaced and become backup partition, got me in trouble once I forgot to slice the main Junos partition to backup. So how this works in the new Junos version?

     

    what happened to existing  Junos version when we upgrade to the new one, for example, upgrading from 18.3r3 to 18.4, is there any chance the 18.3 will active once the current 18.4 is corrupt as it was happened in the legacy junos. To avoid that I used to run request system snapshot slice alternate command. what is the equivalent command in new Junos? How to avoid the automatical rollback of Junos in the vent of corruption.

     

     

     



  • 6.  RE: ex2300 zombi state where 2 48 port switch in virtual chassis
    Best Answer

     
    Posted 02-09-2020 20:44

    Hello 

     

    From the article we can see that in the new junos we have 2 type of snapshots. 

    The one you are looking for is the recovery snapshot 

     

     

    root@EX34xx> request system snapshot recovery
    fpc0:
    --------------------------------------------------------------------------
    Creating image ...
    Compressing image ...
    Image size is 243MB
    Recovery snapshot created successfully

    Recovery snapshot does the same thing as request system snapshot slice alternate on the older platforms.

     

     

    So if you do the recovery snapshot after the upgrade then you can recover the junos from the OAM volume as follows

     

    root@EX3400> request system recover oam-volume
    
    NOTICE: Recovering the OAM volume ...
    da0p1 deleted
    da0p1 added
    /dev/gpt/oam: 500.0MB (1024000 sectors) block size 32768, fragment size 4096
        using 4 cylinder groups of 125.03MB, 4001 blks, 16128 inodes.
    super-block backups (for fsck_ffs -b #) at:
     192, 256256, 512320, 768384
    Verified oam signed by PackageProductionEc_2016 method ECDSA256+SHA256
    Installing OAM volume contents ...
    The OAM volume is now installed
    NOTICE: Creating a recovery snapshot on the OAM volume ...
    Creating image ...
    Compressing image ...
    Image size is 243MB
    Recovery snapshot created successfully
    NOTICE: OAM volume recovered  >>> Recovered