Junos OS

Hi all,

I have two EX3400-48T switches configured in a VC. I struggle with disk-usage alarms for /var that don't make any sense to me. The system is very new, just the OS was updated to 18.2R3-S2.9 and some configuration was done.

me@XX> show system alarms
2 alarms currently active
Alarm time Class Description
2020-02-26 09:29:42 UTC Minor RE 1 /var partition usage is high
2020-02-20 22:20:00 UTC Minor FPC 0 /var partition usage is high

However there should be enough free space from my understanding:

me@XX> show system storage
fpc0:
--------------------------------------------------------------------------
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/junos 1.3G 919M 337M 73% /.mount
tmpfs 805M 48K 805M 0% /.mount/tmp
tmpfs 324M 420K 324M 0% /.mount/mfs
fpc1:
--------------------------------------------------------------------------
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/junos 1.3G 916M 340M 73% /.mount
tmpfs 564M 48K 564M 0% /.mount/tmp
tmpfs 324M 524K 324M 0% /.mount/mfs

Since there is no separate /var partition visible my guess is this is refering to the main partition mounted at /.mount?

Manually set very low thresholds for the alarms but they did not disappear:

me@XX> show configuration chassis disk-partition /var
level full {
free-space 50 mb;
}
level high {
free-space 100 mb;
}

Any help is appreciated.

good luck: I'll be following this thread. I have the same issue on and off with my EX2300s . 

Michel

---

@LapointeMichel wrote:

good luck: I'll be following this thread. I have the same issue on and off with my EX2300s . 

Michel

---

The EX2300's disk is way too small, unfortunately. I often can't even install an update without using a USB stick, which is not fun for a remote switch.

There's a few tips here that may help:

https://networkdirection.net/articles/routingandswitching/juniper-routers-and-switches/junos-cleanup-storage-space/

Hi Bananarama,

Have you tried rebooting the VC for the system to recalculate?

In case you want to free some space try the following:

Execute storage clean up.

root@juniper> request system storage cleanup all-members

Delete the snapshot on all members:

root@juniper> request system snapshot delete * all-members

Check for residual packages from old codes in the /packages/db/ directory and manually delete them.
Note: This is needed for each member.

Example. The current version this device is running is 181_r3_s4 and the previous version was: 151_x53_d59
root@juniper> start shell user root
root@juniper% cd /packages/db/
root@juniper>% ls -l
total 224
drwxr-xr-x 3 root wheel 512 Feb 12 20:34 jdocs-ex-arm-32-20190315.110124_builder_junos_181_r3_s4
drwxr-xr-x 3 root wheel 512 Feb 12 20:34 jpfe-EX34XX-arm-32-20190315.110124_builder_junos_181_r3_s4
drwxr-xr-x 3 root wheel 512 Feb 12 20:34 jsd-arm-32-18.1R3-S4.2-jet-1
drwxr-xr-x 4 root wheel 512 Feb 12 20:34 jsdn-arm-32-18.1R3-S4.2
drwxrwxrwx 5 root wheel 512 Feb 12 19:27 junos-arm-32-15.1X53-D59.4---> old file
drwxr-xr-x 4 root wheel 512 Feb 12 20:34 junos-arm-32-18.1R3-S4.2
drwxr-xr-x 3 root wheel 512 Feb 12 20:34 junos-daemons-arm-32-20190315.110124_builder_junos_181_r3_s4
drwxr-xr-x 4 root wheel 512 Feb 12 20:35 junos-dp-crypto-support-ex-arm-32-20190315.110124_builder_junos_181_r3_s4
drwxr-xr-x 3 root wheel 512 Feb 12 19:33 junos-dp-crypto-support-mtx-arm-32-20180914.204810_builder_junos_151_x53_d59--> old file

Delete old files using the following command:

root@juniper% rm -R [filename]

Example:

root@juniper% rm -R junos-arm-32-15.1X53-D59.4

root@juniper% rm -R junos-dp-crypto-support-mtx-arm-32-20180914.204810_builder_junos_151_x53_d59

Regards,
Lil Dexx
JNCIE-ENT#863, 3X JNCIP-[SP-ENT-DC], 4X JNCIA [cloud-DevOps-Junos-Design], Champions Ingenius, SSYB

If this solves your problem, please mark this post as "Accepted Solution" so we can help others too \:)/

Thank you all for your replies.

@LilDexx wrote:

Have you tried rebooting the VC for the system to recalculate?

---

I rebooted the device ealier and it did not help. Cannot reboot it again because it is a production system.

---

@LilDexx wrote:

In case you want to free some space try the following:

 

Execute storage clean up.

root@juniper> request system storage cleanup all-members

 

Delete the snapshot on all members:

root@juniper> request system snapshot delete * all-members

---

That was done before already and did not free any significant space.

---

@LilDexx wrote:

Check for residual packages from old codes in the /packages/db/ directory and manually delete them.
Note: This is needed for each member.

Example. The current version this device is running is 181_r3_s4 and the previous version was: 151_x53_d59
root@juniper> start shell user root
root@juniper% cd /packages/db/
root@juniper>% ls -l

---

Thanks for this hint. That actually freed 226MB of space on each device. Shortly afterwards the alarms were cleared. As this solved the problem for me, I will select your answer as the selected answer. Thanks!

However one question remains to me: Why did the threshold of 50mb for the full disk alarm not work? There were still 337mb free, no reason at all for the alarms to fire. Does anyone have an idea?

Hi,

You may check the below KB for directions to troubleshoot

https://kb.juniper.net/InfoCenter/index?page=content&id=KB30860

If you dont have any large size file in /var and the you still have high utilization then it may be because of the memory bloating. In that case a power cycle of device can help to bring the /var utlization down.

Hope this helps