Many (read/write?) errors during Junos OS update

last person joined: 6 days ago

Ask questions and share experiences about Junos OS.

Back to discussions

Expand all | Collapse all

Many (read/write?) errors during Junos OS update

1. Many (read/write?) errors during Junos OS update

0 Recommend
Rinklin Naturkost GmbH
Posted 11-27-2018 02:44

Reply Reply Privately
Hi there,

we just updated Junos OS to version 15.1R7.9. On two virtual chassis everything went fine. On the third one we saw a lot of errors during the process indicating disk problems. The messages often contains some output like

UNEXPECTED SOFT UPDATE INCONSISTENCY

It seems that the switch(es) autocorrect all the errors, and we ended up with a working and updated virtual chassis.

Nevertheless we have "mixed feelings" about this. So I like to know what we can do to check the storage hardware.

We did

root@ourswitch:RE:0% nand-mediack
Media check on da0 on ex platform

the output looks good, doesn't it? Does it check the media on all switches of the virtual chassis?

What do you suggest we should do/check?

Thanks for your support and many greets
Stephan

edit: I forgot to mention that this is about EX3300-Switches.
2. RE: Many (read/write?) errors during Junos OS update

0 Recommend
Nexon
Posted 11-28-2018 11:17

Reply Reply Privately
Hi there,

When you ran the command "nand-mediack", did you see any output?

If you din't see any it means that there were no bad blocks observed.

Had there been any, you would see the o/p like the below:

groza_lab@ex:RE:0% nand-mediack
Media check on da0 on non-srx platforms
Zone 02 Block 0172 Addr 08ac00 : Bad erase
Recovering block
Zone 07 Block 0520 Addr 1e0800 : Bad erase
Recovering block
Zone 07 Block 0631 Addr 1e7700 : Bad erase
Recovering block
Zone 08 Block 0856 Addr 235800 : Bad erase
Recovering block

You can check the boot-logs if there are any messages concerning the bad blocks or file system, if it is clean then there is nothing to worry upon...

Kudos are appreciated! 🙂

//Nex

3. RE: Many (read/write?) errors during Junos OS update

Recommend

Rinklin Naturkost GmbH

Posted 11-30-2018 05:54

Hi,

thanks for your reply!

I found some logs from the upgrade process (copy&paste&notepad++ are king! )

Here we go:

U-Boot 1.1.6 (Nov 22 2009 - 07:01:21)

Board: EX2200-24T-4G 4.12
EPLD:  Version 04.100311 (0x02)
DRAM:  Initializing (512MB)
Flash: 8 MB
USB:   scanning bus for devices... 3 USB Device(s) found
       scanning bus for storage devices... 1 Storage Device(s) found

ELF file is 32 bit
Consoles: U-Boot console

FreeBSD/arm U-Boot loader, Revision 1.0
(Wed Nov 18 14:02:39 PST 2009)
Memory: 512MB
Loading /boot/defaults/loader.conf
/kernel data=0x885980+0xdde78 syms=[0x4+0xf0b60+0x4+0xc81f4]


Hit [Enter] to boot immediately, or space bar for command prompt.
Booting [/kernel]...
Kernel entry at 0x1400100 ...
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
Copyright (c) 1996-2014, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
JUNOS 12.3R7.7 #0: 2014-06-12 14:14:29 UTC
    /volume/build/junos/12.3/release/12.3R7.7/obj-arm/junos/bsd/kernels/JUNIPER-EX-2200/kernel
can't re-use a leaf (all_slot_serialid)!
CPU: Feroceon 88FR131 rev 1 (Marvell core)
cpu53: Feroceon 88FR131 revision   WB enabled EABT branch prediction enabled
  16KB/32B 4-way Instruction cache
  16KB/32B 4-way write-back-locking-C Data cache
real memory  = 536870912 (512 MB)
avail memory = 503222272 (479 MB)
SOC: Marvell 88F6281 rev A0, TClock 200MHz
Security policy loaded: Junos MAC/veriexec (mac_veriexec)
MAC/veriexec fingerprint module loaded: SHA256
MAC/veriexec fingerprint module loaded: SHA1
ETHERNET SOCKET BRIDGE initialising
Initializing EXSERIES properties ...
mbus0: <Marvell Internal Bus (Mbus)> on motherboard
ic0: <Marvell Integrated Interrupt Controller> at mem 0xf1020200-0xf102023b on mbus0
timer0: <Marvell CPU Timer> at mem 0xf1020300-0xf102032f irq 1 on mbus0
gpio0: <Marvell Integrated GPIO Controller> at mem 0xf1010100-0xf101011f irq 35,36,37,38,39,40,41 on mbus0
uart0: <16550 or compatible> at mem 0xf1012000-0xf101201f irq 33 on mbus0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> at mem 0xf1012100-0xf101211f irq 34 on mbus0
ehci0: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1050000-0xf1050fff irq 48,19 on mbus0
usb0: EHCI version 1.0
usb0 on ehci0
usb0: USB revision 2.0
uhub0: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1: vendor 0x04cc product 0x1520, class 9/0, rev 2.00/2.00, addr 2
uhub1: single transaction translator
uhub1: 2 ports with 2 removable, self powered
umass0: STMicroelectronics ST72682  High Speed Mode, rev 2.00/2.10, addr 3
mge0: <Marvell Gigabit Ethernet controller> at mem 0xf1072000-0xf1073fff irq 12,13,14,11,46 on mbus0
miibus0: <MII bus> on mge0
e1000phy0: <Marvell 88E1118 Gigabit PHY> on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
i2c0: <Marvell I2C ARM OnChip Controller> at mem 0xf1011000-0xf101101f irq 29 on mbus0
syspld0: <SYSPLD> on i2c0
8564 rtc0: <8564 RTC> on i2c0
poe0: <POE> on i2c0
poe_attach: re-types method failed
device_attach: poe0 attach returned 19
cfi0: <SPI flash - 8MB> at mem 0xf1010600-0xf101062f,0xf8000000-0xf87fffff irq 23 on mbus0
mpfe0: <Juniper EX-series Packet Forwarding Engine> at mem 0xf4000000-0xf7ffffff irq 113 on mbus0
pcib0: <Marvell 88F6281 PCI-Express host controller> at mem 0xf1040000-0xf1041fff,0xe8000000-0xefffffff irq 9 on mbus0
pci0: <PCI bus> on pcib0
Initializing product: 75 ..
bmeb: bmeb_lib_init done 0xc3386800, addr 0xc1d5d7a4
bme0:Virtual BME driver initializing
Timecounter "CPU Timer" frequency 200000000 Hz quality 1000
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <ST ST72682 2.10> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Kernel thread "wkupdaemon" (pid 42) exited prematurely.
Trying to mount root from ufs:/dev/da0s2a
Invalid time in real time clock.
Check and reset the date immediately!
Attaching /packages/jbase via /dev/mdctl...
Mounted jbase package on /dev/md0...
Verified manifest signed by PackageProduction_12_3_0
Verified jboot signed by PackageProduction_12_3_0
Verified jbase-ex-12.3R7.7 signed by PackageProduction_12_3_0
Mounted fips-mode-arm package on /dev/md1...
Verified manifest signed by PackageProduction_12_3_0
Verified fips-mode-arm-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jcrypto-ex package on /dev/md2...
Verified manifest signed by PackageProduction_12_3_0
Verified jcrypto-ex-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jdocs-ex package on /dev/md3...
Verified manifest signed by PackageProduction_12_3_0
Verified jdocs-ex-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jkernel-ex-2200 package on /dev/md4...
Verified manifest signed by PackageProduction_12_3_0
Verified jkernel-ex-2200-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jpfe-ex22x package on /dev/md5...
Verified manifest signed by PackageProduction_12_3_0
Verified jpfe-ex22x-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jroute-ex package on /dev/md6...
Verified manifest signed by PackageProduction_12_3_0
Verified jroute-ex-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jswitch-ex package on /dev/md7...
Verified manifest signed by PackageProduction_12_3_0
Verified jswitch-ex-12.3R7.7 signed by PackageProduction_12_3_0
Mounted jweb-ex package on /dev/md8...
Verified manifest signed by PackageProduction_12_3_0
Verified jweb-ex-12.3R7.7 signed by PackageProduction_12_3_0
Executing /packages/mnt/jweb-ex-12.3R7.7/mount.post..
Automatic reboot in progress...
Media check on da0 on ex platforms
camcontrol: error sending command
(pass0:umass-sim0:0:0:0): Vendor Specific Command. CDB: f1
(pass0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(pass0:umass-sim0:0:0:0): SCSI Status: Check Condition
(pass0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(pass0:umass-sim0:0:0:0): Write protected
camcontrol: error sending command
** /dev/da0s2a
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 37042 free (10 frags, 4629 blocks, 0.0% fragmentation)
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 1a 5e 80 0 0 20 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
** /dev/da0s3e
** Last Mounted on /var
** Phase 1 - Check Blocks and Sizes
PARTIALLY TRUNCATED INODE I=12363
SALVAGE? yes


CANNOT WRITE BLK: 194176
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 194176, 194177, 194178, 194179, 194180, 194181, 194182, 194183, 194184, 194185, 194186, 194187, 194188, 194189, 194190, 194191, 194192, 194193, 194194, 194195, 194196, 194197, 194198, 194199, 194200, 194201, 194202, 194203, 194204, 194205, 194206, 194207,
-1 BAD I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

1 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

2 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

3 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

4 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

5 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

6 DUP I=12363
UNE(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 1a 5e 54 0 0 1 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
XPECTED SOFT UPDATE INCONSISTENCY

-1 BAD I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

1 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

2 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

3 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

4 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

5 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

EXCESSIVE DUP BLKS I=12363
CONTINUE? yes

INCORRECT BLOCK COU(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 1a 5e 55 0 0 1 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): ILLEGAL REQUEST asc:20,0
(da0:umass-sim0:0:0:0): Invalid command operation code
(da0:umass-sim0:0:0:0): Unretryable error
NT I=12363 (2592 should be 472)
CORRECT? yes

INTERNAL ERROR: dups with -p
UNEXPECTED SOFT UPDATE INCONSISTENCY
** Phase 1b - Rescan For More DUPS

CANNOT WRITE BLK: 192096
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 192096, 192097, 192098, 192099, 192100, 192101, 192102, 192103, 192104, 192105, 192106, 192107, 192108, 192109, 192110, 192111, 192112, 192113, 192114, 192115, 192116, 192117, 192118, 192119, 192120, 192121, 192122, 192123, 192124, 192125, 192126, 192127,
1 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

2 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

3 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

4 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

5 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENCY

6 DUP I=12363
UNEXPECTED SOFT UPDATE INCONSISTENC(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 1a 5e 57 0 0 1 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
Y

** Phase 2 - Check Pathnames
DIRECTORY CORRUPTED  I=12481  OWNER=39 MODE=40775
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY

SALVAGE? yes

MISSING '.'  I=12481  OWNER=39 MODE=40775
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY

FIX? yes

DIRECTORY CORRUPTED  I=12484  OWNER=0 MODE=41777
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY

SALVAGE? yes

MISSING '.'  I=12484  OWNER=0 MODE=41777
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=?

UNEXPECTED SOFT(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6c c0 0 0 20 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): ILLEGAL REQUEST asc:20,0
(da0:umass-sim0:0:0:0): Invalid command operation code
(da0:umass-sim0:0:0:0): Unretryable error
 UPDATE INCONSISTENCY

FIX? yes


CANNOT WRITE BLK: 194128
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 194128, 194129, 194130, 194131,

CANNOT WRITE BLK: 194132
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 194132, 194133, 194134, 194135,
DUP/BAD  I=12363  OWNER=0 MODE=100640
SIZE=1297280 MTIME=Sep 30 10:01 2014
FILE=/log/chassisd

UNEXPECTED SOFT UPDATE INCONSISTENCY

REMOVE? yes

MISSING '..'  I=12481  OWNER=39 MODE=40775
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=/log/ext

UNEXPECTED SOFT UPDATE INCONSISTENCY

FIX? yes

DIRECTORY CORRUPTED  I=12481  OWNER=39 MODE=40775
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY

SALVAGE? yes

MISSING '..'  I=12484  OWNER=0 MODE=41777
SIZE=51(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6c c1 0 0 1 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
2 MTIME=Sep 30 09:15 2014

CANNOT WRITE BLK: 255976
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 255976, 255977, 255978, 255979,

CANNOT WRITE BLK: 194128
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 194128, 194129, 194130, 194131,
DIR=/log/ggsn

UNEXPECTED SOFT UPDATE INCONSISTENCY

FIX? yes

DIRECTORY C(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6c c2 0 0 1 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): ILLEGAL REQUEST asc:20,0
(da0:umass-sim0:0:0:0): Invalid command operation code
(da0:umass-sim0:0:0:0): Unretryable error
ORRUPTED  I=12484  OWNER=0 MODE=41777
SIZE=512 MTIME=Sep 30 09:15 2014
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY

SALVAGE? yes

** Phase 3 - Check Connectivity
UNREF DIR  I=12485  OWNER=0 MODE=41777
SIZE=512 MTIME=Sep 30 09:15 2014
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6c c4 0 0 1 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
RECONNECT? yes


CANNOT WRITE BLK: 194132
UNEXPECTED SOFT UPWARNING: /var was not properly dismounted
DATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 194132, 194133, 194134, 194135,
DIR I=12485 CONNECTED. PARENT WAS I=12484

** Phase 4 - Check Reference Counts
BAD/DUP FILE I=12363  OWNER=0 MODE=100640
SIZE=1297280 MTIME=Sep 30 10:01 2014
CLEAR? yes


CANNOT WRITE BLK: 1216
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225, 1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241,(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6c 60 0 0 4 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): ILLEGAL REQUEST asc:20,0
(da0:umass-sim0:0:0:0): Invalid command operation code
(da0:umass-sim0:0:0:0): Unretryable error
 1242, 1243, 124g_vfs_done():da0s3e[WRITE(offset=573440, length=2048)]error = 22
4, 1245, 1246, 1247,
LINK COUNT DIR I=12484  OWNER=0 MODE=41777
SIZE=512 MTIME=Sep 30 09:15 2014  COUNT 3 SHOULD BE 2
ADJUST? yes

** Phase 5 - Check Cyl groups
FREE BLK COUNT(S) WRONG IN SUPERBLK
SALVAGE? yes

SUMMARY INFORMATION BAD
SALVAGE? yes

BLK(S) MISSING IN BIT MAPS
SALVAGE? yes


CANNOT WRITE BLK: 64
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 64, 65, 66, 67, 68, 69, 70, 71,
1107 files, 2485 used, 60417 free (265 frags, 7519 blocks, 0.4% fragmentation)

CANNOT WRITE BLK: 16
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,

CANNOT WRITE BLK: 1120
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 1120, 1121, 1122, 1123,

CANNOT WRITE BLK: 192064
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 192064, 192065, 192066, 192067, 192068, 192069, 192070, 192071,

CANNOT WRITE BLK: 197208
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 197208, 197209, 197210, 197211,

CANNOT WRITE BLK: 96
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127,

CANNOT WRITE BLK: 192096
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 192096, 192097, 192098, 192099, 192100, 192101, 192102, 192103, 192104, 192105, 192106, 192107, 192108, 192109, 192110, 192111, 192112, 192113, 192114, 192115, 192116, 192117, 192118, 192119, 192120, 192121, 192122, 192123, 192124, 192125, 192126, 192127,

CANNOT WRITE BLK: 192128
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 192128, 192129, 192130, 192131, 192132, 192133, 192134, 192135, 192136, 192137, 192138, 192139, 192140, 192141, 192142, 192143, 192144, 192145, 192146, 192147, 192148, 192149, 192150, 192151, 192152, 192153, 192154, 192155, 192156, 192157, 192158, 192159,

***** FILE SYSTEM STILL DIRTY *****

***** PLEASE RERUN FSCK *****
Computing slice and partition sizes for /dev/da0 ...
savecore: could not be determined
No dump exists
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6c 60 0 0 4 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
** /dev/da0s3d

CANNOT READ BLK: 128
CONTINUE? yes

THE FOLWARNING: /var/tmp was not properly dismounted
LOWING DISK SECTORS COULD NOT BE READ:
** Last Mounted on /var/tmp
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
10 files, 69 used, 188785 free (33 frags, 23594 blocks, 0.0% fragmentation)

CANNOT WRITE BLK: 16
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,

CANNOT WRITE BLK: 3104
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 3104, 3105, 3106, 3107,

***** FILE SYSTEM MARKED CLEAN *****
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 b b1 10 0 0 4 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
** /dev/da0s4d

CANNOT READ BLK: 128
CONTINUE? yes

THE FOLWARNING: /config was not properly dismounted
LOWING DISK SECTORS COULD NOT BE READ:
** Last Mounted on /config
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
42 files, 49 used, 31557 free (37 frags, 3940 blocks, 0.1% fragmentation)

CANNOT WRITE BLK: 16
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? yes

THE FOLLOWING SECTORS COULD NOT BE WRITTEN: 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,

CANNOT WRITE BLK: 608
UNEXPECTED SOFT UPDATE INCONSISTENCY

CONTINUE? (da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 1b 53 60 0 0 4 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
yes

THE FOLLOg_vfs_done():da0s4d[WRITE(offset=311296, length=2048)]error = 13
WING SECTORS COULD NOT BE WRITTEN: 608, 609, 610, 611,

***** FILE SYSTEM MARKED CLEAN *****
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 17 6d 60 0 0 20 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): ILLEGAL REQUEST asc:20,0
(da0:umass-sim0:0:0:0): Invalid command operation code
(da0:umass-sim0:0:0:0): Unretryable error
initiate_write_filepage: already started
initiate_write_filepage: already started
(da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 0 0 18 66 6c 0 0 4 0
(da0:umass-sim0:0:0:0): CAM Status: SCSI Status Error
(da0:umass-sim0:0:0:0): SCSI Status: Check Condition
(da0:umass-sim0:0:0:0): DATA PROTECT asc:27,0
(da0:umass-sim0:0:0:0): Write protected
(da0:umass-sim0:0:0:0): Unretryable error
g_vfs_done():da0s3e[WRITE(offset=33978368, length=2048)]error = 5
panic: initiate_write_inodeblock_ufs1: already started
Uptime: 5m14s
Cannot dump. No dump device defined.
Automatic reboot in 15 seconds - press a key on the console to abort
Rebooting...

Maybe this was "just" a filesystem issue that were fixed successfully?

The boot logs look fine, I would say:

root@ourswitch> show system boot-messages
fpc0:
--------------------------------------------------------------------------
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
Copyright (c) 1996-2018, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
JUNOS 15.1R7.9 #0: 2018-09-11 05:53:42 UTC
    builder@watatsumi.juniper.net:/volume/build/junos/15.1/release/15.1R7.9/obj/arm/junos/bsd/kernels/JUNIPER-EX-3300/kernel
can't re-use a leaf (all_slot_serialid)!
CPU: Early Feroceon 88FR571 rev 0 (Marvell core)
cpu55: Early Feroceon 88FR571 revision   WB enabled EABT branch prediction enabled
  32KB/32B 4-way Instruction cache
  32KB/32B 4-way write-back-locking-C Data cache
real memory  = 1073741824 (1024 MB)
avail memory = 1028313088 (980 MB)
SOC: Marvell MV78100, TClock 200MHz
Security policy loaded: Junos MAC/veriexec (mac_veriexec)
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
MAC/veriexec fingerprint module loaded: SHA1
MAC/veriexec fingerprint module loaded: SHA256
ETHERNET SOCKET BRIDGE initialising
Initializing EXSERIES properties ...
mbus0: <Marvell Internal Bus (Mbus)> on motherboard
ic0: <Marvell Integrated Interrupt Controller> at mem 0xf1020200-0xf102023b on mbus0
timer0: <Marvell CPU Timer> at mem 0xf1020300-0xf102032f irq 8 on mbus0
gpio0: <Marvell Integrated GPIO Controller> at mem 0xf1010100-0xf101011f irq 56,57,58,59 on mbus0
uart0: <16550 or compatible> at mem 0xf1012000-0xf101201f irq 12 on mbus0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> at mem 0xf1012100-0xf101211f irq 13 on mbus0
ehci0: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1050000-0xf1050fff irq 72,16 on mbus0
usb0: EHCI version 1.0
usb0 on ehci0
usb0: USB revision 2.0
uhub0: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
umass0: STMicroelectronics ST72682  High Speed Mode, rev 2.00/2.10, addr 2
ehci1: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1051000-0xf1051fff irq 72,17 on mbus0
usb1: EHCI version 1.0
usb1 on ehci1
usb1: USB revision 2.0
uhub1: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 1 port with 1 removable, self powered
ehci2: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1052000-0xf1052fff irq 72,18 on mbus0
usb2: EHCI version 1.0
usb2 on ehci2
usb2: USB revision 2.0
uhub2: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
mge0: <Marvell Gigabit Ethernet controller> at mem 0xf1072000-0xf1073fff irq 41,42,43,40,70 on mbus0
mge0: hardware MAC address 2c:21:31:c8:dc:bf
miibus0: <MII bus> on mge0
e1000phy0: <Marvell 88E1310 Gigabit PHY> on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
i2c0: <Marvell I2C ARM OnChip Controller> at mem 0xf1011000-0xf101101f on mbus0
8564 rtc0: <8564 RTC> on i2c0
poe0: <POE> on i2c0
i2c1: <Marvell I2C ARM OnChip Controller> at mem 0xf1011100-0xf101111f on mbus0
pcib0: <Marvell MV78100 PCI-Express host controller> at mem 0xf1040000-0xf1041fff,0xe8000000-0xebffffff on mbus0
pci0: <PCI bus> on pcib0
mpfe0: <Juniper EX-series Packet Forwarding Engine> mem 0xe8000000-0xebffffff irq 112 at device 1.0 on pci0
mpfe0: 0x100000 bytes of rid 0x10 res 3 failed (0, 0xffffffff).
syspld0 at mem 0xf9000000-0xf90fffff on mbus0
cfi0: <SPI flash - 8MB> at mem 0xf1010600-0xf101062f,0xf8800000-0xf8ffffff irq 1 on mbus0
Initializing product: 113 ..
Timecounter "CPU Timer" frequency 200000000 Hz quality 1000
Registered AMT tunnel Encap with UDP Tunnel!
 Loading Redundant LT driver
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <ST ST72682 2.10> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Kernel thread "wkupdaemon" (pid 45) exited prematurely.
Trying to mount root from ufs:/dev/da0s1a

fpc1:
--------------------------------------------------------------------------
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
Copyright (c) 1996-2018, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
JUNOS 15.1R7.9 #0: 2018-09-11 05:53:42 UTC
    builder@watatsumi.juniper.net:/volume/build/junos/15.1/release/15.1R7.9/obj/arm/junos/bsd/kernels/JUNIPER-EX-3300/kernel
can't re-use a leaf (all_slot_serialid)!
CPU: Early Feroceon 88FR571 rev 0 (Marvell core)
cpu55: Early Feroceon 88FR571 revision   WB enabled EABT branch prediction enabled
  32KB/32B 4-way Instruction cache
  32KB/32B 4-way write-back-locking-C Data cache
real memory  = 1073741824 (1024 MB)
avail memory = 1028313088 (980 MB)
SOC: Marvell MV78100, TClock 200MHz
Security policy loaded: Junos MAC/veriexec (mac_veriexec)
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
MAC/veriexec fingerprint module loaded: SHA1
MAC/veriexec fingerprint module loaded: SHA256
ETHERNET SOCKET BRIDGE initialising
Initializing EXSERIES properties ...
mbus0: <Marvell Internal Bus (Mbus)> on motherboard
ic0: <Marvell Integrated Interrupt Controller> at mem 0xf1020200-0xf102023b on mbus0
timer0: <Marvell CPU Timer> at mem 0xf1020300-0xf102032f irq 8 on mbus0
gpio0: <Marvell Integrated GPIO Controller> at mem 0xf1010100-0xf101011f irq 56,57,58,59 on mbus0
uart0: <16550 or compatible> at mem 0xf1012000-0xf101201f irq 12 on mbus0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> at mem 0xf1012100-0xf101211f irq 13 on mbus0
ehci0: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1050000-0xf1050fff irq 72,16 on mbus0
usb0: EHCI version 1.0
usb0 on ehci0
usb0: USB revision 2.0
uhub0: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
umass0: STMicroelectronics ST72682  High Speed Mode, rev 2.00/2.10, addr 2
ehci1: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1051000-0xf1051fff irq 72,17 on mbus0
usb1: EHCI version 1.0
usb1 on ehci1
usb1: USB revision 2.0
uhub1: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 1 port with 1 removable, self powered
ehci2: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1052000-0xf1052fff irq 72,18 on mbus0
usb2: EHCI version 1.0
usb2 on ehci2
usb2: USB revision 2.0
uhub2: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
mge0: <Marvell Gigabit Ethernet controller> at mem 0xf1072000-0xf1073fff irq 41,42,43,40,70 on mbus0
mge0: hardware MAC address 2c:21:31:c8:c5:7f
miibus0: <MII bus> on mge0
e1000phy0: <Marvell 88E1310 Gigabit PHY> on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
i2c0: <Marvell I2C ARM OnChip Controller> at mem 0xf1011000-0xf101101f on mbus0
8564 rtc0: <8564 RTC> on i2c0
poe0: <POE> on i2c0
i2c1: <Marvell I2C ARM OnChip Controller> at mem 0xf1011100-0xf101111f on mbus0
pcib0: <Marvell MV78100 PCI-Express host controller> at mem 0xf1040000-0xf1041fff,0xe8000000-0xebffffff on mbus0
pci0: <PCI bus> on pcib0
mpfe0: <Juniper EX-series Packet Forwarding Engine> mem 0xe8000000-0xebffffff irq 112 at device 1.0 on pci0
mpfe0: 0x100000 bytes of rid 0x10 res 3 failed (0, 0xffffffff).
syspld0 at mem 0xf9000000-0xf90fffff on mbus0
cfi0: <SPI flash - 8MB> at mem 0xf1010600-0xf101062f,0xf8800000-0xf8ffffff irq 1 on mbus0
Initializing product: 113 ..
Timecounter "CPU Timer" frequency 200000000 Hz quality 1000
Registered AMT tunnel Encap with UDP Tunnel!
 Loading Redundant LT driver
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <ST ST72682 2.10> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Kernel thread "wkupdaemon" (pid 45) exited prematurely.
Trying to mount root from ufs:/dev/da0s1a

fpc2:
--------------------------------------------------------------------------
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
Copyright (c) 1996-2018, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
JUNOS 15.1R7.9 #0: 2018-09-11 05:53:42 UTC
    builder@watatsumi.juniper.net:/volume/build/junos/15.1/release/15.1R7.9/obj/arm/junos/bsd/kernels/JUNIPER-EX-3300/kernel
can't re-use a leaf (all_slot_serialid)!
CPU: Early Feroceon 88FR571 rev 0 (Marvell core)
cpu55: Early Feroceon 88FR571 revision   WB enabled EABT branch prediction enabled
  32KB/32B 4-way Instruction cache
  32KB/32B 4-way write-back-locking-C Data cache
real memory  = 1073741824 (1024 MB)
avail memory = 1028313088 (980 MB)
SOC: Marvell MV78100, TClock 200MHz
Security policy loaded: Junos MAC/veriexec (mac_veriexec)
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
MAC/veriexec fingerprint module loaded: SHA1
MAC/veriexec fingerprint module loaded: SHA256
ETHERNET SOCKET BRIDGE initialising
Initializing EXSERIES properties ...
mbus0: <Marvell Internal Bus (Mbus)> on motherboard
ic0: <Marvell Integrated Interrupt Controller> at mem 0xf1020200-0xf102023b on mbus0
timer0: <Marvell CPU Timer> at mem 0xf1020300-0xf102032f irq 8 on mbus0
gpio0: <Marvell Integrated GPIO Controller> at mem 0xf1010100-0xf101011f irq 56,57,58,59 on mbus0
uart0: <16550 or compatible> at mem 0xf1012000-0xf101201f irq 12 on mbus0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> at mem 0xf1012100-0xf101211f irq 13 on mbus0
ehci0: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1050000-0xf1050fff irq 72,16 on mbus0
usb0: EHCI version 1.0
usb0 on ehci0
usb0: USB revision 2.0
uhub0: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
umass0: STMicroelectronics ST72682  High Speed Mode, rev 2.00/2.10, addr 2
ehci1: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1051000-0xf1051fff irq 72,17 on mbus0
usb1: EHCI version 1.0
usb1 on ehci1
usb1: USB revision 2.0
uhub1: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 1 port with 1 removable, self powered
ehci2: <88F5XXX Integrated USB 2.0 controller> at mem 0xf1052000-0xf1052fff irq 72,18 on mbus0
usb2: EHCI version 1.0
usb2 on ehci2
usb2: USB revision 2.0
uhub2: Marvell EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
mge0: <Marvell Gigabit Ethernet controller> at mem 0xf1072000-0xf1073fff irq 41,42,43,40,70 on mbus0
mge0: hardware MAC address 2c:21:31:c8:ea:7f
miibus0: <MII bus> on mge0
e1000phy0: <Marvell 88E1310 Gigabit PHY> on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
i2c0: <Marvell I2C ARM OnChip Controller> at mem 0xf1011000-0xf101101f on mbus0
8564 rtc0: <8564 RTC> on i2c0
poe0: <POE> on i2c0
i2c1: <Marvell I2C ARM OnChip Controller> at mem 0xf1011100-0xf101111f on mbus0
pcib0: <Marvell MV78100 PCI-Express host controller> at mem 0xf1040000-0xf1041fff,0xe8000000-0xebffffff on mbus0
pci0: <PCI bus> on pcib0
mpfe0: <Juniper EX-series Packet Forwarding Engine> mem 0xe8000000-0xebffffff irq 112 at device 1.0 on pci0
mpfe0: 0x100000 bytes of rid 0x10 res 3 failed (0, 0xffffffff).
syspld0 at mem 0xf9000000-0xf90fffff on mbus0
cfi0: <SPI flash - 8MB> at mem 0xf1010600-0xf101062f,0xf8800000-0xf8ffffff irq 1 on mbus0
Initializing product: 113 ..
Timecounter "CPU Timer" frequency 200000000 Hz quality 1000
Registered AMT tunnel Encap with UDP Tunnel!
 Loading Redundant LT driver
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <ST ST72682 2.10> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Kernel thread "wkupdaemon" (pid 45) exited prematurely.
Trying to mount root from ufs:/dev/da0s1a

And yes - nand-mediack have no output, so I think it's fine, too.

Do you think there is anything to worry about?

Thanks a lot for your support and many greets

Stephan

Junos OS

Many (read/write?) errors during Junos OS update

Rinklin Naturkost GmbH11-27-2018 02:44

Nexon11-28-2018 11:17

Rinklin Naturkost GmbH11-30-2018 05:54

1. Many (read/write?) errors during Junos OS update

2. RE: Many (read/write?) errors during Junos OS update

3. RE: Many (read/write?) errors during Junos OS update