Dying Hard Disk
From Leo's Notes
This page was last edited on 19 May 2017, at 07:36.
May 18, 2017
Here's a dmesg log of another dying disk.
May 17 15:43:18 nas kernel: ata14.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
May 17 15:43:18 nas kernel: ata14.00: failed command: READ DMA EXT
May 17 15:43:18 nas kernel: ata14.00: cmd 25/00:08:88:08:f6/00:00:3c:00:00/e0 tag 0 dma 4096 in#012 res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
May 17 15:43:18 nas kernel: ata14.00: status: { DRDY }
May 17 15:43:18 nas kernel: ata14: hard resetting link
May 17 15:43:18 nas kernel: ata14: nv: skipping hardreset on occupied port
May 17 15:43:23 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 17 15:43:28 nas kernel: ata14: SRST failed (errno=-16)
May 17 15:43:28 nas kernel: ata14: hard resetting link
May 17 15:43:28 nas kernel: ata14: nv: skipping hardreset on occupied port
May 17 15:43:33 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 17 15:43:38 nas kernel: ata14: SRST failed (errno=-16)
May 17 15:43:38 nas kernel: ata14: hard resetting link
May 17 15:43:38 nas kernel: ata14: nv: skipping hardreset on occupied port
May 17 15:43:43 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 17 15:44:13 nas kernel: ata14: SRST failed (errno=-16)
May 17 15:44:13 nas kernel: ata14: limiting SATA link speed to 1.5 Gbps
May 17 15:44:13 nas kernel: ata14: hard resetting link
May 17 15:44:13 nas kernel: ata14: nv: skipping hardreset on occupied port
May 17 15:44:13 nas kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 17 15:44:13 nas kernel: ata14.00: revalidation failed (errno=-2)
May 17 15:44:19 nas kernel: ata14: hard resetting link
May 17 15:44:19 nas kernel: ata14: nv: skipping hardreset on occupied port
May 17 15:44:19 nas kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 17 15:44:19 nas kernel: ata14.00: revalidation failed (errno=-2)
May 17 15:44:24 nas kernel: ata14: hard resetting link
May 17 15:44:24 nas kernel: ata14: nv: skipping hardreset on occupied port
May 17 15:44:25 nas kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 17 15:44:25 nas kernel: ata14.00: revalidation failed (errno=-2)
May 17 15:44:25 nas kernel: ata14.00: disabled
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] tag#0 Sense Key : Not Ready [current]
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] tag#0 Add. Sense: Logical unit not ready, hard reset required
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: Read(10) 28 00 3c f6 08 88 00 00 08 00
May 17 15:44:25 nas kernel: blk_update_request: I/O error, dev sdi, sector 1022757000
May 17 15:44:25 nas kernel: ata14: EH complete
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Write Protect is on
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Write Protect is off
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Write Protect is on
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Write Protect is off
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:44:25 nas kernel: sd 13:0:0:0: [sdi] Sense not available.
May 17 15:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 15:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 16:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 16:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 16:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 16:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 17:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 17:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 17:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 17:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 18:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 18:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 18:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 18:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 19:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 19:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 19:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 19:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 20:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 20:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 20:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 20:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 20:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 20:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
May 17 21:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 21:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 21:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 21:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 22:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 22:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 22:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 22:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 23:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 23:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 17 23:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 17 23:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 00:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 00:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 00:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 00:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 01:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 01:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 01:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 01:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 02:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 02:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 02:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 02:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
May 18 02:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 02:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 03:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 03:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 03:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 03:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 04:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 04:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 04:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 04:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 05:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 05:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 05:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 05:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 06:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 06:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 06:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 06:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 07:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 07:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 07:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 07:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 07:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 07:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
May 18 08:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 08:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 08:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 08:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 09:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 09:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 09:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 09:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 10:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 10:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 10:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 10:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 11:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 11:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 11:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 11:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 12:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 12:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 12:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 12:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 13:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 13:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 13:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 13:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
May 18 13:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 13:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 14:17:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 14:17:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 14:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 14:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 15:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 15:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 15:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 15:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 16:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 16:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 16:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 16:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 17:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 17:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 17:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 17:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 18:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 18:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 18:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 18:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 18:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 18:47:20 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
May 18 19:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 19:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 19:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 19:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 20:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 20:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 20:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 20:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 21:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 21:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 21:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 21:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 22:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 22:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 22:47:22 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 22:47:22 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 23:17:22 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 23:17:22 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 18 23:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 18 23:47:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 19 00:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 19 00:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 00 00 00 00 00 00 00 00 00 00 e5 00
May 19 00:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 19 00:17:21 nas kernel: sd 13:0:0:0: [sdi] tag#0 CDB: ATA command pass through(16) 85 06 2c 00 da 00 00 00 00 00 4f 00 c2 00 b0 00
May 19 00:51:51 nas kernel: ata14: SATA max UDMA/133 cmd 0x960 ctl 0xb60 bmdma 0xc808 irq 23
May 19 00:51:56 nas kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 19 00:51:56 nas kernel: ata14.00: ATA-8: ST2000DM001-9YN164, CC4C, max UDMA/133
May 19 00:51:56 nas kernel: ata14.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 0/32)
May 19 00:51:56 nas kernel: ata14.00: configured for UDMA/133
May 19 00:51:56 nas kernel: sd 12:0:0:0: [sdi] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
May 19 00:51:56 nas kernel: sd 12:0:0:0: [sdi] 4096-byte physical blocks
May 19 00:51:56 nas kernel: sd 12:0:0:0: [sdi] Write Protect is off
May 19 00:51:56 nas kernel: sd 12:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
May 19 00:51:56 nas kernel: sdi: sdi1
May 19 00:51:56 nas kernel: sd 12:0:0:0: [sdi] Attached SCSI disk
May 19 00:53:41 nas kernel: ata14.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
May 19 00:53:41 nas kernel: ata14.00: failed command: IDENTIFY DEVICE
May 19 00:53:41 nas kernel: ata14.00: cmd ec/00:01:00:00:00/00:00:00:00:00/00 tag 0 pio 512 in#012 res 40/00:01:09:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
May 19 00:53:41 nas kernel: ata14.00: status: { DRDY }
May 19 00:53:41 nas kernel: ata14: hard resetting link
May 19 00:53:41 nas kernel: ata14: nv: skipping hardreset on occupied port
May 19 00:53:47 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 19 00:53:49 nas kernel: ata14: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
May 19 00:53:49 nas kernel: ata14.00: configured for UDMA/133
May 19 00:53:49 nas kernel: ata14: EH complete
May 19 00:54:45 nas kernel: ata14.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
May 19 00:54:45 nas kernel: ata14.00: failed command: IDENTIFY DEVICE
May 19 00:54:45 nas kernel: ata14.00: cmd ec/00:01:00:00:00/00:00:00:00:00/00 tag 0 pio 512 in#012 res 40/00:01:09:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
May 19 00:54:45 nas kernel: ata14.00: status: { DRDY }
May 19 00:54:45 nas kernel: ata14: hard resetting link
May 19 00:54:45 nas kernel: ata14: nv: skipping hardreset on occupied port
May 19 00:54:51 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 19 00:54:55 nas kernel: ata14: SRST failed (errno=-16)
May 19 00:54:55 nas kernel: ata14: hard resetting link
May 19 00:54:55 nas kernel: ata14: nv: skipping hardreset on occupied port
May 19 00:55:01 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 19 00:55:05 nas kernel: ata14: SRST failed (errno=-16)
May 19 00:55:05 nas kernel: ata14: hard resetting link
May 19 00:55:05 nas kernel: ata14: nv: skipping hardreset on occupied port
May 19 00:55:11 nas kernel: ata14: link is slow to respond, please be patient (ready=0)
May 19 00:55:40 nas kernel: ata14: SRST failed (errno=-16)
May 19 00:55:40 nas kernel: ata14: limiting SATA link speed to 1.5 Gbps
May 19 00:55:40 nas kernel: ata14: hard resetting link
May 19 00:55:40 nas kernel: ata14: nv: skipping hardreset on occupied port
May 19 00:55:45 nas kernel: ata14: SRST failed (errno=-16)
May 19 00:55:45 nas kernel: ata14: reset failed, giving up
May 19 00:55:45 nas kernel: ata14.00: disabled
May 19 00:55:45 nas kernel: ata14: EH complete
May 19 01:06:47 nas kernel: ata14: SATA max UDMA/100 mmio m1024@0xfd7ff000 tf 0xfd7ff2c0 irq 17
May 19 01:06:48 nas kernel: sd 9:0:0:0: [sdi] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
May 19 01:06:48 nas kernel: sd 9:0:0:0: [sdi] 4096-byte physical blocks
May 19 01:06:48 nas kernel: sd 9:0:0:0: [sdi] Write Protect is off
May 19 01:06:48 nas kernel: sd 9:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
May 19 01:06:48 nas kernel: sd 9:0:0:0: [sdi] Attached SCSI disk
May 19 01:06:49 nas kernel: ata14: SATA link down (SStatus 0 SControl 310)
May 19 01:13:54 nas kernel: sdi:
May 19 01:15:52 nas kernel: sdi: sdi1
An older Entry
I noticed that one of the drives in my ZFS array went down last month, but I couldn't bother myself with fixing it because it's only one drive and it's acting like a raid 0 (right?).
Unfortunately, my backups failed for a few files and then this turned up:
root@nas:/mnt/nas4/images/christine/2015-01-25 Annual Dinner- we host# zpool status -xv
pool: data
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://zfsonlinux.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 5h45m with 0 errors on Tue Jun 17 00:13:18 2014
config:
NAME STATE READ WRITE CKSUM
data DEGRADED 0 0 6.31K
raidz1-0 DEGRADED 0 0 13.2K
ata-ST3000DM001-1CH166_Z1F4HHZ7 ONLINE 0 0 0
ata-ST3000DM001-1CH166_Z1F4HGS6 ONLINE 0 0 0
ata-ST3000DM001-9YN166_W1F0SJ27 FAULTED 1 2 0 too many errors
errors: Permanent errors have been detected in the following files:
data:<0x0>
/data/images/christine/2015-01-25 Annual Dinner- we host/IMG_3255.JPG
/data/images/christine/2015-01-25 Annual Dinner- we host/IMG_3256.JPG
/data/images/christine/2015-01-25 Annual Dinner- we host/IMG_3257.JPG
/data/images/christine/2015-01-25 Annual Dinner- we host/IMG_3258.JPG
/data/images/christine/2015-01-25 Annual Dinner- we host/IMG_3259.JPG
It seems like I lost 5 images that were recently saved on the storage pool. The drive is so far gone that I couldn't even run smartctl on it:
smartctl 5.43 2012-06-30 r3573 [x86_64-linux-2.6.32-431.17.1.el6.x86_64] (local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net
Vendor: /8:0:0:0
Product:
User Capacity: 600,332,565,813,390,450 bytes [600 PB]
Logical block size: 774843950 bytes
>> Terminate command early due to bad response to IEC mode page
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
The logs show the drive failing:
2268 Jan 25 22:13:54 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0xd0000 action 0xe frozen
2269 Jan 25 22:13:54 nas kernel: ata7: SError: { PHYRdyChg CommWake 10B8B }
2270 Jan 25 22:13:54 nas kernel: ata7.00: failed command: READ DMA EXT
2271 Jan 25 22:13:54 nas kernel: ata7.00: cmd 25/00:00:6f:7c:55/00:02:37:00:00/e0 tag 0 dma 262144 in
2272 Jan 25 22:13:54 nas kernel: res d8/d8:d8:d8:d8:d8/d8:d8:d8:d8:d8/d8 Emask 0x12 (ATA bus error)
2273 Jan 25 22:13:54 nas kernel: ata7.00: status: { Busy }
2274 Jan 25 22:13:54 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2275 Jan 25 22:13:54 nas kernel: ata7: hard resetting link
2276 Jan 25 22:13:59 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2277 Jan 25 22:13:59 nas kernel: ata7.00: configured for UDMA/33
2278 Jan 25 22:13:59 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2279 Jan 25 22:13:59 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2280 Jan 25 22:13:59 nas kernel: Descriptor sense data with sense descriptors (in hex):
2281 Jan 25 22:13:59 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d8 d8
2282 Jan 25 22:13:59 nas kernel: d8 d8 d8 d8
2283 Jan 25 22:13:59 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2284 Jan 25 22:13:59 nas kernel: sd 6:0:0:0: [sdf] CDB: Read(10): 28 00 37 55 7c 6f 00 02 00 00
2285 Jan 25 22:13:59 nas kernel: ata7: EH complete
2286 Jan 25 22:15:09 nas kernel: NFS: v4 server 10.1.1.6 does not accept raw uid/gids. Reenabling the idmapper.
2287 Jan 25 22:15:41 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x90000 action 0xe frozen
2288 Jan 25 22:15:41 nas kernel: ata7: SError: { PHYRdyChg 10B8B }
2289 Jan 25 22:15:41 nas kernel: ata7.00: failed command: READ DMA EXT
2290 Jan 25 22:15:41 nas kernel: ata7.00: cmd 25/00:80:c0:2d:c0/00:01:65:00:00/e0 tag 0 dma 196608 in
2291 Jan 25 22:15:41 nas kernel: res d8/d8:d8:d8:d8:d8/d8:d8:d8:d8:d8/d8 Emask 0x12 (ATA bus error)
2292 Jan 25 22:15:41 nas kernel: ata7.00: status: { Busy }
2293 Jan 25 22:15:41 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2294 Jan 25 22:15:41 nas kernel: ata7: hard resetting link
2295 Jan 25 22:15:46 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2296 Jan 25 22:15:46 nas kernel: ata7.00: configured for UDMA/33
2297 Jan 25 22:15:46 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2298 Jan 25 22:15:46 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2299 Jan 25 22:15:46 nas kernel: Descriptor sense data with sense descriptors (in hex):
2300 Jan 25 22:15:46 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d8 d8
2301 Jan 25 22:15:46 nas kernel: d8 d8 d8 d8
2302 Jan 25 22:15:46 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2303 Jan 25 22:15:46 nas kernel: sd 6:0:0:0: [sdf] CDB: Read(10): 28 00 65 c0 2d c0 00 01 80 00
2304 Jan 25 22:15:46 nas kernel: ata7: EH complete
2305 Jan 25 22:15:46 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0xd0000 action 0xe frozen
2306 Jan 25 22:15:46 nas kernel: ata7: SError: { PHYRdyChg CommWake 10B8B }
2307 Jan 25 22:15:46 nas kernel: ata7.00: failed command: READ DMA EXT
2308 Jan 25 22:15:46 nas kernel: ata7.00: cmd 25/00:00:40:2f:c0/00:03:65:00:00/e0 tag 0 dma 393216 in
2309 Jan 25 22:15:46 nas kernel: res d8/d8:d8:d8:d8:d8/d8:d8:d8:d8:d8/d8 Emask 0x12 (ATA bus error)
2310 Jan 25 22:15:46 nas kernel: ata7.00: status: { Busy }
2311 Jan 25 22:15:46 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2312 Jan 25 22:15:46 nas kernel: ata7: hard resetting link
2313 Jan 25 22:15:51 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2314 Jan 25 22:15:51 nas kernel: ata7.00: configured for UDMA/33
2315 Jan 25 22:15:51 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2316 Jan 25 22:15:51 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2317 Jan 25 22:15:51 nas kernel: Descriptor sense data with sense descriptors (in hex):
2318 Jan 25 22:15:51 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d8 d8
2319 Jan 25 22:15:51 nas kernel: d8 d8 d8 d8
2320 Jan 25 22:15:51 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2321 Jan 25 22:15:51 nas kernel: sd 6:0:0:0: [sdf] CDB: Read(10): 28 00 65 c0 2f 40 00 03 00 00
2322 Jan 25 22:15:51 nas kernel: ata7: EH complete
2323 Jan 25 22:15:51 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0xe frozen
2324 Jan 25 22:15:51 nas kernel: ata7: SError: { PHYRdyChg }
2325 Jan 25 22:15:51 nas kernel: ata7.00: failed command: READ DMA EXT
2326 Jan 25 22:15:51 nas kernel: ata7.00: cmd 25/00:14:70:0f:5e/00:00:68:00:00/e0 tag 0 dma 10240 in
2327 Jan 25 22:15:51 nas kernel: res d0/d0:d0:d0:d0:d0/d0:d0:d0:d0:d0/d0 Emask 0x12 (ATA bus error)
2328 Jan 25 22:15:51 nas kernel: ata7.00: status: { Busy }
2329 Jan 25 22:15:51 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2330 Jan 25 22:15:51 nas kernel: ata7: hard resetting link
2331 Jan 25 22:15:56 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2332 Jan 25 22:15:56 nas kernel: ata7.00: configured for UDMA/33
2333 Jan 25 22:15:56 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2334 Jan 25 22:15:56 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2335 Jan 25 22:15:56 nas kernel: Descriptor sense data with sense descriptors (in hex):
2336 Jan 25 22:15:56 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d0 d0
2337 Jan 25 22:15:56 nas kernel: d0 d0 d0 d0
2338 Jan 25 22:15:56 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2339 Jan 25 22:15:56 nas kernel: sd 6:0:0:0: [sdf] CDB: Read(10): 28 00 68 5e 0f 70 00 00 14 00
2340 Jan 25 22:15:56 nas kernel: ata7: EH complete
2341 Jan 25 22:15:57 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0xd0000 action 0xe frozen
2342 Jan 25 22:15:57 nas kernel: ata7: SError: { PHYRdyChg CommWake 10B8B }
2343 Jan 25 22:15:57 nas kernel: ata7.00: failed command: READ DMA EXT
2344 Jan 25 22:15:57 nas kernel: ata7.00: cmd 25/00:14:77:10:5e/00:00:68:00:00/e0 tag 0 dma 10240 in
2345 Jan 25 22:15:57 nas kernel: res d8/d8:d8:d8:d8:d8/d8:d8:d8:d8:d8/d8 Emask 0x12 (ATA bus error)
2346 Jan 25 22:15:57 nas kernel: ata7.00: status: { Busy }
2347 Jan 25 22:15:57 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2348 Jan 25 22:15:57 nas kernel: ata7: hard resetting link
2349 Jan 25 22:16:02 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2350 Jan 25 22:16:02 nas kernel: ata7.00: configured for UDMA/33
2351 Jan 25 22:16:02 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2352 Jan 25 22:16:02 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2353 Jan 25 22:16:02 nas kernel: Descriptor sense data with sense descriptors (in hex):
2354 Jan 25 22:16:02 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d8 d8
2355 Jan 25 22:16:02 nas kernel: d8 d8 d8 d8
2356 Jan 25 22:16:02 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2357 Jan 25 22:16:02 nas kernel: sd 6:0:0:0: [sdf] CDB: Read(10): 28 00 68 5e 10 77 00 00 14 00
2358 Jan 25 22:16:02 nas kernel: ata7: EH complete
2359 Jan 25 22:16:02 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0xd0000 action 0xe frozen
2360 Jan 25 22:16:02 nas kernel: ata7: SError: { PHYRdyChg CommWake 10B8B }
2361 Jan 25 22:16:02 nas kernel: ata7.00: failed command: READ DMA EXT
2362 Jan 25 22:16:02 nas kernel: ata7.00: cmd 25/00:80:c0:3a:c0/00:02:65:00:00/e0 tag 0 dma 327680 in
2363 Jan 25 22:16:02 nas kernel: res d8/d8:d8:d8:d8:d8/d8:d8:d8:d8:d8/d8 Emask 0x12 (ATA bus error)
2364 Jan 25 22:16:02 nas kernel: ata7.00: status: { Busy }
2365 Jan 25 22:16:02 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2366 Jan 25 22:16:02 nas kernel: ata7: hard resetting link
2367 Jan 25 22:16:07 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2368 Jan 25 22:16:07 nas kernel: ata7.00: configured for UDMA/33
2369 Jan 25 22:16:07 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2370 Jan 25 22:16:07 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2371 Jan 25 22:16:07 nas kernel: Descriptor sense data with sense descriptors (in hex):
2372 Jan 25 22:16:07 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d8 d8
2373 Jan 25 22:16:07 nas kernel: d8 d8 d8 d8
2374 Jan 25 22:16:07 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2375 Jan 25 22:16:07 nas kernel: sd 6:0:0:0: [sdf] CDB: Read(10): 28 00 65 c0 3a c0 00 02 80 00
2376 Jan 25 22:16:07 nas kernel: ata7: EH complete
2377 Jan 25 22:16:07 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0xe frozen
2378 Jan 25 22:16:07 nas kernel: ata7: SError: { PHYRdyChg }
2379 Jan 25 22:16:07 nas kernel: ata7.00: failed command: WRITE DMA EXT
2380 Jan 25 22:16:07 nas kernel: ata7.00: cmd 35/00:03:d6:04:48/00:00:69:00:00/e0 tag 0 dma 1536 out
2381 Jan 25 22:16:07 nas kernel: res d0/d0:d0:d0:d0:d0/d0:d0:d0:d0:d0/d0 Emask 0x12 (ATA bus error)
2382 Jan 25 22:16:07 nas kernel: ata7.00: status: { Busy }
2383 Jan 25 22:16:07 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2384 Jan 25 22:16:07 nas kernel: ata7: hard resetting link
2385 Jan 25 22:16:12 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2386 Jan 25 22:16:12 nas kernel: ata7.00: configured for UDMA/33
2387 Jan 25 22:16:12 nas kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2388 Jan 25 22:16:12 nas kernel: sd 6:0:0:0: [sdf] Sense Key : Aborted Command [current] [descriptor]
2389 Jan 25 22:16:12 nas kernel: Descriptor sense data with sense descriptors (in hex):
2390 Jan 25 22:16:12 nas kernel: 72 0b 47 00 00 00 00 0c 00 0a 80 00 00 00 d0 d0
2391 Jan 25 22:16:12 nas kernel: d0 d0 d0 d0
2392 Jan 25 22:16:12 nas kernel: sd 6:0:0:0: [sdf] Add. Sense: Scsi parity error
2393 Jan 25 22:16:12 nas kernel: sd 6:0:0:0: [sdf] CDB: Write(10): 2a 00 69 48 04 d6 00 00 03 00
2394 Jan 25 22:16:12 nas kernel: ata7: EH complete
2395 Jan 25 22:16:13 nas kernel: ata7: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0xe frozen
2396 Jan 25 22:16:13 nas kernel: ata7: SError: { PHYRdyChg }
2397 Jan 25 22:16:13 nas kernel: ata7: hard resetting link
2398 Jan 25 22:16:18 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2399 Jan 25 22:16:18 nas kernel: ata7.00: configured for UDMA/33
2400 Jan 25 22:16:18 nas kernel: ata7: EH complete
2401 Jan 25 22:16:24 nas kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x90000 action 0xe frozen
2402 Jan 25 22:16:24 nas kernel: ata7: SError: { PHYRdyChg 10B8B }
2403 Jan 25 22:16:24 nas kernel: ata7.00: failed command: READ DMA EXT
2404 Jan 25 22:16:24 nas kernel: ata7.00: cmd 25/00:00:c0:0b:c3/00:03:65:00:00/e0 tag 0 dma 393216 in
2405 Jan 25 22:16:24 nas kernel: res d8/d8:d8:d8:d8:d8/d8:d8:d8:d8:d8/d8 Emask 0x12 (ATA bus error)
2406 Jan 25 22:16:24 nas kernel: ata7.00: status: { Busy }
2407 Jan 25 22:16:24 nas kernel: ata7.00: error: { ICRC UNC IDNF }
2408 Jan 25 22:16:24 nas kernel: ata7: hard resetting link
2409 Jan 25 22:16:29 nas kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
2410 Jan 25 22:16:29 nas kernel: ata7.00: configured for UDMA/33
... this goes on for another 12 minutes.
So, lesson learned. Test backups and replace dead drives ASAP.