VTL - some of the VTL Prime processes do not start and stop->starting the VTL services causes Solaris to crash (Doc ID 1528302.1)

Last updated on OCTOBER 05, 2016

Applies to:

Sun StorageTek VTL Prime System - Version 1.0 - Build 1813 to 1.1 - Build 2076 [Release 1.0]
Oracle Solaris on x86-64 (64-bit)
- some of the VTL processes do not start and stop->starting the VTL causes Solaris to crash

[SERVER STATUS]

Sun Microsystems VTL Server v5.10 (Build 2076)
Copyright (c) 2003-2009 FalconStor Software. All Rights Reserved.
Status of VTL SNMPD Module..................... [RUNNING]
Status of VTL System Helper Module............. [RUNNING]
Status of VTL QLogic Module.................... [RUNNING]
Status of VTL Authentication Module............ [RUNNING]
Status of VTL Server (Compression) Module...... [RUNNING]
Status of VTL Server (FSNBase) Module.......... [RUNNING]
Status of VTL Server (Upcall) Module........... [RUNNING]
Status of VTL Server (Event) Module............ [RUNNING]
Status of VTL Server (Path Manager) Module..... [RUNNING]
Status of VTL Server (Application)............. [RUNNING]
Status of VTL Server VTL Upcall Module......... [RUNNING]
Status of VTL Server VTL Upcall Daemon......... [RUNNING]
Status of VTL Server VTL Module................ [RUNNING]
Status of VTL Server Hosted Backup Module...... [RUNNING]
Status of VTL FC Target Module................. [RUNNING]
Status of VTL iSCSI Target Module.............. [RUNNING]
Status of VTL iSCSI (Daemon)................... [RUNNING]
Status of VTL Server IMA Daemon................ [STOPPED]
Status of VTL Server RDE Daemon................ [STOPPED]
Status of VTL Communication Module............. [STOPPED]
Status of VTL CLI Proxy Module................. [STOPPED]
Status of VTL Logger Module.................... [STOPPED]
Status of VTL Email Alerts Module.............. [STOPPED]
Status of VTL SIR Module....................... [STOPPED]
Status of VTL SIR Node......................... [STOPPED]
Status of VTL FSVSHOST Module.................. [RUNNING]
Status of VTL SIR GC Dispatcher................ [STOPPED]
Status of VTL Self Monitor Module.............. [STOPPED]

---


"Adapter 6" LUNs which are currently inacessible by VTL

The below logs show up repeatedly, please check the storage on adapter 6

messages:Feb 8 14:42:18 VTL01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: [COMMAND = 9E] [SCSI_ADDR = 6 0 0 36] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
messages:Feb 8 14:42:31 VTL01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: (Repeated once) [COMMAND = ReadCapacity] [SCSI_ADDR = 6 0 0 36] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
messages:Feb 8 14:42:31 VTL01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: (Repeated once) [COMMAND = 9E] [SCSI_ADDR = 6 0 0 36] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
messages:Feb 8 14:42:35 VTL01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: [COMMAND = ReadCapacity] [SCSI_ADDR = 6 0 0 37] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
messages:Feb 8 14:42:35 VTL01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: [COMMAND = 9E] [SCSI_ADDR = 6 0 0 37] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
messages:Feb 8 14:42:48 VTL01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: (Repeated once) [COMMAND = ReadCapacity] [SCSI_ADDR = 6 0 0 37] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]



The adapter is








Reviewed logs and as pointed out issue appears to be with Adapter 6. This started back on Feb 4:

================
Feb 4 21:11:45 E0100DaBckVtl01 qla2x00fs: [ID 924796 kern.notice] hba(1): PORT_UPDATE reset LOOP_DOWN flag
Feb 4 21:11:45 E0100DaBckVtl01 qla2x00fs: [ID 973166 kern.notice] start_srb(1), loop_transition=262h,iocb_cmd=1h
Feb 4 21:11:45 E0100DaBckVtl01 qla2x00fs: [ID 389396 kern.notice] qla_configure_loop(1) call qla_configure_hba
Feb 4 21:11:46 E0100DaBckVtl01 qla2x00fs: [ID 657045 kern.notice] qla_configure_loop(1) call qla_configure_local_loop
Feb 4 21:11:46 E0100DaBckVtl01 qla2x00fs: [ID 746498 kern.notice] <<<<< qla(1): LOOP or N_Port-to-N_Port skips config_fabric >>>>>
Feb 4 21:11:46 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>IOCORE1 log_scsi_error, acsl 6 0 0 40, cmd 2A 00 01 07 AF 3F 00 04 00 00 00 00 00 00 00 00 target 01 host 00 driver 08 sense 72 06 29 04 00 00 00 B7 00 0A 80 00 00 00 00 00 len 524288 ofs 0 type 2 in 0
Feb 4 21:11:46 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: [COMMAND = Write10] [SCSI_ADDR = 6 0 0 40] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Unit attention] [ASC.ASCQ = 29.04]
================

This continued and after reboot, still occurring:

===============

Feb 8 14:06:32 E0100DaBckVtl01 genunix: [ID 540533 kern.notice] ^MSunOS Release 5.10 Version Generic_120012-14 64-bit
Feb 8 14:06:32 E0100DaBckVtl01 genunix: [ID 943907 kern.notice] Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved.
...
Feb 8 14:31:56 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>IOCORE1 log_scsi_error, acsl 6 0 0 27, cmd 25 60 00 00 00 00 00 00 00 00 00 00 00 00 00 00 target 01 host 00 driver 08 sense 72 02 04 01 00 00 00 B7 00 0A 80 00 00 00 00 00 len 8 ofs 0 type 1 in 1
Feb 8 14:31:56 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: [COMMAND = ReadCapacity] [SCSI_ADDR = 6 0 0 27] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
...
Feb 8 14:42:35 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>IOCORE1 log_scsi_error, acsl 6 0 0 37, cmd 9E B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 target 01 host 00 driver 08 sense 72 02 04 01 00 00 00 B7 00 0A 80 00 00 00 00 00 len 32 ofs 0 type 1 in 1
Feb 8 14:42:35 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>FSNALIAS alias_perform_readcap(): 6:0:0:37 capacity failed totalsectors 0 sectorsize 0
Feb 8 14:42:35 E0100DaBckVtl01 genunix: [ID 885766 kern.notice] <6>IOCORE1 SCSI_ERROR: [COMMAND = 9E] [SCSI_ADDR = 6 0 0 37] [HOST_STAT = No error] [DRIVER_STAT = Additional sense] [TARGET_STAT = Check condition] [SENSE = Device not ready] [ASC.ASCQ = Unit becoming ready]
Feb 8 14:42:37 E0100DaBckVtl01 iscsi: [ID 702911 daemon.notice] IPSTOR||1360352557||I||0x0000c351||Login to the target %1 from the initiator %2||iqn.2000-03.com.sun:vtl.E0100DaBckVtl01.E0001DaBckVtl01-SIRReplica-2||iqn.2000-03.com.sun:vtl.E001DaBckVtl01.4ab9d16a
Feb 8 14:42:37 E0100DaBckVtl01 fsiscsi: [ID 359633 kern.info] fffffe8002f60c80 SocketGetPeername: so_faddr_sa(0x3c032f0a:a6aa)
Feb 8 14:42:37 E0100DaBckVtl01 fsiscsi: [ID 359633 kern.info] fffffe8002f60c80 SocketGetPeername: ipaddr(0x3c032f0a:a6aa)
Feb 8 14:42:37 E0100DaBckVtl01 fsiscsi: [ID 359633 kern.info] fffffe8002f60c80 client 8 initiator iqn.2000-03.com.sun:vtl.E001DaBckVtl01.4ab9d16a type 2 login request to target iqn.2000-03.com.sun:vtl.E0100DaBckVtl01.E0001DaBckVtl01-SIRReplica-2 from 10.47.3.60, conn 49679488, new tcp session
Feb 8 14:42:37 E0100DaBckVtl01 fsiscsi: [ID 359633 kern.info] fffffe8002f60c80 conn 49679488 assume reconnect from client due to ExpCmdRN=433.

Feb 8 14:42:40 E0100DaBckVtl01 fsiscsi: [ID 359633 kern.info] fffffe8002f60c80 conn 49679488 session info still not loaded

Symptoms

VTL not all modules are starting

VTL services not starting completely

Solaris creates a panic and core dumps

Changes

 None

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms