STB Suite

May 2010

8.2 0 New Features, Updates, Release Notes
New Products

  • New version of STBTrace driver
    • Improved error/exception catching
  • Multi-threaded Jukebox test GUI application
    • A new test which simultaneously tests a library and up to eight tape drives. The test does random moves of media within the library, loads and unloads media from all drives under test, and executes user defined write tests to all drives.
    • Support for testing mailboxes
  • Command Probability Sequencer GUI Application
    • The CPS GUI Application functions allow specifying a list of CDB’s, assigning a probability percentage to each CDB. The CDB’s in the list are then issued to as many drives as are specified, according to each CDB’s probability. One use is to simulate real-world I/O environments where “random” CDB’s can be asyncronously executed in the middle of I/O sequences.
  • New SCSI/FC/SAS DoD Purge DMM External test
    • 3x speed improvement

Developer Toolbox (DTB)

  • Command Probability Sequencer into VCPSSL
    • The CPS functions allow specifying a list of CDB’s, assigning a probability percentage to each CDB. The CDB’s in the list are then issued to as many drives as are specified, according to each CDB’s probability. One use is to simulate real-world I/O environments where “random” CDB’s can be asynchronously executed in the middle of I/O sequences.
  • Add new API to tell you the current driver and to be able to set the driver

 

Bus Analyzer Module (BAM) Bug Fixes

  • The “file-save” function is fixed so it doesn’t have duplicate phases at the end of a trace file.
  • In Performance Analysis and IO Statistic pages, only devices that have been selected are used for the data displayed

SCSItoolbox Suite (STB) Additional Features & Tests

  • Converted all “LBA” numbers to 64-bit numbers
  • Added dedicated Mode Page edit page for Background Functions
    • (Page 0x1C subpage 1)
  • Added dedicated Mode Page edit page for POWER CONDITON
    • (page 0x1A)

SCSItoolbox Suite (STB) Bug Fixes

  • Fixed DMM SATA command bug with LSI 6G HBAs
  • Fixed SAS WWN bug

 

Manufacturing Module (DMM) Additional Features & Tests

  • Save any test thresholds (SMART/Performance/Log Pages) to seq file
    • All thresholds are now stored and reloaded with test sequence files.
  • Drive performance profiling using statistics
    • I/O performance (transfer rates and CDB completion times) and number of blocks transferred per test are logged to the DMM log files.
      • Example:

        Transfer Rate: 22.06 MB/sec

        I/O Per Second: 353.01 IO/sec

        Number of Blocks Transferred: 144410880

        Fastest Command Completion Time: 1.205 ms

        Slowest Command Completion Time: 78.234 ms

        Average Command Completion Time: 2.183 ms

        Standard Deviation of Command Completion Times: 0.447 ms

  • Converted all “LBA” numbers to 64-bit numbers
  • Record all EVPD pages External test
    • All INQUIRY EVPD pages of all drives saved to text file
  • SCSI/FC/SAS Seek time External test
    • Full Stroke, Third-stroke, and single-track seek times recorded to log files
  • SATA SET FEATURES External test
    • Allows any SATA FEATURE to be set or reset
  • Added “Key” Overlay and “Time” Overlay
    • On the Advanced Options dialog, you can input a 32-bit value (the “key”) to all blocks of data. The “key” overlay is placed after the LBA overlay. In addition to the “key” overlay, you can also have a “Time” overlay – the current time will be overlaid on each block. The “Time” overlay is placed after the “key” overlay. If a data compare is being done on blocks that have a key/time overlay, you must input the key/time-frame for the block. The “time-frame” consists of a time the data was written PLUS a time-window.

      For example, if the data written was on Apr 3, 2010 15:39 then you could enter on the Advanced Options dialog that the time the data was written was Apr 3, 2010 12:00 Plus/Minus 4 hours.

 

 

 

Ready for your 8.2 STB Suite Upgrade?
STB 8.2 Ready?In order to install and use STB Suite version 8.2 you must have current Performa coverage. To check your Performa status do the following on each system, and with each of your USB license keys.

 

    1. Run the program “STBCustomerKeyUpdateVer81.exe” (Download available here:STBCustomerKeyUpdateVer81.zip)
    2. Check the Performa Expiration Date field as shown here:

  1. As long as your Performa Expiration date is in the future you will be able to install and run STB Suite version 8.2
  2. If your Performa Expiration Date is showing a date in the past you should contactsales@scsitoolbox.com or call 720-249-2641 to renew your Performa coverage.

 

 

 

Ask Dr. SCSI – How to analyze log files efficiently
The real Dr. SCSIQ.“I see these detailed log files generated by DMM – there is lots of good information in them. But – I test many drives at a time…how can I automate analyzing the data in these files?”

A. “That’s a good question – the answer is you need to use your computer to analyze these files for you”

Introduction

A real-world example occurred recently – a customer was using an application which was called as a DMM test step (using the External Program test type) that calculated the single-track, third-stroke, and full-stroke seek times for drives. The application writes the results of these measurements back into the DMM log files for each drive tested like so:

STB DMM Seek Test Results:

Test Run 02/15/10 10:28:54

Single-track Seek Time = 2.79 ms

Third-Stroke Seek time = 8.96 ms

Full-Stroke Seek time = 14.74 ms

Dr. SCSI has been around computers for a long time – but had never tried any of the common text processing languages/tools. This was a perfect opportunity to learn about them while solving a real customer issue. Dr. SCSI loves to solve problems, and loves to share those solutions – so here is a rare glimpse into Dr. SCSI’s diary …“Dear Diary, today I decided to learn to use Python…”

Why Python?

It’s free, it’s well documented with lots of tutorials available on-line, it can be interactive and thus is very easy to learn and use, it was designed as a language for dealing with text… plenty of good reasons to use it.

Where Python?

Go to www.python.org in the download section. You’ll have to decide whether you want to get the older version 2.6.4 or the newer 3.1.1. Most of the tutorials I found were based on v2. There are significant syntactical differences between the two – but I decided since I was starting from the ground up I would try the newest version. Once I learned the differences it wasn’t too hard to adjust the older tutorials to the new syntax.

Go to the Documentation section of the web site – there is a LOT available, you can read it online or download it to read offline. Get the tutorial for the version you decided on.

Install your Python package, bring up the tutorial and work through it.

Solving the problem

Dr. SCSI can’t teach you all about program design, how to define the problem and steps to the solution, and the details of how to implement the solution in Python…but he can tell you that if you have done any programming before and have these foundation skill you will find Python very easy to learn, and you will find that the solutions can be surprisingly easy to implement. Even if you are starting from scratch and have never programmed before you can have success by following the tutorials!

Because it is important it stands repeating here – get the tutorials and walk through them.

By means of our example we will jump forward one day in Dr. SCSI’s diary and see a real-world solution to the “analyzing the seek time in logs” problem. Please note – Dr. SCSI had one day of Python experience when this little tool was written. It most likely is not elegant, and may not even be in “good” Python form. But – it does work.

Here is the script – it will find all “.log” files in its folder. It will prompt the operator to enter the seek time thresholds for each of the three seek time measurements. It then walks through all the .log files, finds the seek information, does the comparison with the thresholds, and prints an error message to the screen indicating the drive that failed and why it failed.

import glob

import os

import time

KEYWORDS = “Full-Stroke”,”Single-track”,”Third-Stroke”

print(“**************** Start of log-file scan “,time.strftime(‘%m/%d/%Y %H:%M:%S’),”***********************”)

Singlelimit = float(input(“Enter Single-Track seek time cutoff = “))

Thirdlimit = float(input(“Enter Third-Stroke seek time cutoff = “))

Fulllimit = float(input(“Enter Full-Stroke seek time cutoff = “))

logfiles = glob.glob(‘*.log’)

for x in range(0,len(logfiles)):

print(‘\n****************\nLog file = ‘,logfiles[x])

logfile = open(logfiles[x], “r”).readlines()

for line in logfile:

if line.find(‘Device’) != -1:

driveid = line

logfile = open(logfiles[x], “r”).readlines()

for line in logfile:

for word in line.split():

if word == “Full-Stroke”:

t = line.split(‘=’)[1].split(‘ms’)[0].split(‘ ‘)

tf = float(t[1])

# print (line)

if tf > Fulllimit:

print (‘\n***Drive ‘,driveid,”*****FAILED Full-Stroke Time”,tf,’ – number is too high – cutoff is’,Fulllimit)

if word == “Third-Stroke”:

t = line.split(‘=’)[1].split(‘ms’)[0].split(‘ ‘)

tf = float(t[1])

# print (line)

if tf > Thirdlimit:

print (‘\n***Drive ‘,driveid,”*****FAILED Third-Stroke Time”,tf,’ – number is too high – cutoff is’,Thirdlimit)

if word == “Single-track”:

t = line.split(‘=’)[1].split(‘ms’)[0].split(‘ ‘)

tf = float(t[1])

# print (line)

if tf > Singlelimit:

print (‘\n***Drive ‘,driveid,”*****FAILED Single-track Time”,tf,’ – number is too high – cutoff is’,Singlelimit)

print(‘\n*****NEXT FILE********\n\n’)

print(‘\n\nFinished **************************************’)

print(“**************** End of log-file scan “,time.strftime(‘%m/%d/%Y %H:%M:%S’),”***********************\n\n”)

 

Look at the results

Here is a look at what the little script outputs:

**************** Start of log-file scan 02/22/2010 10:17:22 ***********************

Enter Single-Track seek time cutoff = 6

Enter Third-Stroke seek time cutoff = 9

Enter Full-Stroke seek time cutoff = 14

****************

Log file = 03-001-00-3EK2NPM500002250Z589.Log

***Drive Device: 3:1:0 IBM-ESXS ST373405LC !# Serial: 3EK2NPM500002250Z589

*****FAILED Full-Stroke Time 14.48 – number is too high – cutoff is 14.0

*****NEXT FILE********

****************

Log file = 03-008-00-3EK1DZ8100007239GC5D.Log

***Drive Device: 3:8:0 IBM-ESXS ST373405LC !# Serial: 3EK1DZ8100007239GC5D

*****FAILED Full-Stroke Time 14.74 – number is too high – cutoff is 14.0

****************

Log file = 04-006-00.Log

*****NEXT FILE********

Finished **************************************

**************** End of log-file scan 02/22/2010 10:17:32 ***********************

Summary & Challange

Quick, easy, versatile – Python definitely rules for analyzing DMM log files!

44 lines of code and the seek time analysis/reporting problem has a solution.

Load it up, play with it – you’ll be an expert in no time!

Now here’s the challenge

DMM log files record the transfer rate for read and write tests.

The problem: screen drives based on read and write transfer rate, and report some statistics.

The first person who sends in a Python example that:

  1. Screens on transfer rate similar to the seek time example
  2. Has the operator enter the date and time range to analyze – for the case where a log file contains multiple test runs
  3. prints the high, average, and low transfer rate of all .log files in a folder, and prints the drive information for those three metrics
  4. and prints a total showing how many .log files were processed

…will receive a one year free Performa upgrade*!

That’s worth a bunch! You will also have your name and script published in the STB Newsletter – you will be famous! People will be calling from around the world to ask you programming questions – no wait, forget you heard that…

Ready – set – go!

 

 

 

DMM Add-on Grab bag!
STB Suite 8.1 iconIntroduction

One of the most powerful features of the STB Suite Disk Manufacturing Module (DMM) is the External Program Test Step. This feature allows you to call any executable program as a DMM test step.

This allows DMM to be simply and easily expanded to incorporate new tests.

If you need a custom test step – anything that isn’t present in the existing Test Step types – just add your own!

Some free extensions with version 8.2

Several of these new DMM external tests are available to customers running STB Suite version 8.2. Here is the list – details for each are below:

  1. SCSI/FC/SAS Seek Profile Test
  2. SCSI/FC/SAS SuperFast DoD Disk Purge
  3. EVPD Dump Test
  4. SATA Set Features Test

SCSI/FC/SAS Seek Profile Test

This test measures and logs a drives seek times. Single-track, Third-Stroke, and Full-Stroke Seeks are executed and measured, with the results being entered into the DMM log files. Here is an example:

STB DMM Seek Test Results:

Test Run 04/20/10 09:10:54

Single-track Seek Time = 1.83 ms

Third-Stroke Seek time = 5.82 ms

Full-Stroke Seek time = 9.84 ms

Since the SATA drive specification has retired the SEEK command this test will not give real times when run against SATA drives – sorry!

 

SCSI/FC/SAS Super-Fast DoD Purge Test

This test accelerates the DoD 5220 Purge speed. Purges execute 2-4 times faster!

Here is an example DMM log file after purging a 36G drive. This example was run on 23 drives at once –

================================================================================

>> SCSI Toolbox32, Version 8. 2. 0 (build) 100415, running on \\STB-BS-S2003 <<

================================================================================

Test Date: 04/17/2010 11:26:37

Test Pass: 1

Device: 7:6:0 SEAGATE ST318304FC Serial: 3EL005MD0000704301YL

Results:

04/17/2010 11:26:37 TEST 1 of 2:

External Program Test, executable = C:\dmmwritesame.exe

04/17/2010 11:56:23 Test Completed Successfully

Return code from External Program: 00000000

String Data from the External Program follows:

Executed Dod Purge Process – per Dod 5220.22-M specification (version 1.0)

04/17/2010 11:56:23 PASSED

——————————————————————————

04/17/2010 11:56:23 TEST 2 of 2:

Read Test; Sequential; for 35964301 Blocks

Fixed-Length Transfers of 128 (0x0080) Blocks

Start Block: 0

Data Pattern: Random; Data Compare ON

Queue Depth = 1

FUA = OFF

04/17/2010 12:16:55 Test Completed Successfully

Transfer Rate: 14.25 MB/sec

I/O Per Second: 228.06 IO/sec

Number of Blocks Transferred: 35964301

Fastest Command Completion Time: 1.017 ms

Slowest Command Completion Time: 16.190 ms

Average Command Completion Time: 3.598 ms

Standard Deviation of Command Completion Times: 1.311 ms

04/17/2010 12:16:55 PASSED

——————————————————————————

Notice the logged times – the purge was begun at 11:26 and completed at 12:16 – 50 minutes for 3 full write passes plus one full read pass with data compare/verify!

 

 

SATA Set Features Test

This test allows you to set or reset any SATA FEATURE – such as WRITE CACHE, setting ACOUSTIC MANAGEMENT or POWER MANAGEMENT, etc. If the FEATURE you chose to manipulate needs a COUNT field you may enter that on the command line parameters line.

The FEATURE which you change is logged to the DMM log files – here’s the example of turning WRITE CACHE on:

EVPD Dump Test

This test dumps all of a disk drives INQUIRY EVPD pages to a text file. EVPD pages contain all kinds of useful information – have a look at the T10 specification or your drives manufacturers drive documentation to see what is available. Here is an example of what you might see:

************************************

Output from DMM GetEVPD program Version 1.0- 04/06/10 13:47:00

Drive = SEAGATE, ST373455SS, Serial Number 3LQ1271W

EVPD Pages reported by drive = 0x00 0x80 0x82 0x83 0xC0 0xC1 0xC2 0xC3 0xD1 0xD2 0xDC

Full INQUIRY Data Page – Length = 0x00

0000: 00 00 00 0B 00 80 82 83 C0 C1 C2 C3 D1 D2 DC 00 …………….

EVPD Page 0x00 – Length = 0x0B

0000: 00 00 00 0B 00 80 82 83 C0 C1 C2 C3 D1 D2 DC 00 …………….

EVPD Page 0x80 – Length = 0x08

0000: 00 80 00 08 33 4C 51 31 32 37 31 57 00 00 00 00 ….3LQ1271W….

EVPD Page 0x82 – Length = 0x1D

0000: 00 82 00 1D 1C 54 31 30 2F 31 34 31 36 2D 44 20 …..T10.1416.D.

0001: 52 65 76 69 73 69 6F 6E 20 37 20 20 20 20 20 20 Revision.7……

0002: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

EVPD Page 0x83 – Length = 0x48

0000: 00 83 00 48 01 03 00 08 50 00 C5 00 04 94 5C AF …H….P…….

0001: 61 93 00 08 50 00 C5 00 04 94 5C AD 61 94 00 04 a…P…….a…

0002: 00 00 00 01 61 A3 00 08 50 00 C5 00 04 94 5C AC ….a…P…….

0003: 63 A8 00 18 6E 61 61 2E 35 30 30 30 43 35 30 30 c…naa.5000C500

0004: 30 34 39 34 35 43 41 43 00 00 00 00 00 00 00 00 04945CAC……..

EVPD Page 0xC0 – Length = 0x38

0000: 00 C0 00 38 31 32 31 33 30 35 31 33 32 30 30 36 …8121305132006

0001: 44 34 39 30 30 30 30 30 30 30 30 30 44 34 39 30 D49000000000D490

0002: 32 30 30 36 32 30 30 36 44 34 39 32 44 34 39 32 20062006D492D492

0003: 32 30 30 36 30 30 30 30 32 31 30 33 00 00 00 00 200600002103….

EVPD Page 0xC1 – Length = 0x10

0000: 00 C1 00 10 30 35 31 39 32 30 30 37 31 32 31 33 ….051920071213

0001: 32 30 30 36 00 00 00 00 00 00 00 00 00 00 00 00 2006…………

EVPD Page 0xC2 – Length = 0x02

0000: 00 C2 00 02 00 00 00 00 00 00 00 00 00 00 00 00 …………….

EVPD Page 0xC3 – Length = 0xF6

0000: 00 C3 00 F6 04 10 00 53 54 33 37 33 34 35 35 53 …….ST373455S

0001: 53 20 20 20 20 20 20 01 20 FF 00 8A 00 79 11 71 S…………y.q

0002: 00 00 00 00 FF 00 04 0F 03 21 00 00 00 00 00 00 …………….

0003: 00 00 00 00 00 00 00 00 00 80 00 42 10 00 00 A0 ………..B….

0004: 80 00 00 00 00 E0 10 00 80 00 08 50 E0 00 00 E2 ………..P….

0005: 00 00 00 80 00 00 00 60 00 00 0D 00 00 00 00 00 …………….

0006: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

0007: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

0008: 00 00 00 01 00 00 17 17 00 00 16 82 08 00 08 00 …………….

0009: 0E 00 00 41 04 4C 28 28 14 12 14 0B 05 0B 0B 00 …A.L……….

000a: F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

000b: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

000c: 80 C0 09 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

000d: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

000e: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

000f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

EVPD Page 0xD1 – Length = 0xF0

0000: 00 D1 00 F0 39 5A 33 30 36 36 2D 30 35 31 20 20 ….9Z3066.051..

0001: 20 20 20 20 31 30 30 34 34 34 33 31 39 20 20 20 ….100444319…

0002: 20 20 20 20 45 45 37 34 33 35 41 43 41 53 20 20 ….EE7435ACAS..

0003: 20 20 20 20 34 34 37 33 33 30 36 43 4D 34 34 35 ….4473306CM445

0004: 37 30 30 30 45 37 34 33 35 41 43 41 53 31 30 31 7000E7435ACAS101

0005: 41 44 42 31 30 30 34 35 31 30 34 32 41 20 20 20 ADB100451042A…

0006: 20 20 20 20 31 30 30 33 37 38 31 39 35 20 20 20 ….100378195…

0007: 20 20 20 20 31 30 30 33 37 38 31 39 36 20 20 20 ….100378196…

0008: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

0009: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000a: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000b: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000c: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000d: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000e: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000f: 20 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 …………….

EVPD Page 0xD2 – Length = 0xF0

0000: 00 D2 00 F0 32 30 30 36 44 34 39 30 20 20 20 20 ….2006D490….

0001: 20 20 20 20 31 32 31 33 30 35 31 33 20 20 20 20 ….12130513….

0002: 20 20 20 20 31 30 30 34 32 37 33 39 36 20 20 20 ….100427396…

0003: 20 20 20 20 45 20 20 20 20 20 20 20 20 20 20 20 ….E………..

0004: 20 20 20 20 39 37 34 35 43 4E 48 44 20 20 20 20 ….9745CNHD….

0005: 20 20 20 20 37 34 35 20 20 20 20 20 20 20 20 20 ….745………

0006: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

0007: 20 20 20 20 33 4C 51 31 32 37 31 57 30 30 30 30 ….3LQ1271W0000

0008: 39 37 34 35 20 20 20 20 20 20 20 20 20 20 20 20 9745…………

0009: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000a: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 …………….

000b: 20 20 20 20 30 30 4C 34 31 37 34 20 20 20 20 20 ….00L4174…..

000c: 54 30 37 20 31 35 20 20 20 20 20 20 20 20 20 20 T07.15……….

000d: 20 20 20 20 57 45 4A 20 20 20 20 20 20 20 20 20 ….WEJ………

000e: 20 20 20 20 30 30 4D 36 39 30 35 20 20 20 20 30 ….00M6905….0

000f: 37 31 32 20 00 00 00 00 00 00 00 00 00 00 00 00 712………….

EVPD Page 0xDC – Length = 0x1E

0000: 00 DC 01 1E 44 45 4C 4C 20 20 20 20 53 54 33 37 ….DELL….ST37

0001: 33 34 35 35 53 53 20 20 20 20 20 20 53 35 31 33 3455SS……S513

0002: 33 4C 51 31 32 37 31 57 20 20 20 20 20 20 20 20 3LQ1271W……..

Summary

To request any of these DMM external tests please contact Jeremy Wolfe at jeremyw@scsitoolbox.com or 720-249-2641. Have your four-digit serial number found on your USB key available. These tests require that your Performa coverage is current and that you are running STB Suite version 8.2.

 

 

 

Customized training with an STB technician

STB TrainingDo you have questions about how to best use the STB Suite in your business? STB is happy to work with you in an interactive “live” environment to help you get the most out of your Toolbox. The cost? If you are a current Performa customer it is free! The commitment? Training sessions run between 30 and 60 minutes.

Here is a list of some recent customer training sessions that STB has conducted – live, interactive web sessions presented by STB programmers:

  1. SSD Manufacturing
  2. How to troubleshoot tape drive problems
  3. DOD disk purging
  4. Multi-drive SATA firmware downloading with the STB Suite
  5. Compliance testing

Contact Jeremy Wolfe at (720) 249-2641 today to schedule your own custom training session!