Dual Port testing for SAS and FC Devices

STB Suite | The Industry Standard in Peripheral Testing.

Introduction

All SAS and Fibre Channel Enterprise disk drives are built with dual-port capability.

Dual-Port means that the drive has two independent physical connections to which the Initiator or host system can connect. Each of the drives’ ports has a separate PHY connection which has its own unique PHY address and WWN.

The host system can  connect to either port, or to both ports, and can maintain I/O to/from each port independently and simultaneously.

Also, the connection through one of the ports may “claim” the drive by using the RESERVE CDB. When one port has issued a RESERVE to the drive then the other port will appear as “off line” and any commands sent to it will be aborted with a SCSI RESERVATION CONFLICT message. We will show an example illustrating this later in the article.

Connecting and testing dual-ported FC drives is mostly identical to connecting & testing dual-ported SAS drives. We will reference SAS drives in this article.

You may also have dual-ported SCSI drives & enclosures. These are a much older technology, but if you do need to test in that environment everything in this article should apply.

Also note that SATA drives by themselves to not have dual-ports available. However you may have SATA drives which connect to your dual-ported drive enclosure using a “SAS Interposer”. This is a little card which plugs into the back data & power connectors on a SATA drive and simulates a dual-port setup. This can work, but we hear of customers having many issues and problems when using these interposers in their test environment. Especially when running very high-speed I/O tests.

Just beware that interposers can cause intermittent problems.

Connecting to both ports

The Enclosure

In order to connect to both ports on the drive you must use a drive enclosure which supports dual-port connections. There will be two sets of connectors on this enclosure, one for Port A or 0, the other for Port B or 1. In addition most drive enclosures will have an “In” and an “Out” connection for each port. Do not confuse the “Out” connector of Port A with the Port B connector. Your enclosure should have these connectors labled.

Cabling

You will need a separate cable to connect each Port of your drive enclosure to an HBA. Be sure to follow all cable quality & speed recommendations for each port. In other words, if you are connecting a 12G SAS HBA to 12G SAS drives in a 12G enclosure be sure to only use 12G rated cables for each port.

HBA(s)

You can connect to each port in a few different ways.

Separate Independent HBAs

If you have to separate HBAs in your test system you can connect each one of them to one port on your drive enclosure.

This would also be the method you would use to connect two test systems to a dual ported enclosure.

Bridged or Concatenated HBAs

Most two-port SAS HBAs will concatenate the drives connected to both HBA ports and present a view which shows all drives and all ports as if they are all on one SAS bus or connection.

In this case each drive will appear two times in the device display – like this

Each port of each drive appears as a separate Target in the device display.

For example, selecting Target 263 will access the physical ST4000 drive, port A.

And selecting Target 264 will access the same physical drive but going through Port B.

Another connection option

You could also connect to both ports using only one HBA SAS or FC port by bridging a cable from your enclosure Port A “Out” connector to your enclosures Port B “In” connector.

 

A few Test Examples

1. Checking Dual-Port information

You will use the STB Suite device display, the Mode Page Edit/View function, and the Log Page view function to examine information about the dual-port status of your drive.

Each port will have its own WWN which you can check. You will also be able to see the WWN of the SAS port that each drive port is connected to.

1. First, double-click on one of your drive ports (each port will be a separate Target #) in your STB Suite device display – in this example we will be examining the drive connected at Targets 263 (ST4000 Port A) and Target 264 (same ST4000, Port B).

Double-clicking on Target 263 shows –

Note what the SAS/WWN is for this drive/port.

2. Now, right-click on the same Target in the device display and choose Edit Mode Pages. Choose Page 0x19 SAS/SSP Page and click Edit Page. This will show detailed information about each port of your drive –

Note that Phy 0 (Port A) shows the same SAS Address as we saw in step #1 above.

In addition you can see the address of the SAS PHY that this drive/port is connected to. This will be a PHY in the SAS Expander in your enclosure.

You will also see what the minimum and maximum allowable connection speed of each drive/port is as well as what speed the drive/port and HBA have actually negotiated to.

Click OK and then Return to return to the device display.

 

 

3. Again Right-Click on the target and this time choose View Log Pages. Your drive should have a Log Page 18 – Protocol Specific Page which will show information about each port which has been recorded during drive use.

Again, the SAS address of each port is displayed along with various error counters for each port. You can scroll to see the information for each port and you can save the information on both ports to a text file.

Checking this Log Page before and after a test session can provide very valuable information as far as connection problems which can cause loss of sync, invalid SAS data, etc. If your drive/port is showing high numbers of errors accumulating during testing you should test the drive in a different enclosure slot to determine if the drive has a problem or if you have a bad or flakey enclosure slot.

 

 

 

 

2. Testing RESERVE/RELEASE

This example will test that your dual-ported drive correctly implements the RESERVE & RELEASE CDBs.

Test steps:

1. Right-Click on the Target representing Port A, and choose User Defined CDBs.

2. Scroll through the list of CDBs and double-click on the RESERVE CDB. This will populate the CDB bytes and values like this –

Click the Send CDB button to issue the CDB to the drive/port.

 

 

You should see this CDB Result –

3. Click the Scan Bus button – the Target representing Port B should now show Off Line

4. Right-Click on the Target for Port B – Target 264 in our case – and choose User Defined CDBs again. Scroll down the CDB list to the Test Unit Ready CDB and double-click it. Then click the Send CDB button.

The CDB Result should show the following failure –

This shows that the RESERVE command we issued to Port A did work and in fact did reserve the drive to be only accessible via Port A until the drive is either power-cycled or freed with a corresponding RELEASEcommand.

You can go back to Target 263 and use the User Defined CDBs function to issue a RELEASE command to the drive. This will let the drive be accessible by either port, the Targets representing both ports in the device display will both show the current drive capacity, and either Target/port may be used for testing.

 

3. General I/O Testing

You can select either Target/Port to run any test to. This can check data integrity to insure that both ports of the drive work properly.

For instance,

  • You can select either or both Target/Ports in a single DMM session.
  • You can run multiple instances of DMM with each instance testing one or both Target/Ports.
  • You can run separate instances of the STB Suite on separate test systems, each connected to either or both Target/Ports.

There are just a few things to be aware of:

  1. If you are testing through both ports of a drive you will need to take care if you are doing READ testing with Data Compare On.
  2. You need to be aware of the effect testing both Ports can have on I/O performance. This would happen as each test on each port causes the drive to have to seek to different areas of the drive.