Internet SCSI (iSCSI) is the combining of the SCSI command protocol that storage devices use with a TCP/IP network as the transport mechanism in order to provide “block” storage connectivity over an existing network. The iSCSI technology is implemented as a layer above the TCP layer in the TCP/IP protocol stack. Using the SCSI nomenclature of initiator and target, a host server would be the iSCSI initiator and a logical storage device or subsystem would be the iSCSI target. The iSCSI target can be implemented in hardware or software.
Once the connection is established between the iSCSI initiator and the iSCSI target, the operating system on the iSCSI initiator sees the storage as a local storage device that can be formatted, read and written in the usual manner. For example, Windows “Disk Management” sees disk volumes the same way whether they are connected via iSCSI or connected via more traditional means. Some restrictions on the iSCSI initiator are listed on page 19.
The iSCSI hardware and software components comprise an iSCSI storage area network (SAN). Hardware components required to implement iSCSI include a network adapter, a network switch, and an iSCSI target storage device or subsystem. The network adapter used for iSCSI traffic can be the same adapter used for traditional network traffic, but in many cases one or more separate adapters are used for iSCSI traffic. Software components required to implement iSCSI include iSCSI initiator software.
Because Ethernet infrastructure already exists in many environments, adding iSCSI components to an existing Ethernet infrastructure can be relatively inexpensive. Typically the management expertise and Ethernet network infrastructure (network switches and cabling) is already in place in most organizations.
Best practices for iSCSI SANs are to separate iSCSI storage traffic normal LAN traffic through the use of virtual LAN technology or by deploying iSCSI traffic to a physically separate network. This is because adding iSCSI traffic to existing LAN traffic may cause degraded overall network performance due to the different nature of iSCSI storage traffic.
Hardware requirements for iSCSI include a Gigabit Ethernet adapter in the host server connected to a Gigabit Ethernet switch. The iSCSI target storage device must also be connected to the Gigabit Ethernet switch.
As many environments have already moved to a Gigabit Ethernet infrastructure for their basic networking needs, iSCSI can be added for no additional hardware costs. For those who do not have a Gigabit Ethernet infrastructure, it can be created relatively inexpensively. Low-end, unmanaged, five-port Gigabit Ethernet switches are available for less than $100 today. Low-end, unmanaged, eight-port Gigabit Ethernet switches are available today for less than $200. Some unmanaged 16-port and 24-port Gigabit Ethernet switches are available today for less than $300. Category 5E or category 6 cabling is recommended for Gigabit Ethernet networks, which is present in many environments. Most of the newer servers have onboard Gigabit Ethernet network interfaces. Simple Gigabit Ethernet Network Interface Cards (NICs) are widely available for less than $50 and specialized, dual-port, server Gigabit Ethernet NICs with advanced networking features are available for less than $200.
Best practice for iSCSI is to use enterprise class, managed switches that support jumbo frames. Enterprise switches are generally designed to be used in higher-traffic networks and are better choices than low-cost switches for iSCSI traffic.
Some switches are designed with “oversubscription,” which is a design that takes advantage of the fact that average utilization of Ethernet links tends to be significantly less than full-bandwidth. As such, these switches cannot run at full-bandwidth on all ports at the same time. Care must be taken when deploying iSCSI traffic on this type of switch so that the switch is not oversubscribed.
Because iSCSI runs over standard Ethernet networks, there is virtually no distance limitation in the basic technology. Wide Area Networks (WANs) can be used to implement iSCSI technology, and iSCSI technology can be used to bring a remote or “stranded” server into an existing storage infrastructure. However, many applications and operating systems do not have a high tolerance for latency, so response time on storage devices should be considered. It is possible that a very large distance (thousands of miles) may generate a response time that is unacceptable. Some iSCSI storage providers recommend a network latency of less than 5 milliseconds, resulting in a distance of approximately 100 kilometers.
Microsoft provides a free iSCSI software initiator, which can be downloaded from the Microsoft web site. There are other iSCSI initiators available, but this report will only discuss the Microsoft iSCSI software initiator. The focus of this report is primarily on iSCSI targets.
The iSCSI initiator works in combination with the network adapter. There are four basic combinations of iSCSI initiators and network adapters available today. Varying degrees of processing can be offloaded to the adapter hardware, depending on the type of network adapter deployed. These combinations of iSCSI initiator and network adapter are:
Software iSCSI initiator with standard network card
Software iSCSI initiator with advanced network card that supports Receive-side Scaling
Software iSCSI initiator with network card that includes a TCP/IP Offload Engine (TOE)
Hardware iSCSI host bus adapter (HBA) that provides offloaded TCP/IP and iSCSI processing.
The performance aspects of these types of iSCSI initiators are discussed below in the iSCSI performance section. The hardware iSCSI HBAs were not tested for this report but they do help to optimize CPU utilization in servers.
There are a variety of iSCSI target solutions available. One way to organize iSCSI target solutions is by the underlying technology. Some of these solutions are based on the Microsoft iSCSI target software and run on a Microsoft Windows Server platform. Other solutions run on a non-Microsoft platform. Some specific examples are discussed in further detail in subsequent sections of this document. Target solutions are available in a wide variety of storage capacities, performance levels, and prices.