By Steve Moore

Rack servers and storage controllers often need extra slots to expand the number of I/O cards that can be deployed. PCI Express (PCIe) riser cards allow this expansion by providing a method of mounting cards parallel to the system board, rather than the traditional vertical mount as seen in tower systems and full-height racks. Riser cards provide I/O expansion, increasing the number of peripherals or I/O devices in a system. Typically, the I/O in a given system is fixed and cannot be increased, due mainly to limitations in the physical design of the enclosure or chassis. This problem is intensified in 1U and 2U rack servers.

While most riser cards simply provide a mechanical repositioning of an existing I/O slot to allow an add-in card to be mounted parallel to the system board, what do you do if you need to connect a function that is not available in a card with a PCIe interface? PCIe is the current industry-standard I/O interconnect, but many cards are still only available with PCI-X and conventional PCI interfaces. In addition to providing the parallel mounting scheme necessary in 1U and 2U rack servers, riser cards fitted with a PCIe-to-PCI-X bridge can provide the protocol translation necessary to allow a PCIe-only system board to work with PCI-X add-in cards. This allows the storage-system manufacturer to address a wider applications base and provide broader functionality to the marketplace.

This article explores PCIe bridge-based riser card designs that are reaching the market today, including forward- and reverse-bridging models ranging from simple, low-cost x1 to 32-bit PCI risers for low-end server applications, through high-performance adapters intended for enterprise storage and networking.

Passive vs. Active Riser Cards
The rise of PCIe as the predominant I/O interconnect for servers and workstations has created challenges for server makers. One challenge is the difficulty of expanding I/O channels in a very confined rack form factor. With yesterday's systems' I/O employing PCI and PCI-X, passive riser cards have been deployed with multiple PCI (or -X) slots, as PCI is a parallel multi-drop bus architecture. This works fine, given that the loading restrictions called out by the PCI spec are observed. However, since PCIe is a point-to-point interconnect, only one I/O channel can be connected to any one port. This means the passive PCIe riser (see figure 1) can only have one PCIe slot.

Overcoming the space constraint requires active riser cards in PCIe systems. Figures 2 and 3 illustrate two active riser card designs that deploy multiple slots. Figure 2 solves two problems: the first being that server makers may need to deploy certain I/O endpoints that may not be available in PCIe versions. Several functions, such as parallel ports and SCSI ports, seem to be at the end of the list of devices to have PCIe implementations in silicon and at the add-in card level. The PCIe-to-PCI-X bridge in figure 2 solves this legacy add-in card connectivity issue. The second problem solved is that of I/O expansion, since two PCI-X cards can be added to this single riser card.

If a server only requires PCIe connectivity, then the problem of limited I/O expansion can be solved by placing a PCIe switch on the riser card, as in Figure 3. A 32-lane three port PCIe switch connects a x16 upstream PCIe port with two x8 ports downstream, thus aggregating up to 2GB/s per port into an upstream port capable of passing 4GB/s to the host.

Reverse-Mode Riser
System upgrades often require opening up the server rack and replacing NICs or HBAs or other I/O cards. If a system is more than two years old, it likely has PCI or PCI-X I/O slots. However, the latest NIC/HBA performance levels and feature sets are only found in the latest generation I/O cards, and these tend to use the PCIe interconnect and not PCI-X. That's where a reverse-mode riser card comes in handy. The secret is deploying a PCIe-to-PCI-X bridge on the riser card in reverse-mode.

When the PCIe to PCI bridge was originally envisioned, it was in a forward-mode application - that is, for bridging a PCIe root complex to one or more PCI (or PCI-X) bus(es). This function was included in the first PCIe chipsets for providing PCI/PCI-X connectivity on early system boards. The most common example is the use of a forward-mode bridge to create PCI slots on a PC motherboard. Another common example is the use of a PCIe-to-PCI bridge to allow deployment of PCI-based endpoints on PCIe system boards. The functionality of a forward-mode bridge is defined in the PCI Express Bridge Specification Rev. 1.0. An appendix (Appendix A) defines the Conventional PCI/PCI-X-to-PCIe bridge functionality. Most of the first-generation PCIe-to-PCI bridges do not include reverse-mode capability. More recently, however, several devices have become available with reverse-mode, such as the PEX 8114 and PEX 8111 bridges from PLX.

With the reverse-mode riser, legacy motherboards with PCI-X (or PCI) slots (and no PCIe slots) can host the latest generation of add-in cards. Figure 4 shows a reverse-mode bridge on a riser card with PCI-X connectivity to the host and PCIe connectivity to the endpoint. A typical bandwidth match for the single PCI-X segment is a x4 PCIe link. Maximum throughput in this configuration is close to 800MB/s -- of course, dependent on the payload size and other factors.

Riser Cards for Low-Profile Desktop PCs and Docks
Many low-cost add-in cards, such as TV tuners, graphics cards, I/O combo cards, and USB 2.0 adapters, are hitting the market for multimedia upgrades to desktop PCs. Most of these applications find that 32-bit PCI has plenty of bandwidth (typically up to 125 MB/s). Many desktop systems share the space constraints found in low-profile rack servers, as the desktop PC gets smaller and smaller. Extremely small riser cards are now available for these applications, connecting a x1 PCIe port to a 32-bit PCI bus. In fact, riser cards no larger than a PCI slot are possible with the PEX 8111, as its 10mm x 10mm package allows it to mount right next to the PCI connector on the riser (Figure 5). The low vertical profile of this design enables designs supporting both vertically stacked and horizontally stacked add-in cards.

Conclusion
Riser-card applications and form factors are expanding to meet the needs of the ever-decreasing space allowed for add-in cards in server and desktop computers. Whereas riser cards were once a simple way to create a horizontal mount in 1U and 2U servers, they now also provide protocol translation and I/O expansion by including PCIe-to-PCI bridges and PCIe switches. This in turn is fueling a richer I/O capability in increasingly dense chassis environments.

Steve Moore is the senior product marketing manager for PLX Technology.
www.plxtech.com