Introduction

IBM develops and manufactures a diverse set of rackable servers, from the Personal Systems Group (PSG) Netfinity* product line to the Enterprise Systems Group (ESG) product line, which includes entry, midrange, and high-end AS/400* and RS/6000*, RS/6000 SP brand systems, as well as high-end S/390* mainframes. A server consists of processor units, cooling components, input and output (I/O) devices, a power distribution unit, memory devices, and hard-drive drawers, installed in one or more frame structures. Not all servers contain all of the subassemblies listed above; for example, an I/O server will not possess a processor unit, a memory device, or a hard-drive drawer. Many IBM customers, particularly governmental agencies such as the Federal Aviation Administration and those in the telecommunications industry, require adequate retention of rack-mounted systems so that personal injury and system damage do not occur, and so that system availability and functionality are not compromised during seismic events. In some areas where severe earthquakes frequently occur, such as in Japan and California, earthquake retention or tie-down of rack-mounted computer systems is required by law or local regulations. Customers in the telecommunications industry often require that rack-mounted systems and equipment be certified to be in compliance with the Bellcore Network Equipment-Building System (NEBS) Requirements: Physical Protection GR-63-CORE specification [1]. This specification imposes significant challenges in the design and development of racks (also referred to as frames) and the equipment installed in them. A complicating issue is the fact that S/390 products are typically installed in raised-floor facilities, whereas Netfinity, RS, AS/400, and SP systems are installed in both raised-floor and nonraised-floor facilities.

Earthquake simulation test specifications

IBM earthquake standard

General description
In February 1992, the IBM Earthquake Committee, comprising IBM personnel from the United States and Japan, published IBM Corporate Bulletin C-B 1-9711-009, “Earthquake Resistance for IBM Hardware: Product Guidelines” [2, 3]. This bulletin specifically defined a set of standardized, repeatable earthquake simulation test profiles for computer equipment, defined pass/fail criteria, and provided guidelines for equipment structure design and installation. Two test levels were defined for simulating the earthquake-induced motions at the floor of a computer room. The first test level specified, Level 1, defined a 0.1-g peak ground acceleration (PGA) to provide assurance that the computer equipment would function satisfactorily during a mild to moderate earthquake with no occurrence of hard (unrecoverable) errors. The second specified test level, Level 2, defined a 0.1- to 0.4-g PGA, which simulates a moderate to severe earthquake.

Test method
The test is conducted sequentially in all three principal axes, with an individual test duration of twenty seconds. Typically, a worst-case system configuration that presents a high center of gravity and high height-to-weight ratio is subjected to both Level 1 and Level 2 test profiles. During the test, the server is functionally tested, and the test software is able to record any transient problems. The system under test is anchored to a test bed in a manner representative of a true field installation. The server is typically installed on a reinforced concrete slab for nonraised-floor applications. For raised-floor application testing, a segment of raised floor is installed on the reinforced concrete slab, and the server is installed on the raised floor in the typical manner.

Performance and functional criteria
When the server is subjected to an IBM Level 1 test, the server should function continuously with no occurrence of unrecoverable errors. Soft errors, such as hard-drive read or write inhibits, are permitted, providing the server recovers in an unassisted manner. For the Level 2 test, the server should not sustain significant physical damage and should not tip over. Functionally, after being subjected to a Level 2 test profile, the server should function satisfactorily with only minor repairs, such as card reseating.

NEBS standard

General description
The Bellcore Network Equipment-Building System (NEBS) Requirements: Physical Protection GR-63-CORE standard defines five earthquake zones from Zone 0, which represents no substantial earthquake risk, to Zone 4, which represents the highest and most severe earthquake risk. Different earthquake zones are assigned to the different geographic regions. Server equipment is tested in accordance with the particular earthquake zone in which the equipment is to be installed. In some installations earthquake compliance is not a requirement. However, to provide the broadest market appeal and allow installation in any designated earthquake risk zone, server equipment should be qualified to the NEBS Zone 4 standard.

Test method
The NEBS earthquake test procedure consists of two primary components. The first component is a swept sine survey to determine the primary lateral resonances of the server and its frame structure under test. The second component is a transient vibration sequence in which the server and its frame under test are subjected to a controlled input motion that is representative of a prescribed earthquake zone. Testing is performed for a prescribed server configuration that is anchored to a test bed in a manner representative of a true field installation. The server and frame under test are typically installed on a reinforced concrete test bed using seismic expansion anchors to retain the system in the test bed.

Swept sine survey
The server and its frame under test are subjected in each of three principal axes to a swept sine input at the concrete test bed base of 0.2 g amplitude from 1 Hz to 50 Hz at a sweep rate of 1.0 octave (doubling of frequency) per minute. Sensors for measuring the acceleration (accelerometers) are mounted onto the frame, usually one at the bottom, one at the midpoint, and one at the top of the frame, to monitor the frame response acceleration (Figure 1).

Figure 1

Earthquake transient
The frame and equipment under test are subjected to a transient time waveform that is representative of a particular NEBS earthquake zone designation. The Test Response Spectrum (TRS) of the test system input, which is generated from a response spectrum analyzer using 2% damping, must meet or exceed the Required Response Spectrum (RRS) for the particular earthquake zone level under consideration. A typical TRS and RRS for Zone 4 are illustrated in Figure 2. For up to 5 Hz, the magnitude of the TRS spectrum more or less matches that of the RRS spectrum; above 5 Hz the magnitude of the TRS spectrum is higher than that of the RRS spectrum. During the earthquake transient test, a means must be provided for measuring the motion of the top of the frame and equipment under test with respect to the base, or anchor point, of the frame. This is typically accomplished by means of the Linear Variable Differential Transformer (LVDT) sensors, which are spring-loaded string potentiometers installed on the top and base of the frame, as shown in Figure 1.

Figure 2

Performance and functional criteria
The performance criteria for the frame and equipment under test are specified in the Bellcore NEBS GR-63-CORE document; they include the following:

1. All server equipment shall be constructed to sustain the waveform testing without permanent structural or mechanical damage. Permanent structural damage is defined to be deformation of any load-bearing element or any connection failure. Examples would include bent or buckled frame corner posts or uprights, deformed bases, cracks, and failed anchors or fastening hardware. Mechanical damage is defined as the dislocation or separation of components, and would include disengaged circuit cards and opened (or partially opened) doors, drawers, or covers.
2. The frame and equipment under test must be adequately retained such that during the waveform testing the maximum single-amplitude deflection at the top of the frame, relative to the base, does not exceed three inches.
3. The frame and equipment under test shall have a natural mechanical frequency greater than 2.0 Hz as determined in the swept sine survey, to ensure that the amplitude of the deflection at the top of the frame is less than three inches. Preferably, the equipment under test shall be rigid and have a natural mechanical frequency greater than 6.0 Hz.

Comparison of NEBS and IBM standards
The differences between IBM and NEBS specifications are presented in Table 1 and plotted in Figure 3.

Table 1  Comparison of IBM and NEBS seismic test specifications.

Specification IBM NEBS   Excitation method Power spectral density Swept sine and transient test waveform Response spectrum control damping 5% 2% Maximum relative displacement of frame top relative to the base floor Not specified 3 in. for nonraised-floor installation

Note: The NEBS standard specifies a lower critical damping value than does the IBM standard. As a result, for a similar input spectrum the NEBS response spectrum amplification factor will be higher. Response spectrum control damping is a damping factor.

Figure 3

Specifying a fixed time domain transient test waveform, as is required in the NEBS standard, permits a system subjected to the NEBS test waveform to experience test sequence levels that are consistent over time and consistent between test facilities. Specifying a power spectral density (the limiting mean square value of gravitational acceleration per unit frequency [4]) excitation, as is specified in the IBM standard, will produce test levels statistically similar to those of a product under test. However, the test sequences are random in time.

In general, the NEBS requirement limiting the maximum frame response to 3 in. is a tighter requirement than that required by the IBM specification. A tall server may pass the functional requirement defined by IBM specification but may not meet the maximum 3-in. deflection at the top of the frame relative to the base as required by NEBS. In addition, for taller and heavier servers, a more rigid and robust frame structure is required to meet the maximum displacement limitation.

Frame structural design

Development implications
The NEBS requirement limiting maximum frame deflection to 3 in. during the seismic event simulation imposes substantial structural stiffness requirements on the frame design. Generally, adequate frame stiffness must be designed into the frame to achieve an effective solution. Structural add-ons to provide additional stiffness to a base frame design can sometimes be problematic. However, such add-ons are sometimes more cost-effective in that a base frame design can be shared by NEBS and non-NEBS applications.

In addition tostructural stiffness add-ons, the lateral stiffness of a frame with component drawers installed can be enhanced if certain guidelines are followed. Specifically, subassembly drawers should be designed for front flange mounting to the vertical rails of the frame, and the number of mounting screws (per side) used to retain the subassembly drawer should be equal in number to the unit height of the drawer. [The unit, U for short, is defined and standardized by Electronic Industries Alliances (EIA). One EIA unit height is equal to 44 mm.] Drawer subassemblies should also be anchored to their horizontal support rails at the rear of the drawer. The modular server frame is designed to be used across the full spectrum of IBM product lines, from low-end Netfinity and PC products to midrange AS and RS servers to high-end S/390 mainframes and SP nodes, as well as for OEM applications. The modular server frame has been structurally designed to meet, as is, the rigors of a wide application spectrum. Specifically, the frame floor, corner posts, casters, and leveler/tie-downs have been structurally designed to allow shipping of fully populated IBM products. The newly developed 19-in. (482.6-mm)-width Enterprise frame is the common frame for RS/6000, AS/400, and Netfinity systems, and the 24-in. (609.6-mm) frame structure has been designed to accommodate S/390 mainframe and SP applications. The total height of subassemblies that can be accommodated in these frames is 36 U or 42 U. The types and dimensions of the various flexible frame configurations are listed in Table 2.

Table 2  Comparison of 19-in. and 24-in. frame designs.

Description Width (mm) Length (mm) Height (mm) Caster separation (side-to-side × front-to-rear) (mm) Loading per unit height (lb/U)       19-in. Enterprise frame 36-Unit EIA 623 1016 1775 463.7 × 774.5 35 42-Unit EIA 623 1016 2014 463.7 × 774.5 35 24-in. frame for S/390 and SP 36-Unit EIA 750 1016 1775 590.7 × 879.7 50 42-Unit EIA 750 1016 2014 590.7 × 879.7 50 19-in. input/output frame 36-Unit EIA 623 623 1775 463.6 × 393.6 35

The caster separation affects the frame stability, or its resistance to tipping over: The wider the distance, the more stable the frame. The loading per unit height shows the maximum allowable weight per U of successfully tested subassemblies.

Each of the frame designs listed in Table 2 is modular and flexible; the frames can each be configured in 36-U and 42-U versions by bolting a top 6-U extender section onto the base 36-U frame (Figure 4). The I/O frame is the shallowest of the three types and does not have a height-adjustment feature. All frames are similarly constructed by welding the two vertical corner posts, each formed from sheet metal into a tubular cross section, onto a flat sheet metal panel to form a side. Two side assemblies are then welded to a sheet metal base to create a U-shaped frame without a top. Two unique top assemblies are constructed in a similar manner, and one or the other can be bolted to the base frame to provide an overall frame height of either 36 U or 42 U. To achieve the same lateral rigidity in the 19-in. frame and the 24-in. frame, several design modifications were implemented:

• The flexural stiffness of the four vertical support tubes was increased by adding stiffening ribs (additional bends in the sheet metal) to the tube, as shown by the tube cross section (Figure 5). Additionally, horizontal stiffening members were welded into the interior of the support tube, partitioning it into five sections and providing additional stiffness.
• The cable brackets (the structural channels that connect the vertical support tubes from front to rear) were welded instead of riveted in place (Figure 4).
• To limit lateral motion during earthquake events, a triangular brace was added to the rear of the frame (Figure 6). This brace is easily installed on hinge pins attached to one corner post and is restrained on the opposite corner post by a single fastener. The hinged brace provides easy access to the rear of systems installed in the frame.

Figure 4 Figure 5 Figure 6

All of these frames feature the ability to ship a server in a fully populated configuration on casters, without incurring the additional cost of pallets. The frames support load densities up to 30 lb/U for the standard 623-mm-wide frame, and up to 45 lb/U for the 750-mm-wide frame. They have been certified to meet the IBM shipping [5] and earthquake test specifications [2] and the Bellcore Zone 4 earthquake test specifications [1] of the Network Equipment-Building System (NEBS) GR-63-CORE requirements. The base frame configuration (in either 36-U or 42-U versions) can meet certain earthquake test specifications without modification (specifically, the NEBS GR-63 Zone 1 and Zone 2 earthquake specification).

The standard basic frame can be converted for use as an earthquake-resistant frame by adding a structural triangular brace and an earthquake retention feature, as shown in Figures 7 and 8 and discussed in the next section. This simple structural add-on (triangular brace installed on the rear of the frame; see Figure 6) provides extended performance in more severe earthquake environments (specifically, NEBS GR-63 Zone 4 earthquake specification and IBM Level 2). The earthquake structural add-on (triangular brace) is a cost-effective means of providing high-end earthquake performance capability without building the cost into the basic frame design. This provides a pricing advantage for low-end applications that do not require severe earthquake performance capability.

Figure 7 Figure 8

Frame retention

Two primary methods of frame retention are used in the industry today for earthquake-prone installations. The first is a flexible restraint, in which the frame is not rigidly attached to the floor. Such a flexible restraint decouples the frame from the floor and thereby limits the amount of energy that can be transmitted into the frame, but allows significant lateral motion of the populated frame during a seismic event. This approach is not suitable for installations which accommodate multiple-frame systems, because the flexibly restrained frames can collide during seismic motion and thereby generate internal impact forces that can disrupt the system functionality. An example of a flexible restraint is a frame system that is simply tethered to the floor, or one in which four spring tie-downs are connected to the frame base. A flexible-restraint method does not meet the requirements of Bellcore GR-63-CORE.

A second method of frame retention is a fixed-restraint design, in which a frame is rigidly anchored to the floor surface. This method minimizes the potential movement of a frame during an earthquake and meets the requirements of Bellcore GR-63-CORE. Two fixed-restraint designs are described here: The first is primarily applicable to large S/390 and SP systems built around a 24-in.-frame form factor and installed on raised floors; the second is a hybrid leveler and anchor system more applicable to midrange systems built around a 19-in.-frame form factor and installed in either raised-floor or nonraised-floor facilities. Common to both of these designs is the requirement for ease of installation for both new and upgraded systems, and the ability to install the retention system without interruption of system function.

The fixed-restraint design for 24-in.-form-factor frames installed on a raised floor is shown in Figure 7. Four triangular plates are mounted to the bottom of the frame structure by means of three bolts and nuts to distribute the load. An eyebolt is welded to each of the plates. The plates can be pre-installed or installed in the field to upgrade an existing system. An upper yaw is installed which passes through a rubber bushing inserted into an opening in a raised floor tile. A lower yaw is attached to another eyebolt that is anchored to the concrete floor beneath the raised floor. Frame retention is achieved by connecting the two yaw rods through a turnbuckle. Rotating the turnbuckle draws the two yoke rods together, creating the anchoring forces. Additionally, the turnbuckle range of motion allows this frame retention mechanism to accommodate a range of raised-floor heights.

The hybrid leveler and anchor system shown in Figure 8 is more applicable to midrange systems built around a 19-in.-frame form factor and installed in either raised-floor or nonraised-floor facilities. The leveler/tie-down hardware is structurally integrated into the floor structure of the frame to provide greater strength and simplicity of design. It rigidly retains the frame on an installed floor and limits frame response during seismic events, while concurrently allowing for height and leveling adjustments of the frame. The coaxial nature of the leveler/tie-down hardware eliminates bending forces that are typical of more traditional leveler/tie-down hardware designs, which are functionally segregated and dimensionally separated. The coaxial design provides the reduced frame floor stiffness required in order to achieve and maintain the high retention forces required in some installations, specifically those in installations prone to earthquakes. The assembly secures the frame to the floor surface and provides a residual compression load to the lower part of the frame. This residual load is key to maintaining small lateral deflection when the system is subjected to earthquake simulation tests.

The tie-down assembly consists of a floor plate, the leveling element, lock nut, anchor bolt, and upper and lower insulators (Figure 8). Leveling is achieved using a hex-shaped foot with a threaded portion that mates to a threaded anchor block that is welded into each corner of the frame floor. The hex shape accommodates a wrench for height adjustments and has a central hole through which the anchor bolt passes. Additionally, the wall of the leveler foot provides lateral support to the anchor bolt. An added feature of the leveler/tie-down is that the anchor bolts are easily accessible for installation, removal, or maintenance (e.g., retorquing) and do not interfere with installation or removal of lower drawers or subassemblies in the frame. This leveler/tie-down also provides electrical isolation of the frame, which is a requirement of the Bellcore NEBS GR-63-CORE specification.

The floor plate is installed directly onto the floor surface and contains numerous holes to facilitate installation. The length of the anchor bolts can be varied to accommodate both raised- and nonraised-floor facilities. Additionally, this type of retention method obviates the use of kick plates, which violate the installation requirements of some major telecommunications customers. The tie-down has been tested and is capable of restraining a populated system of up to 2100 lb total weight.

Earthquake simulation testing of frames

The development of a robust 19-in. server frame and retention system involved three earthquake simulation tests which were conducted at non-IBM test facilities. Testing was performed in order to assess the structural effectiveness at various points in the design process, from early prototype to final design, as well as to validate the responses anticipated from finite element modeling. In all of the test sequences, the systems under test were subjected to uniaxial swept sine waveform stress testing followed by uniaxial NEBS Zone 2 and/or NEBS Zone 4 and IBM Level 1 and Level 2 earthquake test profiles in all three independent axes.

Summary of prototype testing
The original 19-in. frame design was simply a dimensional derivative of an existing modular 24-in. frame, with no structural enhancements. Seismic testing was performed on the original 19-in. frame prototypes to evaluate the need for structural redesign. In some of the test configurations, frame stabilizers were used (these are L-shaped plates mounted to the front and rear bottom frame to enhance the stability of the frame when subassemblies are extended for service). The tested configurations of the 19-in. test frame are described in Table 3. In this test sequence, the frames under test were not mounted to a concrete slab, but rather to 6-in. tubular steel beams that were welded directly to the seismic table. The frames were attached to the tubular beams using the hybrid coaxial retention hardware previously described.

The seismic test results, summarized in Table 4, indicated that although the frame structure was stiff enough to provide seismic performance for a 24-in. form factor, it was inadequate for a 19-in. form factor. From this test, the frame joint weakness was identified and design corrections were implemented. Specifically, the flexural stiffness of the four vertical support tubes was increased by adding stiffening ribs (additional bends in the sheet metal) to the tube cross section (see Figure 5). Additionally, horizontal stiffening plates were welded into the interior of the support tubes, partitioning them into five sections and providing additional stiffness, and additional welds were designed into the floor assembly. The floor-plate mounting-bolt pattern was widened to increase the apparent stiffness between the frame tie-down anchors and the floor.

Summary of intermediate design testing
Structural design modifications were implemented and prototypes procured for evaluation. The tested configurations of the 19-in. test frame are shown in Table 5. In this test sequence, the frames under test were mounted directly to a concrete slab using the hybrid coaxial retention hardware previously described. The floor plate to which the hybrid coaxial retention hardware attaches was secured to the concrete using seismic anchors.

The seismic test results are summarized in Table 6. No functional problems were encountered during this test sequence, since the added triangular brackets stiffened the frame significantly. This test sequence indicated that although the structural modifications implemented were helpful, additional stiffening enhancements in the form of a triangular brace were required to meet the functional requirements of severe seismic environments. It was also determined that the use of additional threaded fasteners (approximately one fastener per U of drawer height) to retain the flange-mounted drawer units in the vertical rails provided improved lateral stiffness. Flange-mounted drawers are therefore preferable, and even constitute a requirement for severe seismic environments. The addition of rear mounting flanges on the drawers can also improve lateral stiffness.

Summary of final production design testing
Previous prototype-level hardware seismic testing had indicated the need for a triangular brace to meet severe seismic environmental requirements. This brace was implemented in the frame design in the form of a triangular brace (Figure 6) that could easily be installed to provide additional seismic function, while at the same time not disrupting access to the front and rear of the frame. Prior to final frame design testing, the structural gate was engineered and designed using finite element techniques.

Using the I-DEAS Master Series** General Purpose Finite Element Program [6], a finite element model of a fully loaded 19-in. frame was constructed and subjected to quasi-static lateral acceleration loads equivalent to the peak experienced during a NEBS Zone 4 earthquake. In the finite element model, each major frame component (base, corner posts, side braces, rails, doors, covers, etc.) was constructed entirely of shell elements. These components were then assembled as a single model by applying coupled degrees of freedom to represent the various welds and screws in the assembly. Elements representing each leveler/tie-down were fully restrained in order to simulate anchoring the populated frame to a concrete floor. Additionally, a modal analysis of the finite element model was performed to estimate the lateral resonance of each of the modeled frame configurations.

Comparisons of the maximum lateral displacement response were made for various scenarios, including a base frame design, and configurations with both single and dual triangular braces (or gates) installed. Comparisons of the frame lateral natural frequency were also made. These comparisons are summarized in Table 7. These results indicated that a single structural gate attached at the rear of the frame would provide adequate stiffness to meet the NEBS Zone 4 earthquake resonance and displacements requirements.

Three final production design frames were tested (Table 8). Two of the frames were 36-U and 42-U versions of the 19-in.-form-factor design, and the third was a 42-U 24-in. frame. These tests were conducted nonfunctionally, since previous tests had demonstrated no functional concerns. The 19-in. frames were installed on a nonraised concrete floor, whereas the 24-in. frame was tested in a raised-floor configuration. It should be noted that for raised-floor configurations, neither IBM nor NEBS specifications specify a maximum displacement limit.

The seismic test results are summarized in Table 9. The relative displacement of configuration 7 is more than 3 in.; however, for raised-floor installation, the NEBS standard does not have a requirement for a maximum relative displacement [7]. Typical response spectra for 19-in. and 24-in. frames are shown in Figures 9, 10, and 11, respectively. In these figures, Bell Y is the NEBS Zone 4 input reference. No functional problems were encountered during this test sequence. For the 19-in. 36-U frame, only a single rear structural gate is needed in order to meet the NEBS lateral displacement requirement for a Zone 4 earthquake, as suggested by the finite element analysis.

Figure 9 Figure 10 Figure 11

It should be noted that the frame is robust enough to withstand eight sequential earthquake stress series (uniaxial NEBS Zone 4 test in all three principal axes, uniaxial IBM Level 2 in all three principal axes, triaxial IBM Level 2, and triaxial NEBS Zone 4).

Summary

This paper has demonstrated that through design, analysis, and test sequences, a robust mechanical frame design with adjustable height can be achieved that will provide exceptional structural stability in both raised-floor and nonraised-floor environments during severe seismic events. Unique retention methods have been demonstrated that provide ease of installation, are serviceable, and minimize the structural stiffness requirements in the floor of a frame. Finally, a unique structural add-on (triangular brace and latch) is demonstrated which is a cost-effective means of providing high-end earthquake performance capability without building the cost into the basic frame design, allowing pricing advantages for low-end applications that do not require severe earthquake performance capability.

Acknowledgments

The authors would like to thank Udo Jourdan, David Linkstrom, William Winkler, and James Gutelius for their assistance in conducting the test.

*Trademark or registered trademark of International Business Machines Corporation.

**Trademark or registered trademark of Structural Dynamics Research Corporation.

References

Received July 28, 2000; accepted for publication July 20, 2001