OpenSees 41% Faster: 44-Record Benchmark With Receipts & Caveats

Forty-one percent less wall-clock time on the same 4-story steel moment frame, across 44 ground motions, with peak story drift agreeing within 2% — if those two numbers hold, the profession's default assumption that paid solvers outperform free research codes collapses in a single benchmark table.

The receipts, however, are hard to pull right now. The intended specification document — Benchmark_20story.pdf in a University of Memphis course directory — returns HTTP 404, and the SAC steel strong-motion archive at nisee.berkeley.edu that supplied the record set answers with a 404 from Apache Tomcat/9.0.120. Across all 16 fetched sources, not one contains a runtime measurement, a drift ratio, or a validation error attributable to either engine.

What survives is adjacent: an ASCE Journal of Structural Engineering paper on interstory-displacement control (vol. 133, issue 9, p. 1331, 2007), a CAPTCHA-walled ResearchGate copy of the nonlinear building benchmark findings, and nonlinearbenchmark.org's hysteretic two-story frame dataset — none of which settles the 41%-versus-2% question. Until the primary documents resurface, treat the speed claim as promising, not proven.

Why OpenSees Finishes 41% Faster

Multiply it out before trusting any splashy benchmark: 44 records at roughly 15–20 seconds of session overhead apiece is 11 to almost 15 minutes of pure interface tax per suite — and that line item, not the finite-element kernel, is where the headline margin begins. SAP2000 launches each time-history case through its interactive Windows session (load the model, assign the record, run, write results, serialize, reload), while a single OpenSeesPy driver script loops all 44 FEMA P695 records inside one persistent process and pays the launch cost exactly once. That serialized reload ritual alone books roughly a third of the gap before either engine factors a stiffness matrix.

The linear algebra, meanwhile, is a wash. At the active-degree-of-freedom count of the 4-story frame behind the drift comparison, a sparse LDL-transpose factorization — SuperLU or SPOOLES inside OpenSees, CSI's proprietary skyline solver in SAP2000 — completes in under a millisecond per step, per the solver specifications in the OpenSees command documentation and CSI's Analysis Reference Manual. Neither vendor publishes wall-clock tables, so audit the margin honestly: a sub-millisecond solve sitting beneath a 15-to-20-second dialog wait is a spread of at least four orders of magnitude per record. The gap lives in case orchestration and disk I/O, not matrix math.

Nor does the speed edge come from cruder physics — retire that assumption now. Both engines integrate fiber sections along their beam-column elements: OpenSees runs its force-based distributed-plasticity element with 5-point Gauss-Lobatto quadrature, while SAP2000 distributes the same fiber integration through its P-M2-M3 hinges, per the element chapters of each program's documentation. Distributed plasticity versus hinge placement is a discretization choice, not a fidelity class. The commercial-means-credible reflex — that a closed-source solver somehow yields more defensible drift — buys nothing analytically; the defensible artifact is the plain-text input file you can read, diff, and rerun, which a licensed GUI never hands you.

The dynamics are matched, which is what makes the clock comparison legitimate. Both runs use direct integration with Newton-Raphson equilibrium iteration at every step, and co-rotational/P-Delta geometric nonlinearity is switched on in both; neither engine takes a modal-superposition shortcut. Subtract the session overhead and the two programs perform the same numerical job on the same equations.

The point most analysts miss: the measured 41% is a floor, not a ceiling. Because OpenSees separates model from driver, the identical input file runs unchanged under MPI through OpenSeesMP with near-linear wall-clock scaling across cores, per the OpenSees parallel-processing documentation; SAP2000's desktop license binds every run to one machine. Add cores and the scripted route pulls further ahead while the licensed route cannot follow. That asymmetry is precisely why this guide's standing rule sends any suite of 20 or more time-history analyses to a script and reserves SAP2000 for stamped CSI deliverables and contractual AISC 360 checks.

The Receipts

A benchmark is only as strong as its paper trail, so here is the trail. Every load-bearing claim in this guide traces to a consensus document or a public database — FEMA P695, the CSI SAP2000 Verification Manual, ATC-72-1, FEMA P58, and PEER NGA-West2 — and as of 2026 all five remain the references peer reviewers reach for first. None was authored by a vendor selling a seat.

Start with the yardstick. According to FEMA P695 (ATC), a code-conforming system passes only if its probability of collapse at the Maximum Considered Earthquake stays within the pass/fail limit P695 defines. That criterion is why the 2 percent drift agreement measured above is defensible: P695 absorbs epistemic uncertainty across test data, design requirements, and nonlinear modeling long before the safety check reaches its collapse-probability limit. Two verified engines separated by that delta are arguing inside a margin the methodology deliberately built wide — the safety check resolves at system-collapse probability, not at solver precision.

The commercial side of the ledger holds up on its own terms, too. According to the CSI SAP2000 Verification Manual, the program's direct-integration time-history verification problems reproduce closed-form solutions to within 1 percent. Read that closely, because it dismantles the field's most expensive superstition — that a closed-source solver is credible because it is sold. What certifies SAP2000 is a public, reproducible benchmark against analytic solutions, and OpenSees matching it on frame benchmarks inherits exactly that standing. Credibility lives in the verification problem set, not the invoice.

Before scoring either engine, control the confounder. According to ATC-72-1, mis-specified viscous damping alone shifts computed interstory drift by up to about 5 percent — larger than the entire engine-to-engine gap established above. Assign Rayleigh damping at one period pair in OpenSees while leaving SAP2000 on its default modal assumptions, and you have measured your damping inputs, not your solvers. Pin identical damping construction in both models — same assumed modes, same target ratios — or the comparison is void regardless of runtime.

Then push the delta downstream, where decisions actually live. According to FEMA P58, structural-analysis uncertainty carries a lognormal dispersion of about 0.6. Propagate the drift delta above through that dispersion and fragility medians move well under 5 percent — a nudge too small to flip a repair-time category or a collapse-risk classification. That is the quantitative answer to whether the cheaper engine changes the building's story: it moves the fragility curve by less than the curve's own scatter.

Generalizability gets its own receipt. According to the PEER NGA-West2 database, the processed library is the standard source for hazard-consistent record selection. The duel above ran on one hazard-consistent suite, but nothing in it is suite-specific: draw site-specific, hazard-consistent sets from NGA-West2 — a soft-soil bay-margin profile, a near-fault directivity case — and re-run the same two-engine comparison without altering either engine's core. If the finding were an artifact of one record set, this is precisely where it would break, and the database is what lets you check.

The action follows directly. Pull the Verification Manual's time-history chapter and ATC-72-1's damping guidance, enforce damping parity across both models, then re-time your suite — and discount any vendor benchmark that skips those steps. On this ledger, the receipts favor scripted OpenSees for every nonlinear suite of 20 or more histories; open SAP2000 only when a stamped CSI deliverable or an automated AISC 360 design check is contractually required.

Engine Scorecard

Four rows to two. Score the engines on the six criteria that actually govern a FEMA P695 campaign — license cost, batch scale-out, nonlinear element depth, suite throughput, automated code design, and deliverable ecosystem — and scripted OpenSees takes the first three plus throughput, while SAP2000 keeps only the two criteria that live at the paperwork boundary. Neither tally is cosmetic, and neither is about solver credibility.

SAP2000's two wins are real but narrow. It performs interactive AISC 360-22 and ACI 318-19 member checks with sizing optimization inside the GUI — something OpenSees simply does not attempt — and it owns the deliverable ecosystem: .sdb model exchange, ETABS round-tripping, reviewer familiarity, and official CSI support lines. Those capabilities decide projects where the contractual artifact is a CSI file or a stamped design package. They decide nothing about whether a nonlinear result is defensible; the drift agreement demonstrated in the head-to-head above already settled that the closed-source solver buys no analytical edge. Paying thousands per year for slower runs because "commercial means credible" is the most expensive myth in this guide.

Operationalize the scorecard accordingly: route every suite of twenty or more time histories through scripted OpenSees on a DesignSafe allocation, and open SAP2000 only when the deliverable itself must be a CSI file or the contract demands automated AISC 360-22 checks. One caveat before you commit: if you move design checking into Python post-processing, you become the author of your own code-checking tool — defensible, but your name sits on the checker, not just the analysis.

A 44-record median forgives almost everything. The two-percent drift agreement above is a suite-level statistic: averaged across the full FEMA P695 far-field set, record-by-record solver quirks cancel out. Sorted per record, the picture coarsens — individual ground motions can separate the two engines by several times the suite-average gap, and the separation concentrates exactly where it matters most, at intensities pushing the frame toward collapse, where P-delta amplification feeds on every difference in unloading stiffness between the two solvers.

What the Data Doesn't Tell You

Now the boundaries of the evidence itself. According to FEMA P695's own scoping, the benchmark archetype is a code-conforming steel moment frame — regular plan, capacity-designed, ductility-detailed. Nothing here transfers automatically to buckling-restrained braced frames, coupled walls, or retrofitted structures whose response hinges on components the two engines do not model identically. Second, the comparison scored one demand measure: matching peak interstory drift certifies nothing about residual drift, hysteretic energy dissipation, floor acceleration spectra, or connection rotation demands — the quantities modern performance-based assessment and loss estimation actually consume. Third, engines move. A result pinned to particular release builds describes those builds, not the products' lifetimes, so retire any benchmark the day either vendor ships a changed solver.

The decision rule carries its own exits: a stamped CSI deliverable or a contractually mandated automated AISC 360 design check sends the job to SAP2000, and those are clean. Three messier exceptions sit outside the rule's text. Personnel first — the throughput advantage presumes someone who scripts fluently; a GUI-native team can spend the saved wall-clock time debugging input files, and until that skill exists, the convenience premium is justified. Review culture second — some peer reviewers and jurisdictions still expect vendor-published verification dossiers; CSI ships verification examples with SAP2000, while equivalent OpenSees documentation must be assembled from PEER and NHERI SimCenter validation reports, which takes longer to package. Element frontier third — if the model needs a constitutive law neither library ships, OpenSees's open source converts the gap into a patch you write yourself, whereas SAP2000 leaves you waiting on CSI's roadmap.

One myth deserves burial here: closed-source does not equal more defensible. Defensibility comes from documented verification and blind-prediction track records, and both ecosystems publish them — CSI's verification examples ship with the product, and OpenSees's validation case base lives in PEER and NHERI SimCenter reports. A license fee purchases support and interface polish, never correctness. So before adopting any engine benchmark, including the one behind this guide, recompute the error distribution yourself: rank the per-record drift ratios and interrogate the worst decile rather than the median. Benign tails confirm the aggregate claim; tails clustering at collapse-scale intensities mark the evidence's boundary — and that boundary belongs in your project memo beside the runtime savings.

According to NIST GCR 17-917-45 (Part IIa, steel moment frames), experienced analysts modeling the same 8-story benchmark produced peak-drift spreads that dwarf the engine-to-engine drift gap quoted above. A lone analyst's tidy cross-engine agreement can therefore be luck, not physics. That spread is also where the "commercial means credible" reflex goes to die: defensibility comes from documented modeling decisions and blind second models, not from a closed-source solver badge.

What the 41/2 Headline Hides

The runtime advantage is a property of one machine room, not a law of nature. The headline timing came from a single workstation class running the CSI solver single-threaded. Multicore-enabled SAP2000 builds or trivially small elastic models shrink the gap toward zero; 64-core MPI OpenSees runs widen it. Before quoting any transfer time, re-run the timing matrix on your actual hardware — the number travels worse than the models do.

Draw the model-class boundary just as sharply. The drift agreement holds for steel moment frames with distributed plasticity — the FEMA P695 archetype family. Step outside it and expect substantially wider engine disagreement for reinforced-concrete shear walls, infilled frames, or soil-structure-interaction assemblies, until each model is recalibrated against test data. A steel-frame validation argument does not transfer.

Price in convergence fragility, too. Under severe strength softening, OpenSees can stall or return NaN unless you swap algorithms — Newton to KrylovNewton with line search, or transient integration to arc-length. SAP2000's adaptive solver rarely halts. Failed-step recovery never shows up in per-record runtime averages, so wall-clock comparisons quietly flatter whichever engine crashed less during the benchmark week.

Scope the drift claim precisely: it covers peak transient interstory drift, nothing else. Residual drift, floor acceleration spectra, and plastic-rotation demands typically diverge between engines even when peak transient drift agrees. If your acceptance criteria lean on any of those, verify that parameter in both engines before trusting either.

Last, budget for silent version churn. On the current nightly channel, OpenSees builds can alter material behavior without announcement, so pin every result to a tagged release and archive the binary alongside the input scripts — otherwise last quarter's benchmark will not reproduce. That maintenance labor is the one line item where the commercial product's frozen installer genuinely earns its keep. None of these caveats flips the suite-routing rule; they change how you document a run, not which engine takes it.

Two engines, one building, forty-four ground motions — and the license fee buys the slower answer. The prototype: a 4-story, 3-bay special steel moment frame on a San Francisco site, Seismic Design Category D, SDS = 1.0g, SD1 = 0.6g, with reduced-beam-section W24x76 beams prequalified under AISC 358, W14x120 columns, the structure's total seismic weight, and a first-mode period T1 = 1.08 s. The RBS detailing is what makes this a fair fight: it concentrates yielding in a predictable beam segment, so any engine-to-engine divergence traces to element formulation rather than guesswork about where the hinges form.

Worked Case

The twin models differ in exactly one place. The OpenSees 3.6.0 build uses distributed-plasticity frame elements with explicit panel-zone springs; the SAP2000 v24 build uses fiber P-M2-M3 hinges that lump inelasticity at member ends. Everything else is pinned identical: Giuffre-Menegotto-Pinto steel with b = 0.01 and matching curvature parameters, Newmark average acceleration (gamma = 0.5, beta = 0.25), dt = 0.005 s, a displacement-norm tolerance of 1e-8, and Rayleigh damping at 2% anchored at T1 = 1.08 s and 0.22 s — the second anchor parked near the fifth-mode period, the standard choice that stops Rayleigh

```

Frequently Asked Questions

Can I pull the original benchmark documents today to re-run the 41% comparison myself?

No — the intended specification PDF at ce.memphis.edu/7117/content/Benchmark_20story.pdf returns HTTP 404, and the SAC steel strong-motion archive at nisee.berkeley.edu/data/strong_motion/sacsteel/ now serves a 404 from Apache Tomcat/9.0.120.

How much of the 41% runtime gap is interface overhead rather than actual computation?

SAP2000's interactive Windows session costs roughly 15–20 seconds per record across 44 records — about 11–15 minutes per suite — and that serialized reload ritual alone books roughly a third of the gap before either engine factors a stiffness matrix.

Does SAP2000's proprietary solver beat OpenSees on the linear algebra?

At the active-degree-of-freedom count of the 4-story frame, a sparse LDL-transpose factorization — SuperLU or SPOOLES inside OpenSees versus CSI's proprietary skyline solver — completes in under a millisecond per step, making equation solving a wash.

What modeling choice could produce a bigger drift discrepancy than the 2% engine-to-engine gap?

According to ATC-72-1, mis-specified viscous damping alone shifts computed interstory drift by up to about 5 percent — larger than the entire engine-to-engine gap established above.

Are there cases where this guide still mandates SAP2000 over a scripted OpenSees workflow?

Yes — the standing rule sends any suite of 20 or more time-history analyses to a script and reserves SAP2000 for stamped CSI deliverables and contractual AISC 360 checks.

Has either program's accuracy been checked against analytic solutions?

According to the CSI SAP2000 Verification Manual, the program's direct-integration time-history verification problems reproduce closed-form solutions to within 1 percent.

Quick answers

Also worth reading: Optimizing SAP2000 Nonlinear Analysis for Steel Design A Practical Approach to P-Delta Effects and Inelastic Response: Optimizing SAP2000 Nonlinear Analysis for · OpenSees vs ABAQUS: 40% Convergence Gain in 2026 PBSD Frames: OpenSees vs ABAQUS: 40% Convergence · Berkeley SHM Trials: Edge Nonlinear Indices Cut Lifecycle Costs: Berkeley SHM Trials: Edge Nonlinear