Data Center Solution September 14, 2026 22 min read

HPC Topology Cabling: Convert Fat-Tree, Dragonfly, and Torus Designs into a Link Register

Plan HPC topology cabling by converting Fat-tree, Dragonfly, or torus designs into an auditable link register with exact ports and change controls.

HPC topology cabling begins before cable selection. Convert every approved logical edge into one fabric link-register row with a stable link ID, exact endpoint ports, topology role, rail or group, design state, and revision. Validate that register first; add rack routes, media, connector details, and cable part numbers in separate downstream records.

Use this workflow when the topology family, participating devices, and adjacency rules have already been approved, but the physical cable design is not yet frozen. If the team is still comparing Fat-tree, Dragonfly, torus, or another architecture, complete that design decision first. This guide starts at the next control point: turning the approved graph into exact port assignments.

Already have an approved topology? Start with the blank CSV header below, create one row for every approved edge, and resolve blocked rows before requesting physical cabling input.

Procurement summary

Procurement should begin with a released fabric link register, not a product list. Before requesting a cable schedule or quotation, confirm that every active topology edge has a stable link ID, exact source and destination ports, topology role, design state, validation result, and approved revision. Then join the register to the rack overlay and exact endpoint interfaces. Suppliers can compare physical options only after route, media, connector mapping, length, quantity, acceptance, documentation, and commercial assumptions are controlled. Unresolved fields should remain visibly blocked instead of becoming supplier guesses.

Five controls to keep

  1. One approved topology edge maps to one canonical register row.
  2. Exact device-port pairs are validated before cable attributes are added.
  3. Logical topology roles do not determine cable media, connector, or length.
  4. Every supplier response references the same register and BOM revisions.
  5. Any change is reviewed against the affected stable link IDs.

Separate four artifacts before cable selection

An approved topology can still produce a bad cabling handoff when the team jumps from a drawing to a purchase list without closing the port map.

Keep four artifacts distinct:

ArtifactQuestion it answersMinimum contentWhat it must not silently decide
Topology modelWhich nodes and switches must be connected?Nodes, logical edges, topology rules, rail/tier/group/coordinate assignments, design revisionPhysical route, cable type, connector or order length
Fabric link registerWhich two exact ports implement each required edge?Stable link ID, source port, destination port, link role, topology metadata, state, approvalMedia, component chain or commercial part number
Rack-placement overlayWhere are the two endpoints and what physical route class may apply?Site/room/row/rack positions, pathway class, physical-design ownerFinal length, slack, pulling or acceptance method
Cable schedule and BOMWhat assembly or component chain will implement the approved link?Exact interfaces, route, media, length, connectors, mapping, quantity and acceptance requirementsChanges to the approved network topology

The stable link ID joins the design, rack, cable, label, and validation records. Keep it when a cable or supplier part number changes. If the topology edge changes, retire or redefine the old record through revision control.

NVIDIA's cabling documentation separates the network design, point-to-point map, and cable BOM. Its P2P guidance records the source, destination, ports, bundling, and labels. The BOM adds cable attributes. See NVIDIA network documentation guidance.

The fabric link register ends where the physical cable schedule begins. Keeping that boundary visible prevents connector, length, or product fields from hiding an unresolved topology error.

Freeze the topology contract

Assign physical ports only after the topology rules have an owner and an approved revision.

Record:

  1. the topology family and the exact implementation or generator used;
  2. every endpoint, switch or router that belongs to the fabric;
  3. the approved port inventory for each device;
  4. the allowed link roles and adjacency rules;
  5. rail, tier, group or coordinate definitions used by the design;
  6. the state vocabulary, such as active, reserved, deferred and retired;
  7. the expansion scope that is designed but not yet cabled;
  8. the network authority who approves exceptions.

Use the approved implementation rather than a generic Fat-tree or Dragonfly diagram. NVIDIA's current topology generator asks for topology, switch, uplink, node, and rail inputs, then reports node-to-leaf and leaf-to-spine cable counts for that configuration. See the NVIDIA InfiniBand Topology Generator.

Assign one source of record to each port identifier. A label copied from a slide may differ from the name in the vendor configuration, rack elevation, or commissioning tool. Normalize the displayed name, but retain the source identifier and document revision.

Keep planned expansion edges in the register with a non-active state. An unused switch port, a deferred topology edge, and a spare cable are different records and should not be counted together.

In graph terms, devices are vertices and required connections are edges. The register converts each approved edge into one canonical row.

FieldControl purpose
Link IDStable key shared by design, rack, cable, label and validation records
Topology revisionIdentifies the approved graph that created the edge
Fabric / railSeparates independent fabrics or traffic rails
Topology familyFat-tree, Dragonfly variant, torus/mesh or another approved model
Link roleEndpoint, uplink, local, global, neighbor, wraparound or project-defined role
Source node and source portOne exact endpoint of the edge
Destination node and destination portThe other exact endpoint of the edge
Topology metadataTier, group, router, coordinate, dimension, direction or other model-specific values
Design stateActive, reserved, deferred, retired or another controlled value
Validation statusPass, blocked or exception with a named owner and evidence reference
Change referenceLinks the row to the approved change that created or modified it

Use one canonical endpoint order for the ID. This stops the same edge from appearing twice with its endpoints reversed. Keep a separate operational direction field if the approved design needs one.

For example, an ID such as FABRIC-A-UPLINK-A can identify the fabric and link role. Leave the supplier part number, length, and connector out of the permanent ID because those details may change while the network edge stays the same.

Use this seven-step conversion workflow:

  1. enumerate the approved vertices;
  2. enumerate the required edges from the controlled topology source;
  3. bind both ends of each edge to exact device ports;
  4. attach rail, tier, group or coordinate metadata;
  5. assign the design state;
  6. generate the stable link ID and run logical validation;
  7. freeze the register revision before rack and cable teams add downstream data.

Keep parallel edges as separate rows. Two links between the same devices still use two distinct port pairs.

Start with a source-controlled CSV header

The register can begin with a portable header that separates common fields from topology-specific metadata:

link_id,topology_revision,fabric,rail,topology_family,link_role,source_node,source_port,destination_node,destination_port,source_tier,destination_tier,source_group,destination_group,source_coordinate,destination_coordinate,dimension,direction,wraparound,design_state,validation_status,exception_owner,evidence_reference,change_reference

Avoid filling every topology-specific column just to eliminate blanks. A Fat-tree row may need tier fields but not coordinates; a torus row may need coordinates and wraparound state but not Dragonfly groups. Define not_applicable, unknown, and missing-required-value states in the data dictionary so they cannot be treated as the same condition. Set validation_status only after the applicable checks have run.

Preserve topology-specific metadata

Topology metadata required in an HPC link register

The common fields make rows consistent. The topology-specific fields let a reviewer compare those rows with the approved fabric.

Topology familyMetadata to preserveCheck enabled by the metadata
Fat-tree / Clos implementationtier, leaf/spine/core role, parent/child or allowed-tier adjacency, fabric/rail, approved uplink setDetects an edge assigned to the wrong tier or rail and compares actual port bindings with the approved adjacency model
Dragonfly implementationgroup ID, router ID, terminal/local/global role, peer group, fabric/railDetects a local link that crosses groups, a global link that stays inside one group, or a terminal link assigned to the wrong role
Torus or coordinate-based meshcoordinate tuple, dimension, positive/negative direction where used, neighbor coordinate, wraparound flagDetects a neighbor relationship that does not match the approved coordinate rule

Fat-tree: record tier and adjacency

For a Fat-tree, record the tier or role at both ends instead of calling every switch-to-switch edge an "uplink." Add the allowed adjacency rule and the fabric or rail ID from the approved design.

Switch radix, uplink allocation, blocking or oversubscription, and tier count are design inputs, not generic constants. Compare register totals with the approved generator or architecture output for the project.

For each approved revision, confirm that every required endpoint and inter-tier edge appears once on an allowed port pair.

Dragonfly variants differ. Record the exact implementation reference, and keep terminal-facing, within-group, and between-group edges in separate roles.

A Sandia/OSTI paper describes ports to compute nodes, routers in the same group, and routers in other groups. It calls the last two categories local and global links. See the OSTI Dragonfly topology paper. HPE publishes separate architecture documentation for Cray EX systems, so a Cray design should use its named system documents rather than a generic Dragonfly template. See the HPE Cray EX system architecture.

For each row, retain the source group, destination group, and link role. A local link with different group IDs fails the check unless the project definition uses different terms. Review a global link with identical group IDs for the same reason.

Torus: retain coordinates, dimension, and wraparound state

Device names are not enough to audit a torus unless they preserve the topology coordinates. Store the system's coordinate tuple, the neighbor dimension and direction where applicable, and whether the edge crosses the coordinate boundary.

IBM documented a three-dimensional torus for Blue Gene/L and a five-dimensional torus for Blue Gene/Q. Use the coordinate model of the actual system rather than treating either dimension as universal. See IBM Blue Gene/L torus research and IBM Blue Gene/Q network research.

A wraparound edge is a logical neighbor, not necessarily a short cable path. Rack placement and facility pathways determine its physical route class.

Run HPC port-map validation before physicalization

The register should pass topology checks before cable type, length or connector information is added.

CheckPass conditionFailure signalOwner
Vertex coverageEvery required endpoint and switch appears in the approved roleMissing device or unexpected deviceNetwork architect
Edge completenessEvery required edge has one canonical rowMissing required row or unexplained extra rowNetwork architect
Port exclusivityA physical port is not assigned to two active point-to-point edgesDuplicate active port assignmentPort-map owner
Reversed-edge duplicateEndpoint A to B and B to A are not stored as separate copies of one edgeSame port pair appears twiceRegister owner
Role validityLink role matches tier, group or coordinate relationshipLocal/global, inter-tier or neighbor mismatchTopology owner
State separationActive, reserved, deferred and retired edges are counted separatelyPlanned expansion mixed into active cable demandDesign-control owner
Revision traceabilityEvery row points to an approved topology revision and change recordOrphan row or undocumented changeDocument-control owner

Counts alone do not prove correctness. A register can have the expected number of rows and still contain a duplicated port plus a missing port. Run both total checks and row-level rules.

Hidden failureWhy a simple row count can miss itControl that exposes it
One required edge is missing and one unapproved edge is addedThe total stays unchangedCompare the canonical edge set, not only the count
One active port is used twice while another port is unusedThe row total and device total may still balanceEnforce active-port exclusivity
The same edge appears in both endpoint ordersBoth rows look individually completeCanonicalize endpoint pairs before duplicate testing
A link role is relabeled to hide a tier or group mismatchThe edited label may pass a text reviewRecompute role validity from topology metadata
Deferred edges are counted as activeThe full register total may match the future designReconcile totals by state, fabric, rail, and role
A row points to an obsolete topology revisionPorts and roles may look valid in isolationJoin every row to the approved revision and change record

What should a validation release contain?

A single green status is not enough for handoff. Freeze four outputs:

  1. control totals by fabric, rail, link role, and design state;
  2. a row-level failure list with the failed rule and affected link IDs;
  3. an approved exception register with owner, evidence, and disposition;
  4. the released register export with its topology revision and change reference.

The released export becomes the comparison point for rack overlay, cable scheduling, installation labels, and later discovery checks. If any row changes, issue a new controlled revision instead of silently replacing the file.

If the release still contains blocked rows without owners, stop at this checkpoint. A cable supplier should not be asked to price around an unresolved topology record.

After deployment, discovery tools can compare the installed fabric with the intended topology. NVIDIA documents Fat-tree and Dragonfly+ validation in ibdiagnet; commands and features vary by version. See the NVIDIA ibdiagnet feature overview. This operational check does not replace the approved design register.

For an exception, record the failed rule, affected link IDs, owner, supporting source, and disposition. Changing a topology label without an approved design change does not resolve the failure.

Passing these checks proves only that the register is internally consistent with the approved topology and port inventory. It does not establish endpoint compatibility, cable reach, optical margin, system performance, or installation acceptance. Those decisions require exact endpoint, route, component, and project evidence in the downstream physical design.

Add rack placement as an overlay

After logical approval, add endpoint locations without turning the register into a product order.

Useful overlay fields include:

  • site, room or data hall;
  • row and rack;
  • rack unit or chassis position where needed;
  • pathway zone or route-class identifier;
  • route-survey owner and status;
  • physical-design revision.

Classify each link as intra-rack, inter-rack, inter-row, or another project-defined route class. The class groups work for route survey; it is not an ordered cable length.

NVIDIA's current cabling guide warns that cables ordered too short can delay deployment, while excess cable can create long-term maintenance problems. Treat route class and length as provisional until the physical path has been surveyed for the named project.

The official abstract for ISO/IEC 14763-2:2019 covers planning, documentation, administration, testing, and operation. The standard does not define this link-register schema. Use the edition and requirements named in the project contract.

Keep logical and physical changes separate. A rack move can change route classes and ordered lengths without changing topology edges or port bindings. A switch replacement can change the register even when the rack stays in place.

A topology term does not specify the cable medium. For example, the word "global" in a Dragonfly design does not define an interface or route. The approved endpoint interface and facility design control the cable assembly.

Control changes by affected edge

Hidden-failure checks for an HPC fabric port map

Record every change against the affected link IDs instead of issuing a spreadsheet with silent edits.

ChangeLink-register impactDownstream impact to review
Port reassignment on the same devicesUpdate endpoint port and re-run exclusivity/adjacency checksLabels, cable schedule, installation and validation records
Switch replacementReconfirm device identity, port inventory, topology role and all affected edgesInterface, media, component, compatibility and BOM decisions
Rack relocationTopology edge may remain unchanged; rack overlay changesRoute survey, length, pathway and installation plan
Rail or group reassignmentUpdate topology metadata and re-run role rulesLabels, operational validation and possibly routing/configuration records
Expansion activationChange reserved/deferred rows to active only under an approved revisionQuantities, cable definition, installation and commissioning scope
Edge removalRetire the row; do not reuse its identity for a different edgeRemove or disposition downstream assemblies and records

The change record needs the reason, requestor, approver, affected link IDs, old and new values, and downstream records that require review. A topology change may leave the cable untouched. A rack or route change may alter the cable record without changing the topology.

Hand off each downstream decision to its owner

The link register is ready for handoff when the topology edges and port assignments are approved. Use the same link ID in every downstream record.

When the design reaches this decisionUse this guideCarry forward
Broad data-center media and connectivity optionsdata center fiber connectivity solutionsProvide approved link IDs, exact endpoints and route classes
MPO cable family and assembly typeMPO cable types guideApply only to rows whose approved physical design uses MPO
MPO component route, lane mapping and approvalMPO polarity channel checklistCarry the stable link ID into the channel record
Pre-terminated route measurement, pulling and receipt acceptancepre-terminated MPO deployment guideAdd measured route and installation evidence downstream
MPO trunk fiber-count and loss decisionsMPO trunk design guideUse only after the physical channel architecture is approved

Use the approved register as a procurement gate

The link register does not replace a complete cable specification, supplier qualification process, or purchase order. It controls when each of those activities can begin and gives every downstream response a stable design reference.

Procurement stageDecision to closeEvidence or controlled outputHandoff rule
1. Recognize the problemA topology drawing is not yet an orderable cable scopeApproved topology source plus unresolved-edge listKeep cable attributes out until exact ports are assigned
2. Understand the product decisionIdentify which links still need a physical assembly decisionApproved link IDs, endpoint interfaces and owner-page handoffSelect the product family downstream; do not infer it from the topology name
3. Define the applicationEstablish fabric, rail, topology role, endpoint locations and route classReleased register plus rack overlayTreat logical role and physical route as separate facts
4. Confirm technical inputsClose exact interfaces, media, connector mapping, route, length, quantity and acceptance basisApproved cable schedule or BOM linked to each link IDA blank or assumed field remains blocked, not supplier discretion
5. Compare suppliersMake each response address the same approved scopeSupplier response mapped to register and BOM revisionsCompare like with like and list every exclusion or alternative
6. Evaluate price and deliveryConfirm what the quotation includes and which schedule assumptions it usesQuote revision, quantities, commercial terms, validity and requested delivery planPrice and delivery are not comparable across different revisions or inclusions
7. Request documents or qualification itemsDecide whether a datasheet, drawing or sample plan is requiredItem-specific document set and approved qualification planA sample is useful only when its configuration and evaluation method are defined
8. Verify quality evidenceDefine the required inspection, test and traceability recordsContract-specific acceptance criteria and required reportsA generic certificate or untied report does not prove the proposed item meets the project requirement
9. Send the inquiryGive the supplier enough controlled information to respond without guessingApproved inquiry pack plus named owners for open itemsKeep unresolved design questions visible
10. Enter order or project executionFreeze the accepted scope and control later changesPurchase-order or project revision linked to the released register, BOM and approved deviationsReassess affected link IDs whenever scope changes

What should a buyer send to a cabling supplier?

Send one controlled inquiry pack rather than separate screenshots and unversioned spreadsheets:

  • the released link register, including control totals, approved exceptions and revision;
  • exact device, port and interface references from the project authority;
  • the rack overlay and route-survey status;
  • the approved physical cable schedule or a visible list of fields still awaiting approval;
  • quantities separated by active demand, planned expansion and any separately governed spares;
  • project-specific labeling, documentation, inspection and acceptance requirements;
  • the delivery destination, requested schedule and any required packaging or logistics instructions;
  • the commercial response format, including the revision and validity information the buyer expects.

The supplier should identify missing inputs instead of silently selecting a connector, polarity, length, fiber type, product substitution or acceptance limit.

How should supplier responses be compared?

Comparison controlWhat to checkReject or clarify when
Scope identityThe response names the same link-register and BOM revisionsThe supplier priced a different revision or cannot identify the source scope
Row coverageEvery included line maps to a link ID, with exclusions listedQuantities are presented only as an unexplained total
Alternatives and deviationsEach proposed change identifies affected link IDs and the reasonAn alternative is described as equivalent without interface, route and acceptance review
DocumentsDatasheets, drawings and required reports identify the proposed configurationA generic family document is used in place of item-specific evidence
Price basisQuantities, inclusions, exclusions, commercial terms and validity are alignedA lower total depends on omitted items or different terms
Delivery basisProduction, shipment and split-delivery assumptions are statedA date is offered without the conditions or scope revision behind it
Change controlRevised responses retain a visible revision and deviation historyA new file silently replaces an earlier offer

Which documents should be closed before an order?

The exact document set depends on the contract, product and qualification plan. Common control points are:

  1. approved link register and rack overlay;
  2. approved cable schedule or BOM tied to stable link IDs;
  3. item-specific datasheet and drawing for the proposed configuration;
  4. approved deviation and substitution record;
  5. sample or qualification plan, when the project requires one;
  6. inspection, test, acceptance and traceability requirements, including which reports must accompany the order;
  7. quotation revision, validity, commercial terms and agreed schedule assumptions;
  8. order acknowledgment that references the accepted specification and revision.

Keep one rule visible throughout the purchase: approved register first, cable specification second. When asking BWNFiber to discuss a cabling scope, send the controlled inquiry pack rather than a topology screenshot or a generic request for an "HPC cable." A sample, datasheet or quotation request belongs to the downstream item specification and qualification stage, not to an unresolved topology record.

Add an international procurement annex without changing the topology

HPC cabling procurement release gate

The destination region does not determine the cable type, fire class, fiber type, connector, test limit, certification, or installation method. Those decisions require the exact site, pathway, endpoint, contract, and local-authority evidence. Keep them out of the logical link register and add a controlled procurement annex after the register is released.

Project regionConfirm in the annexDo not infer from the region name
North AmericaApplicable project and authority requirements, labels and units, site delivery constraints, inspection documents, and named approversA universal cable listing, fire rating, medium, or acceptance limit
EuropeDestination-specific regulatory and conformity documents, required language, labeling, declarations, transition dates, and contract ownerThat one declaration or product class applies in every country, building, or pathway
Middle EastProject and authority specifications, indoor or exposed route conditions, storage and handling plan, import documents, and delivery accessA standard jacket, environmental rating, or installation method for the whole region
Southeast AsiaCountry and site requirements, controlled versus exposed pathways, customs documents, local-language needs, and phased-delivery planThat climate alone selects the cable construction or test requirement
Latin AmericaOperator, EPC, country and site requirements, document language, customs responsibilities, site access, and delivery sequenceA regional certification, connector, or fiber specification
AfricaCountry, operator and project requirements, route and environment evidence, import responsibilities, storage plan, and document ownersThat an ISP or FTTH project belongs on this HPC page or that one outside-plant design fits all sites

At minimum, record the destination country and site, governing document and revision, named approval authority, language and units, route and environment status, packaging and import instructions, required documents and inspections, requested delivery sequence, commercial basis, owners, and open items. If the contract uses an Incoterms rule, identify the named place and edition rather than writing an unexplained three-letter code; the ICC describes Incoterms 2020 as allocating delivery obligations, cost, and risk. If raw-wood packaging is proposed, confirm destination requirements and the applicable IPPC ISPM 15 evidence instead of assuming every packaging material is covered.

Climate, installation, OEM/ODM, bulk supply, export packaging, test-report availability, MOQ, price, and lead time are downstream product, supplier, contract, or logistics facts. They do not change the approved topology edge. Record them only after the responsible party supplies current evidence for the exact project and proposed item.

Before contacting a cable supplier, have the network authority approve the topology revision and fabric link register. Then add the rack-placement overlay and the exact endpoint equipment list.

For an initial cabling-scope inquiry, send:

  • the approved link register with stable link IDs;
  • the exact device and port inventory;
  • rack elevations or endpoint locations;
  • route-class definitions and route-survey status;
  • which downstream cable decisions have already been approved;
  • a list of blocked fields and their owners.

Ask BWNFiber to confirm whether that package is sufficient to discuss a proposed cabling scope and which cable-definition inputs remain missing. Any response should not be treated as approval of the topology, routing behavior, system performance or third-party compatibility.

Contact us through the BWNFiber cabling-scope contact form only when the approved register is ready.

Frequently asked questions

A topology diagram shows the intended relationships. The register controls each approved edge with a stable ID, exact ports, role, state, and revision.

No. The register closes the logical port assignment; the cable schedule adds the physical interface, route, media, assembly, length, quantity, and acceptance requirements. The stable link ID joins them.

How should multiple HPC rails be represented?

Give each rail a controlled ID and store it on every applicable vertex and edge. Then test each active row against the approved rail assignment.

Which Fat-tree fields belong in the register?

Record the implementation reference, endpoint and switch roles, tier or allowed adjacency, uplink set, rail, exact ports, and state. Tier count, radix, and blocking are project inputs.

Retain both group IDs and the project's link role. Check whether the group relationship matches that role for the named implementation.

What must be retained for a torus topology?

Retain the system's coordinate tuple, neighbor dimension and direction where used, and wraparound state. Device names alone may omit the data needed for the neighbor check.

How do you detect duplicate port assignments?

Normalize each device-port ID and flag any active port used by more than one edge. Canonicalize endpoint pairs to catch reversed duplicates.

Are reserved ports the same as spare cables?

No. A reserved port is device capacity, a deferred edge belongs to a future topology state, and a spare cable is inventory. Track them separately.

Does a rack move require a new topology revision?

Not always. If the endpoints and ports stay the same, revise the rack overlay and downstream route records. Apply the project's change policy to the topology revision itself.

When should media and connector selection enter the workflow?

Select them after the edges and exact ports are approved and both endpoint interfaces are documented. They belong in the physical cable design, not the topology check.

Do equal logical hops require equal cable lengths?

No. Hop count describes logical adjacency, not physical distance. Use the approved system and route design for length, skew, or latency requirements.

Revalidate after a change to ports, devices, rails, tiers, groups, coordinates, edge state, or the source topology revision. A rack-only change still triggers route and cable review.

When are two HPC cabling quotations comparable?

They are comparable only when both responses reference the same approved register and BOM revisions, quantities, inclusions, commercial terms, document requirements and delivery assumptions. Resolve deviations before comparing totals.

Should a buyer request a cable sample before topology approval?

No. First approve the topology and exact port map, then define the proposed cable configuration and evaluation method. Request a sample only when the project qualification plan requires it and the sample represents the configuration being assessed.

Does the project region determine the HPC cable type?

No. The exact endpoints, route, site conditions, governing documents, and approval authority determine the physical cable requirements. Region is an annex field, not a substitute for project evidence.

Record the destination site, governing document and revision, approval authority, language and units, route and environment status, packaging and import instructions, required documents and inspections, requested delivery sequence, commercial basis, owners, and unresolved items.

Technical notice: This guide describes a document-control method for translating an approved topology into a link register. It is for reference only and is subject to the exact system documentation and project engineering authority. It is not a topology, routing, compatibility, performance or procurement guarantee.

Need product support related to this topic?

Share your application, target product line, or OEM requirement, and BWNFiber will recommend the most suitable supply direction.

Talk to BWNFiber