IB to Ethernet Storage routing

Investigate getting Ceph to the IB interface of Cheaha compute nodes and consuming GPFS from the Ethernet segments used by the OpenStack Cheaha compute nodes.

added Backlog label

added 1 design

A sketch of the required ib-to-eth storage network

We need to use the skyway routers to route IP packets between clients on the IB and Ethernet segments.

Need to see if the skyway os (mellonox) now supports multiple vlans, if so can do the routing there. If not we need an explicit router to route across the 192.168.5.0/24 Ethernet segment network from the Ceph public to the IPoIB devices.

Here is the page describing how to route traffic from IB to an ethernet based storage system.

https://docs.nvidia.com/networking/display/SkywayUM/Networkwide+Deployment+Guidelines

This diagram of the network highlights the components involved.

We will need to configure the "storage-net router" in our diagram to route between the storage ethernet and the skyway ethernet segment. We will target our TIC spine switches for this configuration.

This explicit device is necessary since the skyway is not capable of using multiple vlans on the ethernet side. It uses just one segment and expects an access vlan configured on the switch side.

added 1 design

changed title from Ceph IB to Virtual Cheaha nodes to IB to Ethernet Storage routing

changed the description

added Sprint 24-8 label and removed Backlog label

assigned to @mhanby

added Sprint 24-10 label and removed Sprint 24-8 label

added Sprint 24-11 label and removed Sprint 24-10 label

added Sprint 24-12 label and removed Sprint 24-11 label

added Sprint 24-13 label and removed Sprint 24-12 label

added Sprint 24-14 label and removed Sprint 24-13 label

added Sprint 24-15 label and removed Sprint 24-14 label

added Sprint 24-17 label and removed Sprint 24-15 label

added Sprint 24-18 label and removed Sprint 24-17 label

added Sprint 24-20 label and removed Sprint 24-18 label

added Sprint 24-22 label and removed Sprint 24-20 label

added Sprint 24-23 label and removed Sprint 24-22 label

added Sprint 24-25 label and removed Sprint 24-23 label

added Sprint 25-02 label and removed Sprint 24-25 label

added Sprint 25-03 label and removed Sprint 25-02 label

added Backlog label and removed Sprint 25-03 label

added Sprint 25-04 label and removed Backlog label

added Sprint 25-05 label and removed Sprint 25-04 label

added Sprint 25-06 label and removed Sprint 25-05 label

added Backlog label and removed Sprint 25-06 label

added Sprint 25-07 label and removed Backlog label

added Sprint 25-08 label and removed Sprint 25-07 label