I started another thread as a specific question about setup of the 10G network to configure Ceph without "getting into the ditch". So while I'm asking I thought I'd share a zoomed out view of "The Lab" (those who are fans of Space Balls - The Movie will get the inside joke) 
in order to create an overall view of our team's sandbox environment. Aside from the need for management of VMs and containers for various purposes (some of which is alluded to in the "mgr" portion of the diagram) we also will be setting up common services such as NATS.io, Postgres and of course local S3 for object storage. So I'd really like to have this under our collective belts before we get too far down the road with one-off storage setups that would just cause more friction down the road.
Eventually I plan to isolate the 10G Ceph network with VLANs to segment traffic. But for now I'd be happy if I could simply get "hello world.txt" to store in a bucket and get retrieved by an application. My hope is that this picture will be worth 1000 words - and give interested viewers a succinct view into what I would consider to be a pretty garden-variety network configuration (according to Ceph). I've been reading some threads and they often wander pretty deeply into some pretty specialized weeds and that tends to "lose the plot" as it drifts so far from my use case that I can't "connect it back". I'm all for deepening my knowledge of storage config, but the rough edges are still snagging me enough to pause and ask for help here. Thanks in advance for reading with a sympathetic eye!
So if anyone has a suggestion around Ceph setup that lays the ground work for a sequestered backhaul on 10G, with a separate pool for radosgw/S3 support, I'd be greatly appreciative. I'd love a pointer to an actual "paved" or "garden" path that is more directly *pre* scriptive for this scenario rather than broadly *de* scriptive of all possible options that can read like 'sharp edges' to the unseasoned eye. And of course, if the only real option is to trip through the generic RTFM then I'll muscle through it on my own and eventually build a garden path that others can follow. I just thought I'd chime in here with a picture and request in case there's someone that's already created that path in way that's helpful to my use case. Thanks!
in order to create an overall view of our team's sandbox environment. Aside from the need for management of VMs and containers for various purposes (some of which is alluded to in the "mgr" portion of the diagram) we also will be setting up common services such as NATS.io, Postgres and of course local S3 for object storage. So I'd really like to have this under our collective belts before we get too far down the road with one-off storage setups that would just cause more friction down the road.Eventually I plan to isolate the 10G Ceph network with VLANs to segment traffic. But for now I'd be happy if I could simply get "hello world.txt" to store in a bucket and get retrieved by an application.
My hope is that this picture will be worth 1000 words - and give interested viewers a succinct view into what I would consider to be a pretty garden-variety network configuration (according to Ceph). I've been reading some threads and they often wander pretty deeply into some pretty specialized weeds and that tends to "lose the plot" as it drifts so far from my use case that I can't "connect it back". I'm all for deepening my knowledge of storage config, but the rough edges are still snagging me enough to pause and ask for help here. Thanks in advance for reading with a sympathetic eye!So if anyone has a suggestion around Ceph setup that lays the ground work for a sequestered backhaul on 10G, with a separate pool for radosgw/S3 support, I'd be greatly appreciative. I'd love a pointer to an actual "paved" or "garden" path that is more directly *pre* scriptive for this scenario rather than broadly *de* scriptive of all possible options that can read like 'sharp edges' to the unseasoned eye.
And of course, if the only real option is to trip through the generic RTFM then I'll muscle through it on my own and eventually build a garden path that others can follow. I just thought I'd chime in here with a picture and request in case there's someone that's already created that path in way that's helpful to my use case. Thanks!