Hi there,
This is a bit of an edge case, though being so short on time it's taking me a while to understand the inner workings of IPv6.
There's a reason they say "throw out your understanding of v4 as it has no bearing with v6", holy moly are they right! It's great but needs to be understood.
To preface the below, we are hoping to structure the IPv6 /32 by using the /48-level as category defining, so we can influence routing (via BGP communities) for different services, international/domestic status, etc. So ideally we would not use /48 per-PVE rather per-category, and maintain flexibility to move VMs between PVE hosts without having to re-address a VM's IPv6. However there's a good chance this is impossible and if so, I will come around!
We have an IPv6 /32 configured on 2x Hardware Routers that each receive it over fibre from upstream.
- From there, we have OSPFv3 configured (potentially wrong) to pass that down to the Switching Stack
- From the Switching Stack, we're looking to have a /48 configured per-PVE, then VMs use a /64 within
- From there, each VM can assign /128s to each hosted web asset, and from there IPv6 should work?
From what I understand, at the moment we have a partial functionality state where outbound routing is workable though not inbound, it stops at the gateway address which we have set as
I think we're closer now, and have changed OSPFv3 to have the backbone running on the rtr1/rtr2 /48s, and then set VMs to use the rtr1/rtr2 as their respective GW. This should mean - I think - we can have the VM IPs be in category-based /48s pointing to the router IPs as their GWs, cross-/48.
Avoiding frequent changes to hardware router configs, is there a sane and simple way to get bi-directional routing from VMs/Sites to Internet and back, hopefully delivering on the desire to be able to influence routing at /48 level to differentiate service types? I am out of my depth!
On the routers now, aside from BGP filters we don't have the VM-category /48s configured on them. Just the router v6 addresses in those 2x /48s. I'm not sure how things like ULAs tie into configs, and whether we're trying to do something that's entirely impossible.
ie. We want the below:
Router 1:
Router 2:
Hypervisors: No IPv6 required
VMs on PVE:
VMs on PVE:
Website on VM:
Website on VM:
and so on.
So you could have
How much config (OSPFv3 etc) do we need and where? The hypervisors have no v6 config at the moment, nor any router VMs. Will we need a /48-per-PVE instead? That would invoke a need to re-address when moving VMs around, or do we simply flatten the structure and lose out on routing options per-/48 via BGP?
OSPFv3 is configured, to some extent at least, on the routers and switches (though switches don't yet have v6 address)
(or do we go down the route of redistributing BGP routes into OSPFv3 and try to push down that way? are our goals still impossible?)
Apologies for the poorly structured post. The concept is a struggle and if the whole idea isn't viable, then a minimal-router-changes method to accomplish something close to what we're after would be appreciated!
Thanks,
Linux
This is a bit of an edge case, though being so short on time it's taking me a while to understand the inner workings of IPv6.
There's a reason they say "throw out your understanding of v4 as it has no bearing with v6", holy moly are they right! It's great but needs to be understood.
To preface the below, we are hoping to structure the IPv6 /32 by using the /48-level as category defining, so we can influence routing (via BGP communities) for different services, international/domestic status, etc. So ideally we would not use /48 per-PVE rather per-category, and maintain flexibility to move VMs between PVE hosts without having to re-address a VM's IPv6. However there's a good chance this is impossible and if so, I will come around!
We have an IPv6 /32 configured on 2x Hardware Routers that each receive it over fibre from upstream.
- From there, we have OSPFv3 configured (potentially wrong) to pass that down to the Switching Stack
- From the Switching Stack, we're looking to have a /48 configured per-PVE, then VMs use a /64 within
- From there, each VM can assign /128s to each hosted web asset, and from there IPv6 should work?
From what I understand, at the moment we have a partial functionality state where outbound routing is workable though not inbound, it stops at the gateway address which we have set as
lir:pref:rtr1::1/48 and lir:pref:rtr2::2/48 which is separate to the /48s we're using for categories, with VMs within them.I think we're closer now, and have changed OSPFv3 to have the backbone running on the rtr1/rtr2 /48s, and then set VMs to use the rtr1/rtr2 as their respective GW. This should mean - I think - we can have the VM IPs be in category-based /48s pointing to the router IPs as their GWs, cross-/48.
Avoiding frequent changes to hardware router configs, is there a sane and simple way to get bi-directional routing from VMs/Sites to Internet and back, hopefully delivering on the desire to be able to influence routing at /48 level to differentiate service types? I am out of my depth!
On the routers now, aside from BGP filters we don't have the VM-category /48s configured on them. Just the router v6 addresses in those 2x /48s. I'm not sure how things like ULAs tie into configs, and whether we're trying to do something that's entirely impossible.
ie. We want the below:
Router 1:
abcd:abcd:rtr1::1/48 (gateway 1)Router 2:
abcd:abcd:rtr2::2/48 (gateway 2)Hypervisors: No IPv6 required
VMs on PVE:
abcd:abcd:type:vm12::1/64VMs on PVE:
abcd:abcd:type:vm11::1/64Website on VM:
abcd:abcd:type:vmid::11/128Website on VM:
abcd:abcd:type:vmid::12/128and so on.
So you could have
abcd:abcd:vps:1234::1/128 through to ::whatever for the websites on that VM.How much config (OSPFv3 etc) do we need and where? The hypervisors have no v6 config at the moment, nor any router VMs. Will we need a /48-per-PVE instead? That would invoke a need to re-address when moving VMs around, or do we simply flatten the structure and lose out on routing options per-/48 via BGP?
OSPFv3 is configured, to some extent at least, on the routers and switches (though switches don't yet have v6 address)
(or do we go down the route of redistributing BGP routes into OSPFv3 and try to push down that way? are our goals still impossible?)
Apologies for the poorly structured post. The concept is a struggle and if the whole idea isn't viable, then a minimal-router-changes method to accomplish something close to what we're after would be appreciated!
Thanks,
Linux
Last edited: