[SOLVED] Download mirrors for download.proxmox.com?

CRCinAU · May 6, 2020

Hi guys,

Are there any mirrors for download.proxmox.com around?

I never seem to get over 100Kb/sec from there in Australia.... Right now (3:35am Melbourne time), I'm getting a whole 65Kb/sec...

fabian · May 7, 2020

usually such issues are temporary routing problems somewhere between you and our CDN node. which node does 'download.proxmox.com' resolve to for you (e.g., host -a download.proxmox.com)?

CRCinAU · May 7, 2020

Sadly, its been since I first downloaded the proxmox ISO - including the CT containers, and any updates...

Code:

$ host -a download.proxmox.com
Trying "download.proxmox.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63219
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 6

;; QUESTION SECTION:
;download.proxmox.com.          IN      ANY

;; ANSWER SECTION:
download.proxmox.com.   900     IN      CNAME   download.cdn.proxmox.com.

;; AUTHORITY SECTION:
proxmox.com.            81585   IN      NS      ns7.kapper.net.
proxmox.com.            81585   IN      NS      ns3.kapper.net.
proxmox.com.            81585   IN      NS      ns2.kapper.net.
proxmox.com.            81585   IN      NS      ns5.kapper.net.

;; ADDITIONAL SECTION:
ns7.kapper.net.         2987    IN      A       69.10.149.116
ns3.kapper.net.         2987    IN      A       94.136.2.4
ns2.kapper.net.         287     IN      A       92.51.129.95
ns5.kapper.net.         2987    IN      A       62.116.84.18
ns3.kapper.net.         167980  IN      AAAA    2a02:ab8:4::13
ns2.kapper.net.         167980  IN      AAAA    2a01:488:66:1000:5c33:815f:0:1

Received 267 bytes from 127.0.0.1#53 in 1438 ms

CRCinAU · May 7, 2020

Code:

wget -4 http://download.proxmox.com/iso/proxmox-mailgateway_6.2-1.iso
--2020-05-07 18:34:22--  http://download.proxmox.com/iso/proxmox-mailgateway_6.2-1.iso
Resolving download.proxmox.com (download.proxmox.com)... 51.91.38.34
Connecting to download.proxmox.com (download.proxmox.com)|51.91.38.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 915961856 (874M) [application/octet-stream]
Saving to: ‘proxmox-mailgateway_6.2-1.iso’

proxmox-mailgateway_6.2-1.iso     0%[                                                        ]   4.45M  63.5KB/s    eta 2h 33m

CRCinAU · May 7, 2020

Trying to drill down further.... the CNAME chain seems to go:

download.proxmox.com -> download.cdn.proxmox.com -> oc.cdn.proxmox.com -> fr.cdn.proxmox.com which has a txt:

Code:

;; QUESTION SECTION:
;fr.cdn.proxmox.com.            IN      ANY

;; ANSWER SECTION:
fr.cdn.proxmox.com.     60      IN      TXT     "your query is coming from a rest of the world IP, redirecting to a server in France"
fr.cdn.proxmox.com.     60      IN      AAAA    2001:41d0:203:7470::34
fr.cdn.proxmox.com.     60      IN      A       51.91.38.34

fabian · May 7, 2020

if the problem is permanent, it likely means your uplink has bad peering to our provider(s). you can try whether the North American node works better for you (na.cdn.proxmox.com). we don't have a node in Australia unfortunately.

CRCinAU · May 7, 2020

It seems like I always get punted to france - no matter what I try....

I would suggest punting the OC region to the NA mirror instead of falling back to france:

Code:

wget -4 http://na.cdn.proxmox.com/iso/proxmox-mailgateway_6.2-1.iso
--2020-05-07 19:15:58--  http://na.cdn.proxmox.com/iso/proxmox-mailgateway_6.2-1.iso
Resolving na.cdn.proxmox.com (na.cdn.proxmox.com)... 66.70.154.81
Connecting to na.cdn.proxmox.com (na.cdn.proxmox.com)|66.70.154.81|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 915961856 (874M) [application/octet-stream]
Saving to: ‘proxmox-mailgateway_6.2-1.iso’

proxmox-mailgateway_6.2-1.iso    16%[========>                                               ] 142.82M  5.19MB/s    eta 3m 30s

fabian · May 7, 2020

the problem is that this depends on your uplink - some Australian ISPs have better connectivity with Europe, some with NA.

CRCinAU · May 7, 2020

As a long time Aussie network guy - there's no direct peering at all with the EU region - just about every transit goes via the US... Our 'long path' is back through Asia...

In fact, the majority of the countries bandwidth goes via the Southern Cross link: https://en.wikipedia.org/wiki/Southern_Cross_Cable

I doubt there's many (if any) that have a better path to the EU than the US...

EDIT: If you want to confirm this yourself, you can use some of the Aussie looking glasses to check BGP / Traceroutes from a lot of the ISPs here: https://www.ausnog.net/tools/lg

EDIT 2: In the mean time, I've changed the repo URLS from download.proxmox.com to na.cdn.proxmox.com and its soooooo much faster... I wonder if these should be mentioned here: https://pve.proxmox.com/wiki/Package_Repositories

CRCinAU · May 7, 2020

@fabian - Sorry - I do have one more question re the mirrors.... How do I make that apply to the Templates dialog etc for downloading LXC images?

I found `/var/lib/pve-manager/apl-info` - which seems to be the data I'm looking for - but not sure how / when / where that gets populated, and if its possible to change the source of those?

fabian · May 8, 2020

that's unfortunately hard-coded (https://git.proxmox.com/?p=pve-mana...7898853326201618c2fad110c3b0646e;hb=HEAD#l195) - easiest way is probably to locally redirect download.proxmox.com yourself (DNS/Proxy?)

CRCinAU · May 8, 2020

Thanks @fabian - I've already got an RPZ zone in bind for serving NXDOMAIN for ~164,000 spam / scam domains - so I'll look into changing the CNAME for download.proxmox.com -> na.cdn.proxmox.com until a better solution is found...

Ben B · Jul 19, 2021

We've got a local mirror based in Australia now and would like to work with @fabian (or anyone else within the Proxmox org) to get something 'more official' going on

Our mirror is essentially a clone of the (na|fr).cdn.proxmox.com endpoints.
You can utilise it by adding an entry in your hostfile:

Code:

# AU Proxmox Mirror - DataMossa
103.76.40.66    au.cdn.proxmox.com download.proxmox.com
2407:6180:0:a:f89c:8aff:fec1:4888    au.cdn.proxmox.com download.proxmox.com

CRCinAU · Jul 19, 2021

This is great - thanks!

I use a bind RPZ to alias download.proxmox.com to something else - which means it takes effect for everything using that DNS server.

It does look like the DNS zones aren't quite there yet though:

Code:

$ host download.proxmox.com
download.proxmox.com is an alias for au.cdn.proxmox.com.
au.cdn.proxmox.com is an alias for fr.cdn.proxmox.com.
fr.cdn.proxmox.com has address 51.91.38.34
fr.cdn.proxmox.com has IPv6 address 2001:41d0:203:7470::34

EDIT: Oh wait, you said *unofficial*

Ben B · Jul 19, 2021

CRCinAU said:
This is great - thanks!

I use a bind RPZ to alias download.proxmox.com to something else - which means it takes effect for everything using that DNS server.

It does look like the DNS zones aren't quite there yet though:

Code:

$ host download.proxmox.com download.proxmox.com is an alias for au.cdn.proxmox.com. au.cdn.proxmox.com is an alias for fr.cdn.proxmox.com. fr.cdn.proxmox.com has address 51.91.38.34 fr.cdn.proxmox.com has IPv6 address 2001:41d0:203:7470::34

EDIT: Oh wait, you said *unofficial*

Correct, but if you set up a host file entry for now across those boxes you want a local mirror for, it'll work.

For your RPZ implementation, you can likely set "download.proxmox.com CNAME mirror.datamossa.io" which should be fine so long as the APT-HTTP still requests http://download.proxmox.com in the header

t.lamprecht · Jul 19, 2021

CRCinAU said:
This is great - thanks!

I use a bind RPZ to alias download.proxmox.com to something else - which means it takes effect for everything using that DNS server.

It does look like the DNS zones aren't quite there yet though:

Code:

$ host download.proxmox.com download.proxmox.com is an alias for au.cdn.proxmox.com. au.cdn.proxmox.com is an alias for fr.cdn.proxmox.com. fr.cdn.proxmox.com has address 51.91.38.34 fr.cdn.proxmox.com has IPv6 address 2001:41d0:203:7470::34

EDIT: Oh wait, you said *unofficial*

Rather just use na.cdn.proxmox.com (North America) or sg.cdn.proxmox.com (Singapore) as hosts, if the GeoIP lookup is not correct for you.

Those are actually from official ones from us and Australia is connected to both via good sea cable connections, at least better than the current fallback fr.cdn which one gets if they try (currently) non-existent servers like au.cdn....

Ben B · Sep 28, 2021

I'm happy to donate a VM with appropriate storage/specs etc for you guys to use as au.cdn.proxmox.com if you want. Happy for you to manage it etc. Just PM/email me with desired specs and a pubkey.

Our network is well connected domestically so should be fast for all here and in NZ.
https://bgp.he.net/AS138466

t.lamprecht · Sep 28, 2021

Ben B said:
I'm happy to donate a VM with appropriate storage/specs etc for you guys to use as au.cdn.proxmox.com if you want. Happy for you to manage it etc. Just PM/email me with desired specs and a pubkey.

Thanks for the offer, definitively appreciated! I cannot comment on it yet, but I'll forward it for internal discussion.

Search

Search

[SOLVED] Download mirrors for download.proxmox.com?

CRCinAU

Well-Known Member

fabian

Proxmox Staff Member

CRCinAU

Well-Known Member

CRCinAU

Well-Known Member

CRCinAU

Well-Known Member

fabian

Proxmox Staff Member

CRCinAU

Well-Known Member

fabian

Proxmox Staff Member

CRCinAU

Well-Known Member

CRCinAU

Well-Known Member

fabian

Proxmox Staff Member

CRCinAU

Well-Known Member

Ben B

Active Member

CRCinAU

Well-Known Member

Ben B

Active Member

t.lamprecht

Proxmox Staff Member

Ben B

Active Member

t.lamprecht

Proxmox Staff Member