Search results

  1. A

    Poor write performance on ceph backed virtual disks.

    I was just reading the latest ceph benchmark PDF from the proxmox folks here for any possible insight. In the single thread sequential write test on a Windows VM, they're only getting 600MB/s on drives that would do near 3000MB/s if directly attached. I'm seeing a similar relative...
  2. A

    Poor write performance on ceph backed virtual disks.

    I don't understand the question but will try to offer some clarification. The enclosure is a 24 X 3.5" bay direct-attach (non-expander) design. groups of 4 drive bays are mapped to 6 X mini-SAS connections. Each node of the cluster is directly attached to 4 of the drive bays in the enclosure...
  3. A

    Poor write performance on ceph backed virtual disks.

    Hello! I've had this issue on my home cluster since I built it in late 2019, but always figured it was caused by being on old/slow hardware and using a mixed bag of consumer SSD/HDDs... ~50MB/s is as good as I can get out of ceph write performance from any VM, to any pool type (SSD or HDD)...
  4. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    I don't know if you have any way to "import" a purchase over the boarder (maybe a friend who could act as a middle-man?), but I found something on ebay that might be of interest...
  5. A

    Small office server tips

    If you're familiar with reloading a firewall config from backup, you know that very complex configurations with numerous packages installed often don't recover properly, so bare hardware is not a great place to configure a complex firewall unless you can afford to lose the config. I like a true...
  6. A

    Small office server tips

    Hi WhiteTiger, Some suggestions: 1. Use server hardware for server purposes. You want a machine that can run for weeks or months between maintenance reboots without any compute or memory errors when hosting lots of VM's, and you also want something that can hold a lot of memory, which is...
  7. A

    Cpu cores and threads.

    Hi Joseph, Modern "big core" CPU architectures have very wide instruction pipelines that often have idle resources when working on a single thread. In order to make use of these idle resources, modern CPU core design has "doubled" the front-end of the CPU so it can schedule 2 threads on...
  8. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    You might want to keep the i9 rig to use as a starting point for the hackintosh stuff. For a "workstation" hosting machine, a fast workstation/gaming computer is still a good thing to maximize application performance. It's just not a good arrangement for hosting server VM's from.
  9. A

    [SOLVED] Setting up multiple nic Network and routing on Proxmox

    I've always found it easier to deal with all the vlan definition right down at the hypervisor node config, rather than up in the VM configs. I think it's more versatile as it allows you to connect a VM to multiple networks by adding more virtual NICs, you know, like if you want to virtualize a...
  10. A

    [SOLVED] Setting up multiple nic Network and routing on Proxmox

    You can carry management vlan1 on the same 10G port with the others if you want. Tagged or untagged, however you want to set it up. Just match it on that port to the switch. It does help with uploading big ISOs ;)
  11. A

    [SOLVED] Setting up multiple nic Network and routing on Proxmox

    Worth pointing out that you don't need to use up both NIC ports for these internal network connections... proxmox can do vlans on an individual interface very easily... Personally, to make things easy to keep track of, I always create a vmbr1 and use it as the management network for the...
  12. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    Upon review I suppose it is using ZFS, though there's no mirror or stripe going on for the pool, just a single large virtual disk.... Come to think of it I remember now I had to disable write sync in the VM and set the disk image in proxmox to "write back" to get decent write performance to...
  13. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    There's no requirement on TrueNAS to configure a storage pool over multiple disks. When running truenas as a VM I just give it a boot disk and a bulk storage disk for each pool I want to define within it. I'll have to check but I don't recall if I'm using ZFS within TrueNAS... Likely not.
  14. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    IMO the best "NAS" is a virtualized instance of TrueNAS or similar NAS OS, running on a proxmox cluster with ceph backed storage. Point in case.... lets say you're thinking about an update from FreeNAS version 11 to TrueNAS version 12. You've read that this doesn't always go smoothly. On bare...
  15. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    Ceph requires a cluster of computers. 4 is a good minimum size cluster. As I said, I have had many HDD failures, added drives, removed drives, and performed several major configuration changes (both physical and logical) of the ceph pools over the last year, but I have not lost any data or...
  16. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    My cluster hums along at about 400W most of the time (includes the switches). That's about $30/mo where I live. In Hawaii it would be closer to $100/mo....
  17. A

    Community Help Request, Purchasing Suggestions and Options (All Opinions Welcome)

    That's quite the post. Wow... lots of thoughts coming to mind... If you want to build a hackintosh, do that as a separate matter. I would not advise running your various server/services/firewall ambitions from the same box, as you're likely to have lots of outages impacting your environment by...
  18. A

    How to change the drive bus?

    After changing the bus type by removing and re-adding the virtual disk, the boot option will have "lost" the drive. Navigate to "options" for the VM and set the disk as the first boot device. You may need to attach as an IDE or SATA device first, set the boot option, and install virtio drivers...
  19. A

    Scale max CPU when migrating

    Assuming both nodes have SMT enabled cores, have you considered reducing the number of virtual cores to 16 on this VM so that it can run on your 8 core nodes? On a 16 core node this still allows the VM access to ~70-80% of compute resources of the node. In practice, it's not usually a good idea...