Setting up HA with Proxmox: A comprehensive guide for production clusters

How to provide reliable availability (HA) with Proxmox for production clusters?

Launch HA with Proxmox Its goal is to reduce service downtime and ensure availability of virtual machines and containers in the face of hardware or software failures. Written for server administrators, DevOps, cloud service providers, and technical teams, this guide covers the fundamentals, prerequisites, practical configuration, and security tips. Real-world Linux examples and commands are provided, and topics such as storage sharing (Ceph/NFS/iSCSI), networking, fencing (STONITH), and various practical solutions are covered.

Requirements and overall architecture for Proxmox HA

To have stable HA on Proxmox you need the following:

• At least 3 physical nodes To ensure quorum in normal mode. In the case of two nodes, you must QDevice Or have a third-party mediator.
• One layer Shared storage For VMs: Ceph RBD It is the best option for HA and live-migration. NFS, iSCSI Or DRBD There are other options too.
• A separate management network for cluster traffic and corosync With low bandwidth and latency.
• Mechanism Fencing/STONITH To avoid split-brain.
• Backup scheduling and monitoring.

Proposed architecture:

• Management Network (Cluster Network): 2x10Gb or 1Gb + LACP, private addressing (e.g. 10.10.0.0/24).
• Storage Network (Ceph): Separate interfaces with MTU 9000 (jumbo).
• Service network: public/VLAN for customer and service access.
• At least 3 Proxmox nodes with Ceph MONs and OSDs or an NFS storage tier with UPS and NAS HA.

Why do we recommend Ceph?

The main reasons for proposing Ceph:

• RBD As a distributed block storage, it allows live migration without the need to move disk files.
• Absence single-point-of-failureVertical and horizontal scalability.
• Native integration with Proxmox via pveceph.

Practical steps: from installation to HA activation

In this example, we will set up a 3-node cluster with Ceph and HA for VMs. The main steps are as follows:

1) Install Proxmox base on each node

Install on any Proxmox VE server (versions 6/7/8 compatible). After installation, set up the management and SSH address and update the system:

sudo apt update && sudo apt dist-upgrade -y

2) Create a cluster (on the first node)

Create the cluster on the first node and check the status:

sudo pvecm create my-cluster
pvecm status

3) Adding nodes to the cluster

Run the following command on each secondary node and then check the cluster status:

sudo pvecm add <IP-نود-اول>
pvecm nodes
pvecm status

4) CoreSync Network and MTU

Make sure bindnetaddr is on /etc/pve/corosync.conf You have a properly configured and low latency network. For Ceph, use MTU 9000 Use it and enable it on the network card and switch.

5) Installing and setting up Ceph (simple example)

Basic commands to install and configure Ceph on each node:

pveceph install
pveceph init --mon-ip <IP-نود>
pveceph createosd /dev/sdb
ceph -s

6) Activate HA Manager and add VM to HA

In the web interface: Datacenter → HA → Add → select resource (e.g. vm:100 or ct:101) and specify policy (restart, migrate).

You can also add the resource from the CLI with pvesh:

pvesh create /cluster/ha/resources -resource vm:100 -state started -max_restart 3
ha-manager status
pvesh get /cluster/ha/status

7) Failover testing

To simulate a node power outage and check if VMs come up on other nodes:

ssh root@node1 "shutdown -h now"
ha-manager status
qm list

Recommended network settings

Recommendations for network design:

• Two separate physical networks for management and storage. VLANs for traffic isolation.
• Bonding (LACP) for redundancy and increased bandwidth. Sample configuration:

auto bond0
iface bond0 inet manual
    bond-slaves eth0 eth1
    bond-mode 802.3ad
    bond-miimon 100

For Ceph, it is recommended that the MTU be set to 9000 and that the network switch support jumbo frames.

Fencing (STONITH) and preventing split-brain

Fencing is necessary to isolate a faulty node. Proxmox supports fence-agents.

apt install fence-agents

Example IPMI command for fencing:

fence_ipmilan -a <IP> -l <USER> -p <PASS> -o poweroff

In cloud or data center environments, you can use IPMI/BMC or network-based fencing (e.g. fence_virt) used. In case of two nodes, be sure to create a QDevice with the command pvecm qdevice setup Or have a third witness.

Tips for all types of applications (trading, gaming, AI/GPU, rendering)

VPS for trading: Choosing a location close to stock exchanges (e.g. London, Frankfurt, New York), using low-latency and anti-DDoS servers, and HA with a fast restart policy.

Gaming VPS: Low ping and high bandwidth are important. HA is useful for league/continuous servers; but for competitive matches, it is common to replicate and snapshot before the match.

GPU Cloud (AI and Rendering): HA is challenging for GPUs because failover of GPU to another node requires vGPU support or re-attach. Recommendation: Use job-level management (e.g. Kubernetes + GPU scheduling) to reschedule jobs in case of node failure.

The service provider offers graphics processing unit (GPU), anti-DDoS server, and VPS for trading and gaming that can be integrated with Ceph and low-latency networking.

Security, backup and monitoring

• Authentication: Enable 2FA for the web interface, manage SSH keys, and restrict access.
• Firewall: Use Proxmox Firewall and strict rules for prots (SSH, corosync, ceph).
• Backup: Define RTO and RPO, snapshot and regular backup to another location (e.g. off-cluster or S3-compatible storage).

vzdump --compress lzo 100 --storage backup-storage 101

Monitoring: Prometheus + Grafana or Zabbix is recommended for monitoring the health of nodes, corosync, ceph, and VMs.

Pre-production checklist

• Failover testing in a test environment before moving to production.
• Verify live migration performance with Ceph RBD and I/O testing.
• Configure and test fencing (disconnect the node and check if it is down or disconnected).
• Specify HA policies: restart vs migrate, max_restart, timeout.
• Backup and archive log program.

Location comparison and selection recommendations

General recommendations for choosing a location based on application:

• Europe (Frankfurt, London): Suitable for European websites and trading; low latency to European financial markets.
• America (Ashburn, NY): Suitable for trading with American markets, CDN and content distribution.
• Asia (Tokyo, Singapore): Suitable for games and services for Asian users and applications requiring processing in Asian regions.

Executive summary and best recommended configuration

• At least 3 Proxmox nodes With separate networks for cluster and storage.
• Ceph RBD As shared storage for HA and live migration.
• IPMI-based fencing or fence-agent and its regular testing.
• Network with bonding, MTU 9000 for Ceph and VLAN for isolation.
• Implement regular monitoring and backups.

Contact the technical and support team

If you want to build a production Proxmox HA cluster or need advice on choosing a location (from 85+ global locations), graphics server (GPU), anti-DDoS server, trading or gaming VPS, or storage advice (Ceph/NFS/iSCSI) — our technical team is ready to design, implement, and test HA for you.

Frequently Asked Questions