Private Cloud on Your Own Hardware: Replicating AWS and Azure On-Prem

June 2, 2026 10 min read Virtualization & Containers

private-cloudvirtualizationinfrastructureopenstackproxmoxhomelab

Contents

There is a particular feeling the first time you use a public cloud. You type a command, and ninety seconds later a virtual machine exists — with an IP address, a firewall, a disk, and an operating system — on hardware you will never see, chosen by software you will never meet. You did not pick the physical server. You did not rack anything. You asked for capacity and the cloud decided where to put it.

That feeling is the thing people want to recreate on their own hardware. Not “a server I can SSH into,” but the cloud experience: self-service, on-demand, API-driven, with the placement decision handled for you. This post is the opening of a series that walks through the open-source platforms that deliver some or all of that on bare metal you own — OpenStack, CloudStack, OpenNebula, Harvester, Incus/LXD, XCP-ng, oVirt, a cloud-like Proxmox, and MAAS — with an honest accounting of each one’s strengths and weaknesses.

Before we get to specific platforms, this post does the groundwork: what capabilities actually make something “a cloud,” the one feature that separates a real cloud platform from a glorified cluster manager, the architectural fork in the road, and why the question is suddenly urgent again in 2026.

What you are actually trying to rebuild

“AWS at home” is a vibe, not a specification. To evaluate platforms you need to break the vibe into concrete capabilities. A public cloud’s IaaS layer gives you roughly eight things:

Capability	What it means	Why it is hard on-prem
Self-service provisioning	Users create resources without filing a ticket	Requires a control plane, not just a hypervisor
API and CLI	Everything scriptable; the UI is just a client of the API	Many on-prem tools bolt an API on as an afterthought
A placement scheduler	The system picks which host runs the workload	This is the part most “cluster managers” simply do not do
Software-defined networking	Virtual networks, security groups, routers, floating IPs on demand	Real SDN (overlays, tenant isolation) is genuinely complex
Software-defined storage	Volumes and object stores created by API, replicated and durable	Covered by Ceph and friends, but integration varies wildly
Multi-tenancy and quotas	Isolated projects, users, and resource limits	Most homelab tools assume a single trusted admin
Identity and access management	Roles, policies, who-can-do-what	Often the weakest area in open-source stacks
Metering and billing	Knowing who used what	Frequently absent or an add-on

No open-source platform nails all eight the way AWS does. The platforms in this series differ mostly in which of these they take seriously. OpenStack tries to do all of them (and pays for it in complexity). Incus does a few of them extremely well and ignores the rest. Knowing which capabilities you actually need is how you avoid deploying ten times more software than your problem requires.

The dividing line: does it have a scheduler?

If you take one idea from this entire series, make it this one. The single feature that separates a cloud platform from a cluster manager is the placement scheduler — the component that, when you ask for a VM, decides on its own which physical host should run it.

It sounds minor. It is the whole game.

A cluster manager (the category Proxmox mostly lives in) lets you run VMs across several nodes, migrate them, and fail them over — but you choose the node at create time. The tool manages the cluster; the human makes the placement decision.
A cloud platform (OpenStack, CloudStack, OpenNebula) takes a request that says nothing about hosts — “give me a 4-vCPU, 8 GB instance” — and a scheduler evaluates every eligible host against filters (does it have capacity? the right hardware? the right network?) and weighers (which host is least loaded? best balanced?), then places the workload automatically.

This is exactly the VMware feature people pay for by name: DRS (Distributed Resource Scheduler), which not only places VMs but continuously rebalances them across the cluster as load shifts. When someone asks “can I do AWS-style provisioning where it just chooses the best machine,” they are asking for a scheduler, whether they know the word or not.

   Cluster manager                       Cloud platform
   ───────────────                       ──────────────
   You: "run this VM on node3"           You: "run this VM" (4 vCPU, 8 GB)
        │                                     │
        ▼                                     ▼
   [places on node3]                    [scheduler evaluates all hosts]
                                         filters:  capacity? hardware? net?
                                         weighers: least loaded? balanced?
                                              │
                                              ▼
                                         [places on the best fit, e.g. node5]

As you read the rest of the series, mentally sort each platform by this test. Some have a real scheduler (OpenStack’s Nova, OpenNebula’s match-making, CloudStack’s planner, Kubernetes’ scheduler under Harvester). Some have only partial or pool-level placement (XCP-ng, oVirt’s policies). Some — notably Proxmox — have essentially none for normal provisioning, and you bolt placement on yourself. None of these is “wrong”; they are different points on a spectrum of how much the software decides for you.

The architectural fork: hyperconverged vs disaggregated

The second decision that shapes everything is how compute and storage relate to each other.

Hyperconverged infrastructure (HCI) fuses compute and storage onto the same nodes. Every server runs both your VMs and a slice of the distributed storage (Ceph, Longhorn, vSAN). Add a node and you add both compute and storage at once.

Strengths: dense, simple to start, fewer moving parts, the default homelab pattern. Three identical boxes become a self-contained cloud.
Weaknesses: compute and storage scale together whether you want that or not; a storage rebuild competes with your VMs for CPU and network; a noisy workload can starve the storage layer.

Disaggregated infrastructure separates the tiers: dedicated compute nodes, a dedicated storage cluster, dedicated network gear, each scaled independently.

Strengths: clean blast radius, independent scaling, predictable performance, the pattern real clouds use.
Weaknesses: more hardware, more to operate, overkill below a certain size.

Most of this series’ platforms can do either, but they have defaults and personalities. Harvester is HCI to its core. OpenStack assumes disaggregation at scale. Proxmox+Ceph is the canonical homelab HCI. There is no universally right answer — there is only the right answer for your node count, your workloads, and your tolerance for operating storage and compute as separate systems.

Where the commercial incumbents sit — and why this matters again now

For two decades the answer to “private cloud on-prem” was, for most enterprises, VMware vSphere with DRS for placement, vSAN for storage, and NSX for networking — or an appliance like Nutanix that packaged HCI into a turnkey product. These are excellent, mature, and genuinely do the automatic-placement thing better than any open-source option. They are also why this series exists in 2026.

After Broadcom’s acquisition of VMware, the licensing model changed in ways that reopened a question many shops considered settled:

Perpetual licenses are gone. Everything is subscription now — annual or multi-year, with mandatory support bundled in.
Per-core pricing replaced per-socket, with a 16-core-per-CPU minimum, which inflates costs on modern high-core CPUs. (Broadcom floated a 72-core minimum, then walked it back after backlash.)
Editions were collapsed into bundles — vSphere Standard and Essentials were discontinued; customers are pushed toward VMware vSphere Foundation (VVF) or the full VMware Cloud Foundation (VCF) stack, paying for storage and networking components even if they do not use them.
A 20% retroactive penalty for late renewals concentrated the mind further.

The net effect: a lot of organizations that would never have considered open-source virtualization three years ago are now actively migrating, and a lot of homelabbers who used to run ESXi for free have already left. Nutanix benefits from some of this exodus, but it is a commercial product with its own licensing. The open-source field — long capable but operationally demanding — suddenly has a much larger and more motivated audience. That is the backdrop for everything that follows.

This series will not cover vSphere or Nutanix in depth; they are the well-documented commercial baseline you are likely comparing against. The honest framing throughout is: the open-source options ask you to trade money for operational effort. Whether that trade is worth it depends entirely on your scale and your appetite for running the platform yourself.

The field this series covers

Here is the map. Each of these gets its own deep-dive post with full strengths and weaknesses; this is the one-paragraph orientation so you know where each one is heading.

OpenStack — the full reference cloud. Nova compute with a real scheduler, Neutron SDN, Cinder block, Keystone IAM, Horizon UI. It is the closest thing to a private AWS and the most operationally demanding by a wide margin; you survive it via a deployment tool like Kolla-Ansible or Canonical’s Sunbeam. Current release as of 2026 is Gazpacho (2026.1).
Apache CloudStack — the underrated workhorse. Far simpler to stand up than OpenStack, with a web UI, full API, multi-tenancy, and a real planner; supports KVM, XCP-ng, and VMware as hypervisors. Quietly runs a lot of real hosting providers.
OpenNebula — the pragmatic private cloud. A genuine scheduler and clean UI without OpenStack’s sprawl; the sweet spot for “I want a real cloud but I want to sleep at night.”
Harvester — cloud-native HCI. KVM via KubeVirt, Longhorn storage, all managed as Kubernetes, tight Rancher integration. The most modern architecture here, if you are willing to inherit Kubernetes’ weight.
Incus / LXD — the lightweight cluster that auto-places. System containers plus VMs under one excellent CLI, clustering that lands instances on the least-loaded member, minimal overhead. Not a full multi-tenant cloud, but the best low-effort path to self-service-with-placement for mostly-Linux workloads.
XCP-ng + Xen Orchestra — the XenServer successor. A polished, fully open Xen platform with an excellent web UI; the natural home for refugees from Citrix Hypervisor.
oVirt — enterprise KVM management with real scheduling policies (affinity, power-saving, balancing), but a heavyweight architecture and an uncertain future after Red Hat’s RHV end-of-life.
Making Proxmox cloud-like — for the many readers already on Proxmox VE: how far OpenTofu, the API, and HA get you toward self-service, and the hard truth that Proxmox does not pick the best node for you at provision time.
MAAS — bare-metal as a service. Not VMs at all: it treats physical machines like cloud instances, allocating “the most appropriate machine” from a pool by hardware constraints. The foundation layer under several of the above.

The series ends with a head-to-head showdown — a strengths-and-weaknesses matrix scored on setup effort, scheduler quality, networking, storage integration, multi-tenancy, UI/API maturity, and community health — plus concrete recommendations by scenario.

Setting honest expectations

Three things are worth saying plainly before anyone goes and deploys OpenStack on a weekend.

None of these is “AWS in a box.” Public clouds are operated by large teams who hide staggering complexity behind a clean API. Reproducing even a slice of that on-prem means you become that team. The API is the easy part; the operations — upgrades, capacity planning, troubleshooting a wedged control plane at 2 a.m. — are the real product you are buying with your time.

The operational cost is the actual price. Open-source private cloud is not free; it is “no license fee, significant labor.” For a three-node homelab that labor might be a fun weekend and occasional maintenance. For a business it is a real headcount decision. Be honest about which side of that line you are on, because it changes the answer completely. A small shop is often better served by Proxmox or even staying on a commercial product than by standing up OpenStack.

Match the platform to the need, not the ambition. It is tempting to deploy the most powerful option because it is the most cloud-like. Resist it. If you need to run twenty VMs across three nodes and occasionally not care which node, Incus or a scripted Proxmox will serve you better than a multi-service cloud you spend more time maintaining than using. The right platform is the smallest one that covers the capabilities you genuinely need from that table near the top.

With that framing in place, the rest of the series goes platform by platform. Next up: OpenNebula, the pragmatic private cloud — the one that, for a lot of people, hits the best balance of real cloud features against the effort to run them.

Sources: