The transition to Apple Silicon brought about many exciting things, but one of the capabilities left behind was access to the world of Intel-based virtual machines. X86_64 just isn't a trivial architecture to emulate, and it may never be feasible to do so on Apple Silicon at a useful speed as the operating systems you wish to run gain more and more complexity and become more and more demanding.
You have plenty of great options for emulation, like UTM, but the performance penalty is significant, and that rules out many use-cases.
I have always used virtual machines as my window into the past. I started emulating Windows 95 via Connectix' Virtual PC a lifetime ago, and in my teenage years explored the exciting years of then-lost alternative operating systems, like BeOS, OS/2, NEXTSTEP, as well as keeping vaguely up to date with Microsoft's doomed 'Longhorn' experiments. Apple's transition to Intel in 2006 opened up whole new opportunities, and spawned VMware Fusion and Parallels for Mac. VMware in particular provided a great, Mac-like experience that really resonated with me, and I've used the app ever since.
But say you are somebody who has used VMware Fusion on their Mac for a while — you might have a library of virtual machines you need to preserve for various work or productivity-related tasks, like a Windows 7 install with Microsoft Office, or Visual Studio. Or you're a Mac developer who needs to be able to run an older verson of macOS to test the backwards-compatibility of their apps. Now, you've just bought a brand new Apple-Silicon-based Mac, and there is just simply no way to run your VMs any more. VMware no longer boots anything under Rosetta, and the VMware Fusion Apple Silicon preview only supports virtualizing Linux as Microsoft's licensing story for Windows on ARM does not include the Mac.
There is, however, another way.
VMware have an enterprise-level operating system called ESXi, which is a bare metal hypervisor. What does that mean in practice? If you have a spare PC and Ethernet, you can install ESXi as its OS, configure it over the network through a web browser, and have it become the dedicated engine that runs your VMware virtual machines. With VMware Fusion Pro, you can connect to that ESXi machine via its IP address, and then be able to run/manage your virtual machines mostly the same as when you could run them locally. VMware Fusion Pro is not free, though if you've been keeping up to date over the years an upgrade license is about $100. ESXi itself offers a free license for home use. To be clear: you can run and control your VMs entirely from ESXi's free web UI, if you don't want to pay for Fusion Pro, and/or use the built-in Screen Sharing or Remote Desktop features of your guest OSes instead.
Not all features of VMware are supported in this manner. You can't use the 'Unity' mode to interleave your Mac and Windows apps, or use the quick launcher menu, for example, and there are quirks like getting black thumbnails for suspended VMs. I found VMware Fusion also tabs windows, which is annoying (fix). It's also kinda frustrating that all the VMs on ESXi are provided as a flat list, with no way to group them or put them in folders — but that's a first-world problem for people with a ton of VMs, not your average user.
What you do get, though, is near-native performance, without taking up any of the resources of the Mac you're working on. Most of the same configuration and management UIs are available, but for those that aren't there is always the browser-based management interface to your ESXi box, which has tons of advanced settings beyond anything VMware usually exposes. You can boot, shut down, suspend, add, delete VMs remotely. You can drag and drop your existing VMs to the server. You can even stream local disc images from VMware Fusion to the VM on the server. Really, the only tell that it's coming over the network at all is a little bit of h264 compression when things are in motion.
One advantage very specific to the low-level way ESXi works, because there's no underlying operating system to get in the way, is the ability to 'pass through' real hardware to the virtual machine — say a specific network card, or a USB hub. Or most importantly for macOS: a GPU. If all you want to do is run Windows or Linux, no need to read on — life is easy.
Things get technical below.
Preamble
From what I gather ESXi has multiple web management interfaces: the free one, and the expensive VSphere/vCenter Server. Trying to figure this stuff out from the VMware website as a home user is near-impossible: it's completely obtuse, enterprise-focused, and there's no ELI5 option. Almost everybody writing about this topic is using the non-free UI, because they are virtualization professionals. I am not; I have used the basic, free UI for everything — so while the pre-existing guides all still apply, the location and wording of the options is slightly different. Use them to inform you, but be prepared to learn as you go. I had to piece together a whole lot of disparate blog and reddit posts to get this set up the way I want, and now that I have I just want to record it all here for posterity (and so I don't have to keep it in my own head, either; 🧵 Twitter thread). Once you've done everything once, the rest is trivial.
Virtualizing macOS
ESXi has the same ability to run macOS VMs that VMware Fusion has, with an important caveat: this only works out of the box if you have ESXi running on a Supported Mac, like the 2018 Mac mini, or a pre-2019 Mac Pro (both of which are still available from Apple.com for the time being). If you have a spare Mac mini, great!
I, however, decided to base my ESXi server on an Intel NUC, a little PC barely much bigger than a Raspberry Pi. Specifically a tenth-generation NUC10i5FNHN in my case; I chose a tenth-generation NUC because from what I read, GPU passthrough wasn't fully working on the NUC11 (yet?), and there is precious little information to go on out there to try and figure out if that situation has changed. All of its hardware seems to be fully supported in the latest versin of ESXi, including onboard networking. You can also buy a whole lot of NUCs for the price of a Mac mini, and they all have upgradable NVMe, RAM, and a 2.5" SATA drive slot, which are huge advantages over most Macs.
While this is not a tutorial, I wanted to link to the sources I used in putting together my setup. In particular, this guide runs through almost everything I needed to get macOS running — use it as a starting point, then come back, because I go beyond below.
There are a couple of elements to running macOS in ESXi on non-Apple hardware:
- Turning off Secure Boot on your ESXi PC to enable the running of scripts and modification of the OS
- Patching ESXi using esxi-unlocker
- Creating an ISO from any given macOS installer
- Creating the macOS virtual machine
- Providing some extra properties (like a real Mac's serial number) to enable the Mac App Store, iMessage, etc
- Installing and setting up macOS
Apple helpfully provides an entire library of legacy versions of macOS as free downloads — How to get old versions of macOS. However, except for macOS 11 which has Apple Silicon support, most of them will need to be downloaded on a supported Intel Mac. From those downloads, you will create the ISOs as per the linked guide — they can't be booted as-is.
Passthrough Graphics for macOS VMs
Additionally, if you've ever used a recent macOS in a virtual machine, you will be keenly aware at how badly it performs without graphics acceleration. If you have a supported Mac running ESXi, like a Mac mini, this part is easy; check out this guide. But in short:
- Ensure networking is set up in your VM
- Enable Screen Sharing (because you will no longer be able to interact with this VM via VMware Fusion when the graphics card takes over the display)
- Turn on passthru for your graphics chip using the ESXi web UI
- Add a 'PCI device' to your VM, with the ESXi web UI, and point it at your passthru graphics chip
- You should use the VMware Tools to set specific resolutions for the VM, if you want Retina/HiDPI.
If you're on a Mac mini, you might be done here; simply wait for your virtual machine to show up in the Network list on your own Mac, then connect to it via Screen Sharing like you might any other machine.
Guides mention some extra passthrough options through the VSphere interface that I can't find in the free UI, but from what I've gathered almost everything is available through the ESXi shell if not exposed in the web interface. Again, you will find bits and pieces of this documented in random helpful blog posts.
I can't remember if this was important, but I set it and forgot about it:
esxcli graphics host set --default-type SharedPassthru
Note also this command from the guide linked earlier, which stops ESXi from automatically claiming the GPU/display for itself at startup:
esxcli system settings kernel set -s vga -v FALSE
Unsupported GPUs
The GPU in the NUC10 I'm using isn't natively recognized by macOS, so, after a lot of experimentation, I opted to put together a secondary disk with the Clover bootloader that injects a series of kexts (FakePCIID) that spoof the device ID such that macOS will light it up. I had to add the NUC10's graphics chip's device-id into the Info.plist in the couple of kexts, and modify the Clover config.plist with the right vendor and framebuffer IDs. The Clover config also specifically spoofs a 2018 Mac mini model ID. Almost all the Hackintosh-specific guides I found told me to use OpenCore instead of Clover, and Lilu/WhateverGreen, but I had precisely zero success with this in ESXi and wasted a day along those lines of exploration.
If that sounds way over your head, don't worry: this disk image, now I've made one, is pretty much standalone, in that I can copy it into new Mac VMs and boot from it and accelerated graphics will 'just work'. It doesn't otherwise screw with macOS or require any weird modifications, and shouldn't get in the way of software updates.
Ready-made disk: 💽 Clover-ESXi-NUC10-PassthroughGPU-BootMe.zip (67MB)
- Attach ready-made Clover disk as hard drive 2 to the VM
- 'Power On To Firmware' in VMware, Enter Setup, and change the boot order to boot from drive 2 first
- Boot!
- [Optional] Edit fstab inside the macOS VM to prevent the 'msdos' Clover disk from mounting at boot. Out of sight, out of mind.
If your ESXi PC has PCIe slots, or an eGPU enclosure attached over Thunderbolt, you could just as easily find a fully-Mac-compatible GPU to dedicate to your VM, or even one for each… (down this road lies madness). There are some undocumented ESXi options that might help.
Virtualizing Mac OS X 10.4 "Tiger"–10.6 "Snow Leopard"
"Lion" 10.7 was the first client verson of Mac OS X that was officially supported in VMware, without any hacks. Mac OS X Server 10.5 and 10.6 were supported, but not the consumer releases. As a result, there are some additional steps to follow to get older versions to boot. In summary:
- Add the extra properties to your VM config as above
- Find and copy
efi32-srvr.romandefi64-srvr.rominto your VM's container with ESXi's Datastore browser - For Tiger, use the efi32 variant as per the steps linked above. For Leopard and Snow Leopard, use 'efi64' in place of 'efi32'.
As an aside: if you have an older Apple Developer account, your 'More Downloads' archive may still have Universal Intel/PowerPC disk images available for 10.4.7 Server, 10.5.0 (9A581), and 10.6.0 (10A432), if you don't really want to trust random ISOs from the internet. Once set up, Mac VMs should have no issues installing OS updates, so you can patch them up to latest.
Conclusion
Moving my VMware Fusion collection to ESXi and getting it set up just right has been a marathon, but I'm so glad I did it. Not only does this bring my VMs with me into an Apple Silicon world, but it also opens up new opportunities like running fully-graphics-accelerated versions of macOS for testing my apps. The NUC is small enough to squirrel away somewhere, but powerful enough to run virtually any Intel-based OS I want. In fact, since there is now a preliminary version of ESXi for ARM (which can run on Raspberry Pi 4, no less), you could even expose ARM-based operating systems, Linux or Windows-on-ARM, to an Intel-based Mac, and have them listed alongside the rest of your VMs in VMware Fusion Pro.
At least now I can comfortably work on Apple Silicon without losing my VMware library.
✌️