It is approximately 3 years since I have tried to install the VmWare ESXI 6.0 and 6.5 on my small home server consisting of Zotac CI323 nano. The problem I was facing at that time was that the installation was unable to start and was stuck at “Kernel Loading Modules” screen. I have tried to use the Proxmox Hypervisor, but there were some difficulties with the linux graphic drivers as well. I ended up with with using Windows 10,installed Hyper-V and launched the VMs such as this webserver thought the Hyper-V manager. I was aware that this might be not the optimal solution, but then everything worked great and really fast. The only thing I disliked was that windows ate a lot of memory, which was wasted.

Quite recently, I began to notice strange server down-times and was not sure why. Obviously, many hardware related-problems and blue-screens are caused by bad memory, so I ran the MemTest, but without any errors. The unexpected shutdowns began to be happening quite often, so I went to the Windows Event Viewer and realized that there were some critical errors related to “Kernel Power” however there were no power-outages, so I checked the AC adapter with a multi-meter. Still Everything was okay, so I took one evening working on this issue directly with the PC, when suddenly Blue Screen – and the PC rebooted into BIOS instead of windows. Quite strange, perhaps windows somehow managed to change some BIOS settings? So I went to bios and realized, that my Intel SSD 535 was not found during boot. As you can guess, I have immediately started to backing up data, which was to be honest in the last minute I believe. The backup was fortunately successful and I was really glad that I can backup the data with a speed of 500MB/s to another SSD. Really I though that SSDs are the most safe data locations, but after this accident, I begin to rethink this theory as this was in fact my second SSD, which has died (First one being the 128GB OCZ Vertex 2 – which died because of a firmware bug, causing the SSD to be not found in bios as well).

 

Nevertheless,I have decided to buy a new SSD, this time however from another vendor with better MTBF (Mean time Between Failures) and TBW (Tera Bytes Written) properties, as I was unsure what caused the havoc. So that right now,the page you are seeing is served from the Samsung 860 PRO. Yes I have decided to spent extra funds on a better SSD as losing my data (Which of course have previously backup on a per month-basis) seems like not a good deal. Of course I do know, that there are SSDs dedicated for server usage, but after all, my small server with 200 visitors/day and 1 email account really doesn’t need the best on the market hardware – not to mention, that I was quite disappointed by the Intel SSD hard-drives at this time. Since 3 years have passed since I have tried the ESXI, I decided to give it another try with the 6.7 version. (Yes,7.0 version has been released a few weeks ago, but from my experience so far,its not a good decision to stay with the most-up to date systems and usually, being 1-2 versions back works better, since you spent less time solving problems on them).

There were basically 4 problems before I have finished the work on the ESXI:

  1. “Kernel Loading Modules”.
  2. “ESXI – No Networks adapters found”.
  3. “Moving and Converting Hyper-V .vhd files into ESXI .vdmk files”.
  4. Fixing issues inside the VMs such as unsupported network drivers.

Kernel Loading Modules

This is the easiest to fix. When ESXI boots, press Shift + O, leave the default boot parameters and append “ignoreHeadless=TRUE“. For example, if you follow this post and change the boot parameters as stated, the ESXI boots, but as a system and not as an installer, which is likely what you want if you are about to install ESXI. This is however only the first part of the problem, as you need to modify this settings permanently. Install ESXI as you wish and after you remove the installation media, do this trick with IgnoreHeadless again. This time, ESXI should successfully boot from HDD. Now its time to make the change permanent:

  1. Press F2 to customize settings
  2. Login (as root)
  3. Enable ESXI Shell in ” troubleshooting options”.
  4. Launch the ESXI Shell & login there. To launch,press Alt + F1 as stated here.
  5.  Set Ignore Headless Permanently:
  6. Verify that the settings have been applied

No Network Adapters Found

This is something, you wouldn’t expect during installation as each kernel is usually packed with plenty of various drivers. Unfortunately not for VmWare’s own VMKernel. I suggest you take a coffee before proceeding. Basically, we need to inject a corresponding networking driver into the default ESXI .iso. This sounds like a simple step, but its not. First of all, you have to understand, that these drivers are not officially supported, which also means, that if you would like to use the latest ESXI version, you are going to need corresponding drivers, If someone has done this for you or if the older drivers work, you are okay, otherwise you are kind of screwed and need to somehow compile the drivers for the newer ESXI. This is mainly the reason why I don’t like being in line with the most up-to-date systems. Drivers for CI323 ,that do work with ESXI 6.7 are fortunately here. I have tried them on ESXI  7.0 and they don’t work, so don’t bother yourself with this idea. 

Now comes the funny part, there is a script for creating the image for you HERE. I have tried it, but in my case, it always fails with a network error (I am not sure, whether this is because of the script, temporal download server overload or something on my side …). You can give it a try and if this works for you, then most of your work is done and I have to apologize for the unused coffee. Anyway, lets assume you have kind of the same amount of luck, as myself … In that case, you have to download the VmWare’s power CLI and follow the official guide for updating images:

  1. Download ESXI 6.7 R3 from here such as “update-from-esxi6.7-6.7_update03.zip” make sure NOT TO download the .iso ,but “bundle – .zip” version.
  2. Download any drivers also in the form of bundle – .zip files – create a folder somewhere and store all the data there (Such as C:/ESXI67ImageBuilder).
  3. Launch Windows PowerShell and navigate to the corresponding folder.

Displays the available images in the offline bundle you downloaded and added to Software Depot:

Clone reference Image located in the bundle and change acc.level to support our drivers:

List available packages filtered by “Realtek and VFrontDe” params:

Add the displayed packages and drivers (SATA + Network) into the cloned image:

Create the bootable .ISO with included drivers:

Hurray,you can give it a try and boot-up the .ISO!

I do recommend using rufus for creating a bootable USB drive.


Converting Images to VDMK

This step is required only if you need to convert some images,If you dont have one, you can start using ESXI! Othervise, you have to face additional problems. Although ESXI is free with limitations (Such as maximum RAM, Amount of CPUs, VmWareSupport), you would expect VmWare to provide and option to convert VHD files to VDMK files. Well There is and there isnt. You can download (Dont even do that): vCenter Convreter Standalone client, which suports these conversions – BUT: At first, you have to have the VM running on a Hyper-V which doesnt comes handy especially if your Hyper-V is dead for good. I have tried to fix this by Enabling Hyper-V on my desktop and creating new VMs with the backupted .VHD files. This worked, the vCenter Converter was able to detect the VMs and begin the conversion. This is where the additional problem arose. There are 2 types of VDMKs (Read here) – Thin and Thick. In short, if you have a VHD, that is dynamically alocated for up to 400GB, but eats right now only for example 200GB,then this disk is Thin. If you preallocate 400GB for the drive and dedicate the space, then this is called Thick. Of course it makes sense to use Thin disk, but conversion of Thin discs is not supported in free version of ESXI. I have given a try to the StarWind ‘s V2V converter, which is a great free utility and much better than vCenterConverter (At least it doesnt require VM Running).

Still, even the V2V converter fails due to the licence, you are left with 2 options: Buy vSphere for 700 Euro for personal use or navigate yourself to sites such as THIS or THIS (Or any other). On one hand, I really dont like sharing such tips, but on the other hand, this functionality should be in my opinion included in the non-paid version, as it makes sense for VmWare to allow for individuals to move from Hyper-V to ESXI especially, if its intention is to allow the use for individuals.

Fixing problems inside VMs

Hopefully, you convert or move the VM onto the ESXI and it works. But it doesnt have to be so easy! I am right now using the 18.0.4.4 LTS Ubuntu as a server and well to be honest, there is some problem with the networking drivers. The VM boots,but is unable to serve any request,as the VMXNET3 virtual adapter doesnt work by default . I do think that this is quite unfortunate for a server OS.

 For 18.04.4,I have successfully fixed this by creating a systemd service with the following script (netup.sh):

In order for the service to work: Create /etc/systemd/system/esxi_network.service (sudo nano) and paste:

You have to enable the service by

After all the struggles with Installation, error fixing and bug fixing and so on, I think that it was a good decision to move from Windows10’s Hyper-V to ESXI6.7 and I am even looking forward to move to ESXI 7.0 and Kubernetes as well. ESXI is a great tool with a good and sufficient web UI and you have a full control over the VMs and their customizations. Under normal circumstances, even the free licence is sufficient, what I would however love to see is that the cross-platform functionality be improved on VmWare’s side as their driver support is not great and the migration tool is just a mess. Of course there are other alternatives to consider such as the Windows Hyper-V Server (Which is also free) or the Proxmox. I guess only time will tell whether I switch my VMs to different platform, although the imagination of all these struggles again makes me really wonder and I honestly don’t believe, that moving to another platform will be easier.

 

My final though is that when I am about to build a new server machine, I  will likely go for a, lets say, more mainstream PC (Like i3/i5/i7) with enough space for Hardware 2x/4x RAID, which I would love to use after these SSD failures. And by the way, yes,I don’t think I need an Intel Xeon as long as I am not an IT provider, but rather an FPGA engineer :)