Windows error 2 occurred while loading the Java VM

This problem occurred on certain installations of Windows 10, but theoretically could appear on other platforms. After starting the installation, the error message: "Windows error 2 occurred while loading the Java VM" appeared.

The known workaround is to start cmd with elevated privileges (as an admin) and start the installation from the command line. Then everything should work OK.