Monday, February 11, 2013

Shut down ESXi 5.1 guest VMs and the host (free edition) via SSH - the easy way!

Thanks go out to reader Everett for pointing out an easier way to gracefully shutdown guest VMs and the host on a VMware ESXi 5.1 (free) server. 

This is much easier than the method described in the previous post I made on the subject, although that method may have been necessary at the time due to issues with VMware 5.0 U1, or still may be necessary in special cases (clustering, etc.).  The previous method also gives you a tiny bit more control, but it is not supported.

You may want to gracefully shut down your guest VMs and host ESXi 5.1 server via SSH, for example, on the triggering of a UPS power outage event or something similar.

The method is as follows:

1) Install VMware Tools in all guest VMs.

2) Make sure each guest VM is setup to perform the shutdown action "Guest Shutdown" (or you could also use a suspend, if you wanted to) in the virtual host settings "Virtual Machine Startup and Shutdown" section.

3) The following two commands, run in sequence, will shutdown the properly configured guest VMs and the host server also:

/sbin/shutdown.sh && /sbin/poweroff

These commands can be run in sequence via an SSH connection from another system (for example, a batch file and plink on Windows, on a machine running a UPS).  The poweroff will only run if the shutdown.sh script runs successfully.

4) That's it!

Thanks Everett!

32 comments:

  1. Hey Thanks for your Guide!

    I still have one problem. I have the remote-shutdown.sh script lying on "/usr/local/Dell/UPSLocalNodeManager/bin/virt_tools/". If I start it manually it works fine and shuts down all VMs and then the ESXi itself. I'm using Dells LNM to receive the shutdowntrigger after a power outage. If I put in the path to the script in the field "Shutdown script" and run a test nothing happens. Where do I put in the path to the script, so that the ULNM is able to start it? And what exactly do I have to put in there!? Just the path or anything else?

    Thanks in advance!

    ReplyDelete
    Replies
    1. Actually I put the shutdown.sh script on the ESXi datastore and wrote another shellscript to SSH the ESXi to start shutdown.sh ... so im putting in the path to my small SSH script in the ULNM ... Sorry for that, hope you're not confused by now :-)

      Delete
    2. Problem solved ... automated SSH login was only permitted as vi-admin, but not as root, whereas the script is being executed as root :)

      thanks for your good work, you solution works fine!!! :)

      Delete
  2. Is there a way to shutdown the VM's gracefully instead of shutting them off completely? I mean like actually initiate a HALT and SHUTDOWN.EXE -S then shutdown ESXi?

    ReplyDelete
    Replies
    1. Have you installed the VMware tools in the guest VMs? If you do that, then you can configure each guest VM to do a graceful shutdown or save instead of a hard power off.

      The commands in the blog post above are for initiating a graceful shutdown of the VMware host. If you have installed the VMware tools in your guest VMs and configured them to shutdown gracefully when the host is shutdown, then everything should be good.

      Delete
    2. it looks like setting the guest to do a guest shutdown is not enough, I have to set it to auto startup to make it shutdown gracefully with the host.

      Delete
  3. the command /sbin/shutdown.sh && /sbin/poweroff, does shut down the VMs, but the host never shuts off but instead becomes unreachable via vshpere client, although it is pingable, Also, the console screen of the host is still on, but when I try to access it via F2, an error displays saying 'access to the console has been disabled by the administrator'
    am I missing something obvious?
    thanks

    ReplyDelete
    Replies
    1. Hmm - what version of ESXi, and what kind of setup do you have (clustering, shared storage?).

      It's possible that maybe something is hanging or preventing the shutdown script from completing, or maybe it is taking a very long time to shut down a particular virtual or the host itself.

      The error message you're quoting sounds like "Lockdown Mode" - not sure how your system would have gotten into that state unless it was intentionally done. Check the following URL -> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008077

      You might this -> try manually shutting down all the guests on your box by hand. With them all shut down, then try issuing the two commands and see what happens to the host. If the host shuts down cleanly that way, I'd take a guess that one of the guests is preventing the host from shutting down or taking a long time to do so.

      Delete
    2. This has happened to me if there are any VM's that are not part of the Automatic Startup (EG: "Any Order" vs "Manual Startup". Put the VM in the Any Order group, and then test the script. If the system is hung, reboot it first. You can also view the contents of the sub-script called by shutdown.sh and issue the services start command to get the machines back up and running

      Delete
  4. Did you get to the bottom of this problem? Could you post the final steps if so. Thx MIT

    ReplyDelete
  5. Hello to all,
    can someone please post the whole procedure?
    I have to shutdown ESXi 5.1 free edition, and as stated in Williams blog http://www.virtuallyghetto.com/2011/06/dreaded-faultrestrictedversionsummary.html that kind of operations are no longer allowed in FREE version.
    I have HP power protect tools instaled in VMA, and from there I was planing to execute script shutdownESXi.pl but I get error
    SOAP fault:
    Fault string: Current license or esxi version prohibits execution of the requested operation.
    Thank you in advance, BR

    ReplyDelete
    Replies
    1. You won't be able to use the shutdownESXi.pl script on ESXi 5.1 free edition, as you have discovered.

      However, you can use a shell script called esxidown.sh that I developed - read the following:
      - http://www.nojokeit.com/2012/11/graceful-shutdown-of-esxi-51-host-free.html
      - esxidown.sh - works on 5.1 free - https://github.com/sixdimensionalarray/esxidown

      Delete
    2. Hi,
      tnx for reply 6DA,
      I came accross that what you're suggesting, and saw the script. Just I never worked with plink, and I don't know how to trigger it with UPS signal, that's why I was hoping for some kind of example of that..
      I'll do my research, and get back with concrete questions if needed..

      Delete
  6. How did You manage to traverse the USB-port from APC Smart-UPS on ESXi VM?

    ReplyDelete
    Replies
    1. You should be able to pass-through a USB port as a virtual USB adapter to one of your guest VMs (the physical USB port can only be shared with a single guest at a time though). There are articles on Google about using APC Smart-UPS and passing them through to the APC management software running in a guest VM. That guest VM could be one you already have, or you might also be able to use the vSphere Management Assistant (vMA) image which is free from VMware.

      Hope that helps!

      Delete
  7. Hi I have created a script on the vMA to ssh in to Esxi and perform the /sbin/shutdown.sh && /sbin/poweroff.

    I have set up password free access to the Esxi with keys.

    The scrip is

    ssh root@xxx.xxx.xxx.xxx

    /sbin/shutdown.sh && /sbin/poweroff

    I have added to the vMA sudoers VI-ADMIN:ALL

    I run the script with

    sudo /home/vi-admin/ups.sh

    It stops after each line after loging in to the ESXI....Im missing something here ???

    Tony

    ReplyDelete
  8. Tony,

    1) Have you verified that the ssh root@xxx.xxx.xxx.xxx command is working without anything else? Here are some suggested setup steps for ESXi specifically:
    http://blogs.vmware.com/vsphere/2012/07/enabling-password-free-ssh-access-on-esxi-50.html

    2) If the passwordless SSH is working, how are you calling the script in the SSH line? You can do it all in one line, something like:

    ssh root@xxx.xxx.xxx.xxx "/sbin/shutdown.sh && /sbin/poweroff"

    However, note that after the command has been sent, the SSH shell does not exit until the ESXi server shuts down and breaks the connection. That's not usually a problem, but you may not like that behavior.

    You could get around that problem too - look at the async.sh script found here->
    https://github.com/sixdimensionalarray/esxidown
    That's a way to run commands/scripts in the background, so you can ssh in, run the script/commands as a background job, and then terminate the SSH connection and the server should continue shutting down even in the ssh connection is lost.

    Hope that helps!

    ReplyDelete
  9. Guys, what a fantastic guide you have put together here.

    I have two ESX hosts running 8 VM's, and another physical server. Everything is connected into one APC Smart-UPS 3000, which is connected only to the physical server via an APC USB interface cable (no network management card in the UPS). So to get around that, I've setup Powerchute on the physical server to monitor the UPS status, and when AC power is lost and I have about 5 minutes run time left in the batteries, I've configured it to run a command file. The command file is configured to ssh into the ESX hosts and initiates graceful shutdowns.

    My testing has been absolutely fantastic... it takes about 45-60 seconds to shut down each ESX host.

    Once the command file is finished executing, powerchute proceeds to shut down applications on the physical server and power it down as well.

    Finally, I've set the servers' BIOS up so that they all automatically power back up when AC is restored.

    No need for the paid edition of Powerchute, no need for the APC network management card, just standard out of box gear and a little creativity thank to you gents.

    Thank you!

    ReplyDelete
  10. I always see people using VM host to attach to your UPS and send the commands, but isn't there a way to install the UPS mon daemon and the USB driver for the UPS onto ESXi and have it do the whole thing?

    ReplyDelete
  11. Script can be improved by shutting down the virtual machines in paralell by calling shutdown on each running machine before going into the wait loop. Exit the wait loop when no virtual machines are left running and proceeed to shutdown the host.

    ReplyDelete
  12. Is there a way to shutdown the VMs and then the host but on a schedule (for example, turning the VMs off at 2:30 AM followed by the host itself)?

    Thanks

    ReplyDelete
  13. Hi, I need some help getting the shutdown script you created to work. My VM IDs are 2, 4 and 6(cyberpower ups). Can you provide some sort of steps on how to add that script to my esxi host and any changes I need to make to have the script run to shutdown my VMs and host at 2:30AM everyday? Will these changes stay once the ESXI host reboots or is there any changes I need to make to do this?


    Thanks

    ReplyDelete
  14. If anyone is wanting to Beta-Test the solution we developped on this topic, you can get it here: www.opmonis/en/downloads
    We have encountered numerous times the same issues with our clients and were annoyed by using workarounds and unsafe scripts using unencrypted login data. Well we finished our ready-to-go solution and invite everybody to test it.

    ReplyDelete
  15. Are you trying to money into a bank account, the overall quantity sometimes can't be straightaway accessed; there could also be a waiting amount for the check to clear? we tend to square measure the corporate

    ReplyDelete
  16. I would like to thank you for the efforts you have made in writing this article.
    super smash flash 2 unblocked | strike force kitty | super smash flash | super smash flash 2

    ReplyDelete
  17. This scheme is useful in times of urgency or immediate need of cash funds. Getting payday loan is very easy from the lender as it requires a very limited amount of paperwork and documentation.

    ReplyDelete
  18. This is regardless of the way that they acknowledge postdated check with inadequate assets.

    ReplyDelete
  19. I'm looking for a few bugs in my post. But I think I should have someone look and point out it.
    colorswitchplay.com

    ReplyDelete
  20. You can get the assets in a couple of hours. There is no holding up period required by any stretch of the imagination.

    ReplyDelete