Using Linux Run Levels for VM termination tasks

I’m writing this article because recently I was looking into a solution where I had to upload some VM logs(an AWS EC2 instance logs) to an external storage endpoint(S3 bucket) when the VM is getting terminated. Even though the task looks pretty simple, identifying the triggering point(shutting-down state) and executing a Shell script(to upload the logs) was the BIG QUESTION!

As a solution, I previously did an architectural improvement to my AWS stack where I added an AWS lifecycle hook to the instance autoscaling group, created a Lambda function, and executed an SSM document on the EC2 instance. This worked pretty well. But there are a couple of disadvantages. Those are, a lifecycle hook can be attached into an autoscaling group only, this architecture is specific only to AWS cloud also this will be an additional cost(Lambda function). I will write a separate article about this architecture later on.

After a while, I heard something called Linux Run Levels. This is an inbuilt mechanism in all Linux Kernal Based machines. This was the best fit to get my task done without an additional cost. Also, this solution does not add any extra cost. But the number of learning materials in this area is very limited. Therefore I thought of adding my two cents here.

First, let's see what Linux Run Level is…

Run Levels are based on numbers from 0–6. This number defines the current state of the Linux-based machine. Each run level designates a different system configuration and allows access to different combinations of processes. The following are the definitions of each number(taken from https://www.geeksforgeeks.org/run-levels-linux/).

• 0- System halt i.e the system can be safely powered off with no activity.
• 1- Single user mode.
• 2- Multiple user mode with no NFS(network file system).
• 3- Multiple user mode under the command line interface and not under the graphical user interface.
• 4- User-definable.
• 5- Multiple user mode under GUI (graphical user interface) and this is the standard runlevel for most of the LINUX-based systems.
• 6- Reboot which is used to restart the system.

Now let’s see how run levels are being used in a system…

When a system is booted up the init process will be started and executes the scripts that are required to initialize the hardware, bring up the network and start the GUI. Then it will look for the default run level. The default run level can either be 5 or 1. This default run level can be changed according to your preference too. To find out the current run level you can execute $ runlevel command. This will prompt the current run level that you are in. To manually switch the run level you can execute the following command. This will reboot your system$ sudo telinit 6.

When a run level is changed, the related scripts to that particular run level will be executed in an order. The scripts related to each run level can be found on /etc/rc0.d, /etc/rc1.d, /etc/rc2.d, /etc/rc3.d, /etc/rc4.d, /etc/rc5.d and /etc/rc6.d. If the run level is switched to 0(shutting down), the scripts that are in /etc/rc0.d directory will be executed. rc stands for Run Command.

If we go to these directories(/etc/rc0.d) already there are a couple of scripts. When naming the script there is a standard naming convention. You can have a look at the available scripts that are located in the /etc/rc0.d directory. The scripts start with either S or K. The file name is in S|K[01–99]filename format. Eg:- K01instance-shutdown. The idea behind this name is as follows.

S: Start the service, if it’s not already started.

K: Kill the service, if it’s running.

01–99 is the priority for that start or kill task

If we take the above file name(K01instance-shutdown) it defines a kill service(something which is done at the end). The priority is 01 which means this has a high priority than anything else. If you want another script to be executed after this script then it may take 02 priority.

If we take a closer look in /etc/rc#.d directories we can see that most of these files are symbolic links. The actual file is situated in the /etc/init.d/ directory. A symbolic link of this file is created in the correct rc directory.

Now let’s see a sample LSB file that can be used to create a custom Run Level script. From this script, we can automatically add the execution scripts to the related rc directory.

Let’s try to understand what each line does…

Required-Start: Any script that is required to start this script.

Required-Stop: The facility provided by this script should stop before the listed facilities are stopped to avoid conflicts.

For both the options we have used $local_fs which means we have mounted the local file system. For more details refer to this link.

Provides: This tells about the boot facilities that this script provides so that those will be counted as already present when another script requires it.

Default-Start and Default-Stop: This defines the run-levels in which the service has to be started or stopped. In the above sample, we have added 0, 6 to the Default-Stop. That means when the system is getting shutdown, the commands under the stop case will get executed.

Description: Provide a brief description of the actions of the init script.

In the switch case statement, we have defined the case stop. When the script is executed with the parameter stop, the commands underneath the stop case will get executed. In my case, the log upload commands will be executed when the stop case is triggered.

This script needs to be added to the /etc/init.d directory first. This script will be named as instance-shutdown in this sample. No file extension is needed.

Now let’s make this script executable.

$ chmod +x /etc/init.d/instance-shutdown

We can automatically add the symbolic links with the following command. The following commands will only work if the links were not created before. Otherwise, errors will be reported. By reading the above script the symbolic links will be created in the respective directories. It will use the Default-Start and Default-Stop tags when creating the symbolic links in the respective directories.

$ /lib/systemd/systemd-sysv-install enable instance-shutdown

You can now ls the /etc/rc0.d and the other respective directories to see if the link is properly created. The file name can be seen as K01instance-shutdown.

Then you need to definitely restart the systemctl, for this to get applied.

$ systemctl restart instance-shutdown

Now when you restart the instance or shutdown the instance the K01instance-shutdown script will execute and get your task done.

Hope you got an idea about Linux Run Levels. Cheers and see you back again with another write-up!