5 Simple Steps to Implement System Call in Debian

Ever thought of doing a function that a computer performs, on your own? Fascinated by what happens within the kernel? Well, let’s get started by learning how to implement system call in Debian.

Open source distributions allows us to manipulate the original source code and customize it to our need. It’s a great platform to experiment with the conventional methods and processes done by the operating system. Debian Linux distribution has been used here.

What is a system call?

Before we start implementing system calls, let’s just understand what it actually is. A system call is a way of requesting the OS kernel to do something on your behalf.  Rather than jumping to some code, your program has to ask the CPU to switch into the kernel mode and then go to a predefined location within the kernel to handle your system call. Since all you need is a table entry and a few codes here and there, it’s quite easy to implement your system call. Worth a shot, right?

A note for the readers:

Since this tutorial involves getting a copy of Linux source code, manipulating and finally booting it; if you’re not confident enough to directly try it out in your system, it’s best to try this on a Virtual Machine (VM). To setup a VM in your Linux pc : Read

How to implement system call in Debian:

Step 1: Get the kernel source using APT

The following command will download the kernel source code with Debian patches:

sudo apt-get install linux-source-3.16

Next we’ve to create a new directory to extract the kernel source into

mkdir ~/kernel-src

and change into that directory

cd ~/kernel-src

Extract the kernel source into the current directory

tar -xaf /usr/src/linux-source-3.16.tar.xz

Step 2:Add the new system call source code

Change into the linux source directory

cd ~/kernel-src/linux-source-3.16

Make a new directory for the system call source files and change into it. Let’s call it helloworld

mkdir helloworld && cd helloworld

We need to put a Makefile inside this directory so that the make build automation will be able to compile our source code. Make is a build automation tool that automatically builds executable programs from source code by reading Makefiles which specify how to derive the target program.

touch Makefile

Now let’s add the source code and name it hello.c

The next step is to modify the Makefile. This will ensure that our source file hello.c will be compiled and included in the kernel.

Then, add the helloworld directory to the kernel’s Makefile. So change back to the linux-source directory and edit the Makefile.

cd ~/kernel-src/linux-source-3.16/

Find the line (#842) in the Makefile which says

and change it to

After that we’ve to register the system call in the linux system calls table. So change into the syscalls directory.

cd ~/kernel-src/linux-source-3.16/arch/x86/syscalls/

For 64 bit systems

Edit syscall_64.tbl and append the following line at the end

Note: The syscall number might be different in your case. Just add one to the syscall number in the previous entry (319 + 1 = 320). Also keep note of the syscall number for later use.

For 32 bit systems

Edit syscall_32.tbl and use i386 instead of 64 as ABI, the rest is same for both.

Finally we’ve to include our system call in the syscalls header file, for this:

Change into include directory and find syscalls.h

cd ~/kernel/linux-source-3.16/include/linux/

Edit syscalls.h and append the following line at the end of the file before #endif

asmlinkage long sys_hello(void);

Install Build Tools

sudo apt-get install fakeroot build-essential

Step 3:Configuring the Kernel

Since we’re compiling the same version of the kernel with our system call, the configuration will most likely be the same. So we can use the same configuration proposed by Debian without too many modifications.

Copy the current configuration file.

cp /boot/config-3.16.0-4-amd64 ~/kernel-src/linux-source-3.16/.config

Note: The config filename might be different depending on the kernel version

Update the configuration by running

make oldconfig

If you hadn’t used an old configuration then run the following command to configure everything manually

make menuconfig

Step 4:Compiling the Kernel

Once the configuration process is done we’re ready to compile the kernel. The following command will start the compilation process

make -j5

where N in -jN is the number of build jobs to be run in parallel (which is usually the no. of cores+1).

The process usually takes 2-3 hours to complete.

Packaging the Kernel

In Debian it is possible to install the kernel, just like any other software, from a deb package. Using a deb package is easier and safe. So we’ll package the compiled kernel.

The following command will generate the required Debian packages

make deb-pkg

Step 5:Installing the new Kernel

Now change to the parent directory

cd ~/kernel-src/

Install the generated Deb packages

sudo dpkg -i *.deb

After installation reboot the system. Choose the new kernel while booting.

Testing the System Call

Compile and run the following C program.

Copy the contents of the file :

If you’ve done everything correctly “Hello World!” will be printed in the kernel log, which can be accessed using the following command,

dmesg

The final Output after implementing system call will look like this:

custom-syscall-output

 

Congratulations! You’ve implemented and tested your own system call!

Have fun with kernel development 🙂