Tag: demystifying kernel threads

Kernel Threads – Part 1 : Kthread Introduction and its api workflow

How kernel threads are created

There are already ample of resources available on the web and books explaining the apis to create kernel threads. But just for the sake of mentioning it explicitly, following are the apis throught which kernel threads are spawned and scheduled.

kthread_create(threadfn, data, namefmt, arg...)
kthread_run(threadfn, data, namefmt, ...)

But most of us are oblivious to what happens behind the scenes when kthread is created.

One might think how a kernel thread is created and how the thread function gets executed by the kernel. Who manages the kernel threads and who decide to execute them. Well its simple. Process management subsystem(PMS) of the OS, it is !

So here I will try to explain how kthreads are created and how scheduler picks these threads and execute them. The explanation will also unravel some basics and will help resolving some common mistakes which, kernel beginners like me , make while using kernel threads.

kthread_run() internally calls kthread_create() and wake the newly created thread. Ultimately kthread_create() calls

13 #define kthread_create(threadfn, data, namefmt, arg...) \
14         kthread_create_on_node(threadfn, data, -1, namefmt, ##arg)

The function will create a new request for creating a thread and insert it into a linked list of request.


    267 struct task_struct *kthread_create_on_node(int (*threadfn)(void *data),
    268                                            void *data, int node,
    269                                            const char namefmt[],
    270                                            ...)
    271 {
..
    274         struct kthread_create_info *create = kmalloc(sizeof(*create),
    275                                                      GFP_KERNEL);
    276
    277         if (!create)
    278                 return ERR_PTR(-ENOMEM);
    279         create->threadfn = threadfn;
    280         create->data = data;
    281         create->node = node;
    282         create->done = &done;
    283
    284         spin_lock(&kthread_create_lock);
    285         list_add_tail(&create->list, &kthread_create_list);
    286         spin_unlock(&kthread_create_lock);

and wakes a kernel thread “kthreadadd_task

    288         wake_up_process(kthreadd_task);

Some background on when kthreadadd_task was created during system initialization, just after init process.

Its pid is 2.


[root@kashish ~]# ps aux | head -5
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1   2828  1116 ?        Ss   Sep06   0:06 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Sep06   0:00 [kthreadd]    <<----
root         3  0.0  0.0      0     0 ?        S    Sep06   0:00 [migration/0]
root         4  0.0  0.0      0     0 ?        S    Sep06   0:01 [ksoftirqd/0]

Check function rest_init() in init/main.c. It executes thread function kthreadd()

Now kthreadd() will dequeue the request from the linked list and call create_kthread()


    483 int kthreadd(void *unused)
    484 {
    485         struct task_struct *tsk = current;
    486
    487         /* Setup a clean context for our children to inherit. */
    488         set_task_comm(tsk, "kthreadd");
    489         ignore_signals(tsk);   <<---why ignoring signals of current(parent). see next blog
..
    495         for (;;) {
..
    501                 spin_lock(&kthread_create_lock);
    502                 while (!list_empty(&kthread_create_list)) {
    503                         struct kthread_create_info *create;
    504
    505                         create = list_entry(kthread_create_list.next,
    506                                             struct kthread_create_info, list);
    507                         list_del_init(&create->list);
    508                         spin_unlock(&kthread_create_lock);
    509
    510                         create_kthread(create);
    511
    512                         spin_lock(&kthread_create_lock);
    513                 }
    514                 spin_unlock(&kthread_create_lock);
    515         }

create_thread() calls kernel_thread() to create new kthread, similar to how kthreadadd_task was created.

    223 static void create_kthread(struct kthread_create_info *create)
    224 {
    225         int pid;
..
    230         /* We want our own signal handler (we take no signals by default). */
    231         pid = kernel_thread(kthread, create, CLONE_FS | CLONE_FILES | SIGCHLD);
..

kthread() is another internal function which adds up more magic. Most work is done by this function to create a kernel process

   1699 pid_t kernel_thread(int (*fn)(void *), void *arg, unsigned long flags)
   1700 {
   1701         return do_fork(flags|CLONE_VM|CLONE_UNTRACED, (unsigned long)fn,
   1702                 (unsigned long)arg, NULL, NULL);
   1703 }

now do_fork() will call kthread() after new thread is created. kthread() will call thread function of newly created kthread.


    175 static int kthread(void *_create)
    176 {
    177         /* Copy data: it's on kthread's stack */
    178         struct kthread_create_info *create = _create;
    179         int (*threadfn)(void *data) = create->threadfn;
    180         void *data = create->data;
...   
    205         if (!test_bit(KTHREAD_SHOULD_STOP, &self.flags)) {
    206                 __kthread_parkme(&self);
    207                 ret = threadfn(data);
    208         }
    209         /* we can't just return, we must preserve "self" on stack */
    210         do_exit(ret);
    211 }

Note : At line 210 we call do_exit() after thread function returns. I will highlight its significance in next section.

Significance of kthread_stop()

I decided to keep a separate section for this api is because most of the times we miss what this api does and how it is linked with thread function execution.

In previous section, I explained what happens when thread function returns – PMS calls do_exit()

Quoting from the kernel –

threadfn() can either call do_exit() directly if it is a
standalone thread for which no one will call kthread_stop(), or
return when 'kthread_should_stop()' is true (which means
kthread_stop() has been called)
This should be, in general, kept in mind before using kthread_stop().

kthread_stop() can only be used when kthread is already running and flag KTHREAD_SHOULD_STOP is set. Otherwise, system will crash.

The above statement will make sense after understanding what kthread_stop() does


    460 int kthread_stop(struct task_struct *k)
    461 {
    462         struct kthread *kthread;
    463         int ret;
    464
    465         trace_sched_kthread_stop(k);
    466
    467         get_task_struct(k);
    468         kthread = to_live_kthread(k);
    469         if (kthread) {
    470                 set_bit(KTHREAD_SHOULD_STOP, &kthread->flags);
    471                 __kthread_unpark(k, kthread);
    472                 wake_up_process(k);
    473                 wait_for_completion(&kthread->exited);
    474         }
    475         ret = k->exit_code;
    476         put_task_struct(k);
    477
    478         trace_sched_kthread_stop_ret(ret);
    479         return ret;
    480 }

The function sets KTHREAD_SHOULD_STOP and wakes up the process(incase it is not active)
, sets the exit code.

KTHREAD_SHOULD_STOP flag is checked by function kthread_should_stop(). This function can be used to loop and let thread function check this flag at every iteration.
When kthread_stop() will be called, thread function will come to know that request to stop this thread has arrived and thread function will safely be stopped.

//thread func of k1
int k1_func(void *arg)
{
        set_current_state(TASK_INTERRUPTIBLE);
        while (!kthread_should_stop()) {
                schedule_timeout(msecs_to_jiffies(1000));
                if (business logic) {
                      ....
                }
                set_current_state(TASK_INTERRUPTIBLE);
        }
        set_current_state(TASK_RUNNING);
        return 0;
}

void __exit cleanup(void)
{
        printk("cleaning module \n");
        if (k1)
            kthread_stop(k1);
}

If thread function has already returned and do_exit() is called on it by kthread(), then call to kthread_stop() might result in system crash.
Therefore caller before calling this function must ensure that task_struct can’t go away.

Uff… Too much information. Have to take rest now. Will try to explain signal handling for kthreads in next section.

Advertisements