BIOS and OS

DISCLAIMER: This post talks about linux os version which use to rely on BIOS. Latest Linux kernel directly drives the hardware and does not use the BIOS

In my school days, I use to play around a lot with my home PC. I use to get intrigued with a screen with something like below. We are gonna talk about BIOS in this post.

When we boot our system it must start and we don’t know if we even need an OS at this point. But somehow later it should load any variant of an OS from some storage(can be a floppy disk, pen drive, etc). This pre-OS environment is a pretty nascent stage and having even proper disk I/O would be a luxury. So we have a program that initiates what we know as an OS running on the system, This program is called BIOS.

BIOS

On PC, the initial boot code is generally called BIOS(Basic input-output system). The built-in BIOS is already configured for devices that live on the motherboard like disks, network interface, serial ports, etc.

• BIOS we use today are a little smarter as they allow us to enter a configuration mode at boot time by holding down one or two special keys. On windows its ESC(multiple times) + F10 ( Yes! its that blue screen with grey tab and dropdowns)
• This configuration allows to select which devices to boot from (Can boot from a pen drive, floppy disk, CD-ROM, etc), configure and test storage devices, and select boot options, set passwords and security settings for system devices, configure power management features, etc.
• Once we decide on a device to boot from then it BIOS loads the first 512 bytes of a disk (This is called MBR- master boot record)
• MBR contains the program which tells the computer from which disk partition to load a secondary boot program called Bootloader

Bootloaders

A program that loads an OS when a computer is turned on. There are two mainly used bootloaders in Linux :

• GRUB (Grand Unified Bootloader: GRUB loads the kernel from /boot/vmlinuz-xx.x and plays a splash screen from the file /boot/grub/splash.xpm.gz.

  • Most of us including me have used this as this is the default boot loader on Fedora, Red Hat, SUSE, windows, etc.
  • By default GRUB reads its configuration from /boot/grub/grub.conf or grub.cfg

  $ cd /boot
  $ ls
  grub vmlinuz-3.13.0-44-generic otherfiles
  $ cd grub
  $ ls
  default  grubenv
  fonts     grub.cfg    
  $ head -n 5 grub.cfg
  #
  # DO NOT EDIT THIS FILE
  #
  # It is automatically generated by grub-mkconfig using templates
  # from /etc/grub.d and settings from /etc/default/grub
  $
  

  GRUB command-line options

  reboot	Reboot the system
  root	Specify a the root device(partition)
  find	Find a file on all mountable partitions
  kernel	Load kernel from the root device
  boot	Boot system from the specified kernel image
  help	interactive help for commands

  For more details about GRUB and its command-line options refer here

• LILO(Traditional Linux Boot Loader) : It picks up its config from /etc/lilo.conf.

  • Used by ubuntu.
  • Slow compared to GRUB If you have an error in lilo.conf then lilo won’t discover the problem until halfway through.

  Both the bootloaders allow command-line flags passing to the kernel. Example of flags

  Option	What does it do?
  init = /sbin/init	Tells the kernel to user /sbin/init as its init program.
  init = /bin/bash	Debug flag which tells to start bash shell
  init = /dev/cupcakes	Tells the kernel to user /dev/cupcakes as root device

These bootloaders then start the kernel which is described in the below sections.

Bootstrapping

• Process during which Kernel is loaded into main memory and beings to execute a variety of tasks. Boot time is a vulnerable period as errors in config files, missing equipment, damaged filesystems, etc can prevent a computer from starting up.

• When a computer is turned on, it executes boot code stored in ROM. That code figures out how to load and start kernel code.

• The Kernel looks over the system hardware and if all goes fine starts the system’s init process which is always process number 1.

  • Init is responsible for doing a lot of steps like filesystem mount, starting system daemons, etc.
  • Init does this with help of series of shell scripts called rc scripts. The manner in which these scripts are executed may vary from OS to OS.

• Order of operations while booting

  • Load and init kernel
  • Hardware device identification and configuration.
  • Creation of kernel threads.
  • Execution of system startup scripts(RC scripts).
  • Multiuser operation

• Admin has control only over the RC scripts using which she/he can bootstrap configuration.

• systemd is a software suite that provides basic building blocks for a Linux OS. All the above startup software is provided in systemd beforehand. The systemctl command is used to examine and control the state of “systemd” system

  systemctl [OPTION] [SERVICE]
  

  Step 1 : Kernel Init

  Kernel code is located usually at /boot/vmlinuz or related names(do cd /boot). In my system directory structure was :

  hack@solodynamo:/boot$ ls
  vmlinuz-3.16.0-4-amd64
  extlinux
  

  • So kernel init has 2 steps i.e First a small program on ROM loads kernel code then kernel loads and performs memory tests to find out available RAM. Also, kernel consists of static data structures so it knows beforehand how much memory it will need so it sets aside a fixed amount of real memory for itself when it starts. This memory is reserved from the user-level process(processes we run). If you notice kernel prints on the console a message reporting the total amt of physical memory and amt available for user processes(hint: If u do ssh to some Unix server..this is the first message we see usually) ex:

    * Documentation:  https://help.ubuntu.com/
    
    System information as of Tue Jan  8 00:48:50 IST 2019
    
    System load:  0.02               Processes:           223
    Usage of /:   64.0% of xx.xxGB   Users logged in:     1
    Memory usage: 49%                IP address for eth0: xxx.xxx.xx.xx
    Swap usage:   0%
    
    Graph this data and manage this system at:
    https://landscape.canonical.com/
    
    Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud
    

  Step 2: Hardware Configuration

  There is already a device configuration present in kernel code. But this is the minimum hardware it requires for execution and in addition kernel by using buses for devices asks the appropriate drivers for information. The driver which doesn’t respond gets disabled.

  Step 3: Kernel Threads

  The Kernel creates several spontaneous processes(non-forks), init is followed by other memory and kernel handler processes. The Import thing to note is that these threads aren’t your normal programmable threads i.e.

  • Kernel itself manages these threads which never block (each thread has its own Process Control Block).
  • These threads favor thread heavy processes but have a high management footprint(slow in context switching).
  • User level threads are managed by a user-level thread lib like libpthreads.a and have a single Process Control Block so If one thread blocks then a process blocks. These threads have a much lower management footprint.

  Below are all the processes in init call which runs on kernel threads(complex names!)

  init	Runs rc scripts
  kjournald	Commits ext3 journal updates to disk
  kswapd	Swaps processes when physical memory is low
  kreclaimd	Reclaims memory pages not been used recently
  ksoftirqd	Handles soft interrupts
  khubd	Configures USB devices

  • A Little intro to above k prefixed processes:

    • kjournald: ext3, or third extended filesystem commonly used by the Linux kernel. Its journaled file system i.e. keeps track of changes not yet committed to the file system’s main part by recording the intentions of such changes in a data structure known as a journal. It’s known to use a lot of CPU under certain loads.

    • kswapd: When physical memory becomes scarce then the Linux memory management subsystem attempts to free physical pages. This task falls to the kernel swap daemon (kswapd).

      • It uses two variables, freepageshigh and freepageslow to decide if it should free some pages. So long as the number of free pages in the system remains above freepageshigh, the kernel swap daemon does nothing.
      • For seeing swap memory used by a processes run cat /proc/swaps No running process in my case

  hack@solodynamo:/proc$ cat /proc/swaps
  Filename                Type        Size    Used    Priority
  

  For seeing all devices(memory disks etc) on which paging and swapping will be done run cat /etc/fstab

  hack@ankitsingh:/proc$ cat /etc/fstab
  # /etc/fstab: static file system information.
  proc    /proc    proc    nodev,noexec,nosuid    0    0
  /dev/vda1    /    ext3    errors=remount-ro    0    1
  
  //xx.xxx.xxx.xx/data_shared /data1 cifs uid=0,credentials=/home/hyfr/.smbcredentials,file_mode=0777,dir_mode=0777,noperm 0 0
  

  Even if there is still available RAM, the Linux Kernel will move memory pages which are hardly ever used into swap space so if you don’t have enough memory, swap will be used quite often and noticeably more during any memory requirement spikes.

  kswapd goes home if there is no swap? No, there are processes that have a memory that is backed by a file..so that data can be swapped to that file.

• kreclaimd: A running system will eventually use all available page frames for inode entries (Everything in Linux is a file and each file has an inode, and this inode contains all of the file’s metadata), etc. Old pages can be freed and invalidated for new uses before physical memory(RAM) is exhausted.

  Page replacement policy used by keclaimd is not a strict LRU based algorithm. LRU consists of two lists active_list and inactive_list

  active_list: Contains all the active set of processes.

  inactive_list: Contains a set of processes that are candidates for reclamation. There are a lot more things that can be discussed which would be better to do in a separate post.

• ksoftirqd: Interrupts is one of the commonly used techniques for multitasking. There are two types of interrupts soft(software) interrupt and hard(hardware) interrupts. No hardware interrupts are limited to the number of interrupt request lines(IRQ) to the processor(CPU) from hardware connectors. ksoftirqd runs where the CPU is under a heavy load of a software interrupt. Hardware interrupts are given priority over software interrupts.

  Usually comes situations when IRQs come very very fast one after the other and the operating system cannot finish servicing one before another one arrives.

  Such interrupts are queued by the operating system for later processing by a special internal process named ksoftirqd. To view run ps -e | grep “ksoftirqd”

  hack@ankitsingh:~$ ps -e
    PID TTY          TIME CMD
      1 ?        00:02:43 systemd
      2 ?        00:00:00 kthreadd
      3 ?        02:31:22 ksoftirqd/0 (here!)
      5 ?        00:00:00 kworker/0:0H
  

• khubd : When a USB device is attached to the system async messages called USB request block(URB) are passed by calling the usbsubmit_urb method.

  • When initially usbhubinit is called it creates daemon kernel thread called khubd. If interested here is the source
  • The khubd thread is responsible for configuring devices.

TLDR;

BIOS ---calls---> Bootloaders ----calls---> kernel ---calls---> init(lot of things happen)\

Thanks!