Binary configuration support

What is Binary configuration?

Binary configuration allows configuration of post built binaries to generate custom image, without a need to rebuild from source

Features that can be supported includes, however the list is tentative.

Selects/deselects and integrates binary packages, i,e., OS packages & Various Applications packages from package pool, to generate the final image ready to deploy.
Binary level offline configuration includes
Configure the users/passwords

Configure the network type etc

Configure the host name

Configure the serial console

Select the services to be started by default

Select the modules to be loaded by default

Security related configuration

configure exportfs format in ramfs/ext3/... partitioning etc

More...

It has following benefits

User need not modify the poky layers content or build configuration for few configuration requirements.
Makes packages and Linux image more configurable, gives platform for managing and extending binary configuration.
Provides clean configuration interface to user.
Allows selection of binary packages to be installed and supports package specific configuration.
Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom Linux image.
Can be further extended to make configuration graphical user interface.

Refer bug [3252]

How it works?

Binary configuration can be used in two ways

Binary configuration integrated to current poky: Useful for current users where target image is generated from poky while building from sources.
Post build binary configuration: Useful for users who want to generate custom configured images from already built package binaries.

Binary configuration integrated to current poky

This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.

Config Commands:

These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.

Ex: shadow package provides adduser config command.

Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section #Defining a new config command.

Config-Library:

All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.

Binary level user configuration (file)

Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections

Selecting/omitting required packages to be installed into target image.
Ex: addpkg(shadow)
Configuring the content
Ex: adduser(test, test123, life=999)
Exporting the rootfs
Ex: diskimage( ext3(size=10240) )

Refer #Binary configuration file format.

Post build binary configuration

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. config-library which contains configuration commands is packed along with SDK and is used in applying the configuration. So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to section #Binary level user configuration (file) for more details regarding Binary level user configuration.

Design details

Supporting Binary level configuration requires:

Supporting a new configuration format.
Parser to parse that format.
Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

Config parser

Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.

Configuration Interface

cquery

Provides a query interface to read the Binary Configuration values given by user.

Ex: to get install packages list.

cwrite

Provides a write interface to write or modify the configuration data. Provided for future use.

Ex: Useful for GUI based configuration editor implementation

capply

Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

Defining a new config command

Config commands are implemented in python in <package_name>_cfg.py file which is kept under <package>/files/configlib/pkgconf folder. Adding a new config command requires

Deriving new config command class from ConfigCmd class.
Add __init__( ) method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
Add/override execute() method and add logic to apply the configuration. Shell functions can also be called from here by using shellexec python library.
Add inherit binary-config-package to shadow bb file

Sample code:

Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
    """config command to add user"""
    uname=""
    passwd=""
    life=""

    def __init__(self,uname, passwd="", life=99999):
        #any input validation can be done here
        self.uname = uname
        self.passwd = passwd
        self.life = life

    def execute(self):
        #write logic to create user
        print "User :"+self.uname+" added"

Appendix

Binary configuration file format

#   Author        : venkatarg@huawei.com
#   Description   : This is a sample format, only for reference however actual functions configuration funtions will differ
#   version       : 0.1
#
#   BNF for conf file:
#
# sectiondef	: '[' sectioninfo ']' [statements]         
# statements	: [ statement ]*                           
# statement 	: operation '(' [paramlist] ')'              
# operation 	: identifier
# paramlist 	: param [, param ]*                         
# param     	: directparam | fullparam | embstatement   
# directparam 	: nondelimiters | '"' anycharcters '"'    
# fullparam   	: identifier '=' (directparam | embstatement) 
# embstatement 	: statement                             
# identifier  	: [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
	openssh,
	busybox,
	update-modules,
	# Additional Package Configuration
	binutils,
	coreutils,
	cracklib,
	#Added for disk Management
	e2fsprogs-e2fsck,
	e2fsprogs-mke2fs,
	e2fsprogs-ext4,
	elfutils,
	elfutils-libebl,
	iproute2,
	iputils,
	logrotate,
	nfs-utils,
	numactl-2.0.3-r0.x86_64,
	numactl-2.0.3-r0.x86,
	libstdcxx,
	libpam,
	pam-plugin-*,
	!pam-plugin-motd,
	!pam-plugin-permit,
	pciutils,
	portmap,
	procps,
	sed,
	shadow,
	sysconfig,
	syslog-ng,
	sysvinit,
	sysvinit-pidof,
	hw-breakpoint,
	#Added for disk Management
	util-linux,
	vlan,
	perf,
	lock-task )

adddebugpkg( strace,
	     ethtool,
	     file,
	     binutils-debug,
	     oprofile,
	     iproute2-debug
	    )


[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
	   package=kernel-module-ramfs,
	   status=e,
	   param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 ) 
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin) 
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
	 shell=/usr/bin,
	 group="mail,ssh,rtos" ) 



[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0) 
ifconfig( eth1, ffe::200 , mode=6) 
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage( 
	partitions (
		partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
		partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
		)
	)
# or
diskimage(
	ext2(size=10240)
	)
# or
flashimage(
	jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
	)

# or
flashimage(
	jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
	cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
	)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd) 

[others]