Yocto Project - User contributions [en]

Binary configuration support

2013-07-17T15:47:05Z

Venkata Ramana Gollamudi: /* What is Binary configuration? */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries to generate custom image, without a need to rebuild from source

Features that can be supported includes, however the list is tentative.
# Selects/deselects and integrates binary packages, i,e., OS packages & Various Applications packages from package pool, to generate the final image ready to deploy.
# Binary level offline configuration includes
#:Configure the users/passwords
#:Configure the network type etc
#:Configure the host name
#:Configure the serial console
#:Select the services to be started by default
#:Select the modules to be loaded by default
#:Security related configuration
#:configure exportfs format in ramfs/ext3/... partitioning etc
#:More...

It has following benefits
* User need not modify the poky layers content or build configuration for few configuration requirements.
* Makes packages and Linux image more configurable, gives platform for managing and extending binary configuration.
* Provides clean configuration interface to user.
* Allows selection of binary packages to be installed and supports package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom Linux image.
* Can be further extended to make configuration graphical user interface.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section [[#Defining a new config command]].

=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration (file)===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

Refer [[#Binary configuration file format]].

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to section [[#Binary level user configuration (file)]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

Binary configuration support

2013-07-17T15:15:26Z

Venkata Ramana Gollamudi: /* Binary level user configuration (file) */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section [[#Defining a new config command]].

=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration (file)===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

Refer [[#Binary configuration file format]].

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to section [[#Binary level user configuration (file)]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

Binary configuration support

2013-07-17T15:13:45Z

Venkata Ramana Gollamudi: /* Post build binary configuration */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section [[#Defining a new config command]].

=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration (file)===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to section [[#Binary level user configuration (file)]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

Binary configuration support

2013-07-17T15:10:25Z

Venkata Ramana Gollamudi: /* Binary level user configuration */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section [[#Defining a new config command]].

=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration (file)===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to sections [[3.1]], [[3.2]], [[3.3]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

Binary configuration support

2013-07-17T15:06:30Z

Venkata Ramana Gollamudi: /* Config Commands: */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section [[#Defining a new config command]].

=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration ===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to sections [[3.1]], [[3.2]], [[3.3]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

Binary configuration support

2013-07-17T14:59:12Z

Venkata Ramana Gollamudi: /* Config Commands: */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package to define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.

Please refer section [[Defining a new config command]].

=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration ===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to sections [[3.1]], [[3.2]], [[3.3]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

File:Configcmd class diagram.jpg

2013-07-17T14:49:53Z

Venkata Ramana Gollamudi: Class design explaining how to add a new config command. It allows to define how each package it can be configured. These are stored along with each package metadata. Ex: shadow package provides “adduser” config command

Class design explaining how to add a new config command. It allows to define how each package it can be configured. These are stored along with each package metadata.
Ex: shadow package provides “adduser” config command

Binary configuration support

2013-07-17T14:46:53Z

Venkata Ramana Gollamudi: /* Design details */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.
=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration ===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to sections [[3.1]], [[3.2]], [[3.3]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

=== Defining a new config command ===
Config commands are implemented in python in ''<package_name>_cfg.py'' file which is kept under ''<package>/files/configlib/pkgconf'' folder. Adding a new config command requires
# Deriving new config command class from ''ConfigCmd'' class.
# Add ''__init__( )'' method with method parameters as per the arguments to be passed to config command, including default arguments. Any validation on command input can also be added here. Store the parameters as class members.
# Add/override ''execute()'' method and add logic to apply the configuration. Shell functions can also be called from here by using ''shellexec'' python library.
# Add <code>inherit binary-config-package</code> to shadow bb file

[[File:configcmd_class_diagram.jpg]]

Sample code:
<pre>
Ex: shadow/files/configlib/lib/pkgconf/shadow_cfg.py

from configcmd import *
class users_adduser (ConfigCmd):
"""config command to add user"""
uname=""
passwd=""
life=""

def __init__(self,uname, passwd="", life=99999):
#any input validation can be done here
self.uname = uname
self.passwd = passwd
self.life = life

def execute(self):
#write logic to create user
print "User :"+self.uname+" added"
</pre>

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

Binary configuration support

2013-07-16T16:48:44Z

Venkata Ramana Gollamudi: /* Binary configuration file format */

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.
=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration ===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to sections [[3.1]], [[3.2]], [[3.3]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser: ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[export]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>

File:Binary configuration design flow.jpg

2013-07-16T16:45:55Z

Venkata Ramana Gollamudi: Binary configuration design flow and interfaces to image generation.

Binary configuration design flow and interfaces to image generation.

File:Post build binary configuration.jpg

2013-07-16T16:43:43Z

Venkata Ramana Gollamudi: Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to gener

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs.

File:Binary config integrated with poky.jpg

2013-07-16T16:41:29Z

Venkata Ramana Gollamudi: This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.

This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.

Binary configuration support

2013-07-16T16:36:24Z

Venkata Ramana Gollamudi: This page explains about binary configuration feature , how it works and design.

== What is Binary configuration? ==
Binary configuration allows configuration of post built binaries. It gets the following features and benefits
* Makes packages and linux image more configurable, gives platform for extending and managing binary configuration.
* Provides clean configuration interface to user.
* Allows post build package selection and package specific configuration.
* Facilitate sharing the package feed containing built packages, so that they can be used to generate a custom linux image.

Refer bug [3252]

== How it works? ==
Binary configuration can be used in two ways
# '''Binary configuration integrated to current poky''': Useful for current users where target image is generated from poky while building from sources.
# '''Post build binary configuration''': Useful for users who want to generate custom configured images from already built package binaries.

== Binary configuration integrated to current poky ==
This allows Binary configuration to work along with poky. So Binary level user configuration changes will reflect into generated image.
[[File:Binary_config_integrated_with_poky.jpg]]

=== Config Commands: ===
These are the configuration commands which allow each package define how it can be configured. These are stored along with each package metadata.
:Ex: shadow package provides ''adduser'' config command.
Config commands can be overridden by different packages which support the same configuration but varying/extending implementation.
=== Config-Library: ===
All the configuration commands are copied to a single library during build. This library is called config-library. This library contains all the configurations that are supported on a particular build.
=== Binary level user configuration ===
Binary level user configuration file contains configuration values/parameters given by user to configure the image. It calls the configuration commands and passes them with appropriate parameters to configure.

The configurations can be broadly divided into 3 major sections
# Selecting/omitting required packages to be installed into target image.
#:<code>Ex: addpkg(shadow)</code>
# Configuring the content
#:<code>Ex: adduser(test, test123, life=999)</code>
# Exporting the rootfs
#:<code>Ex: diskimage( ext3(size=10240) )</code>

For sample user configuration file refer to appendix A.

== Post build binary configuration ==
[[File:Post_build_binary_configuration.jpg]]

Post build binary configuration aims at configuring and generating the image from the built package binaries/repository. This process involves reading the configuration values from binary level configuration file and applying those configurations to generate rootfs. ''config-library'' which contains configuration commands is packed along with SDK and is used in applying the configuration.
So users now can configure the image by passing various configuration values as part of Binary level user Configuration file. Please refer to sections [[3.1]], [[3.2]], [[3.3]] for more details regarding Binary level user configuration.

== Design details ==
Supporting Binary level configuration requires:
*Supporting a new configuration format.
*Parser to parse that format.
*Interfaces to read, modify and apply the configuration on the rootfs; such that image generation procedure can use these interfaces to generate the custom image.

[[File:Binary_configuration_design_flow.jpg]]
=== Config parser: ===
Parses the Binary level user Configuration file and generates the parsed content into a Config data tree. Also reports syntax and semantic errors if any.
=== Configuration Interface ===
==== cquery ====
Provides a query interface to read the Binary Configuration values given by user.
:Ex: to get install packages list.
==== cwrite ====
Provides a write interface to write or modify the configuration data. Provided for future use.
:Ex: Useful for GUI based configuration editor implementation
==== capply ====
Provides an interface to apply the Binary level user configuration values on to the rootfs to generate custom configured image. Configuration commands from config-library are called to apply the configuration.

== Appendix ==
=== Binary configuration file format ===
<pre>
# Author : venkatarg@huawei.com
# Description : This is a sample format, only for reference however actual functions configuration funtions will differ
# version : 0.1
#
# BNF for conf file:
#
# sectiondef : '[' sectioninfo ']' [statements]
# statements : [ statement ]*
# statement : operation '(' [paramlist] ')'
# operation : identifier
# paramlist : param [, param ]*
# param : directparam | fullparam | embstatement
# directparam : nondelimiters | '"' anycharcters '"'
# fullparam : identifier '=' (directparam | embstatement)
# embstatement : statement
# identifier : [ alphanums_-. ]
# nondelimiters : <anycharcters except [ or ( or " or , or ) or ] >

[general]
version( "1.2" )

[pkgs]
addlib( 32,64 )

#minimal/mandatory pkgs are default added
addpkg( libacl )

#add all kernal module except kernel-module-pchspi
addpkg( kernel-module*, !kernel-module-pchspi )

addpkgs( packagegroup-core-boot,
openssh,
busybox,
update-modules,
# Additional Package Configuration
binutils,
coreutils,
cracklib,
#Added for disk Management
e2fsprogs-e2fsck,
e2fsprogs-mke2fs,
e2fsprogs-ext4,
elfutils,
elfutils-libebl,
iproute2,
iputils,
logrotate,
nfs-utils,
numactl-2.0.3-r0.x86_64,
numactl-2.0.3-r0.x86,
libstdcxx,
libpam,
pam-plugin-*,
!pam-plugin-motd,
!pam-plugin-permit,
pciutils,
portmap,
procps,
sed,
shadow,
sysconfig,
syslog-ng,
sysvinit,
sysvinit-pidof,
hw-breakpoint,
#Added for disk Management
util-linux,
vlan,
perf,
lock-task )

adddebugpkg( strace,
ethtool,
file,
binutils-debug,
oprofile,
iproute2-debug
)

[module-init]
configmod( module_squashfs_modules, package=kernel-module-sqashfs, status=e)
configmod( module_ramfs_modules,
package=kernel-module-ramfs,
status=e,
param="rsize=100,caddress=0x444" )

[users]
addgroup( rtos )
adduser( ramana, pwdhhxx, life=9999 )
adduser( user1, pwdhhyy, life=9999 , shell=/usr/bin)
adduser( uname=user2, pwd=pwdhhyy, life=9999 ,
shell=/usr/bin,
group="mail,ssh,rtos" )

[kernel]
kernelparam( "nmi_watchdog=1 rdinit=/sc_init" )

[network]
ifconfig( eth0, 10.18.89.222, 255.255.255.0)
ifconfig( eth1, ffe::200 , mode=6)
addroute( eth0, 10.18.89.*, 192.168.1.1 )
addroute( eth1, ffe::*, FF2::1 , mode=6 )

[security]
include ( security.conf )

[fs]
# should be more detailed and relooked
diskimage(
partitions (
partition( hd0, /home , ext4, size=3G, flags="r acp", fstab=y ),
partition( hd1, / , ext4, size=max, flags="rw acp" , fstab=y )
)
)
# or
diskimage(
ext2(size=10240)
)
# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512 )
)

# or
flashimage(
jffs2( entrypoint=0x80000,size=10240,blocksize=512, fs=/home),
cramfs( entrypoint=0x888888880000,size=10240,blocksize=512, fs="/ !/home")
)

# or
archiveimage( type=cpio.tar.gz )

# or
isoimage(dvd)

[others]

</pre>