TipsAndTricks/Tinfoil: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
(Add 2.4+ material for enumerating recipes)
 
(4 intermediate revisions by the same user not shown)
Line 6: Line 6:
= Writing simple utility scripts using Tinfoil =
= Writing simple utility scripts using Tinfoil =


The tinfoil API, which was introduced to BitBake a number of years ago and underwent a major rework in the 2.3 (pyro) release, provides a way for scripts to interact with the metadata and call into BitBake/OpenEmbedded code. It's intended to be an easier-to-use wrapper around BitBake's internal code, hence the name. Some of the scripts that come with BitBake / OpenEmbedded make heavy use of it - <code>bitbake-layers</code>, <code>devtool</code>, <code>recipetool</code> and <code>oe-pkgdata-util</code> are examples.
The tinfoil API, which was introduced to BitBake a number of years ago and underwent a major rework in the 2.3 (pyro) release, provides a way for you to write python scripts to interact with the metadata and call into BitBake/OpenEmbedded code. It's intended to be an easier-to-use wrapper around BitBake's internal code, hence the name. Some of the scripts that come with BitBake / OpenEmbedded make heavy use of it - <code>bitbake-layers</code>, <code>devtool</code>, <code>recipetool</code> and <code>oe-pkgdata-util</code> are examples.


== Getting a variable value ==
== Preamble - loading tinfoil ==


Let's take a look at a simple example which reads a configuration-level variable value - <code>TMPDIR</code> to be specific. This script needs to be placed in the <code>scripts</code> subdirectory to work as-is, although you can modify the import code to make it work from anywhere:
There's a bit of mostly boilerplate that needs to be at the start of any script using tinfoil. This is written assuming the script will be placed in the <code>scripts</code> subdirectory, although you can modify the import code to make it work from anywhere:


<pre>
<pre>
Line 27: Line 27:
import bb.tinfoil
import bb.tinfoil


</pre>
== Getting a variable value ==
Let's take a look at a simple example which reads a configuration-level variable value - <code>TMPDIR</code> to be specific (I've omitted the boilerplate above):
<pre>
with bb.tinfoil.Tinfoil() as tinfoil:
with bb.tinfoil.Tinfoil() as tinfoil:
     tinfoil.prepare(config_only=True)
     tinfoil.prepare(config_only=True)
Line 40: Line 47:


<pre>
<pre>
#!/usr/bin/env python3
import os
import sys
# Set up sys.path to let us import tinfoil
scripts_path = os.path.dirname(os.path.realpath(__file__))
lib_path = scripts_path + '/lib'
sys.path.insert(0, lib_path)
import scriptpath
scriptpath.add_bitbake_lib_path()
import bb.tinfoil
with bb.tinfoil.Tinfoil() as tinfoil:
with bb.tinfoil.Tinfoil() as tinfoil:
     tinfoil.prepare(config_only=False)
     tinfoil.prepare(config_only=False)
Line 76: Line 69:
== Enumerating available recipes ==
== Enumerating available recipes ==


Iterating over available recipes in the system is relatively straightforward:
Iterating over available recipes in the system is relatively straightforward.
 
=== 2.4+ ===


<pre>
<pre>
#!/usr/bin/env python3
with bb.tinfoil.Tinfoil() as tinfoil:
    tinfoil.prepare(config_only=False)
 
    for recipe in tinfoil.all_recipes():
        print(recipe.pn)
</pre>


import os
<code>tinfoil.all_recipes()</code> returns an iterator over <code>TinfoilRecipeInfo</code> objects. These have the following attributes/functions:
import sys


# Set up sys.path to let us import tinfoil
{| class="wikitable"
scripts_path = os.path.dirname(os.path.realpath(__file__))
!Attribute/function
lib_path = scripts_path + '/lib'
!Purpose
sys.path.insert(0, lib_path)
|-
import scriptpath
|<code>pn</code>
scriptpath.add_bitbake_lib_path()
|Recipe name (<code>PN</code>)
|-
|<code>pe</code>
|Recipe epoch (<code>PE</code>)
|-
|<code>pv</code>
|Recipe version (<code>PV</code>)
|-
|<code>pr</code>
|Recipe revision (<code>PR</code>)
|-
|<code>fn</code>
|Recipe file name of the default provider (full path with virtual prefix if applicable)
|-
|<code>fns</code>
|All recipe files which provide this same <code>PN</code>
|-
|<code>inherits()</code>
|Classes which the recipe inherits
|-
|<code>inherit_files</code>
|Class files which the recipe inherits
|-
|<code>depends</code>
|Build-time dependencies for this recipe (derived from <code>DEPENDS</code>)
|-
|<code>provides</code>
|Build-time provides for this recipe (derived from <code>PROVIDES</code>)
|-
|<code>alternates</code>
|Other recipe files which provide this same pn (same as <code>inherit_files</code>, excluding the default provider)
|-
|<code>rdepends</code>
|Runtime dependencies of this recipe (<code>RDEPENDS</code>)
|-
|<code>rrecommends</code>
|Runtime recommends of this recipe (<code>RRECOMMENDS</code>)
|-
|<code>rprovides</code>
|Runtime provides of this recipe (<code>RPROVIDES</code>)
|-
|<code>packages</code>
|Runtime packages for this recipe (<code>PACKAGES</code>)
|-
|<code>packages_dynamic</code>
|Runtime dynamic package expressions assumed satisfied by this recipe (<code>PACKAGES_DYNAMIC</code>)
|}


import bb.tinfoil
=== Pre-2.4 ===


<pre>
with bb.tinfoil.Tinfoil() as tinfoil:
with bb.tinfoil.Tinfoil() as tinfoil:
     tinfoil.prepare(config_only=False)
     tinfoil.prepare(config_only=False)
Line 106: Line 152:
* This doesn't yet work with multiconfig - just the main configuration
* This doesn't yet work with multiconfig - just the main configuration


There are some improvements to be made in this area which should hopefully land in 2.4.
There are some improvements to be made to the API in this area which should hopefully land in 2.4.


== Other useful functions ==
== Other useful functions ==


The bitbake/lib/bb/utils.py, meta/lib/oe/utils.py, and meta/lib/oe/recipeutils.py files all contain useful functions that will help you perform various tasks that might be interesting from a script that uses tinfoil.
Tinfoil also gives you access to quite a bit of BitBake and OE utility code - in particular have a look in <code>bitbake/lib/bb/utils.py</code>, <code>meta/lib/oe/utils.py</code>, and <code>meta/lib/oe/recipeutils.py</code> as they all contain useful functions that will help you perform various tasks that might be interesting from a script that uses tinfoil.

Latest revision as of 00:57, 14 March 2018

NOTE: This is currently a draft, needs additional content & editing - PaulEggleton (talk) 18:22, 30 May 2017 (PDT)

Writing simple utility scripts using Tinfoil

The tinfoil API, which was introduced to BitBake a number of years ago and underwent a major rework in the 2.3 (pyro) release, provides a way for you to write python scripts to interact with the metadata and call into BitBake/OpenEmbedded code. It's intended to be an easier-to-use wrapper around BitBake's internal code, hence the name. Some of the scripts that come with BitBake / OpenEmbedded make heavy use of it - bitbake-layers, devtool, recipetool and oe-pkgdata-util are examples.

Preamble - loading tinfoil

There's a bit of mostly boilerplate that needs to be at the start of any script using tinfoil. This is written assuming the script will be placed in the scripts subdirectory, although you can modify the import code to make it work from anywhere:

#!/usr/bin/env python3

import os
import sys

# Set up sys.path to let us import tinfoil
scripts_path = os.path.dirname(os.path.realpath(__file__))
lib_path = scripts_path + '/lib'
sys.path.insert(0, lib_path)
import scriptpath
scriptpath.add_bitbake_lib_path()

import bb.tinfoil

Getting a variable value

Let's take a look at a simple example which reads a configuration-level variable value - TMPDIR to be specific (I've omitted the boilerplate above):

with bb.tinfoil.Tinfoil() as tinfoil:
    tinfoil.prepare(config_only=True)
    tmpdir = tinfoil.config_data.getVar('TMPDIR')
    print('TMPDIR is "%s"' % tmpdir)

If you are querying multiple variable values this is much more efficient than calling out to "bitbake -e" and parsing its output, which was the old way of getting this kind of information.

Parsing recipes

Here's another example where we parse a couple of recipes:

with bb.tinfoil.Tinfoil() as tinfoil:
    tinfoil.prepare(config_only=False)

    # Parse the gzip recipe
    rd = tinfoil.parse_recipe('gzip')
    print('gzip SRC_URI = "%s"' % rd.getVar('SRC_URI'))

    # Parse the kernel recipe (whichever is selected in your configuration)
    rd = tinfoil.parse_recipe('virtual/kernel')
    print('Kernel recipe is %s' % rd.getVar('PN'))
    # Check if the kernel recipe inherits the kernel class - we'd expect True
    print('Kernel recipe inherits kernel class: %s' % bb.data.inherits_class('kernel', rd))
    # Check if the kernel recipe inherits the cmake class - we'd expect False
    print('Kernel recipe inherits cmake class: %s' % bb.utils.inherits_class('cmake', rd))

Notice that in this second example we pass config_only=False - this tells tinfoil to load the data for all recipes enabled in your configuration, just as bitbake does every time you run a normal build. You don't have to specify config_only=False to parse a recipe file if you know the path to it, but it is required if you want to specify it simply by name and get the preferred version / provider.

The parse_recipe() function will accept recipe names, or providers such as virtual/kernel in the second call above. It returns a datastore (usually seen as d within recipes and other python code within OE) which you can then use to get variable values, check inheritance etc.

Enumerating available recipes

Iterating over available recipes in the system is relatively straightforward.

2.4+

with bb.tinfoil.Tinfoil() as tinfoil:
    tinfoil.prepare(config_only=False)

    for recipe in tinfoil.all_recipes():
        print(recipe.pn)

tinfoil.all_recipes() returns an iterator over TinfoilRecipeInfo objects. These have the following attributes/functions:

Attribute/function Purpose
pn Recipe name (PN)
pe Recipe epoch (PE)
pv Recipe version (PV)
pr Recipe revision (PR)
fn Recipe file name of the default provider (full path with virtual prefix if applicable)
fns All recipe files which provide this same PN
inherits() Classes which the recipe inherits
inherit_files Class files which the recipe inherits
depends Build-time dependencies for this recipe (derived from DEPENDS)
provides Build-time provides for this recipe (derived from PROVIDES)
alternates Other recipe files which provide this same pn (same as inherit_files, excluding the default provider)
rdepends Runtime dependencies of this recipe (RDEPENDS)
rrecommends Runtime recommends of this recipe (RRECOMMENDS)
rprovides Runtime provides of this recipe (RPROVIDES)
packages Runtime packages for this recipe (PACKAGES)
packages_dynamic Runtime dynamic package expressions assumed satisfied by this recipe (PACKAGES_DYNAMIC)

Pre-2.4

with bb.tinfoil.Tinfoil() as tinfoil:
    tinfoil.prepare(config_only=False)

    for pn in tinfoil.cooker.recipecaches[""].pkg_pn:
        print(pn)

A few notes about pkg_pn:

  • It's actually a dictionary and so the keys aren't sorted alphabetically
  • It contains every target, not just every recipe file - i.e. there will be variants created through BBCLASSEXTEND in there as well.
  • There's a corresponding pkg_fn dict that maps files to PN values, but bear in mind this also contains virtual:... entries, again created through BBCLASSEXTEND.
  • This doesn't yet work with multiconfig - just the main configuration

There are some improvements to be made to the API in this area which should hopefully land in 2.4.

Other useful functions

Tinfoil also gives you access to quite a bit of BitBake and OE utility code - in particular have a look in bitbake/lib/bb/utils.py, meta/lib/oe/utils.py, and meta/lib/oe/recipeutils.py as they all contain useful functions that will help you perform various tasks that might be interesting from a script that uses tinfoil.