Kernel Development QA: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
Line 1: Line 1:
[[Category:Kernel Development QA]]
This article is the test plan for kernel development features in Yocto Project.
This article is the test plan for kernel development features in Yocto Project.


= About Kernel Development =
= About Kernel Development =
Describes common tasks you can perform on Yocto Project using kernel tools, and shows you how to use the metadata required to work with the kernel.  
Describes common tasks you can perform on Yocto Project using kernel tools, and shows you how to use the metadata required to work with the kernel.  
For more information you can review the [http://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html Kernel Development manual].
For more information you can review the [http://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html Kernel Development manual].
Line 15: Line 15:


= Scope =
= Scope =
* linux-yocto-custom
 
* local source
== Types of Tests ==
* local source with parallel meta
 
* local source with recipe-space meta
* Manual tests on different platforms
* External source
* No automated tests at the moment
* defconfig
 
* defconfig + fragments
== Features Tested ==
* linux-yocto meta data + local fragments
: 1. linux-yocto-custom
* building external modules (hello-mod)
:: 1.1 local source
:: 1.2 local source with parallel meta
:: 1.3 local source with recipe-space meta
: 2. External Source
: 3. defconfig
: 4. defconfig + fragments
: 5. linux-yocto meta data + local fragments
: 6. building external modules (hello-mod)
 
The complete set of test cases are documented on the [[Kernel Development Test Cases]] wiki. They are also listed on the master Kernel test plan in Testopia, that can be reached following this [https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=94 link].


= Test Strategy =
= Test Strategy =
Line 33: Line 42:


== Test automation ==
== Test automation ==
Tests will be gradually automated whenever it may be possible.
Tests will be gradually automated, after determining the feasibility to do so following discussions with the development team.
== Test Approach ==


=== Sanity testing ===
== Sanity testing ==
* Not covered at this moment
* Not covered at this moment
* TBD following DEV discussions
* TBD following discussions with the development team


==Test Process==
: 1. Follow the steps as indicated on the [[Kernel Development Test Cases]] wiki.
: 2. Verify that all test cases pass. If not, raise bugs properly.
: 3. Update results to corresponding Testopia Test Run.


== Maintaining the test cases ==
== Submitting Bugs ==
== Submitting Bugs ==
Being part of the Yocto Project, Kernel Development follows the same Yocto Project guidelines and principles. The guidelines can be found at https://wiki.yoctoproject.org/wiki/Community_Guidelines.  
Being part of the Yocto Project, Kernel Development follows the same Yocto Project guidelines and principles. The guidelines can be found at https://wiki.yoctoproject.org/wiki/Community_Guidelines.  
Kernel Development bugs are no different and are tracked into Bugzilla, the official Yocto Project bug tracker. Learn more about [https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Tracking our process for reporting bugs].
Kernel Development bugs are no different than other Yocto Project bugs and are tracked into Bugzilla, the official Yocto Project bug tracker. Learn more about [https://wiki.yoctoproject.org/wiki/Bugzilla_Configuration_and_Bug_Tracking our process for reporting bugs].


=Requirements=
=Requirements=
Line 67: Line 79:
*Debian-8.*
*Debian-8.*
*openSUSE-project-13.2
*openSUSE-project-13.2
=Features=
<b> Features to be Tested </b>
: 1. linux-yocto-custom
:: 1.1 local source
:: 1.2 local source with parallel meta
:: 1.3 local source with recipe-space meta
: 2. External Source
: 3. Defconfig
: 4.Defconfig + Fragments
: 5.building external modules (hello-mod)
The complete set of test cases are documented on the [[Kernel Development Test Cases]] wiki. They are also listed on the master Kernel test plan in Testopia, that can be reached following this [https://bugzilla.yoctoproject.org/tr_show_plan.cgi?plan_id=94 link].


= Schedule =
= Schedule =

Revision as of 16:26, 25 October 2017

This article is the test plan for kernel development features in Yocto Project.

About Kernel Development

Describes common tasks you can perform on Yocto Project using kernel tools, and shows you how to use the metadata required to work with the kernel. For more information you can review the Kernel Development manual.

Objectives

Verify all kernel development components are fully functional.

Team members

QA team members involved in Kernel Development testing:

Jair Gonzalez

Scope

Types of Tests

  • Manual tests on different platforms
  • No automated tests at the moment

Features Tested

1. linux-yocto-custom
1.1 local source
1.2 local source with parallel meta
1.3 local source with recipe-space meta
2. External Source
3. defconfig
4. defconfig + fragments
5. linux-yocto meta data + local fragments
6. building external modules (hello-mod)

The complete set of test cases are documented on the Kernel Development Test Cases wiki. They are also listed on the master Kernel test plan in Testopia, that can be reached following this link.

Test Strategy

There are several test approaches for kernel development, such as:

  • Perform test cases agreed upon development during periodic full pass test cycles, according to the schedule.
  • Write new test cases based on developer requests or new features added, as required.
  • Maintain current test cases and update them according to changes.
  • Perform exploratory testing on existing functionalities.

Test automation

Tests will be gradually automated, after determining the feasibility to do so following discussions with the development team.

Sanity testing

  • Not covered at this moment
  • TBD following discussions with the development team

Test Process

1. Follow the steps as indicated on the Kernel Development Test Cases wiki.
2. Verify that all test cases pass. If not, raise bugs properly.
3. Update results to corresponding Testopia Test Run.

Submitting Bugs

Being part of the Yocto Project, Kernel Development follows the same Yocto Project guidelines and principles. The guidelines can be found at https://wiki.yoctoproject.org/wiki/Community_Guidelines. Kernel Development bugs are no different than other Yocto Project bugs and are tracked into Bugzilla, the official Yocto Project bug tracker. Learn more about our process for reporting bugs.

Requirements

Use cases are documented in the corresponding section for each test case on Kernel Development Test Cases wiki.

HW Requirements

  • Any machine able to build Yocto Project.

Software Requirements

  • Poky
  • GNU/Linux environment

Environment Requirements

  • YP 2.3 Release:
  • Ubuntu-14.04
  • Ubuntu-14.10
  • Ubuntu-15.04
  • Fedora-21
  • CentOS-6.*
  • CentOS-7.*
  • Debian-7.*
  • Debian-8.*
  • openSUSE-project-13.2

Schedule

Every cycle depending on Milestone and release candidate

Test execution Cycle

Build Automated Test Manual Test
Daily Master NO NO
Weekly Build TBD Y
Milestone Build TBD Y

Dependencies

  • YP Dependencies
    • poky
    • meta-kerneltest layer

Risk Assumptions

  • For each change in kernel version and kernel cache, the team needs to change many configuration files in order to achieve a correct execution.
  • Just like any software, the kernel is also susceptible to bugs.

Tools

  • bitbake
  • yocto-layer
  • bitbake-layers
  • any available text editor

Release Criteria/ Exit Criteria

  • All test cases pass.
  • No blocking issues are found.