Extensible SDK Test Plan (eSDK): Difference between revisions

From Yocto Project
Jump to navigationJump to search
Line 40: Line 40:


= Test Strategy =
= Test Strategy =
There are several test approaches for Extensible SDK, such as:
Perform test cases agreed upon the development during periodic Manual (full pass) according to the Schedule.
Write new test cases based on developer request or new features as required.
Maintain current test cases and update them accordingly the new changes.
Perform exploratory testing on existing functionalities.




Line 47: Line 52:


=== Sanity testing ===
=== Sanity testing ===
* Not covered at this moment
* TBD following DEV discussions


=== Performance and Stress ===
=== Performance and Stress ===
* TBD


=== Load testing ===
=== Load testing ===
* TBD


=== System Integration Testing ===
=== System Integration Testing ===
* TBD


===Regression===
===Regression===


== Maintaining the test cases ==
== Maintaining the test cases ==

Revision as of 16:23, 31 May 2016

This article is the test plan for eSDK.

About eSDK

Extensible SDK makes it easy to add new applications and libraries to an image, edit the source for an existing component, test the changes on the target hardware, and also allow you to integrate into the rest of OpenEmbedded build system. In order to Setting up the extensible SDK please go to Setting Up to Use the Extensible SDK.

Objectives

Verify all Extensible SDK components to be fully functional. Components to be verified:

Team members

QA Team involved in eSDK testing

Francisco Pedraza 

Scope

Devtol features to be tested:

  • Add a new recipe.
  • Modify the source for an existing recipe.
  • Upgrade an existing recipe.
  • Show work-space status.
  • Search available recipes.
  • Edit a recipe file in your work-space.
  • Get help on configure script options.
  • Build a recipe.
  • Apply changes from external source tree to recipe.
  • Remove a recipe from your work-space.
  • Deploy recipe output files to live target machine.
  • Undeploy recipe output files in live target machine.
  • Build packages for a recipe.
  • Build image including work-space recipe packages.
  • Run QEMU on the specified image.
  • Build a derivative SDK of this one.
  • Extract the source for an existing recipe.
  • Synchronize the source tree for an existing recipe.
  • Update SDK components.
  • Install additional SDK components.

Test Strategy

There are several test approaches for Extensible SDK, such as: Perform test cases agreed upon the development during periodic Manual (full pass) according to the Schedule. Write new test cases based on developer request or new features as required. Maintain current test cases and update them accordingly the new changes. Perform exploratory testing on existing functionalities.


Test automation

Test Approach

Sanity testing

  • Not covered at this moment
  • TBD following DEV discussions

Performance and Stress

  • TBD

Load testing

  • TBD

System Integration Testing

  • TBD

Regression

Maintaining the test cases

Submitting Bugs

Being part of the Yocto Project, eSDK follows the same Yocto Project guidelines and principles. The guidelines can be found at https://wiki.yoctoproject.org/wiki/Community_Guidelines. eSDK bugs are no different and are tracked into Bugzilla, the official Yocto Project bug tracker. Learn more about our process for reporting bugs.

Requirements

1. As a developer I want the Ability to install needed development libraries and headers from published sstate feeds.

2. As a developer I want Distributable sstate cache for individual components.

3. As a developer I want the Ability to generate images out of binary feeds.

4. As a developer I want the ability to Public eSDK with modifications/addons.

5. As a developer I want the abillity to create eSDK image benchmark.

6. As a developer I want to be able to Develop eSDK software benchmark.

7. As a developer I want to be able to install node.js modules' in bitbake output.

8. As a developer I want to be able to create proper recipes for Node.js modules for devtool.

9. As a developer I want to be able to extend c make recipe creation on recipetool.

10. As a developer I want to be able to create a kernel recipe with custom .config for devtool.

11. As a developer I want to be able to store additional detailed information about builds history.

12. As a developer I want to be able to extend autotools recipe creation.

13. As a developer I want to be able to list all the content of bundles using Bitbake.

14. As a developer I want to be able to validate the way how to detect recipe problems where it can become host-dependant.

HW Requirements

  • Required target hardware.
  • GNU/Linux host.

Software Requirements

  • devtool.
  • qemu.
  • GNU/Linux host.

Environment Requirements

    • YP 2.1 Release:
    • Ubuntu-14.04
    • Ubuntu-14.10
    • Ubuntu-15.04
    • Fedora-21
    • CentOS-6.*
    • CentOS-7.*
    • Debian-7.*
    • Debian-8.*
    • openSUSE-project-13.2

Features

Features to be Tested

1. Beggining on a recipe.

1.1 add.

1.2 modify.

1.3 upgrade.


2. Getting Information.

2.1 Status.

2.2 Check.


3. Working on a recipe in the workspace.

3.1 edit-recipe.

3.2 configure-help.

3.3 build.

3.4 update recipe.

3.5 reset.


4. Testing changes on target.

4.1 deploy-target.

4.2 undeploy-target.

4.3 package.

4.4 build-image.

4.5 runqemu.


5. Advanced.

5.1 build-sdk.

5.2 extract.

5.3 sync.


6. SDK maintenance.

6.1 sdk-update.

6.2 sdk-install.

Schedule

Test execution Cycle

Build Sanity Test/E2E Automated (Weekly Test) Manual (Full Pass)
Daily Master TBD TBD TBD
Weekly Build TBD TBD TBD
Milestone Build TBD TBD TBD

Dependencies

  • YP Dependencies
    • Devtool.
    • poky-glibc script.
    • toolchain

Risk Assumptions

Tools

Release Criteria/ Exit Criteria