Distro Testing Plan: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(remove unneeded info)
 
(100 intermediate revisions by 4 users not shown)
Line 1: Line 1:
[[Category:Toaster]]
This article is the test plan for enabling OS distributions at the Yocto Project Autobuilder
This article is the test plan for Distro testing


= About Distro Testing =
= About Distro Testing =


Distro Testing is intended to catch bugs that are distribution specific using the yocto-autobuilder.
Distro Testing is part of the process of enabling an OS distribution at the Yocto Project [[Autobuilder]]. It is intended to catch bugs that are distribution specific and would prevent an Autobuilder worker to use such distribution.


= Objectives =
= Test Objectives =


* Verify that Distro executes oe-selftest  
* Verify that Distro executes oe-selftest  
* Verify that Dsitro is able to execute componenets (Eclipse, Toaster, WIC)
* Verify that Distro is able to execute components (Eclipse, Toaster, WIC)
* Veirfy that Distro is able to build per package type (IPK, DEB, RPM)
* Veirfy that Distro is able to build per package type (IPK, DEB, RPM)
* Verify that Distro is able to build different image types (LSB, non LSB)
* Verify that Distro is able to build different image types (LSB, non LSB)
Line 19: Line 18:
= Test Strategy =
= Test Strategy =


The strategy will be divided in two groups one executed once in GDC autobuilders when the Distro is released and another tested continuously on public autobuilders
The strategy will be divided in two groups:


* Review the test objectives once in an Autobuilder instance when the OS distribution is first-time enabled (one time testing)
* [[SWAT]] monitoring at the Autobuilders


== GDC Autobuilder  ==


The first time that Distro is released it will be installed on GDC autobuilder and a series of build steps going to be applied to that Distro,
== Staging AutoBuilder ==
once very build set is PASS that distro will be added to the list of supported sitros by QA team  


=== Steps ===
When an OS Distribution is chosen for inclusion, it will be setup as a worker of an Autobuilder and a series of build steps will run. Once every Buildset shows no failures, that distro will be added to the list of supported Distros by [[QA]], this is a one-time testing activity, not executed in every milestone o release test cycle.
 
[[File:Distro Steps.PNG]]


== Public Autobuilder ==
== Public Autobuilder ==


Once the distro was first time validated by QA team it will be running on public autobuilder different build set already defined [[]], after that it going to be under the focus of SWAT team
Once the distro was first time validated by QA team it will be running on public autobuilder different [https://autobuilder.yoctoproject.org/main/builders build sets] already defined , after that it going to be under the focus of SWAT team
 


= Process =
= Process =
This section describes the process to add a new Distro as supported on Yocto Project
This section describes the process to add a new Distro as supported on Yocto Project


 
[[File:Distro Steps_v2.PNG]]
== Enable Distro ==




== Execute defined build steps ==
== Open a Bug ==


When a new Distro is identified to be added as a supported [[#Supported OS Distributions Criteria | candidate]] a bug should be filed on [https://bugzilla.yoctoproject.org/ Bugzilla] and has to be assigned to the [[#QA Responsible | QA Responsible ]] of the Distro Testing.


== Update Supported Distros ==
*example of a bug [https://bugzilla.yoctoproject.org/show_bug.cgi?id=11005 bug 11005]


=Requirements=
== Install Distro ==
==HW Requirements==
A host system with a minimum of 50 Gbytes of free disk space that is running a supported Linux distribution


==Software Requirements==
QA responsible should ensure that the new Distro is installed properly on the local AB
*Django 1.6
*South 0.8.4
*argparse 1.2.1
*wsgiref 0.1.2
*beautifulsoup4 4.4.0 or greater
*A release of the Yocto Project: http://git.yoctoproject.org/cgit/cgit.cgi/poky/
*YP [http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html Quick Start] requirements
**Git 1.7.8 or greater
**tar 1.24 or greater
**Python 2.7.3


==Environment Requirements==
=== Process To Install New Distro ===
*Supported Browsers – no official browser selected yet, usually Firefox is used
*Supported operating systems as per SANITY_TESTED_DISTROS listed in  meta-yocto/conf/distro/poky.conf
**YP 2.0 Release:
**Ubuntu-14.04
**Ubuntu-14.10
**Ubuntu-15.04
**Fedora-21
**CentOS-6.*
**CentOS-7.*
**Debian-7.*
**Debian-8.*
**openSUSE-project-13.2


*Latest version of Selenium installed as python module
*YP QS (Maybe, depending where Toaster server is installed)
*SQLITE 3 Client


1. Install the new distro


=Features=
2. Install the requirements from the Yocto Project Quickstart
==Features to be tested==
===Backend===
*Toaster installation and first time configuration
**Download poky
**Checkout on desired version
**Start Toaster with “bitbake/bin/Toaster” command


*Data integrity
3. Start an Autobuilder staging instance
**Data presence - the information is being recorded into the database
 
**Data consistency - the information is being recorded properly when modified from UI
  $ git clone http://git.yoctoproject.org/git/yocto-autobuilder
**Build data - all the information from bitbake process is being transmitted properly to the database
 
*Data transfer between Toaster and Bitbake
  # Ensure the owner of cloned repo is different than root (Poky does not work with root user)
*Toaster and Bitbake complete cycle communication
  $ chown -R ab:ab yocto-autobuilder/


;The test runs templates for Toaster backend are are found in Testopia, as following
  $ cd yocto-autobuilder
*Toaster Backend - Weekly: https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4028
*Toaster Backend - Managed Mode: https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4030


===Frontend===
  # Load buildset configuration with nightly-qa-distro
*Verifying if all view elements work (buttons, tables, sort buttons, etc.)
  $ ln -sf buildset-config.yocto-qa/ buildset-config
*Configurations
**Communication between UI-BackEnd
**Browse and build any layers published in [http://layers.openembedded.org/layerindex/branch/master/layers/ OpenEmbedded Metadata Index]
**Import your own layers for building
**Add/Remove layers from configuration
**Set configurations variables
**Select a target or multiple targets to build
*Building images
**Build base images(core-image-minimal, core-image-sato etc)
**Build images with added layers
**Build images after modifying project settings
*Data Presentation
**All builds report table
**View what was built and what packages were installed into final image
**Browse the directory structure of the image
**View the value of all variables form build configuration
**Examine errors, warnings and trace messages
**View information about the Bitbake task executed and reused during a build
**View dependency relations between recipes, packages and tasks
**View performance informations such as build time, task time, CPU usage and disk I/O
**Build details
***Build configuration page
****Build artifacts
****Bitbake variables table
***Tasks report table
***Metrics report tables
****CPU usage
****Disk I/O
****Time
***Included packages report table
****Package details page
***Recipes used during build
****Recipe details page
**Definition data from JSON file
***file path: “/poky/meta-yocto/conf”
***contains default settings and informations for:
****configuration
****layer source
****releases
****bitbake


;The test runs templates for Toaster frontend are are found in Testopia, as following:
  # setup the autobuilder
*Toaster UI: https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4027
  $ . yocto-autobuilder-setup
*Toaster UI - Managed Mode: https://bugzilla.yoctoproject.org/tr_show_run.cgi?run_id=4029
 
  # Generates password for AB Web
  $ htpasswd -b .htpasswd distrouser distropass


== New Features to be tested 2.1 ==
  # start autobuilder
  $ ./yocto-start-autobuilder both


Image Customization
*Create, edit, delete custom image
*Add/remove packages
*Build image
*Image details page
*Downloading the image custom recipe


== Setup covered by QA tests ==
4. Open a browser and enter to http://localhost:8010
*SQLITE database
  Note: If the browser fails to open the URL review AB logs for errors yocto-controller/twistd.log and yocto-worker/twistd.log
**add MySQL database testing
**add running Toaster as a service with Apache (e.g. production instance)


*YP 2.0 Release:
5. Log in with the distrouser and distropass
**Ubuntu-14.04
**Ubuntu-14.10
**Ubuntu-15.04
**Fedora-21
**CentOS-6.*
**CentOS-7.*
**Debian-7.*
**Debian-8.*
**openSUSE-project-13.2


*Firefox browser
6. Enter to http://localhost:8010/builders/nightly-qa-distro and start (force) a build
*Add Chrome as well


== We don’t cover yet ==
7. Wait to the buildsets to end and review the output, load bugs if failures occur
*Layer integrity (covering all layers is impossible)
**AR Mihai to get Brian’s supported layer list
*Toaster-next branch
**what’s the easiest way to run the automated tests for a person.
**holy grail - make a single framework solution.
*Safari browser
*TBD - to check with DEV what it’s not covered yet.


= Schedule =
8. Stop the staging Autobuilder


*Toaster is tested only at Full Pass by QA Team
  $ ./yocto-stop-autobuilder both
**the plan is to run it according to the Test execution Cycle, after automation is in place.
*Toaster is tested on Toaster-next branch before every merge into Master by Dev team


*Toaster will be tested according to the [https://wiki.yoctoproject.org/wiki/Yocto_2.1_Schedule YP 2.1 Schedule].
== Execute Build Sets ==


Build sets to be executed are defined in below table, to add a Distro as supported all the build sets should be PASS on local AB


==Test execution Cycle==
{| class="wikitable"
{| class="wikitable"
! Build
! style="text-align: center;" | RPM
! Sanity Test/E2E
! style="text-align: center;" | DEB
! Weekly Test
! style="text-align: center;" | IPK
! Full Pass
! style="text-align: center;" | Component
! style="text-align: center;" | Multilib
! style="text-align: center;" | World
! oe-selftest
|-
|-
| Daily Master
| ARM
| Y
| x86_64
| Y
| x86_32
| NO
| Eclipse
| x86 64/32
| x86_64
|  
|-
|-
| Weekly Build
| init
| Y
| systemd
| Y
| systemd
| NO
| Toaster
| x86 64/x32
| systemd
|  
|-
|-
| Milestone Build
| poky_lsb
| Y
| poky
| Y
| poky_tiny
| Y
| buildtools
| core-image-sato
| poky
|
|-
| core-image-lsb
| core-image-sato-sdk
| core-image-minimal
| read_only rootfs
| poky
| core-image-sato-sdk
|
|-
|
|
|
| pam
| init/systemd
|
|
|-
|
|
|
| Log rotate
| RPM,DEB,IPK
|  
|  
|}
|}


== Results ==


The [[#QA Responsible | QA Responsible]] should ensure that all the defined [[#Execute Build Sets | Build Sets]] are executed correctly and raise the proper bugs for the failures that each of the build sets is showing.


;Sanity Test is described [https://wiki.yoctoproject.org/wiki/Yocto_2.1_Overall_Test_Plan#Sanity_Testing here].
== Update Supported Distro List ==
*run on Toaster-next or poky-contrib.
**TBD
***need info from DEV owners on what Sanity testing is done by them
***detail the set of tests. e.g. django unit tests, Toaster oe-selftest
Weekly test
*Weekly testing is performed on a weekly basis on the poky weekly build.
*E2E consists of all automated tests that can be run on Toaster
**oe-selftest Toaster tests
*selenium UI tests
*TBD
**add E2E integration information
**add more information
Full Pass
*On full pass all available tests will be run:
**Automated tests:
***Django Unit Tests
***oe-selftest Toaster tests
***Selenium automated UI tests
**Manual tests
***The non-automated tests from the Toaster test runs.
*Full pass is scheduled to be run on every YP release candidate. The YP is divided in four release milestone structure as described in the YP Schedule.


=Dependencies=
Once all the build sets were executed and all were PASS, then then it can be added to the list of supported Distros, also the unsupported distros should be removed from the list.
*YP Dependencies
**Bitbake works
*Some changes in the Toaster UI might affect running the UI automated tests using Selenium
**e.g. Changing the element ID’s in the Toaster pages reflected in failing all UI related automated TC’s
*recipe image we’re testing to get build data (metadata broken)


To accomplish above step a patch should be sent to the [https://lists.yoctoproject.org/listinfo/poky  poky mailing list] with '''Sanity Tested Distro File''' updated, you can refer to the following guide to [http://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded  send a patch], remember top use the correct list.


=Risk Assumptions=
=== Sanity Tested Distro File (Snapshot)===
*TBD (AlexG)
*Project Risks
meta-poky/conf/distro/poky.conf
**QA and DEV to define this
-----
**Toaster doesn’t affect atm any other YP component (confirmed)
SANITY_TESTED_DISTROS ?= " \
**Changes in Bitbake, bitbake logging might affect Toaster functionality
              poky-1.7 \n \
**Is reliant by host/browser environment
              poky-1.8 \n \
***TBD - add opensuse as well at some point
              poky-2.0 \n \
***TBD - expand the Toaster test framework to the supported host distributions of YP
              Ubuntu-14.04 \n \
*Testing Risks
              Ubuntu-14.10 \n \
**Toaster quality might be affected
              Ubuntu-15.04 \n \
              Ubuntu-15.10 \n \
              Ubuntu-16.04 \n \
              Fedora-21 \n \
              Fedora-22 \n \
              Fedora-23 \n \
              CentOS-6.* \n \
              CentOS-7.* \n \
              Debian-7.* \n \
              Debian-8.* \n \
              openSUSE-project-13.2 \n \


=Tools=
=Requirements=
*Refer to Requirements section
==HW Requirements==
*TBD (need more detailed info of required tools)


= Release Criteria/ Exit Criteria =


{| class="wikitable"
! style="text-align: center;" | Item
! style="text-align: center;" | Hardware requirements
|-
| Processor
| Intel Core i5-6260U (1.9 GHz Turbo, Dual Core, 4 MB Cache, 15W TDP)
|-
| Memory
| DDR4 32GB
|-
| Disk space
| 2 TB of free disk space is required.
|}


==Software Requirements==


To make sure all those packages are installed on the new distro.


{| class="wikitable" style="border: none; background: none;"
! colspan="4"| Packages
|-
| gawk  || make || wget || tar
|-
| bzip2|| gzip || python || unzip
|-
| perl|| patch || diffutils || diffstat
|-
| git|| cpp || gcc || gcc-c++
|-
| glibc-devel || texinfo || chrpath || ccache
|-
| perl-Data-Dumper || perl-Text-ParseWords || perl-Thread-Queue|| socat
|-
| findutils || which || SDL-devel || xterm
|-
| tigervnc || tigervnc-server || python3 || tunctl
|-
| gitpython || python3-pip || python3-devel || qemu-kvm
|-
| perl-bignum ||  ||  || 
|-
|}


= Test execution Cycle =


== QA Responsible ==
TBD
meanwhile you can contact [mailto:jose.perz.carranza@intel.com José Pérez Carranza]


== Public Autobuilder ==


* There will be a continuous execution of random build sets defined on autobuilder, results under foucs of SWAT team


= Exit Criteria for Enabling an OS Distribution =


* All the defined build steps are PASS [[#Execute Build Sets|Execute Build Sets]]
* Distro was added to the supported list [[#Update Supported Distro List|Update Supported Distro]]


= Supported OS Distributions =


The distributions used are:


*Poky
*Fedora
*Ubuntu
*CentOS
*OpenSuse
*Debian


<!--
== Supported OS Distributions Criteria ==
THIS IS THE OLD TESTING PLAN


= Test Areas =
List of supported OS Distributions should be the same at [http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#detailed-supported-distros  Mega Manual]  and in variable mentioned in [https://wiki.yoctoproject.org/wiki/Distro_Testing_Plan#Sanity_Tested_Distro_File_.28Snapshot.29  Distro File] if is not equal should a bug should be filled.
Toaster consists of two big components, as follows:
The following criteria is used to see if a Distro and his specific version is supported


== Backend ==
*'''Poky'''
** Most 2 recent releases
*'''Fedora'''
** Most 2 recent releases
*'''Ubuntu'''
** Most recent release
** Most recent LTS release
** When above are the same, use most recent 2 releases
*'''CentOS'''
** Most recent release
*'''OpenSuse'''
** Most recent release
** Most recent LEAP release
*'''Debian'''
** Most recent release


==== Functionality tests ====
BETA version are considered to enter the cycle of validation as a new distro, but The goal here is to provide some pre-release testing of major OS when their release comes soon after YP release. Once the major release is published the process defined above will be applied and then that distro can be added to the supported list.
*  [[REST_API_Contracts | REST API]]  verification – create Django tests to detect API calls returning incomplete (fields having null values) or wrong data for a certain set of entries from each table - our goal is to have 90% data collected. The tests can be found [http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=andreeap/Toaster-tests&id=7cfd179be5db2a5530d60093fd09d0240138c2fa here];
*  Calculation of the data collection rate - the ratio between the number of the variables having null values ​​and the total number of variables collected (the data collection rate does not include wrong values); Run: ./fail_rate.py path_to_Toaster.sqlite_file. Output: table_name field_name value_that_never_changes and for each table a percentage: (the number of occurences of all the fields that never change their values)/(total number of entries for that table);
*  Verify that all links in the simple UI are available;
*  Verify the quality of the data collected through the simple UI;
*  Verify the quality of the data collected by verifying most of the values from the database by connecting directly to the database - our immediate goal is to have 90% correct data collected;


==== Usability tests ====
=== List ===
*  Verify the easy usage of Toaster ([https://wiki.yoctoproject.org/wiki/WebHob#Installation_and_Running easy to install and start/stop the Toaster server])


== Frontend ==
{| class="wikitable"
 
! style="text-align: center;" | OS Version
==== Functionality tests ====
! style="text-align: center;" | Status
*  Manual testing in the first stage;
*  Automate testing using  [http://www.seleniumhq.org/ Selenium], in the second stage;
 
==== Compatibility tests ====
*  Verify the behavior of the GUI on different browsers and operating systems; TBD
 
==== Usability tests ====
*  Verify if the GUI design is as described here: http://yoctoproject.org/webhob;
*  Friendly graphical user interface;
 
==== Performance tests ====
*  Stress testing (e.g. display appropriate error messages when the system is under stress);
 
= Test Cycle =
{|class="wikitable" style="text-align: center;"
| || || colspan="3" | Test execution cycle
|-
| || || [[#Weekly Test]] || [[#Full Pass Test]] || [[#Release Test]]
|-
| rowspan="2" | Build type || Weekly  || Yes || Yes ||
|-
| Release || Yes || Yes || Yes
|-
|-
| rowspan="2" | [[#Test Areas]] || [[#Backend]] || Yes || Yes || Yes
| Ubuntu-16.04 LTS
| align="center" style="color:green;" | PASS
|-
|-
| [[#Frontend]]  || Yes || Yes || Yes
| Ubuntu-17.04
| align="center" style="color:green;" | PASS
|-
|-
| rowspan="5" | Target machine || qemuarm || || Yes || Yes
| Fedora-25
| align="center" style="color:green;" | PASS
|-
|-
| qemumips|| || || Yes
| Fedora-26
| align="center" style="color:green;" | PASS
|-
|-
| qemuppc|| || || Yes
| CentOS-7
| align="center" style="color:green;" | PASS
|-
|-
| qemux86|| Yes || Yes || Yes
| Debian-9
| align="center" style="color:green;" | PASS
|-
|-
| qemux86-64  || ||  || Yes
| openSUSE-13.2
| align="center" style="color:green;" | PASS
|-
|-
| rowspan="3" | Target image|| core-image-minimal|| Yes || ||
| openSUSE-42.2
|-
| align="center" style="color:green;" | PASS
| core-image-sato-sdk|| || Yes || Yes
|}
|}


== Weekly Test ==
== Supported OS Distributions Release Calendar ==
*; Scope:
** Images built weekly and released through the distribution team.
 
*; Objective:
** Functionality test on most areas with minimum sets of tests;
** Regression test with high probability to find bugs.
 
== Full Pass Test ==
*; Scope:
** Images built as candidates for milestone or final release;
** Passed [[#Weekly Test]]
 
*; Objective:
** Ensure functionality of Toaster component.


== Release Test ==
This section contains the links to the supported distros schedules, should be tracked at beginning of every YP release to check which DIstro may be candidate to be added during that release cycle.
*; Scope:
** Release candidates that pass [[#Full Pass Test]]


*; Objective:
*[https://wiki.ubuntu.com/Releases Ubuntu ]
** All scheduled features are covered, or rescheduled;
*[https://fedoraproject.org/wiki/Releases Fedora]
** All relevant bugs are fixed and verified.
*[https://www.debian.org/releases/ Debian]
-->
*[https://en.opensuse.org/openSUSE:Roadmap OpenSuse]
*[https://wiki.centos.org/Download CentOS]

Latest revision as of 01:28, 6 March 2018

This article is the test plan for enabling OS distributions at the Yocto Project Autobuilder

About Distro Testing

Distro Testing is part of the process of enabling an OS distribution at the Yocto Project Autobuilder. It is intended to catch bugs that are distribution specific and would prevent an Autobuilder worker to use such distribution.

Test Objectives

  • Verify that Distro executes oe-selftest
  • Verify that Distro is able to execute components (Eclipse, Toaster, WIC)
  • Veirfy that Distro is able to build per package type (IPK, DEB, RPM)
  • Verify that Distro is able to build different image types (LSB, non LSB)
  • Verify that Distro is able to build per architecture (arm, x86)
  • Verify that Distro is able to build per bootloader (systed, init)
  • Verify that Distro is able to build poky-tiny
  • Verify that Distro executes multilib

Test Strategy

The strategy will be divided in two groups:

  • Review the test objectives once in an Autobuilder instance when the OS distribution is first-time enabled (one time testing)
  • SWAT monitoring at the Autobuilders


Staging AutoBuilder

When an OS Distribution is chosen for inclusion, it will be setup as a worker of an Autobuilder and a series of build steps will run. Once every Buildset shows no failures, that distro will be added to the list of supported Distros by QA, this is a one-time testing activity, not executed in every milestone o release test cycle.

Public Autobuilder

Once the distro was first time validated by QA team it will be running on public autobuilder different build sets already defined , after that it going to be under the focus of SWAT team

Process

This section describes the process to add a new Distro as supported on Yocto Project

Distro Steps v2.PNG


Open a Bug

When a new Distro is identified to be added as a supported candidate a bug should be filed on Bugzilla and has to be assigned to the QA Responsible of the Distro Testing.

Install Distro

QA responsible should ensure that the new Distro is installed properly on the local AB

Process To Install New Distro

1. Install the new distro

2. Install the requirements from the Yocto Project Quickstart

3. Start an Autobuilder staging instance 

  $ git clone http://git.yoctoproject.org/git/yocto-autobuilder 
  
  # Ensure the owner of cloned repo is different than root (Poky does not work with root user)
  $ chown -R ab:ab yocto-autobuilder/

  $ cd yocto-autobuilder

  # Load buildset configuration with nightly-qa-distro 
  $ ln -sf buildset-config.yocto-qa/ buildset-config

  # setup the autobuilder
  $ . yocto-autobuilder-setup
  
  # Generates password for AB Web
  $ htpasswd -b .htpasswd distrouser distropass

  # start autobuilder
  $ ./yocto-start-autobuilder both


4. Open a browser and enter to http://localhost:8010 

  Note: If the browser fails to open the URL review AB logs for errors yocto-controller/twistd.log and yocto-worker/twistd.log

5. Log in with the distrouser and distropass

6. Enter to http://localhost:8010/builders/nightly-qa-distro and start (force) a build

7. Wait to the buildsets to end and review the output, load bugs if failures occur

8. Stop the staging Autobuilder 

  $ ./yocto-stop-autobuilder both

Execute Build Sets

Build sets to be executed are defined in below table, to add a Distro as supported all the build sets should be PASS on local AB

RPM DEB IPK Component Multilib World oe-selftest
ARM x86_64 x86_32 Eclipse x86 64/32 x86_64
init systemd systemd Toaster x86 64/x32 systemd
poky_lsb poky poky_tiny buildtools core-image-sato poky
core-image-lsb core-image-sato-sdk core-image-minimal read_only rootfs poky core-image-sato-sdk
pam init/systemd
Log rotate RPM,DEB,IPK

Results

The QA Responsible should ensure that all the defined Build Sets are executed correctly and raise the proper bugs for the failures that each of the build sets is showing.

Update Supported Distro List

Once all the build sets were executed and all were PASS, then then it can be added to the list of supported Distros, also the unsupported distros should be removed from the list.

To accomplish above step a patch should be sent to the poky mailing list with Sanity Tested Distro File updated, you can refer to the following guide to send a patch, remember top use the correct list.

Sanity Tested Distro File (Snapshot)

meta-poky/conf/distro/poky.conf
-----
SANITY_TESTED_DISTROS ?= " \
             poky-1.7 \n \
             poky-1.8 \n \
             poky-2.0 \n \
             Ubuntu-14.04 \n \
             Ubuntu-14.10 \n \
             Ubuntu-15.04 \n \
             Ubuntu-15.10 \n \
             Ubuntu-16.04 \n \
             Fedora-21 \n \
             Fedora-22 \n \
             Fedora-23 \n \
             CentOS-6.* \n \
             CentOS-7.* \n \
             Debian-7.* \n \
             Debian-8.* \n \
             openSUSE-project-13.2 \n \

Requirements

HW Requirements

Item Hardware requirements
Processor Intel Core i5-6260U (1.9 GHz Turbo, Dual Core, 4 MB Cache, 15W TDP)
Memory DDR4 32GB
Disk space 2 TB of free disk space is required.

Software Requirements

To make sure all those packages are installed on the new distro.

Packages
gawk make wget tar
bzip2 gzip python unzip
perl patch diffutils diffstat
git cpp gcc gcc-c++
glibc-devel texinfo chrpath ccache
perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue socat
findutils which SDL-devel xterm
tigervnc tigervnc-server python3 tunctl
gitpython python3-pip python3-devel qemu-kvm
perl-bignum

Test execution Cycle

QA Responsible

TBD meanwhile you can contact José Pérez Carranza

Public Autobuilder

  • There will be a continuous execution of random build sets defined on autobuilder, results under foucs of SWAT team

Exit Criteria for Enabling an OS Distribution

Supported OS Distributions

The distributions used are:

  • Poky
  • Fedora
  • Ubuntu
  • CentOS
  • OpenSuse
  • Debian

Supported OS Distributions Criteria

List of supported OS Distributions should be the same at Mega Manual and in variable mentioned in Distro File if is not equal should a bug should be filled.

The following criteria is used to see if a Distro and his specific version is supported

  • Poky
    • Most 2 recent releases
  • Fedora
    • Most 2 recent releases
  • Ubuntu
    • Most recent release
    • Most recent LTS release
    • When above are the same, use most recent 2 releases
  • CentOS
    • Most recent release
  • OpenSuse
    • Most recent release
    • Most recent LEAP release
  • Debian
    • Most recent release

BETA version are considered to enter the cycle of validation as a new distro, but The goal here is to provide some pre-release testing of major OS when their release comes soon after YP release. Once the major release is published the process defined above will be applied and then that distro can be added to the supported list.

List

OS Version Status
Ubuntu-16.04 LTS PASS
Ubuntu-17.04 PASS
Fedora-25 PASS
Fedora-26 PASS
CentOS-7 PASS
Debian-9 PASS
openSUSE-13.2 PASS
openSUSE-42.2 PASS

Supported OS Distributions Release Calendar

This section contains the links to the supported distros schedules, should be tracked at beginning of every YP release to check which DIstro may be candidate to be added during that release cycle.

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Distro_Testing_Plan&oldid=36634"