Quality Assurance yocto project: Difference between revisions

From Yocto Project
Jump to navigationJump to search
No edit summary
No edit summary
 
(23 intermediate revisions by 4 users not shown)
Line 22: Line 22:
TEST_TARGET = "simpleremote"
TEST_TARGET = "simpleremote"
TEST_SERVER_IP = "HOST ip" --> IP of the machine being used to launch tests
TEST_SERVER_IP = "HOST ip" --> IP of the machine being used to launch tests
TEST_TARGET_IP = "DUT ip"  --> IP of the device under test  
TEST_TARGET_IP = "DUT ip"  --> IP of the device under test
TEST_SUITES = " auto"
 
For releases before 2.3 also add
 
TEST_SUITES = " auto"  
 
Remember to add the machine for example, if genericx86 is needed, MACHINE ?= "genericx86"
Remember to add the machine for example, if genericx86 is needed, MACHINE ?= "genericx86"


Line 30: Line 34:


<p>
<p>
5. Edit bblayers.conf, bblayers.conf PATH:~/poky/build/conf
5. In ~poky/build directory, in command line, type the following commands:
</p>
<ul><li>in " BBLAYERS ?= " \ " section add meta-selftest respecting the pattern from above</li></ul>
<p>
 
6. In ~poky/build directory, in command line, type the following commands:
</p>
</p>
<pre>
<pre>
$ bitbake rpm psplash
$ bitbake rpm busybox run-postinsts
$ bitbake package-index
$ bitbake package-index
</pre>
</pre>
<p>
<p>
NOTE: After finishing the tasks, copy the image manifest (core-image-sato-sdk-genericx86.manifest) from autobuilder at the following path:
NOTE: After finishing the tasks, copy the image manifest (core-image-sato-sdk-genericx86.manifest)  and the testdata.json (core-image-sato-sdk-genericx86.testdata.json) from autobuilder at the following path:
"~poky/build/tmp/deploy/images/genericx86/" ( if "/tmp/deploy/images/genericx86" does not exist create new directories with this path)
"~poky/build/tmp/deploy/images/genericx86/" ( if "/tmp/deploy/images/genericx86" does not exist create new directories with this path)
</p>
</p>
Line 49: Line 48:
<p>Then add the key to the host again ssh , SSH TARGETNAME@IPTARGET</p>
<p>Then add the key to the host again ssh , SSH TARGETNAME@IPTARGET</p>
<p>After a while the testing will be finished</p>
<p>After a while the testing will be finished</p>
<p>NOTE: For QEMU images there are additional steps that need to be followed. They are detailed on</p>
<p>https://wiki.yoctoproject.org/wiki/Steps_for_Automated_Testing_on_QEMU</p>


<h2>'''Steps for LSB Automated Testing'''</h2>
<h2>'''Steps for LSB Automated Testing'''</h2>
Line 63: Line 65:
<p>4. Edit local.conf, adding the same configuration as BSP. Just one small changing the distro</p>  
<p>4. Edit local.conf, adding the same configuration as BSP. Just one small changing the distro</p>  
<ul><li>DISTRO ?= "poky-lsb"</li></ul>
<ul><li>DISTRO ?= "poky-lsb"</li></ul>
NOTE: The bblayers configuration is the same as BSP.
<p>From command line type the following commands</p>
<p>From command line type the following commands</p>
<pre>$ bitbake rpm psplash (wait for tasks to be executed)</pre>
<pre>$ bitbake rpm busybox run-postinsts (wait for tasks to be executed)</pre>
<pre>$ bitbake package-index (wait for tasks to be executed)</pre>
<pre>$ bitbake package-index (wait for tasks to be executed)</pre>
NOTE: After finishing the tasks copy the image manifest (core-image-lsb-sdk-machine.manifest) from autobuilder at the following path:
NOTE: After finishing the tasks copy the image manifest (core-image-lsb-sdk-machine.manifest) from autobuilder at the following path:
Line 72: Line 73:
<pre> $ bitbake core-image-lsb-sdk -c testimage </pre>
<pre> $ bitbake core-image-lsb-sdk -c testimage </pre>


<h2>'''pTest and runtime'''</h2>


<p>1. pTest run steps:</p>
<h2>'''Defect/Bug life Cycle'''</h2>
<p>1) Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)</p>
----
* Usually tested on NUC with genericx86-64
[[File:Yp_BugLifeCycle.png]]
<p>2) Install the image on DUT (using legacy boot)</p>
<p>3) Boot the image and copy "ptest-runner.sh" script on DUT</p>
<p>4) In command line run "ptest-runner.sh > ptest.log" and wait for it to finish ( about 5 hours)</p>
<p>* P.S. : I will send you also the script via google drive, so you can open it</p>


<p><strong>If you need more information about bug information and levels, please go to
https://wiki.yoctoproject.org/wiki/Bug_reporting_and_Information_levels</strong></p>


<p>2. Runtime compliance steps:</p>
<p>1) Download lsb image from autobuilder( same image as in LSB weekly testing for genericx86-64-lsb bsp)</p>
<p>* we test compliance on NUC with genericx86-64-lsb, core-image-lsb-sdk</p>
<p>2) Install the image on DUT </p>
<p>3) Configure the network so it be able to work externally: </p>
<p> - edit /etc/resolv.conf and add the gateway ip_address </p>
<p> - add the ip and netmask using "ifconfig" command </p>
<p> - add the route using "route add default gw <ip_address>" </p>
<p> - export the proxy using "export http_proxy=<add your proxy link>" command </p>
<p> * there is a bug and if you make these steps in another order than above, it may be possible not work </p>
<p>4) Copy "compliance_test.py" script on DUT</p>
<p>5) Make sure that your network connection is working </p>
<p>6) Run the script like this: </p>
<p>- make the script executable:</p>
                    <pre>"$ chmod a+x compliance_local.py" </pre>
<p>- run in command line the following command " </p>
                <pre>./compliance_test.py <milestone> <date>" </pre>
<p>- wait until "Configuration done. LSB script must be started from machine." in command line( about 8-12 hours) </p>
<p>7) Run "LSB_test.sh" via ssh or manually and wait for it to finish( about a day) </p>
<p>8) Get the logs from DUT: </p>
<p>- result-<milestone>-data.fulllog </p>
<p>- result-<milestone>-data.log </p>
<p>- result-<milestone>-data.fail </p>
<p>- posix.log (can be found in: /opt/ltp/testcases/open_posix_testsuite) </p>
<p>* the three others are found in /opt/ltp directory, in output, temp, result folders  </p>
<p>* send these log to Yi Zhao specifying the version and the type of image </p>
<p>- In /var/opt/lsb/test/manager/results/x86.../x86....tar.gz </p>
<p>* It has like 18 Mb </p>
<p>* Upload this file on drive and send the link to hongxu.jia@windriver.com and mark.hatle@windriver.com </p>
<p>9) After this, update the tests from Testopia - Runtime test run on passed  </p>


<h2>'''Weekly Testing (automated)'''</h2>


<h2>'''Defect/Bug life Cycle'''</h2>
<p>1. Genericx86 weekly tests:</p>
----
<p>a) download the core-image-sato-sdk image from autobuilder( link available in the point release mail)</p>
[[File:Yp_BugLifeCycle.png]]
<p>b) make checkout on the poky commit from the mail (in command line):</p>
<p>- clone a clean version of poky: "git clone git://git.yoctoproject.org/poky"</p>
<p>- in poky directory checkout on the desired commit: "git checkout <commit-id>"</p>
<p>c) source the environment:</p>
<p>- in poky directory ( "cd ~/poky"): " source oe-init-build-env"</p>
<p>d) add in local. conf :</p>
<p>* local.conf path: ~/poky/build/conf</p>
<p>- at the end of file add:</p>
<pre>
INHERIT += "testimage"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "DUT ip"
TEST_SERVER_IP = "workstation ip"
</pre>
<p>- change the machine: MACHINE ??= "genericx86"</p>
<p>e) in ~poky/build directory, in command line, type the following commands:</p>
<p>- "bitbake rpm busybox run-postinsts" (wait for tasks to be executed)</p>
<p>- "bitbake package-index" (wait for tasks to be executed)</p>
<p> - after finishing the tasks copy the image manifest (core-image-sato-sdk-genericx86.manifest) from autobuilder at the following path:</p>
<p>"~poky/build/tmp/deploy/images/genericx86/" ( if "/tmp/deploy/images/genericx86" does not exist create new directories with this path)</p>
<p>- "bitbake core-image-sato-sdk -c testimage"</p>
<p>If having issues with keys use this command: ssh-keygen -f /home/user/.ssh/known_hosts -R targetip</p>
<p>Then add the key to the host again ssh , SSH TARGETNAME@IPTARGET</p>


<h3>If you need more information about bug information and levels, please go to </h3>
<p>2. Use the same steps as above for Genericx86-64 BSP but take care to download the correct image and modify accordingly the machine in local.conf and path for manifest.</p>
https://wiki.yoctoproject.org/wiki/Bug_reporting_and_Information_levels


<h2> Weekly Testing (automated)</h2>
<p>3. Test generic weekly LSB images (genericx86-lsb and genericx86-64-lsb):</p>
<p>- the #1 steps are followed for lsb images</p>
<p>a) download the core-image-lsb-sdk image from autobuilder( link available in the point release mail)</p>
<p>* the images are available in sato and lsb form( access genericx86-lsb and genericx86-64-lsb directories for lsb images)</p>
<p>b) the checkout step is the same as #1</p>
<p>c) same as #1</p>
<p>d) add in local. conf :</p>
<p>* local.conf path: ~/poky/build/conf</p>
<p>- at the end of file add:</p>
<pre>
INHERIT += "testimage"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "DUT ip"
TEST_SERVER_IP = "workstation ip"
</pre>
<p>- change the machine: MACHINE ??= "genericx86" (or "genericx86-64")</p>
<p>- change the distro : DISTRO ?= "poky-lsb"</p>
<p>e) in command line (~poky/build directory) type the following commands:</p>
<p>- "bitbake rpm busybox run-postinsts" (wait for tasks to be executed)</p>
<p>- "bitbake package-index" (wait for tasks to be executed)</p>
<p>- after finishing the tasks copy the image manifest (core-image-lsb-sdk-machine.manifest) from autobuilder at the following path:</p>
<p>"~poky/build/tmp/deploy/images/<machine_name>-lsb/" ( if "/tmp/deploy/images/<machine_name>-lsb" does not exist create new directories with this path)</p>
<p>- "bitbake core-image-lsb-sdk -c testimage"</p>


1. Genericx86 weekly tests:
a) download the core-image-sato-sdk image from autobuilder( link available in the point release mail)
b) make checkout on the poky commit from the mail (in command line):
- clone a clean version of poky: "git clone git://git.yoctoproject.org/poky"
- in poky directory checkout on the desired commit: "git checkout <commit-id>"
c) source the environment:
- in poky directory ( "cd ~/poky"): " source oe-init-build-env"
d) add in local. conf :
* local.conf path: ~/poky/build/conf
- at the end of file add:
INHERIT += "testimage"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "DUT ip"


TEST_SERVER_IP = "workstation ip"
<h2> '''Compliance Testing'''</h2>
TEST_SUITES = " auto"
- change the machine: MACHINE ??= "genericx86"
e) add in bblayers.conf:
* bblayers.conf path: ~/poky/build/conf
- in " BBLAYERS ?= " \ " section add meta-selftest respecting the pattern from above
f) in ~poky/build directory, in command line, type the following commands:
- "bitbake rpm psplash" (wait for tasks to be executed)
- "bitbake package-index" (wait for tasks to be executed)
- after finishing the tasks copy the image manifest (core-image-sato-sdk-genericx86.manifest) from autobuilder at the following path:
"~poky/build/tmp/deploy/images/genericx86/" ( if "/tmp/deploy/images/genericx86" does not exist create new directories with this path)
- "bitbake core-image-sato-sdk -c testimage"
if having issues with keys use this command: ssh-keygen -f /home/user/.ssh/known_hosts -R targetip
Then add the key to the host again ssh , SSH TARGETNAME@IPTARGET


2. Use the same steps as above for Genericx86-64 BSP but take care to download the correct image and modify accordingly the machine in local.conf and path for manifest.
<p>Runtime compliance steps:</p>
<p>1) Download lsb image from autobuilder( same image as in LSB weekly testing for genericx86-64-lsb bsp)</p>
<p>* we test compliance on NUC with genericx86-64-lsb, core-image-lsb-sdk</p>
<p>2) Install the image on DUT</p>
<p>3) Configure the network so it be able to work externally:</p>
<p>- Export the proxy using "export http_proxy=<add your proxy link>" command eg</p>
<pre>$ export http_proxy=http://proxy-chain.intel.com:911</pre>
<p>And check if you have internet connection, typing on terminal:</p>
<pre>$ wget www.google.com</pre>
<p>* There is a bug and if you make these steps in another order than above, it may be possible not work</p>
<p>4) Copy "compliance_test.py" script on DUT</p>


3. Test generic weekly LSB images (genericx86-lsb and genericx86-64-lsb):
<p>- Make the script executable: "chmod a+x compliance_local.py"</p>
- the #1 steps are followed for lsb images
<p>- Run in command line the following command:</p>
a) download the core-image-lsb-sdk image from autobuilder( link available in the point release mail)
<pre> ./compliance_test.py <milestone> <date>" </pre>
* the images are available in sato and lsb form( access genericx86-lsb and genericx86-64-lsb directories for lsb images)
<p>- Wait until "Configuration done. LSB script must be started from machine." in command line( about 8-12 hours)</p>
b) the checkout step is the same as #1
<p>8) Get the logs from DUT:</p>
c) same as #1
 
d) add in local. conf :
<p>- Results fulllog: /opt/ltp</p>
* local.conf path: ~/poky/build/conf
<p>- results.log: /opt/ltp/results</p>
- at the end of file add:
<p>- results.fail: /opt/ltp/output</p>
INHERIT += "testimage"
<p>- Results posix: /opt/ltp/testcases/open_posix_testsuites</p>
<p>* 18 MB  File: /var/opt/lsb/test/manager/results/x86.../x86....targz </p>
TEST_TARGET = "simpleremote"
<p>* send these log to Yi Zhao specifying the version and the type of image</p>
TEST_TARGET_IP = "DUT ip"
<p>- In /var/opt/lsb/test/manager/results/x86.../x86....tar.gz (you can find it with auto-complete(tab) easily)</p>
<p>* It has like 18 Mb</p>
<p>* Upload this file on drive and send the link to hongxu.jia@windriver.com and mark.hatle@windriver.com</p>
<p>9) Put the tests from Testopia - Runtime test run on passed </p>


TEST_SERVER_IP = "workstation ip"
TEST_SUITES = " auto"
- change the machine: MACHINE ??= "genericx86" (or "genericx86-64")
- change the distro : DISTRO ?= "poky-lsb"
e) in bblayers.conf add meta-selftest as in #1
f) in command line (~poky/build directory) type the following commands:
- "bitbake rpm psplash" (wait for tasks to be executed)
- "bitbake package-index" (wait for tasks to be executed)
- after finishing the tasks copy the image manifest (core-image-lsb-sdk-machine.manifest) from autobuilder at the following path:
"~poky/build/tmp/deploy/images/<machine_name>-lsb/" ( if "/tmp/deploy/images/<machine_name>-lsb" does not exist create new directories with this path)
- "bitbake core-image-lsb-sdk -c testimage"




<h2> Compliance Testing</h2>
<h2>''' pTest Run'''</h2>


Runtime compliance steps:
<p>pTest run steps:</p>
1) Download lsb image from autobuilder( same image as in LSB weekly testing for genericx86-64-lsb bsp)
* we test compliance on NUC with genericx86-64-lsb, core-image-lsb-sdk
2) Install the image on DUT
3) Configure the network so it be able to work externally:
- edit /etc/resolv.conf and add the gateway ip_address – yocto -> Broadcast de maquina, sudo rounte –n
- add the ip and netmask using "ifconfig" command - yocto
- add the route using "route add default gw <ip_address>" -> broadcast de maquina
- export the proxy using "export http_proxy=<add your proxy link>" command eg
Export http_proxy=http://proxy-chain.intel.com:911. And check if you have internet connection. With $wget www.google.com
* there is a bug and if you make these steps in another order than above, it may be possible not work
4) Copy "compliance_test.py" script on DUT
5) Make sure that your network connection is working
6) Run the script like this:
- make the script executable: "chmod a+x compliance_local.py"
- run in command line the following command "./compliance_test.py <milestone> <date>"
- wait until "Configuration done. LSB script must be started from machine." in command line( about 8-12 hours)
7) Run "LSB_test.sh" via ssh or manually and wait for it to finish( about a day)
8) Get the logs from DUT:
- result-<milestone>-data.fulllog
- result-<milestone>-data.log
- result-<milestone>-data.fail
- posix.log (can be found in: /opt/ltp/testcases/open_posix_testsuite)
* the three others are found in /opt/ltp directory, in output, temp, result folders
* send these log to Yi Zhao specifying the version and the type of image
* I'll fwd you an email so you could see an example
- in /var/opt/lsb/test/manager/results/x86.../x86....tar.gz (you can find it with auto-complete(tab) easily)
* it has like 18 Mb
* upload this file on drive and send the link to hongxu.jia@windriver.com and mark.hatle@windriver.com
* also I'll fwd an email to see how it looks
9) Put the tests from Testopia - Runtime test run on passed


*the script can be found at this link: https://drive.google.com/open?id=0B4B8zTi4deOyNjF6NkM0cW01S0E
<p>1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)</p>
<p>* we test it on NUC with genericx86-64</p>


<p>2. Install the image on DUT (using legacy boot)</p>


<h2> pTest Run</h2>
<p>3) Configure the network so it be able to work externally:</p>
<p>- Export the proxy using "export http_proxy=<add your proxy link>" command eg</p>
<pre>$ export http_proxy=http://proxy-chain.intel.com:911</pre>
<p>And check if you have internet connection, typing on terminal:</p>
<pre>$ wget www.google.com</pre>


1. pTest run steps:
<p>4. In command line type</p>
1) Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)
<pre>$ ptest-runner > ptest.log </pre>
* we test it on NUC with genericx86-64
<p>and wait for it to finish ( about 5 hours)</p>
2) Install the image on DUT (using legacy boot)
<p>Note: The "script" is already placed on the distro, you just have to type ptest...TAB and the terminal should autocomplete the command.</p>
3) Boot the image and copy "ptest-runner.sh" script on DUT
<p> "> ptest.log", in order to save the test results in that file. </p>
4) In command line run "ptest-runner.sh > ptest.log" and wait for it to finish ( about 5 hours)

Latest revision as of 15:57, 10 October 2017

Steps for BSP Automated Testing

1. Download the core-image-sato-sdk image from autobuilder( link available in the point release mail)

2. Make checkout on the poky commit from the mail (in command line):

a) Clone a clean version of poky: 

$ git clone git://git.yoctoproject.org/poky"

b)In poky directory checkout on the desired commit: 

$ git checkout <commit-id>

3. Source the environment, from poky directory 

 $ cd ~/poky 

$ source oe-init-build-env

4. Edit local.conf. PATH: ~/poky/build/conf, at the en of the file adding: 

INHERIT += "testimage"
TEST_TARGET = "simpleremote"
TEST_SERVER_IP = "HOST ip" --> IP of the machine being used to launch tests
TEST_TARGET_IP = "DUT ip"  --> IP of the device under test

For releases before 2.3 also add 

TEST_SUITES = " auto" 

Remember to add the machine for example, if genericx86 is needed, MACHINE ?= "genericx86"

5. In ~poky/build directory, in command line, type the following commands: 

$ bitbake rpm busybox run-postinsts
$ bitbake package-index

NOTE: After finishing the tasks, copy the image manifest (core-image-sato-sdk-genericx86.manifest) and the testdata.json (core-image-sato-sdk-genericx86.testdata.json) from autobuilder at the following path: "~poky/build/tmp/deploy/images/genericx86/" ( if "/tmp/deploy/images/genericx86" does not exist create new directories with this path) 

$ bitbake core-image-sato-sdk -c testimage

If you face issues with the keys from your host, use this command: $ ssh-keygen -f /home/user/.ssh/known_hosts -R targetip

Then add the key to the host again ssh , SSH TARGETNAME@IPTARGET

After a while the testing will be finished

NOTE: For QEMU images there are additional steps that need to be followed. They are detailed on

https://wiki.yoctoproject.org/wiki/Steps_for_Automated_Testing_on_QEMU


Steps for LSB Automated Testing

1. In general the steps are very similar to BSP, just make sure to download the correct image and modify accordingly the machine in local.conf and path for manifest.

2. Test generic weekly LSB images (genericx86-lsb and genericx86-64-lsb):

  • The #1 steps are followed for lsb images
  • download the core-image-lsb-sdk image from autobuilder( link available in the point release mail)

NOTE: the images are available in sato and lsb form( access genericx86-lsb and genericx86-64-lsb directories for lsb images)

3. The checkout step is the same as BSP.

4. Edit local.conf, adding the same configuration as BSP. Just one small changing the distro

  • DISTRO ?= "poky-lsb"

From command line type the following commands

$ bitbake rpm busybox run-postinsts (wait for tasks to be executed)
$ bitbake package-index (wait for tasks to be executed)

NOTE: After finishing the tasks copy the image manifest (core-image-lsb-sdk-machine.manifest) from autobuilder at the following path: "~poky/build/tmp/deploy/images/<machine_name>-lsb/" ( if "/tmp/deploy/images/<machine_name>-lsb" does not exist create new directories with this path)

Type the following command from terminal:

 $ bitbake core-image-lsb-sdk -c testimage


Defect/Bug life Cycle

Yp BugLifeCycle.png

If you need more information about bug information and levels, please go to https://wiki.yoctoproject.org/wiki/Bug_reporting_and_Information_levels


Weekly Testing (automated)

1. Genericx86 weekly tests:

a) download the core-image-sato-sdk image from autobuilder( link available in the point release mail)

b) make checkout on the poky commit from the mail (in command line):

- clone a clean version of poky: "git clone git://git.yoctoproject.org/poky"

- in poky directory checkout on the desired commit: "git checkout <commit-id>"

c) source the environment:

- in poky directory ( "cd ~/poky"): " source oe-init-build-env"

d) add in local. conf :

* local.conf path: ~/poky/build/conf

- at the end of file add:

INHERIT += "testimage"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "DUT ip"
TEST_SERVER_IP = "workstation ip"

- change the machine: MACHINE ??= "genericx86"

e) in ~poky/build directory, in command line, type the following commands:

- "bitbake rpm busybox run-postinsts" (wait for tasks to be executed)

- "bitbake package-index" (wait for tasks to be executed)

- after finishing the tasks copy the image manifest (core-image-sato-sdk-genericx86.manifest) from autobuilder at the following path:

"~poky/build/tmp/deploy/images/genericx86/" ( if "/tmp/deploy/images/genericx86" does not exist create new directories with this path)

- "bitbake core-image-sato-sdk -c testimage"

If having issues with keys use this command: ssh-keygen -f /home/user/.ssh/known_hosts -R targetip

Then add the key to the host again ssh , SSH TARGETNAME@IPTARGET

2. Use the same steps as above for Genericx86-64 BSP but take care to download the correct image and modify accordingly the machine in local.conf and path for manifest.

3. Test generic weekly LSB images (genericx86-lsb and genericx86-64-lsb):

- the #1 steps are followed for lsb images

a) download the core-image-lsb-sdk image from autobuilder( link available in the point release mail)

* the images are available in sato and lsb form( access genericx86-lsb and genericx86-64-lsb directories for lsb images)

b) the checkout step is the same as #1

c) same as #1

d) add in local. conf :

* local.conf path: ~/poky/build/conf

- at the end of file add:

INHERIT += "testimage"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "DUT ip"
TEST_SERVER_IP = "workstation ip"

- change the machine: MACHINE ??= "genericx86" (or "genericx86-64")

- change the distro : DISTRO ?= "poky-lsb"

e) in command line (~poky/build directory) type the following commands:

- "bitbake rpm busybox run-postinsts" (wait for tasks to be executed)

- "bitbake package-index" (wait for tasks to be executed)

- after finishing the tasks copy the image manifest (core-image-lsb-sdk-machine.manifest) from autobuilder at the following path:

"~poky/build/tmp/deploy/images/<machine_name>-lsb/" ( if "/tmp/deploy/images/<machine_name>-lsb" does not exist create new directories with this path)

- "bitbake core-image-lsb-sdk -c testimage"


Compliance Testing

Runtime compliance steps:

1) Download lsb image from autobuilder( same image as in LSB weekly testing for genericx86-64-lsb bsp)

* we test compliance on NUC with genericx86-64-lsb, core-image-lsb-sdk

2) Install the image on DUT

3) Configure the network so it be able to work externally:

- Export the proxy using "export http_proxy=<add your proxy link>" command eg

$ export http_proxy=http://proxy-chain.intel.com:911

And check if you have internet connection, typing on terminal:

$ wget www.google.com

* There is a bug and if you make these steps in another order than above, it may be possible not work

4) Copy "compliance_test.py" script on DUT

- Make the script executable: "chmod a+x compliance_local.py"

- Run in command line the following command:

 ./compliance_test.py <milestone> <date>"

- Wait until "Configuration done. LSB script must be started from machine." in command line( about 8-12 hours)

8) Get the logs from DUT:

- Results fulllog: /opt/ltp

- results.log: /opt/ltp/results

- results.fail: /opt/ltp/output

- Results posix: /opt/ltp/testcases/open_posix_testsuites

* 18 MB File: /var/opt/lsb/test/manager/results/x86.../x86....targz

* send these log to Yi Zhao specifying the version and the type of image

- In /var/opt/lsb/test/manager/results/x86.../x86....tar.gz (you can find it with auto-complete(tab) easily)

* It has like 18 Mb

* Upload this file on drive and send the link to hongxu.jia@windriver.com and mark.hatle@windriver.com

9) Put the tests from Testopia - Runtime test run on passed


pTest Run

pTest run steps:

1. Download pTest image from autobuilder( you can find core-image-sato-sdk image in pTest directory)

* we test it on NUC with genericx86-64

2. Install the image on DUT (using legacy boot)

3) Configure the network so it be able to work externally:

- Export the proxy using "export http_proxy=<add your proxy link>" command eg

$ export http_proxy=http://proxy-chain.intel.com:911

And check if you have internet connection, typing on terminal:

$ wget www.google.com

4. In command line type

$ ptest-runner > ptest.log

and wait for it to finish ( about 5 hours)

Note: The "script" is already placed on the distro, you just have to type ptest...TAB and the terminal should autocomplete the command.

"> ptest.log", in order to save the test results in that file.

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Quality_Assurance_yocto_project&oldid=32279"