Bugzilla Configuration and Bug Tracking: Difference between revisions
| Line 29: | Line 29: | ||
To set up an account, go to the '''[http://www.bugzilla.yoctoproject.org/ Yocto Bugzilla]''' home page and click on "New Account" in the footer area. Follow the directions there to set up your account. | To set up an account, go to the '''[http://www.bugzilla.yoctoproject.org/ Yocto Bugzilla]''' home page and click on "New Account" in the footer area. Follow the directions there to set up your account. | ||
=== Classification Product and Components === | === Classification, Product and Components === | ||
Yocto Bugzilla uses these Classifications, Products and Components. Configurations change over time and should be defined by module owners: | Yocto Bugzilla uses these Classifications, Products and Components. Configurations change over time and should be defined by module owners: | ||
Revision as of 20:56, 22 November 2011
Yocto Project Bug Process version
| Version | Modifier | Comments |
| 0.3 | Yongkang You | Initial Version |
| 0.5 | Yongkang You | Apply comments from Richard, Dave and Susie. Add Feature Request Process and Add a new Priority "FEAT" for it. |
| 0.8 | Yongkang You | Internal review and agree bug life cycle and bug Triage process. |
| 0.85 | Saul Wold | Add Bug Status / Sub-Status Definitions. |
| 0.9 | davest | Cleanup for 0.9 release. |
You should also read our Community Guidelines before submitting bugs.
Defect Management
Yocto uses Bugzilla as its defect tracking tool.
Home Address
http://bugzilla.yoctoproject.org
Get an Account
Anyone working on the Yocto project can query existing bugs in Bugzilla. You must have an account to submit a bug, edit a bug, or take action on a bug.
To set up an account, go to the Yocto Bugzilla home page and click on "New Account" in the footer area. Follow the directions there to set up your account.
Classification, Product and Components
Yocto Bugzilla uses these Classifications, Products and Components. Configurations change over time and should be defined by module owners:
+=======================+======================+======================+ | Classification | Product | Components | |=====================================================================| | Yocto Projects | Anjuta Plug-in | General | | +----------------------+----------------------| | | BSPs | General | | +----------------------+----------------------| | | Cross-prelink | General | | | +----------------------| | | | Poky Integration | | +----------------------+----------------------| | | Documentation | Handbook | | | +----------------------| | | | Other | | | +----------------------| | | | PRD | | | +----------------------| | | | SDK | | +----------------------+----------------------| | | Eclipse Plug-in | General | | +----------------------+----------------------| | | Kernel | Build | | | +----------------------| | | | Configuration | | | +----------------------| | | | General | | | +----------------------| | | | Runtime | | +----------------------+----------------------| | | Pseudo | General | | | +----------------------| | | | Poky Configuration | | +----------------------+----------------------| | | Runtime Distribution | Installation | | | +----------------------| | | | System Startup | | +----------------------+----------------------| | | SDK Tools | General | | +----------------------+----------------------| | | Swabber | General | | | +----------------------| | | | Poky Integration | | +----------------------+----------------------| | | Yocto Test Suite | General | |-----------------------+----------------------+----------------------| | Infrastructure | AutoBuilder | General | | +----------------------+----------------------| | | Bugzilla | General | | +----------------------+----------------------| | | Website | General | |-----------------------+----------------------+----------------------| | Poky | ADT | General | | +----------------------+----------------------| | | Build System | BitBake | | | +----------------------| | | | Configuration | | | +----------------------| | | | General | | | +----------------------| | | | Hob | | | +----------------------| | | | Image Creator | | | +----------------------| | | | Layers | | +----------------------+----------------------| | | Security | General | |-----------------------+----------------------+----------------------| | Yocto Metadata Layers | Layers | dlna | | | +----------------------| | | | extra | | | +----------------------| | | | general | | | +----------------------| | | | intel | | | +----------------------| | | | linaro | | | +----------------------| | | | xilinx | | |----------------------+----------------------| | | Meta Recipes | connectivity | | | +----------------------| | | | core | | | +----------------------| | | | demoapps | | | +----------------------| | | | devtools / toolchain | | | +----------------------| | | | general | | | +----------------------| | | | graphics | | | +----------------------| | | | kernel | | | +----------------------| | | | multimedia | | |----------------------+----------------------| | | Sato | General | | | +----------------------| | | | gtk-engine | | | +----------------------| | | | Icon | | | +----------------------| | | | Matchbox | |-----------------------+----------------------+----------------------+
Hardware
- All
- PC
- Netbook
- e-Menlow
- Blacksand
- JasperForest
- TunnelCreek
- Macintosh
- Beagleboard
- mpc8315e-rdb
- Routerstationpro
- Other
Version
The versions can vary depending on the Component and Project. Here is a list of current selections:
- 0.9
- 1.0-beta
- 1.0
- 1.1-beta
- 1.1
- 1.2
Priority and Severity
- The priority classifies bug fixing order. It can not be set when open a bug. It was set when doing Triage. The Priorities should be following 5 kinds.
- High -- Bug fixing is planned within 2 weeks, no later than the up-coming milestone, or critical issue for out of cycle release. e.g. highly reproducible crash issues (system or related to key apps/components need to deliver in the upcoming milestone); issues which cause major function loss of a specific feature which is POR of the up-comping milestone; issues which are easy to hit by user and greatly impact user experience or customer requirements; and urgent security issues which needs to be fixed in last release.
- Medium -- Bug fixing is planned before the milestone and must be fixed before the release is finalized.
- Low -- Bug fixing is not planned for the up-coming project release. Non POR feature request, hard to reproduce will fall into this category.
- Undecided -- New reported bug is undecided, before Triage.
- FEAT -- A feature request bug, which isn't approved for future release yet. It will be changed to real priority after Change Control Board (CCB) approves it.
- Note: High impact but Low Priority bugs can be documented in the release notes.
- The Severity indicates bug's impacting level to reporter. It can be categorized to following 5 kinds. The default is Normal.
- Critical -- Crashes, hang, loss of data, negative impact to other components, memory leak etc.
- Major -- Major loss of functionality of POR
- Normal -- Regular issue, some loss of functionality under certain circumstance
- Minor -- Minor loss of functionality, or issues with easy workaround available
- Enhancement -- Request for enhancement work
Bug Status
- Open -- A new reported bug with default assignee.
- ACCEPTED-- The assigned developer has reviewed and accept the bug.
- Resolved -- bug is fixed or closed by other resolved method.
- Fixed -- This is fixed and in the master branch will be set automatically during build
- NotABug -- This is verified as not a bug
- WontFix -- We will not fix this bug, possibly because a newer package fixes it
- Duplicate -- Duplicate of another bug, possibly different failure mode
- Invalid -- The bug is invalid, sometimes this is used when the author of the bug does not provide accurate information, these will be reviewed by Triage team, and could be changed to NeedInfo
- Obsolete -- The bug is obsolete, old bug that is no longer appropriate, or a package that is depercated.
- WorksForMe -- The bug does not appear when tested by engineer, more appropriately, this may be NeedInfo
- Verified -- bug is double checked and agreed with the resolved method by original reporter.
- Reopen: bug can be reopened, if it is in Resolved, Verified or Close stage.
- NeedInfo: bug needs be set to NeedInfo, if there is important information missing to understand or reproduce bug.
- WaitForUpstream: The owner does bug analysis and finds this bug comes from upstream. Owner can post a new bug in upstream bugzilla, change status to WaitForUpstream and add the upstream bugzilla URL in comment area. When upstream bug is fixed, put the bug fixing into Yocto repository. After the patch is built into weekly build, change its status to Resolved/FIXED.
- Close stage is not used in normal bug life. When reported two same bugs by wrongly click button twice, the 2nd one can be closed. Generally it(close stage) is just for keeping wrong bug out of scrub cycle and statistic.
Bug Life Cycle
A normal Bug's life is started from Open and ended by Verified. Triage team will select "verified: Program Management", after confirm the verification.
Bug should be marked as Resolved/Fixed, after its fixing patch is built into formal build (weekly build or milestone build). There is an automation method to update bugs to "RESOVLED/FIXED" status by following steps.
- Developer fixed bug and check in patch to his contrib branch with special words in commit description: [BUGFIX #4321]
- Developer asked tree gatekeeper to pull patch.
- Release Engineer do formal build. The build script will read new commit description and find out bug ID.
- Build script will notify bugzilla and update bugs status to "RESOLVE/FIXED"
If a weekly build includes a new commit, which reverts another patch, there should be a tool to judge whether the reverted patch summary include any bug fixing ID. If the reverted patch does include any bug fixing, the bug should be reopen.
Defect Submission
Defects may be entered by anyone working on the Yocto. Bug submission should include severity, thorough environment information, proper and clear reproduce steps and logs.
Bug Group
Bug group is used to control bug visibility. Couple of groups will be set up. When reporting or updating bugs, group check box could be edit.
- Security -- Bugs can only be viewed by Security group user. Bugs will be in Security Group, when it is reported to security component.
- No_Group_Check_Box_Selection-- Bugs are public. (Be careful)
- TBD
Bug Reporter
- Provide clear and simple bug subject. It is recommended to put a Fault Component Name with brackets at the beginning of subject as a keyword, e.g. [Poky].
- Validate all fields are correct. If fields are not filled out correctly, the bug might be sent back to the submitter.
- Judge the right Severity.
- Adds any additional information from the bug scrub.
- Assigns the bug to the correct person to disposition the bug.
- If a bug is too vague to make a determination then the Resolution Field of the bug will be set to need more information and sent back the bug submitter.
- Bug Priority can't be set when report a bug. It will be set when doing bug Triage
Bug Owner
- Review bugs to check if there are enough information. If not, set the bug to NeedInfo. Try to reproduce bug, if there are clear steps.
- Based on bug Priority and Severity to fix bugs in time. For example, High Priority bugs usually should be fixed in 2 weeks. High/critical bugs should be fixed as soon as possible, which has impaction to whole system.
- Update bugs in time with comments. Set the bug to Fixed and provide the tree/commit info, after bug fixing. Bug should not be marked as "fix", if patch is not checked into repository.
Bug Tracking
Yocto is a cross many Geo project. Single bug scrub meeting is not easy to maintain. Considering Embedded Linux will be a big project, it brings a concept named Triage to effectively track bug. Triage is mainly for setting new bug priority. Triage virtual team will based on bug's severity, features and schedule to suggest bug fixing priority. Developer teams will receive triage email in time. If reach agreement, bugs will be followed up based on priority. Team manager should be responsible for monitoring the bug fixing schedule to align with related priority. This process is to make the high priority bug be fixed in time and reduce the schedule risk. There are still separated bug review or discussion meeting, which is held through Jabber.
Triage Management Team
- Program Manager
- Architect
- Engineering Manager
- QA leader
Triage Process
The triage process will be held frequently to make sure new coming bugs are set with correct priority in time. Triage chairman scrubs new bugs and set priority everyday. PDT meeting will review past week's overall triage result once per week.
The Bugzilla system will send emails to component owner and related engineers about creating or changing of bugs. Before Triage, new bugs could be discussed and even fixed.
Bug Triage Process
Triage email example
To: Yocto OTC Subject: [Yocto-bug-triage] Bug Triage Review - ww26.1 Manager: MGR_A ================== Bug ID Sev Priority Assignee Summary 1 maj High dev_A [Poky] XYZ Build Failure Manager: MGR_B ==================== Bug ID Sev Priority Assignee Summary 2 maj High dev_B [SDK] ABC button error 3 nor Medium dev_C [X] Can't start X
Responsibility
- Triage Chairman
- Triage new bugs: set initial priority, correct bug component and owner and mark duplicated bugs.
- Monitor overall bug fix progress with indicators: find out aging bugs.
- Send out triage result email to yocto mailing list (or Triage management team member) for review.
- Escalate aging high priority open bugs.
- Triage Management Team
- Development/Distribution managers are mandatory to review triage result and commit to fix
- Development/Distribution Manager
- Ensure developer/distro ACCEPT bugs in bugzilla within 3 days
- Commit developer to fix high priority bugs
- Program Manager
- Coordinate and push bugs to be fixed in time
- Triage Decision Forum (PDT Meeting)
- Make decision on controversial bugs
- Review and Monitor previous decision
- Triage Chairman update bugzilla based on decision
- Discuss and approve bug whose priority needs to be downgraded
Triage Chairman
Triage chairman role will be rotated in distribution leaders, engineer manager, SDK leader and QA leader. Each chairman will own 1 milestone cycle. Triage chairman starts from Yocto 0.9 M2 cycle and rotate in following order.
- Saul Wold -- Distro leader
- Susie Li -- engineer manager
- Kevin Tian -- distro leader
- Jessica Zhang -- SDK leader
- QA Leader
Bug Data Reporting
Defect Data is tracked and presented to the PDT meeting on a weekly basis by QA team. Any escalation items should be discussed and resolved in the PDT meeting.
External Owned Bug Triage
There might be some bugs owned by external developers. These bugs' triage results will be sent to Yocto public mailing list weekly.
Re-Triage for Next Milestone and Release
After a new milestone, there will be a time to review all medium priority bugs and set new Priority for next coming milestone. After a new release, there will be a time to review all low priority bugs and set new Priority for next coming release.
Feature Request Process
New feature request should be applied through Bugzilla as a bug. User could select corresponding Severity to indicate how important the feature is. User needs to follow Feature Request template to submit new request. Triage chairman will mark new feature request bug priority to FEAT.
There will be two important feature request review time per year in Aug. and Jan. (2 months before current release). In the review period, there will be 2~4 times review meetings by CCB. CCB will decide new features which will be included in next release, that will be released 8 months later. If feature request missed the review period, in theory it will be missed in next release. Any approved new feature will be reset Priority value and Target Version value. Non approved feature will keep "FEAT" priority.
Change Control Board
Following people should join CCB to review new feature request. PME(Product Marketing Engineer), PM, Architect, Managers, Key developer and QA leader
Feature Request Bug Requirement
Feature request bug should be added specific Keywords in subject and Keywords field.
Following message should be added at the beginning of subject. [FEAT_REQ Company Yocto_Target_Version]
For example: [FEAT_REQ xxx 1.0] Add Connman for network connection.
Following words should be added in Keywords field. FEATURE_REQUEST
Feature Request Bug Template
1. Feature Description: 2. Feature Present Code Status(including location and version): 3. Code License (e.g. GPLv2): 4. Business Impaction: 5. Other Justification:
