|
|
| (3 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| Place to discuss changes to the Yocto Autobuilder.
| | Obsolete - Can be deleted |
| | |
| We will collaborate with OE people on this. Below is OE's proposal and requirements
| |
| | |
| == New build-testing/QA-testing infrastructure ==
| |
| | |
| We need solution to:
| |
| # Collect and report build env and results statistics per package;
| |
| # Control build task execution on build slaves.
| |
| | |
| Entities we should have:
| |
| # '''Client'''. This is client-side software which should report some build/QA-stats to some master server. Clien-side build may be started by human or by build slave. Should be some bitbake 'plugin' like oe-stats.bbclass/buildstats.bbclass.
| |
| # '''Slave'''. Slave should periodically ask master server for new builds, fetch build params and run appropriate builds. I.e. this is client-side software for 'build farm' nodes or for individuals, who will provide his computer to run community builds.
| |
| # '''Master'''. Master have 3 different purposes and some database behind.
| |
| ## '''Stats collector'''. It should accept and store reports sent by clients. Most loaded part.
| |
| ## '''Slave supervisor'''. It should control slaves and provide work for them.
| |
| ## '''Interface'''. It should provide UI for looking reports from collected stats.
| |
| | |
| NOTE: May be terminology master-slave should be replaced with something because of political correctness.
| |
| | |
| | |
| == Build statistics visualization use cases ==
| |
| | |
| - I want to see 20111109 and 20111110 and compare the time it took to build foo
| |
| - I want to see 20111109 and 20111110 and compare the time it took to do_rootfs
| |
| - I want to see 20111109 and 20111110 and 20090404 and compare the time it took to do everything that took more than 4 seconds
| |
| - I just want to see a graph similar to the pybootchart graph of 20110101 of every bitbake task that took more than 8 seconds
| |
| - I want to run this standalone. I want this to be an easy setup so I can look at http://localhost/buildstats, have it configured to look at my buildstats dir and be able to get graphs on that.
| |
| - I want to be able to run this using my autobuilder.
| |
| - I want to have the option of allowing others to upload their buildstats (for OE mainly) similar to how OEStats does it
| |
| | |
| == Use cases ==
| |
| * Alice is started build on home machine. It fails and show URL where full error info is located. Alice show this URL on irc channel/ML and got help. If Alice doing build offline she may run some tool (push-build-results) when online to upload stats and errors to server and get that URL.
| |
| * Bob is a developer. He is started build which is failed on some package. He is going to Yocto buildstats server to check what builds of that package was succeeded (if any).
| |
| * Claudia have strong machine that mostly idling. She want to allow Yocto project to run builds on her machine when it is idle (at night e.g.).
| |
| * Daniel is core developer but have slow machine under his hands. So he is going to Yocto's build server and requesting build from his git repo/branch and some special build environment. Build master rearrange tasks and put his task to start of queue. After build it sends build short info to Daniel via email/jabber/irc.
| |
| * Elizabeth is sysadmin at some big company which doing some products based on Yocto. Company have large buildfarm. Elizabeth want to manage large set of builds from her workstation or notebook. She have installed build slaves on farm nodes and setup build master on some server. So how she may control it over web interface - add/remove/change builds parameters, wire some builds to nodes, look at build results.
| |
| | |
| == Requirements ==
| |
| Here we are trying to collect requirements to create such solution.
| |
| | |
| === Client ===
| |
| * report build environment to server at build start:
| |
| ** build start timestamp in UTC
| |
| ** target MACHINE
| |
| ** target DISTRO
| |
| ** target IMAGE
| |
| ** build host ARCH and DISTRO
| |
| ** bitbake version
| |
| ** OE metadata version
| |
| ** builder name/id
| |
| ** build type (clean/incremental)
| |
| * report to server per-package
| |
| ** package name
| |
| ** package status (successfull/failed + on which task)
| |
| ** package build duration
| |
| ** log file(s) for failed/QA-failed package
| |
| * report overall build status after build end:
| |
| ** build duration
| |
| ** build status (successfull/failed, QA issues)
| |
| | |
| Good idea to store data in persistent storage and report periodically (when?) or after build finished. Then you may even do a build offline and flush all results to server when connected to internet.
| |
| | |
| === Slave ===
| |
| * announce his capabilities (build environment) to master on connect
| |
| * ask server for build task parameters
| |
| * do a build
| |
| * should have possibility to link slave task with build results, because results are reported by client, not by slave itself.
| |
| | |
| === Master ===
| |
| * control slaves
| |
| * collect info from clients and slaves
| |
| * build reports on collected data:
| |
| ** "waterfall"
| |
| ** "matrix"
| |
| ** something like big table from OE's [http://wiki.openembedded.org/index.php/Testing Testing] page
| |
| * provide ability to search collected data at least by:
| |
| ** builder name/id
| |
| ** target params (MACHINE/DISTRO/IMAGE)
| |
| ** package name
| |
| ** build status (failed builds)
| |
| * server task queue admins should have ability to change order of tasks for slave servers and attach tasks to slaves.
| |
| | |
| == Details ==
| |
| | |
| | |
| == Technical part ==
| |
| | |
| # BuildBot http://trac.buildbot.net/ (used by Yocto)
| |
| # Jenkins http://jenkins-ci.org/, remote access API: http://wiki.jenkins-ci.org/display/JENKINS/Remote+access+API
| |
| # LAVA https://wiki.linaro.org/Platform/Validation/LAVA
| |
| | |
| == Discussions ==
| |
| | |
| === Discussion with RP on #yocto ===
| |
| [23:35] <Jay7> in short we (OE) are discussing now new QA- and build-testing infrastructure<br />
| |
| [23:36] <Jay7> I've collected some requirements here: http://wiki.openembedded.net/index.php/BuildStats<br />
| |
| [23:36] <Jay7> from merging perspective may be good idea to collaborate on this<br />
| |
| [23:39] <RP__> Jay7: Have you seen buildstats.bbclass in poy?<br />
| |
| [23:39] <Jay7> RP__: not yet<br />
| |
| [23:39] <RP__> Jay7: That starts to address collecting the info you're after<br />
| |
| [23:40] <Jay7> I have seen qemu runtime testing part and oestats-client in OE<br />
| |
| [23:40] <RP__> Jay7: If we collect the data, you could then feed it to an oestats style service all in one<br />
| |
| [23:40] <RP__> Jay7: So what I'm saying is that buildstats.bbclass is some effort on the Yocto side to start working on a piece of this problem<br />
| |
| [23:40] <ka6sox> okay this would be after the entire build either works or fails?<br />
| |
| [23:40] <Jay7> RP__: well, I'll look there then<br />
| |
| [23:40] <RP__> ka6sox: Yes<br />
| |
| [23:41] <ka6sox> okay does it include the logs?<br />
| |
| [23:41] <RP__> ka6sox: At present, no<br />
| |
| [23:41] <Jay7> RP__: have it server-side part?<br />
| |
| [23:41] <Jay7> I mean something implemented<br />
| |
| [23:41] <RP__> Jay7: At the moment its there purely to log the data onto disk as files<br />
| |
| [23:42] <Jay7> ah, ok <br />
| |
| [23:42] <RP__> Jay7: We have ideas about post processing it for various reasons, remote transmission is one of those things<br />
| |
| [23:42] <RP__> and obviously what gets transmitted can be discussed<br />
| |
| [23:42] <ka6sox> okay, well, since we are working on common goals comming up with a common solution would be good.<br />
| |
| [23:42] <RP__> ka6sox: agreed, we should try and share what work we can<br />
| |
| [23:43] <Jay7> so, you are using Buildbot as master->slave and buildstats.bbclass+some server to collect data from client builds<br />
| |
| [23:43] <RP__> Jay7: currently we collect success/failure with buildbot<br />
| |
| [23:43] <ka6sox> RP__, so what if we tackle the server side and you tackle the build side and we create a common dataset?<br />
| |
| [23:43] <RP__> Jay7: buildstats is the start of our next generation plans<br />
| |
| [23:44] <Jay7> ah, well..<br />
| |
| [23:44] <RP__> ka6sox: We're open to any help and collaboration on the stats collection/transmission<br />
| |
| [23:44] <Jay7> so have you plan to replace buildbot or integrate it with buildstats anyhow?<br />
| |
| [23:45] <RP__> Jay7: I think we're moving towards collecting data separately from buildbot<br />
| |
| [23:45] <RP__> Its UI to the data is suboptimal<br />
| |
| [23:45] <RP__> its good at triggering builds at the right time and watching scms for changes<br />
| |
| [23:45] <ka6sox> RP__, do you care about console on success or only on failure?<br />
| |
| [23:46] <RP__> ka6sox: failure is certainly the more interesting<br />
| |
| [23:46] <ka6sox> I would think so.<br />
| |
| [23:46] <RP__> ka6sox: success has its uses as a reference but those could be turned off by default<br />
| |
| [23:47] <Jay7> other's success is good to look when you have failure :)<br />
| |
| [23:47] <RP__> What I really want to get to is being able to record what files were in a package, how big the package was, how long a task took to run etc<br />
| |
| [23:47] <ka6sox> okay compression.<br />
| |
| [23:47] <RP__> but not all data we record has to be transmitted for any given protocol<br />
| |
| [23:47] <ka6sox> RP__ me too...right now its each individual task or the WHOLE thing.<br />
| |
| [23:48] <ka6sox> by package would be best..as the way its reported often is minimal, 2008.1, binutils<br />
| |
| [23:48] <ka6sox> by the package.<br />
| |
| [23:48] <RP__> buildstats is simple right now but if you look at the mailing lists you'll see interesting graphs from even that data (http://tim.rpsys.net/bootchart.png)<br />
| |
| [23:48] <RP__> I want to see how these things change over time<br />
| |
| [23:49] <Jay7> cool graph<br />
| |
| [23:49] <Jay7> I have some benchmarks but done with my own scripts :)<br />
| |
| [23:50] <ka6sox> somewhere between "capture nothing" and "capture everything" there is capture what is needed.<br />
| |
| [23:50] <Jay7> capture everything is better than nothing and HDD is cheap now.. and some FS have deduplication feature ;)<br />
| |
| [23:51] <ka6sox> Jay7, okay if you want to keep it on the builders.<br />
| |
| [23:51] <Jay7> I like doing graphs from data and look on it :)<br />
| |
| [23:51] <ka6sox> but BW and HD space on servers that aggregate this is not.<br />
| |
| [23:52] <zeddii> sgw, late pong. I had to go pickup a kid from school and missed the bug scrub.<br />
| |
| [23:52] <ka6sox> I'd like to be able to get the data if needed...but to simply throw everything across the net is wasteful.<br />
| |
| [23:52] <Jay7> I hope Intel provided good infrastucture for yocto ;)<br />
| |
| [23:52] <Jay7> well, but I'm fine with client-only stats<br />
| |
| [23:53] <Jay7> client-only detailed stats and server-based short stats<br />
| |
| [23:53] <ka6sox> RP__, does buildbot allow clients that are behind corp FW's?<br />
| |
| [23:53] <RP__> As I said, this is what I want to write onto disk during a build, not put over the wire which should be something different<br />
| |
| [23:53] <RP__> ka6sox: I suspect not easily<br />
| |
| [23:54] <ka6sox> nothing seems to.<br />
| |
| [23:54] <ka6sox> the only thing I knew that worked fine with that was wanna-build.<br />
| |
| [23:55] <Jay7> RP__: may be dedicate one TSC meeting to this?<br />
| |
| [23:55] <Jay7> or may be non-TSC<br />
| |
| [23:55] <ka6sox> at least its python...we can work with that.<br />
| |
| [23:55] <RP__> Jay7: Its fine to have a meeting, we need people who have time to do the work...<br />
| |
| [23:55] <Jay7> because OE's reaction in ML was empty<br />
| |
| [23:56] <RP__> Jay7: We really struggled in this cycle to sort out buildstats for yocto. It is going to be assigned time for 1.1<br />
| |
| [23:56] <ka6sox> Jay7, we have 3 folks so far...we can start with that.<br />
| |
| [23:56] <RP__> pidge is the person who worked on buildstats for Yocto<br />
| |
| [23:56] <RP__> She is also our release engineer so coming up to a release is a bad time to involve her in the discussions<br />
| |
| [23:57] <ka6sox> RP__ we need to know what is important both to keep and to display.<br />
| |
| [23:57] <Jay7> when release is planned?<br />
| |
| [23:57] <RP__> Jay7: Start of April<br />
| |
| [23:57] <ka6sox> the keeping isn't so bad its the display and collection.<br />
| |
| [23:57] <RP__> We hit hard freeze on friday<br />
| |
| [23:58] <RP__> ka6sox: I'm working on the principle that we can start to collect things whilst we work on the other parts<br />
| |
| [23:58] <RP__> You can see my couple of hours efforts at display :)<br />
| |
| [23:58] <ka6sox> Jay7, lets look @ what buildstats is today and see if OE can use that as a starting place.<br />
| |
| [23:59] <ka6sox> RP__ yes, I can see there is good potential there.<br />
| |
| [23:59] <Jay7> ok.. then let's wait for OE and Yocto releases then have continue this<br />
| |
| [23:59] <ka6sox> Jay7, right, but lets familiarize ourselves with it so that we are ready to talk.<br />
| |
| [23:59] <ka6sox> and ready to take action when its time.<br />
| |
| [00:00] <RP__> ka6sox: The idea is we're planning to expand its scope over time<br />
| |
| [00:00] <RP__> ka6sox: and most likely write something to convert the flat files into something xml like<br />
| |
| [00:00] <RP__> (for storage/transfer purposes)<br />
| |
| [00:00] <ka6sox> RP__, I like the idea of xml...<br />
| |
| [00:00] <Jay7> and I dislike idea of xml :)<br />
| |
| [00:00] <Jay7> but now it does not matters :)<br />
| |
| [00:01] <ka6sox> most important to me are the definitions of what data is important and not.<br />
| |
| | |
| | |
| === Discussion with asac on #linaro ===
| |
| [03:36] <Jay7> seems you are working on build stats tools as well :)<br />
| |
| [03:37] * Jay7 have doing some prototype for Yocto/OE<br />
| |
| [03:37] <Jay7> https://wiki.yoctoproject.org/wiki/Yocto_Buildbot_Autobuilder_Discussions<br />
| |
| [03:37] <Jay7> here are my ideas<br />
| |
| [03:38] <asac> Jay7: our stuff is a cloud setup ... and assume eversything is cross build ... so we dont need slaves querying for new builds<br />
| |
| [03:39] <asac> we just spin up an instance, build, kill it<br />
| |
| [03:39] <asac> oh its build-testing ;)<br />
| |
| [03:39] <asac> hehe<br />
| |
| [03:39] <asac> nevermind<br />
| |
| [03:39] * asac cannot read<br />
| |
| [03:39] <Jay7> there is a lot of things combined<br />
| |
| [03:40] <Jay7> building/testing/collecting statistics/etc<br />
| |
| [03:40] <asac> Jay7: whats the status of that project?<br />
| |
| [03:40] <asac> Jay7: sure, building is often part of testing/validation<br />
| |
| [03:40] <Jay7> asac: 3 different clients, 2 different dead servers and no master/slave :)<br />
| |
| [03:41] <asac> lol<br />
| |
| [03:41] <Jay7> seems we are starting from scratch<br />
| |
| [03:41] <asac> Jay7: who is leading that effort?<br />
| |
| [03:41] <asac> Jay7: why dont you join lava ;)<br />
| |
| [03:41] <Jay7> lava?<br />
| |
| [03:41] <asac> thats our testing/validation infrastructure<br />
| |
| [03:41] <asac> feels quite similar<br />
| |
| [03:41] <Jay7> I'm slowly trying to prototype this<br />
| |
| [03:41] <Jay7> there are 2 people around as well<br />
| |
| [03:42] <asac> we currently focus on image testing, benchmarking etc.<br />
| |
| [03:42] <Jay7> asac: where I can read about lava? :)<br />
| |
| [03:42] <asac> Jay7: we have a team of 6 ;)<br />
| |
| [03:42] <asac> Jay7: https://wiki.linaro.org/Platform/Validation/LAVA<br />
| |
| [03:43] <Jay7> asac: your team are paid to work on this I suspect ;)<br />
| |
| [03:43] <asac> but i think building could be rather done with something like our build service and then injected into the test farm. focus is realyl about executing tests/benchmarks on various images on various boards etc.<br />
| |
| [03:43] <asac> Jay7: yes, thats why i think its better to join here<br />
| |
| [03:43] <Jay7> well<br />
| |
| [03:43] <Jay7> pidge have talk on ELC about this<br />
| |
| [03:44] <Jay7> pidge == Elisabeth Flanagan<br />
| |
| [03:44] <Jay7> from Yocto side<br />
| |
| [03:44] <asac> funny<br />
| |
| [03:44] <asac> plars also talks about LAVA there ;)<br />
| |
| [03:44] <Jay7> cool<br />
| |
| [03:44] <Jay7> I should say pidge about this :)<br />
| |
| [03:44] <asac> is she the projet owner?<br />
| |
| [03:45] <asac> i think we should discuss, yes.<br />
| |
| [03:45] <asac> we build it for android and ubuntu on ARM atm<br />
| |
| [03:45] <asac> so adding another thing would be just "one" more<br />
| |
| [03:45] <Jay7> asac: yes, it would be great imho<br />
| |
| | |
| === Discussion with pidge/Jay7/ka6sox on #yocto-buildstats ===
| |
| * Now talking on #yocto-buildstats
| |
| <pidge> so, the first thing I want to hash out is what we want this to look like. It should be stand alone, which, IMHO, simplifies this quite a bit.
| |
| <pidge> I can set up a repo for it this PM yocto-buildstats or something like that
| |
| <pidge> I'd like to be able to see a graphic representation of: build/package comparisions based on various metrics we collect so we can compare builds.
| |
| <pidge> the metric bit should be configurable so as we add collected data the website automagically picks up that we've added, say, disk io
| |
| <pidge> I would like to keep this python-esque, since we all deal with it anyhow. No sense doing it in java or something.
| |
| <pidge> Jay: ping
| |
| <Jay7> pong :)
| |
| <Jay7> I'm here
| |
| <ka6sox-farfarawa> okay that works for me....
| |
| <pidge> how does this sound to you (since I'm hoping *grin* that you're going to be doing a lot of this)
| |
| * pidge 's web-foo isn't strong
| |
| <Jay7> hehe :)
| |
| <ka6sox-farfarawa> the issue of how and when we collect the stats has to be integrated into a task that isn' there now.
| |
| <ka6sox-farfarawa> so we have to write that already
| |
| <ka6sox-farfarawa> what we collect and how we use it are separate IMHO.
| |
| <ka6sox-farfarawa> those need to be flexible.
| |
| * Jay7 is looking into pidge's email
| |
| <pidge> yocto already does this. I think porting buildstats.bbclass would be straight forward
| |
| <pidge> elizabeth.flanagan@intel.com
| |
| <Jay7> I mean letter :)
| |
| <Jay7> there is some topic list
| |
| <ka6sox-farfarawa> yes, thats what I think that we need to backport into OE.
| |
| <Jay7> btw, I wrote little use cases for buildstats
| |
| <pidge> Jay7: I'd like to see that. I'll get you set up on the git repo asap.
| |
| <Jay7> moment, I'm uploading from notebook
| |
| <pidge> ok
| |
| <ka6sox-farfarawa> okay other than pass/fail and log information what else do you want to look at for metrics?
| |
| <pidge> well, right now, I want disk io. I have that partially done.
| |
| <pidge> I also want image size without extra space
| |
| <pidge> one of the things I want to do (and this I'm still trying to figure out if it'll work or not) is strip out the elf headers and do a diff on the objcopy. Anything with more than %1 or so diff is noted. This will have to be configurable as it would slow things down substantially.
| |
| <Jay7> https://wiki.yoctoproject.org/wiki/Yocto_Buildbot_Autobuilder_Discussions#Use_cases
| |
| <pidge> ka6sox-farfarawa: anything you can think of?
| |
| <Jay7> that is from where requirements come
| |
| <Jay7> I fear we are at other sides of this task :)
| |
| <Jay7> I'm more at user/devel side
| |
| <Jay7> you - at infrastructure side :)
| |
| <ka6sox-farfarawa> Jay7, thats why I want you here...
| |
| <ka6sox-farfarawa> we need different things but I want to make sure we have all of it when we need it
| |
| <pidge> hrm. I want to avoid uploads of buildstats. ka6sox was telling me how that was slowing down his network and I'd like to avoid that here. I would like to see this be standalone so people can make comparative statistics but, for me, I don't care if Alice takes 20 hours to build on her 486. What I do care about is if Alice was taking 20 hours and after a git fetch is taking 40 hours. If it's stand alone, alice can run this on her
| |
| <pidge> machine and identify they issue herself as opposed to us providing all this infrastructure around helping alice.
| |
| <ka6sox-farfarawa> pidge, I want scratch build size, time to build, pass/fail for a particular set.
| |
| <ka6sox-farfarawa> and logs of the build available.
| |
| <pidge> we have the last two.
| |
| <pidge> we need scratch
| |
| <ka6sox-farfarawa> also since we aren't regressing individual packages but a "set"
| |
| <pidge> scratch == image size without extra space, correct?
| |
| <ka6sox-farfarawa> yes
| |
| <pidge> we have pass/fail for each recipe and image set
| |
| <ka6sox-farfarawa> pidge, I don't mind collecting @ the end of *each* package.
| |
| <ka6sox-farfarawa> and also for the entire build.
| |
| <ka6sox-farfarawa> so for example: I want to know about how binutils built...and also the entire build.
| |
| <pidge> ok, so, hold on a sec.
| |
| <pidge> if you look here:
| |
| <pidge> http://autobuilder.yoctoproject.org/nightly/20110325-1/buildstats/buildstats/poky-image-sato-qemuarm/201103251400/base-passwd-3.5.22-r0/
| |
| <Jay7> pidge: OE build stats problem was caused by wrong design
| |
| <pidge> and here:
| |
| <pidge> http://autobuilder.yoctoproject.org/nightly/20110325-1/buildstats/buildstats/poky-image-sato-qemuarm/201103251400/build_stats
| |
| <Jay7> did you want to not collect stats from users at all?
| |
| <pidge> no. if they want to collect their statistics, I want to enable that.
| |
| <pidge> but I want to avoid having a central server to collect everyone's build statistics
| |
| <ka6sox-farfarawa> yes, we want to do that on a package and user basis
| |
| <pidge> at least for now
| |
| <ka6sox-farfarawa> because one recipe can have 4 or 5 patches
| |
| <Jay7> pidge: what about Bob's case?
| |
| <pidge> That one I want to support, however, we don't build every git push. So if he has master cloned out of a commit we didn't build, he may not get that info.
| |
| <pidge> it would help him though if he has a git hash between something that passed and something that didn't
| |
| <Jay7> ah, you still considering only your autobuilder :)
| |
| <ka6sox-farfarawa> pidge, for the nightly's we can use that factory...how do you want to handle the testing builds?
| |
| <pidge> right, because in order for me to get somewhat valid statistics (time, cpu and disk) the hardware should be static.
| |
| <ka6sox-farfarawa> what did you call them?
| |
| <pidge> fuzz?
| |
| <pidge> fuzz builds... hrmm. Since they're always different, buildstats for them are worthless.
| |
| <Jay7> nice stats logged btw
| |
| <pidge> they're just smoke tests
| |
| <pidge> ty
| |
| <Jay7> really useful
| |
| <pidge> hrmm... or maybe they aren't
| |
| * pidge thinks
| |
| <ka6sox-farfarawa> its important
| |
| <Jay7> well.. let's talk about autobuilder things then right now
| |
| <Jay7> w/o considering other builders
| |
| <ka6sox-farfarawa> we need to know if you add patches #889, 994 and 1100 that they don't blow up
| |
| <pidge> right
| |
| <pidge> that's what fuzz is
| |
| <pidge> we're going to be building from staging
| |
| <pidge> once fuzz passes, it'll send a PULL to the list saying "Hey, these passed a fuzz test!"
| |
| <pidge> it's not 100% that they won't break, but it's more than what some developers do
| |
| <Jay7> what repos/branches/arches/distros you will test?
| |
| <pidge> so, if fuzz builds core-image-sato on qemumips and it passes, it sends out info based on that
| |
| <pidge> for fuzz it'll be:
| |
| <Jay7> with oe-core we will have LOT of arch'es and distros
| |
| <Jay7> well, with meta-oe
| |
| <pidge> I'm setting it to build off of a branch right now
| |
| <ka6sox-farfarawa> Jay7, for oe-core I only have 4: qemu-mips, qemu-arm, qemu-ppc, qemu-x86
| |
| <pidge> all the arches we support x all the images the arch supports.
| |
| <ka6sox-farfarawa> thats all
| |
| <pidge> qemux86_64 as well
| |
| <ka6sox-farfarawa> okay 5...sorrgy
| |
| <pidge> still IA but we get some fails between the two
| |
| <Jay7> that's why I'm talking about other builders :)
| |
| * pidge has three new servers coming on line
| |
| <pidge> maybe more
| |
| <Jay7> or you should have very large set of build nodes in datacenter
| |
| * pidge smiles. I'm working on it. I think once I get my new sys admin up to speed, we can do some long needed maintence, fix our disk IO issue and then run two slaves per machine. Which will come out to 10 machines
| |
| <pidge> doh, kicked out my video cable
| |
| <pidge> so, I think we have a general idea of what we're looking for?
| |
| <Jay7> well.. master + large set of slaves
| |
| <Jay7> + interface to
| |
| <pidge> that's autobuilder. I would like this to stand alone so it'll take any buildstats dir from anything (local builder, autobuilder, someone being crazy and running it under hudson) and run with it
| |
| <pidge> does that make sense?
| |
| <ka6sox-farfarawa> okay so for poky-master/poky-slave on 1 box too?
| |
| <Jay7> yes, this may help but we should not care about this very much at this stage
| |
| <pidge> sure. it should be utterly agnostic as to where it get's the data
| |
| <pidge> Jay7: agreed.
| |
| <Jay7> so let's define feature set for october?
| |
| <pidge> sure
| |
| <pidge> 1. visual graph with configurable x/y
| |
| <Jay7> graphs :)
| |
| <pidge> I want to see 20111109 and 20111110 and compare the time it took to build foo
| |
| <pidge> I want to see 20111109 and 20111110 and compare the time it took to do_rootfs
| |
| <pidge> I want to see 20111109 and 20111110 and 20090404 and compare the time it took to do everything that took more than 4 seconds
| |
| <pidge> I just want to see a graph similar to the pybootchart graph of 20110101 of every bitbake task that took more than 8 seconds
| |
| <pidge> all use cases
| |
| <pidge> I want to be able to git clone this thing we're building, set a location to my buildstats dir and run it. I should go to http://localhost/buildstats and see this page
| |
| <Jay7> I want to see that bootchart + cpu + disk io + something else on same chart
| |
| <pidge> yes!
| |
| <Jay7> to see where we are slow :)
| |
| <Jay7> or too hungry
| |
| <pidge> exactly
| |
| <pidge> ok, I think we're on the same page
| |
| <Jay7> package build results cube is nice to see too..
| |
| <pidge> yes
| |
| <Jay7> well..
| |
| <pidge> I think we've got a general path forward here. The question I have is. What do you want to use framework wise?
| |
| <Jay7> django?
| |
| <ka6sox-farfarawa> twisted?
| |
| <pidge> outside of my realm of expertise. I've only done servlets within the past 2 years
| |
| <Jay7> if we will use HTTP only django is enough
| |
| <Jay7> if we need plain tcp/udp/xmpp/irc or something else - then twisted
| |
| <ka6sox-farfarawa> I am a simple guy..and html is about the limit of my web stuff
| |
| <pidge> I think HTTP is ok for now
| |
| <ka6sox-farfarawa> I'd rather just start with twisted if I am going to have to learn anyways.
| |
| <pidge> I think django should be fine. this is visualization, not reporting
| |
| <Jay7> http is good for posting data from behind the proxy
| |
| <ka6sox-farfarawa> okay any of that is fine...
| |
| <ka6sox-farfarawa> first thing is to get the data collected...then we can show it.
| |
| <Jay7> thats only advangate though
| |
| <pidge> brb
| |
| <pidge> phone
| |
| <Jay7> but may be using stand alone collector is good idea
| |
| <ka6sox-farfarawa> we need to make it so that we can collect it for later analysis...
| |
| <ka6sox-farfarawa> analyze what we can now and collect for later analysis once we figure it out.
| |
| <Jay7> but size of data is amazing..
| |
| <pidge> I'm cool with a standalone collector. But yeah, the data does get unwieldy. I'm thinking of putting in some flags like BSTATS_DISK BSTATS_CPU, etc
| |
| <pidge> make it so people don't have to collect everything if they don't want to
| |
| <ka6sox-farfarawa> well...can we bzip it before we send it?
| |
| <Jay7> send is easy task
| |
| <pidge> sending it?
| |
| <Jay7> processing is hard :)
| |
| <pidge> why are we sending it.
| |
| <Jay7> slave -> collector
| |
| <ka6sox-farfarawa> pidge, not yocto...
| |
| <ka6sox-farfarawa> we are thinking later
| |
| <ka6sox-farfarawa> but making it flexible.
| |
| <pidge> ahh
| |
| <Jay7> or client -> collector
| |
| <ka6sox-farfarawa> right that method.
| |
| <ka6sox-farfarawa> in my case we are going to have a "master" which will control builds with "slaves" in the LF rack I think
| |
| <ka6sox-farfarawa> so "sending it"
| |
| <ka6sox-farfarawa> is across Gb wire
| |
| <Jay7> hehe.. you are experienced :)
| |
| <ka6sox-farfarawa> so its 20'
| |
| <ka6sox-farfarawa> but for OE builds (using same method) the are distributed.
| |
| <ka6sox-farfarawa> since buildbot doesn't expect FT connections and will live nicely behind FWs
| |
| <ka6sox-farfarawa> pidge, do you care about / package or just / entire build?
| |
| <Jay7> hm.. are buildbot slaves contacting master?
| |
| <ka6sox-farfarawa> I am still confusec
| |
| <Jay7> or master contacting slaves?
| |
| <ka6sox-farfarawa> slaves call master
| |
| <pidge> I care about both
| |
| <ka6sox-farfarawa> "I'm available for work"
| |
| <pidge> in buildbot, they do both IIRC
| |
| <Jay7> ah, that's work then
| |
| <ka6sox-farfarawa> right
| |
| <ka6sox-farfarawa> it will work for both close and distributed
| |
| <pidge> slave alerts master to it's presence, master sends instructions to slave
| |
| <ka6sox-farfarawa> right...but the master doesn't freak out if slave disappears for a while.
| |
| <ka6sox-farfarawa> and goes off to work.
| |
| <ka6sox-farfarawa> its a variable timeout
| |
| <ka6sox-farfarawa> pidge the waterfall, when you click on a particular package build, does it show you the stats?
| |
| <pidge> right, master just says "whoops, slave ain't there"
| |
| <pidge> no
| |
| <pidge> for stats you have to kind of know where to go
| |
| <Jay7> so will we use buildbot later?
| |
| <pidge> for now, yes
| |
| <pidge> I'm putting in a lot of work the next 6 months to make it not suck
| |
| <Jay7> ah, well
| |
| * pidge writes note to add stats linking to package build IO
| |
| <pidge> the problem is, nothing else meets the requirements that bb meets
| |
| <Jay7> do you pushing changes to upstream? :)
| |
| <Jay7> ah
| |
| <pidge> and it was long neglected
| |
| <Jay7> this is answer :)
| |
| <pidge> so many of the changes are oe/yocto specific
| |
| <Jay7> then we may just use our own fork
| |
| <pidge> really are things best handled in the config file
| |
| <pidge> currently, it's a master of bb 0.83*
| |
| <pidge> keeping it like that and upstreaming when we find something that is flat out broken (and not much is) is optimal
| |
| <ka6sox-farfarawa> we need to use the p2 release as it has fix for git-poller
| |
| <ka6sox-farfarawa> for fuzz builds
| |
| <pidge> ka6sox: we do
| |
| <pidge> yeah, 0.83 for slaves 0.83p2 for master... or reverse. I forget which one is which
| |
| <pidge> it use to be master and slave was the same tarball. they split that up in... 0.82?
| |
| <ka6sox-farfarawa> something like that
| |
| <ka6sox-farfarawa> okay so we have 0.83p2...good
| |
| <pidge> ok, folks. I'm late for a meeting. I think we have a path forward. I say we leave this channel up and use it as a side channel from #yocto.
| |
| <Jay7> well..
| |
| <ka6sox-farfarawa> sounds good...
| |
| <ka6sox-farfarawa> I'd like to use the irc feature too and get it to announce..here.
| |
| <ka6sox-farfarawa> :D
| |
