Daemon loses pushes on mainline if they happen during a workspace update

[g-arjones]

A push to a mainline branch will be lost if they happen in time span that is shorter than the polling period.. That is because the first one will trigger a workspace update (and a build). Since the daemon uses the head commit id to detect pushes to mainline, the remaining pushes will no longer be detected.

An easy fix would be to only do the workspace update at the end of the poll loop

[g-arjones]

That's suboptimal though (we would still lose the ones that happen during the update)

[doudou]

I don't understand how this happens .. can you describe a step-by-step scenario ?

[g-arjones]

Sure..

t0: Mainline push happens on pkgA
t1: Mainline push happens on pkgB
t2: Daemon polls events and adds both to a list to be processed by some hook
t3: pkgA event is processed: The process hook compares the push head sha with the current head sha of the checked out pkgA, since they are different, the daemon triggers a build and performs a workspace update
t4: Now, with the workspace already updated, pkgB event is processed: The process hook compares the push head sha with the current head sha of the checked out pkgB, they are the same now (because of the previous workspace update), the event is then ignored and no build is triggered

[doudou]

I see two possible strategies:

• not compare the current HEAD with the push event SHA to decide whether we should trigger a build (maybe only to decide whether we should run a workspace update)
• only update the package instead of updating the whole workspace (using --no-deps)

[g-arjones]

We have to compare, otherwise we will keep triggering builds until the event stops showing up in the event list (unless we start caching the events as well, which I didn't want to).

I like the second suggestion but I don't know how to update the buildconf only

[doudou]

We have to compare, otherwise we will keep triggering builds until the event stops showing up in the event list (unless we start caching the events as well, which I didn't want to).

You get an event again even when using the Etags ?

I like the second suggestion but I don't know how to update the buildconf only

Rigth now, it is buildconf and package sets. We'd have to add support for it in autoproj.

[g-arjones]

You get an event again even when using the Etags ?

Yes, unless we implement our own middleware (and drop faraday-http-cache)

We'd have to add support for it in autoproj

👍

[doudou]

We'd have to add support for it in autoproj

+1

There's already --config, we could accept a list of configuration "items" to update, e.g.

--config main rock.core

[g-arjones]

How about using Autoproj::CLI::SwitchConfig ?

[doudou]

Not sure how that helps.

My understanding is that you need to restrict the update to the main config and/or the package sets that have been updated. SwitchConfig does not do that.

[g-arjones]

Isn't switching to the current config the same as an update?

[g-arjones]

Ah. It still updates the package sets :(