This commit fixes a bug in the wait.NoError function. If the predicate
function, f, passed to the NoError function would hang for the full
timeout, then the `predErr` would remain nil and so a nil error would be
returned from the function. This commit handles that case.
This commit sends the returned value of pred() inside a goroutine such
that when a predicate function never returns during the first run, the
wait package can timeout properly.
In high CPU usage scenarios such as our parallel itests, it seems that
some goroutines just don't get any CPU time before our test timeouts
expire. By polling 10 times less frequently, we hope to reduce the
overall number of goroutines that are spawned because of the RPC
requests within the polling code.
This commit constructs a helper closure assertAmountSent that can be
reused by other functions. The closure returns an error so that it can
be used with wait.NoError or the new wait.InvariantNoError. The latter
is added since the predicate could otherwise pass immediately for the
sphinx_replay_persistence tests, but change shortly after. It also
rounds out the wait package so that we offer all combinations of
predicate and no-error style waits.
