Commit graph

29 commits

Author SHA1 Message Date
Nick Mathewson
f5a852de91 Note that threadpool_queue_work...() can't actually return NULL 2017-07-27 16:28:59 -04:00
Nick Mathewson
bddea78ded Fix a pair of stale comments in workqueue.c
These comments said that each thread had a separate queue, but we
haven't been using that design for some while.
2017-07-27 16:28:59 -04:00
Nick Mathewson
efadebf7c3 Make the chance for priority inversion thread-specific
Instead of choosing a lower-priority job with a 1/37 chance, have
the chance be 1/37 for half the threads, and 1/2147483647 for the
other half.  This way if there are very slow jobs of low priority,
they shouldn't be able to grab all the threads when there is better
work to do.
2017-07-27 16:28:59 -04:00
Nick Mathewson
10e0bff4ca Add support for multi-priority workqueues
Each piece of queued work now has an associated priority value; each
priority goes on a separate queue.

With probability (N-1)/N, the workers will take work from the highest
priority nonempty queue.  Otherwise, they'll look for work in a
queue of lower priority.  This behavior is meant to prevent
starvation for lower-priority tasks.
2017-07-27 16:28:05 -04:00
Nick Mathewson
44514058b9 Correctly handle fd-drain errors on windows workqueues
Windows doesn't let you check the socket error for a socket with
WSAGetLastError() and getsockopt(SO_ERROR).  But
getsockopt(SO_ERROR) clears the error on the socket, so you can't
call it more than once per error.

When we introduced recv_ni to help drain alert sockets, back in
0.2.6.3-alpha, we had the failure path for recv_ni call getsockopt()
twice, though: once to check for EINTR and one to check for EAGAIN.
Of course, we never got the eagain, so we treated it as an error,
and warned about: "No error".

The fix here is to have these functions return -errno on failure.

Fixes bug 21540; bugfix on 0.2.6.3-alpha.
2017-03-15 12:16:17 -04:00
Nick Mathewson
aae034d13e Write a bunch of module documentation.
This commit adds or improves the module-level documenation for:

  buffers.c circuitstats.c command.c connection_edge.c control.c
  cpuworker.c crypto_curve25519.c crypto_curve25519.h
  crypto_ed25519.c crypto_format.c dircollate.c dirserv.c dns.c
  dns_structs.h fp_pair.c geoip.c hibernate.c keypin.c ntmain.c
  onion.c onion_fast.c onion_ntor.c onion_tap.c periodic.c
  protover.c protover.h reasons.c rephist.c replaycache.c
  routerlist.c routerparse.c routerset.c statefile.c status.c
  tor_main.c workqueue.c

In particular, I've tried to explain (for each documented module)
what each module does, what's in it, what the big idea is, why it
belongs in Tor, and who calls it.  In a few cases, I've added TODO
notes about refactoring opportunities.

I've also renamed an argument, and fixed a few DOCDOC comments.
2016-10-17 10:16:59 -04:00
Nick Mathewson
429d15c529 Mark the unreachable lines in compat_{,p}threads and workqueue
These are all related to failures from functions that either can't
fail as we call them, or where we cannot provoke failure.
2016-06-08 17:30:22 -04:00
Nick Mathewson
23f8c9b32f Add a brief file-level description for everything in src/common 2016-02-27 18:19:57 +01:00
Sebastian Hahn
32220d38c0 Ensure worker threads actually exit when it is time
This includes a small refactoring to use a new enum (workqueue_reply_t)
for the return values instead of just ints.
2015-08-21 10:36:53 -04:00
Nick Mathewson
f724b2e5aa Merge remote-tracking branch 'public/bug16741_026' 2015-08-17 14:40:27 -04:00
Nick Mathewson
720a9ccb2f Check for EINTR correctly on windows
(even though these are nonblocking calls and EINTR shouldn't be possible).

Also, log what error we're seing if drain_fn fails.
2015-08-07 09:12:33 -04:00
cypherpunks
b3ea3c8e2f Switch order of unblocking threads and releasing the mutex.
According to POSIX, the mutex must be locked by the thread calling the signal
functions to ensure predictable scheduling behavior.

Found the issue using Helgrind which gave the warning `dubious: associated lock
is not held by any thread`.
2015-08-04 13:35:02 -04:00
cypherpunks
3b3b447f75 Fix some potential memory leaks in the thread pool code. 2015-07-21 13:57:53 -04:00
Nick Mathewson
a194385d56 Impose an upper limit on threads per threadpool.
Found by Coverity; Fixes CID 1268069
2015-05-28 12:24:29 -04:00
Nick Mathewson
0b46b08225 Check thread count for negative; realloc->reallocarray
CID 1268069
2015-02-17 08:46:11 -05:00
Nick Mathewson
5d2a23397a Fix a few coverity "Use after NULL check" warnings
Also remove the unit test mocks that allowed get_options() to be
NULL; that's an invariant violation for get_options().
2015-02-16 15:40:15 -05:00
Sebastian Hahn
424edd5710 Don't leak a cond var when starting threads in a pool 2015-02-15 11:21:54 +01:00
Nick Mathewson
38b3f9a619 use the correct free fn. spotted by dgoulet 2015-01-21 14:54:38 -05:00
Nick Mathewson
3c8dabf69a Fix up some workqueue/threading issues spotted by dgoulet. 2015-01-21 12:26:41 -05:00
Nick Mathewson
a52e549124 Update workqueue implementation to use a single queue for the work
Previously I used one queue per worker; now I use one queue for
everyone.  The "broadcast" code is gone, replaced with an idempotent
'update' operation.
2015-01-15 11:05:22 -05:00
Nick Mathewson
051ad788e0 Incorporate some comments based on notes from dgoulet 2015-01-14 11:31:14 -05:00
Nick Mathewson
1e896214e7 Refactor cpuworker to use workqueue/threadpool code. 2015-01-14 11:23:34 -05:00
Nick Mathewson
e5f8c772f4 Test and fix workqueue_entry_cancel(). 2015-01-14 11:17:46 -05:00
Nick Mathewson
c51f7c23e3 Test a little more of compat_threads.c 2015-01-14 11:05:56 -05:00
Nick Mathewson
b2db3fb462 Documentation for new workqueue and condition and locking stuff 2015-01-14 11:05:54 -05:00
Nick Mathewson
4abbf13f99 Add a way to tell all threads to do something. 2015-01-14 11:01:21 -05:00
Nick Mathewson
51bc0e7f3d Isolate the "socketpair or a pipe" logic for alerting main thread
This way we can use the linux eventfd extension where available.
Using EVFILT_USER on the BSDs will be a teeny bit trickier, and will
require libevent hacking.
2015-01-14 11:01:19 -05:00
Nick Mathewson
c7eebe237d Make pending work cancellable. 2015-01-14 10:56:27 -05:00
Nick Mathewson
a82604b526 Initial workqueue implemention, with a simple test.
It seems to be working, but more tuning is needed.
2015-01-14 10:56:24 -05:00