Commit graph

23105 commits

Author SHA1 Message Date
Alexander Færøy
b0d268a822 Add process_reset_environment() to the Process subsystem.
This patch adds a new function that allows us to reset the environment
of a given process_t with a list of key/value pairs.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
4f611a1df7 Add process_terminate().
This patch adds support for process termination to the Process
subsystem.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
338137221c Make sure we call process_notify_event_exit() as the last thing in different callbacks.
This patch makes sure that we call process_notify_event_exit() after we
have done any modifications we need to do to the state of a process_t.
This allows application developers to call process_free() in the
exit_callback of the process.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
e982fb1dae Add documentation for the is_socket and error argument of read_to_chunk().
See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
89393a77e5 Add process_get_pid() to the Process subsystem.
This patch adds support for getting the unique process identifier from a
given process_t. This patch implements both support for both the Unix
and Microsoft Windows backend.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
bb784cf4f3 Add Windows backend for the Process subsystem.
This patch adds support for Microsoft Windows in the Process subsystem.

Libevent does not support mixing different types of handles (sockets,
named pipes, etc.) on Windows in its core event loop code. This have
historically meant that Tor have avoided attaching any non-networking
handles to the event loop. This patch uses a slightly different approach
to roughly support the same features for the Process subsystem as we do
with the Unix backend.

In this patch we use Windows Extended I/O functions (ReadFileEx() and
WriteFileEx()) which executes asynchronously in the background and
executes a completion routine when the scheduled read or write operation
have completed. This is much different from the Unix backend where the
operating system signals to us whenever a file descriptor is "ready" to
either being read from or written to.

To make the Windows operating system execute the completion routines of
ReadFileEx() and WriteFileEx() we must get the Tor process into what
Microsoft calls an "alertable" state. To do this we execute SleepEx()
with a zero millisecond sleep time from a main loop timer that ticks
once a second.  This moves the process into the "alertable" state and
when we return from the zero millisecond timeout all the outstanding I/O
completion routines will be called and we can schedule the next reads
and writes.

The timer loop is also responsible for detecting whether our child
processes have terminated since the last timer tick.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
2e957027e2 Add Unix backend for the Process subsystem.
This patch adds the Unix backend for the Process subsystem. The Unix
backend attaches file descriptors from the child process's standard in,
out and error to Tor's libevent based main loop using traditional Unix
pipes. We use the already available `waitpid` module to get events
whenever the child process terminates.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Alexander Færøy
35509978dd Add new Process subsystem.
This patch adds a new Process subsystem for running external programs in
the background of Tor. The design is focused around a new type named
`process_t` which have an API that allows the developer to easily write
code that interacts with the given child process. These interactions
includes:

- Easy API for writing output to the child process's standard input
  handle.
- Receive callbacks whenever the child has output on either its standard
  output or standard error handles.
- Receive callback when the child process terminates.

We also support two different "protocols" for handling output from the
child process. The default protocol is the "line" protocol where the
process output callbacks will be invoked only when there is complete
lines (either "\r\n" or "\n" terminated). We also support the "raw"
protocol where the read callbacks will get whatever the operating system
delivered to us in a single read operation.

This patch does not include any operating system backends, but the Unix
and Windows backends will be included in separate commits.

See: https://bugs.torproject.org/28179
2018-12-17 16:39:28 -05:00
Taylor R Campbell
ed71e1e89c Create a temporary directory for tor's DataDirectory in test_rebind.
Fixes #28562.

While here, put the argument count test and usage message _before_ we
attempt to read from sys.argv.
2018-12-17 10:32:28 -05:00
Nick Mathewson
315c21d2e2 test_rebind: wait for tor to timeout, even if it is logging a lot
Fixes bug 28883; bugfix on 0.3.5.4-alpha.
2018-12-17 09:53:17 -05:00
rl1987
5b2acbec0e Refrain from closing connection if found one unrecognized address in NETINFO cell 2018-12-16 10:19:37 +02:00
rl1987
3bec371d04 Refrain from hardcoding address length and type in netinfo.trunnel 2018-12-16 10:05:35 +02:00
Nick Mathewson
3dd1f064a7 Rewrite the core of parse_short_policy() to be faster.
The old implementation did some funky out-of-order lexing, and
tended to parse every port twice if the %d-%d pattern didn't match.

Closes ticket 28853.
2018-12-14 16:07:10 -05:00
Nick Mathewson
6dc90d290d Use 25% less RAM for base64-encoded directory objects
We were allocating N bytes to decode an N-byte base64 encoding,
when 3N/4 would have been enough.
2018-12-14 13:51:51 -05:00
Nick Mathewson
3c35c0d441 Add a function to provide an upper bound on base64 decoded length 2018-12-14 13:51:51 -05:00
Nick Mathewson
cf7342ab6f Add a benchmark for parsing a microdescriptor 2018-12-14 13:51:51 -05:00
Rob Jansen
325348b360 allow any value for HearbeatPeriod in testing Tor networks 2018-12-14 09:22:23 -05:00
Nick Mathewson
f8dac5c900 Merge branch 'maint-0.3.5' 2018-12-13 19:01:29 -05:00
Nick Mathewson
94a7998158 Merge remote-tracking branch 'tlyu-github/ticket28731-035' into maint-0.3.5 2018-12-13 18:57:00 -05:00
Nick Mathewson
69264f96f3 Merge branch 'dormant_persist_squashed' 2018-12-13 08:26:10 -05:00
Nick Mathewson
e3b7fd2a81 Unit tests for back-end functions for persistent dormant state 2018-12-13 08:25:54 -05:00
Nick Mathewson
b5c04173c8 Change interaction between dormant mode and clock jumps.
When the clock jumps, and we have a record of last user activity,
adjust that record.  This way if I'm inactive for 10 minutes and
then the laptop is sleeping for an hour, I'll still count as having
been inactive for 10 minutes.

Previously, we treat every jump as if it were activity, which is
ridiculous, and would prevent a Tor instance with a jumpy clock from
ever going dormant.
2018-12-13 08:25:54 -05:00
Nick Mathewson
4afc6b172a Merge branch 'ticket28755_v2_squashed' 2018-12-12 11:07:38 -05:00
Nick Mathewson
845e8dbe59 Fuzzing module for various string operations, currently focusing on
encoding and decoding.

There are bunches of places where we don't want to invest in a full
fuzzer, but we would like to make sure that some string operation
can handle all its possible inputs.  This fuzzer uses the first byte
of its input to decide what to do with the rest of the input.  Right
now, all the possibilities are decoding a string, and seeing whether
it is decodeable.  If it is, we try to re-encode it and do the whole
thing again, to make sure we get the same result.

This turned up a lot of bugs in the key-value parser, and I think it
will help in other cases too.

Closes ticket 28808.
2018-12-12 11:07:08 -05:00
Nick Mathewson
f0a8664677 Add code to parse K=V lines into config_line_t format.
Closes ticket 28755
2018-12-12 11:07:08 -05:00
Nick Mathewson
b915b6cd21 Merge remote-tracking branch 'github/prop297' 2018-12-11 09:44:57 -05:00
Nick Mathewson
6506b1ee9f Merge branch 'maint-0.3.3' into maint-0.3.4 2018-12-11 09:41:05 -05:00
Nick Mathewson
e1273d7d1b Merge branch 'maint-0.3.4' into maint-0.3.5 2018-12-11 09:41:05 -05:00
Nick Mathewson
27e4269929 Merge branch 'maint-0.3.5' 2018-12-11 09:41:05 -05:00
Nick Mathewson
c1f9191581 Merge branch 'maint-0.2.9' into maint-0.3.3 2018-12-11 09:41:04 -05:00
Nick Mathewson
ce501a529f Merge remote-tracking branch 'catalyst-github/ticket27402' 2018-12-11 09:37:41 -05:00
Taylor Yu
1b855af5e3 Log bootstrap tag names
Add the bootstrap tag name to the log messages, so people
troubleshooting connection problems can look up a symbol instead of a
number.  Closes ticket 28731.
2018-12-10 17:22:28 -06:00
teor
4991b29311 Fallbacks: Update the hard-coded fallback list in December 2018
Merge Phoul's two lists into teor's list.

Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in
January 2018 (of which ~115 were still functional), with a list of
157 fallbacks (92 new, 65 existing, 85 removed) generated in
December 2018.

Closes ticket 24803.
2018-12-10 17:02:19 +10:00
teor
78e177d622 Fallbacks: Update the hard-coded fallback list in December 2018
Replace the 150 fallbacks originally introduced in Tor 0.3.3.1-alpha in
January 2018 (of which ~115 were still functional), with a list of
148 fallbacks (89 new, 59 existing, 91 removed) generated in
December 2018.

Closes ticket 24803.
2018-12-07 16:43:10 +10:00
Nick Mathewson
2ccb9e9444 Merge branch 'maint-0.3.5' 2018-12-06 09:26:34 -05:00
Nick Mathewson
c1f86f7492 Merge branch 'maint-0.3.4' into maint-0.3.5 2018-12-06 09:26:33 -05:00
Nick Mathewson
c4f7953d8b Merge branch 'maint-0.3.3' into maint-0.3.4 2018-12-06 09:26:32 -05:00
Nick Mathewson
00341d97f3 Merge branch 'maint-0.2.9' into maint-0.3.3 2018-12-06 09:26:32 -05:00
Karsten Loesing
57798eb1cb Update geoip and geoip6 to the December 5 2018 database. 2018-12-05 21:02:39 +01:00
Nick Mathewson
46a321fbdd Merge branch 'maint-0.3.5' 2018-12-05 10:25:12 -05:00
Nick Mathewson
967efc0d28 Merge remote-tracking branch 'tor-github/pr/546' into maint-0.3.5 2018-12-05 10:23:28 -05:00
Nick Mathewson
1eb3719a62 Merge remote-tracking branch 'public/prop298' 2018-12-05 09:43:03 -05:00
Nick Mathewson
1f95e80351 Merge branch 'prop293_squashed' 2018-12-05 09:24:51 -05:00
Nick Mathewson
a2f81b644b Write tests for mark_my_descriptor_dirty_if_too_old() 2018-12-05 09:24:45 -05:00
Nick Mathewson
00509aaafa Merge remote-tracking branch 'tor-github/pr/553' 2018-12-05 08:28:54 -05:00
Nick Mathewson
ca4b86f90a Merge remote-tracking branch 'tor-github/pr/508' 2018-12-05 08:19:02 -05:00
Nick Mathewson
c01507a5fe remember why we are doing getsockopt() 2018-12-05 08:14:21 -05:00
Nick Mathewson
4f55884315 Add an option to start tor in dormant mode for the first time. 2018-12-04 12:08:24 -05:00
Nick Mathewson
b25b8150c2 Remember in our state file how long we've spent since user activity
Rather than initializing the "Dormant" status to "off" and the "last
activity" count to "now", initialize them based on our state file:
stay dormant if we were dormant, or remember the amount of time
we've spent inactive.
2018-12-04 11:59:11 -05:00
Nick Mathewson
31a6d9f499 Add tests for parsing each routerstatus flag. 2018-12-03 13:40:08 -05:00