Exclude relays that have been down for 1 or more days from the fallback
candidate list.
When a relay operator has multiple relays, this prioritises relays that are
up over relays that are down.
Fixes issue #20926.
7 days is a tradeoff between the expected time between major Tor releases,
which is 6 months, and the number of relays with enough stability.
Relays whose OnionOO stability timer is reset on restart by bug #18050
should upgrade to Tor 0.2.8.7 or later, which has a fix for this issue.
Closes ticket #20880; maintains short-term fix in e220214 in tor-0.2.8.2-alpha.
The functions it warns about are:
assert, memcmp, strcat, strcpy, sprintf, malloc, free, realloc,
strdup, strndup, calloc.
Also, fix a few lingering instances of these in the code. Use other
conventions to indicate _intended_ use of assert and
malloc/realloc/etc.
Only some very ancient distributions don't ship with Libevent 2 anymore,
even the oldest supported Ubuntu LTS version has it. This allows us to
get rid of a lot of compat code.
If we manually remove fallbacks in C by adding '/*' and '*/' on separate
lines, stem still parses them as being present, because it only looks at
the start of a line.
Add a comment to this effect in the generated source code.
Remove a fallback that changed its fingerprint after it was listed
This happened after to a software update:
https://lists.torproject.org/pipermail/tor-relays/2016-June/009473.html
Remove a fallback that changed IPv4 address
Remove two fallbacks that were slow to deliver consensuses,
we can't guarantee they'll be fast in future.
Blacklist all these fallbacks until operators confirm they're stable.
Operators have sent emails asking to have their relays added or
removed from the fallback list. Since none of the blacklisted
relays are in the hard-coded falback list, it does not need to
be changed.
Log a notice just before the script is about to perform a
potentially time-consuming operation
Clarify the warning when py2-ipaddress isn't found
Make log levels more consistent
No behavioural change (just logging)
As well as the existing reports of IPv6 address additions or removals,
the script now warns when keys change but IPv4:ORPort or
IPv6:IPv6ORPort remain the same.
Existing checks for other whitelist detail changes have also
been re-worded and upgraded to warnings.
This makes it easier for changes to be identified so operators can
be contacted to confirm whether the change is stable.
Also add misbehaving relays to updateFallbackDirs.py blacklist,
but leave them commented out in case it's a transient issue,
or it's been resolved by the download check fixes. (These
relays hang stem's downloader. It's unlikely they'll ever help
clients.)
Use IP address, effective family, and contact info to
discover and limit fallbacks to one per operator.
Also analyse netblock, ports, IP version, and Exit flag,
and print the results. Don't exclude any fallbacks from
the list because of netblocks, ports, IP version, or
Exit flag.
But as advertised bandwidth is controlled by relays,
use consensus weight and median weight to bandwidth ratio
to approximate measured bandwidth.
Includes minor comment changes and parameter reordering.
Previously, we would cut the list down to 100 fallbacks,
then check if they could serve a consensus, and comment
them out if they couldn't.
This would leave us with fewer than 100 active fallbacks.
Now, we stop when there are 100 active fallbacks.
Also count fallbacks with identical contact info.
Also fix minor logging issues.
Give each fallback a set weight of 10.0 for client selection.
Fallbacks must have at least 3000 consensus weight.
This is (nominally) 100 times the expected extra load of
20 kilobytes per second (50 GB per month).
Fixes issue #17905.
Improve the download test:
* Allow IPv4 DirPort checks to be turned off.
* Add a timeout to stem's consensus download.
* Actually check for download errors, rather than ignoring them.
* Simplify the timeout and download error checking logic.
Tweak whitelist/blacklist checks to be more robust.
Improve logging, make it warn by default.
Cleanse fallback comments more thoroughly:
* non-printables (yes, ContactInfo can have these)
* // comments (don't rely on newlines to prevent // */ escapes)
