Merge branch 'debian-merge' into debian

* debian-merge: (95 commits)
  New upstream version: 0.2.2.2-alpha
  downgrade a log severity, since this event has been known
  Update to the "September 4 2009" ip-to-country file.
  bump to 0.2.2.2-alpha
  Revert "Teach connection_ap_can_use_exit about Exclude*Nodes"
  fix grammar / add changelog for the torify commit
  Fix compile on Snow Leopard
  Fix build warnings on OSX 10.5.8
  Change the condition on the nonlive timeout counting.
  Add a couple of time helper functions.
  Fix typos and comments, plus two bugs
  Implement and document new network liveness algorithm.
  Fix some precision-related asserts in unit tests.
  replace contrib/auto-naming with a readme saying where it went
  clarify our rules for assigning the Named flag
  disable the end of circuitbuildtimeout units tests
  draw in a lot of 0.2.1.20 changelog items into 0.2.2.2-alpha
  Fix compile on freebsd
  Let our config abbreviations rewrite more than once
  a mish-mash of stuff in my sandbox
  ...
This commit is contained in:
Peter Palfrader 2009-09-21 13:16:44 +02:00
commit b69f6fe82d
60 changed files with 12692 additions and 20732 deletions

154
ChangeLog
View File

@ -1,3 +1,72 @@
Changes in version 0.2.2.2-alpha - 2009-09-21
o Major features:
- Tor now tracks how long it takes to build client-side circuits
over time, and adapts its timeout to local network performance.
Since a circuit that takes a long time to build will also provide
bad performance, we get significant latency improvements by
discarding the slowest 20% of circuits. Specifically, Tor creates
circuits more aggressively than usual until it has enough data
points for a good timeout estimate. Implements proposal 151.
We are especially looking for reports (good and bad) from users with
both EDGE and broadband connections that can move from broadband
to EDGE and find out if the build-time data in the .tor/state gets
reset without loss of Tor usability. You should also see a notice
log message telling you that Tor has reset its timeout.
- Directory authorities can now vote on arbitary integer values as
part of the consensus process. This is designed to help set
network-wide parameters. Implements proposal 167.
- Tor now reads the "circwindow" parameter out of the consensus,
and uses that value for its circuit package window rather than the
default of 1000 cells. Begins the implementation of proposal 168.
o Major bugfixes:
- Fix a remotely triggerable memory leak when a consensus document
contains more than one signature from the same voter. Bugfix on
0.2.0.3-alpha.
o Minor bugfixes:
- Fix an extremely rare infinite recursion bug that could occur if
we tried to log a message after shutting down the log subsystem.
Found by Matt Edman. Bugfix on 0.2.0.16-alpha.
- Fix parsing for memory or time units given without a space between
the number and the unit. Bugfix on 0.2.2.1-alpha; fixes bug 1076.
- A networkstatus vote must contain exactly one signature. Spec
conformance issue. Bugfix on 0.2.0.3-alpha.
- Fix an obscure bug where hidden services on 64-bit big-endian
systems might mis-read the timestamp in v3 introduce cells, and
refuse to connect back to the client. Discovered by "rotor".
Bugfix on 0.2.1.6-alpha.
- We were triggering a CLOCK_SKEW controller status event whenever
we connect via the v2 connection protocol to any relay that has
a wrong clock. Instead, we should only inform the controller when
it's a trusted authority that claims our clock is wrong. Bugfix
on 0.2.0.20-rc; starts to fix bug 1074. Reported by SwissTorExit.
- We were telling the controller about CHECKING_REACHABILITY and
REACHABILITY_FAILED status events whenever we launch a testing
circuit or notice that one has failed. Instead, only tell the
controller when we want to inform the user of overall success or
overall failure. Bugfix on 0.1.2.6-alpha. Fixes bug 1075. Reported
by SwissTorExit.
- Don't warn when we're using a circuit that ends with a node
excluded in ExcludeExitNodes, but the circuit is not used to access
the outside world. This should help fix bug 1090, but more problems
remain. Bugfix on 0.2.1.6-alpha.
- Work around a small memory leak in some versions of OpenSSL that
stopped the memory used by the hostname TLS extension from being
freed.
- Make our 'torify' script more portable; if we have only one of
'torsocks' or 'tsocks' installed, don't complain to the user;
and explain our warning about tsocks better.
o Minor features:
- Add a "getinfo status/accepted-server-descriptor" controller
command, which is the recommended way for controllers to learn
whether our server descriptor has been successfully received by at
least on directory authority. Un-recommend good-server-descriptor
getinfo and status events until we have a better design for them.
- Update to the "September 4 2009" ip-to-country file.
Changes in version 0.2.2.1-alpha - 2009-08-26 Changes in version 0.2.2.1-alpha - 2009-08-26
o Security fixes: o Security fixes:
- Start the process of disabling ".exit" address notation, since it - Start the process of disabling ".exit" address notation, since it
@ -87,6 +156,22 @@ Changes in version 0.2.2.1-alpha - 2009-08-26
them, and they provided another avenue for detecting Tor users them, and they provided another avenue for detecting Tor users
via application-level web tricks. via application-level web tricks.
o Packaging changes:
- Upgrade Vidalia from 0.1.15 to 0.2.3 in the Windows and OS X
installer bundles. See
https://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.2.3/CHANGELOG
for details of what's new in Vidalia 0.2.3.
- Windows Vidalia Bundle: update Privoxy from 3.0.6 to 3.0.14-beta.
- OS X Vidalia Bundle: move to Polipo 1.0.4 with Tor specific
configuration file, rather than the old Privoxy.
- OS X Vidalia Bundle: Vidalia, Tor, and Polipo are compiled as
x86-only for better compatibility with OS X 10.6, aka Snow Leopard.
- OS X Tor Expert Bundle: Tor is compiled as x86-only for
better compatibility with OS X 10.6, aka Snow Leopard.
- OS X Vidalia Bundle: The multi-package installer is now replaced
by a simple drag and drop to the /Applications folder. This change
occurred with the upgrade to Vidalia 0.2.3.
Changes in version 0.2.1.20 - 2009-??-?? Changes in version 0.2.1.20 - 2009-??-??
o Major bugfixes: o Major bugfixes:
@ -96,6 +181,9 @@ Changes in version 0.2.1.20 - 2009-??-??
patch. Bugfix on the 54th commit on Tor -- from July 2002, patch. Bugfix on the 54th commit on Tor -- from July 2002,
before the release of Tor 0.0.0. This is the new winner of the before the release of Tor 0.0.0. This is the new winner of the
oldest-bug prize. oldest-bug prize.
- Fix a remotely triggerable memory leak when a consensus document
contains more than one signature from the same voter. Bugfix on
0.2.0.3-alpha.
o New directory authorities: o New directory authorities:
- Set up urras (run by Jacob Appelbaum) as the seventh v3 directory - Set up urras (run by Jacob Appelbaum) as the seventh v3 directory
@ -105,6 +193,41 @@ Changes in version 0.2.1.20 - 2009-??-??
- Fix a signed/unsigned compile warning in 0.2.1.19. - Fix a signed/unsigned compile warning in 0.2.1.19.
- Fix possible segmentation fault on directory authorities. Bugfix on - Fix possible segmentation fault on directory authorities. Bugfix on
0.2.1.14-rc. 0.2.1.14-rc.
- Fix an extremely rare infinite recursion bug that could occur if
we tried to log a message after shutting down the log subsystem.
Found by Matt Edman. Bugfix on 0.2.0.16-alpha.
- Fix an obscure bug where hidden services on 64-bit big-endian
systems might mis-read the timestamp in v3 introduce cells, and
refuse to connect back to the client. Discovered by "rotor".
Bugfix on 0.2.1.6-alpha.
- We were triggering a CLOCK_SKEW controller status event whenever
we connect via the v2 connection protocol to any relay that has
a wrong clock. Instead, we should only inform the controller when
it's a trusted authority that claims our clock is wrong. Bugfix
on 0.2.0.20-rc; starts to fix bug 1074. Reported by SwissTorExit.
- We were telling the controller about CHECKING_REACHABILITY and
REACHABILITY_FAILED status events whenever we launch a testing
circuit or notice that one has failed. Instead, only tell the
controller when we want to inform the user of overall success or
overall failure. Bugfix on 0.1.2.6-alpha. Fixes bug 1075. Reported
by SwissTorExit.
- Don't warn when we're using a circuit that ends with a node
excluded in ExcludeExitNodes, but the circuit is not used to access
the outside world. This should help fix bug 1090. Bugfix on
0.2.1.6-alpha.
- Avoid segfault in rare cases when finishing an introduction circuit
as a client and finding out that we don't have an introduction key
for it. Fixes bug 1073. Reported by Aaron Swartz.
- Work around a small memory leak in some versions of OpenSSL that
stopped the memory used by the hostname TLS extension from being
freed.
o Minor features:
- Add a "getinfo status/accepted-server-descriptor" controller
command, which is the recommended way for controllers to learn
whether our server descriptor has been successfully received by at
least on directory authority. Un-recommend good-server-descriptor
getinfo and status events until we have a better design for them.
Changes in version 0.2.1.19 - 2009-07-28 Changes in version 0.2.1.19 - 2009-07-28
@ -201,6 +324,37 @@ Changes in version 0.2.1.17-rc - 2009-07-07
further bugs for relays on dynamic IP addresses. further bugs for relays on dynamic IP addresses.
Changes in version 0.2.0.35 - 2009-06-24
o Security fix:
- Avoid crashing in the presence of certain malformed descriptors.
Found by lark, and by automated fuzzing.
- Fix an edge case where a malicious exit relay could convince a
controller that the client's DNS question resolves to an internal IP
address. Bug found and fixed by "optimist"; bugfix on 0.1.2.8-beta.
o Major bugfixes:
- Finally fix the bug where dynamic-IP relays disappear when their
IP address changes: directory mirrors were mistakenly telling
them their old address if they asked via begin_dir, so they
never got an accurate answer about their new address, so they
just vanished after a day. For belt-and-suspenders, relays that
don't set Address in their config now avoid using begin_dir for
all direct connections. Should fix bugs 827, 883, and 900.
- Fix a timing-dependent, allocator-dependent, DNS-related crash bug
that would occur on some exit nodes when DNS failures and timeouts
occurred in certain patterns. Fix for bug 957.
o Minor bugfixes:
- When starting with a cache over a few days old, do not leak
memory for the obsolete router descriptors in it. Bugfix on
0.2.0.33; fixes bug 672.
- Hidden service clients didn't use a cached service descriptor that
was older than 15 minutes, but wouldn't fetch a new one either,
because there was already one in the cache. Now, fetch a v2
descriptor unless the same descriptor was added to the cache within
the last 15 minutes. Fixes bug 997; reported by Marcus Griep.
Changes in version 0.2.1.16-rc - 2009-06-20 Changes in version 0.2.1.16-rc - 2009-06-20
Tor 0.2.1.16-rc speeds up performance for fast exit relays, and fixes Tor 0.2.1.16-rc speeds up performance for fast exit relays, and fixes
a bunch of minor bugs. a bunch of minor bugs.

View File

@ -4,7 +4,7 @@ dnl Copyright (c) 2007-2008, The Tor Project, Inc.
dnl See LICENSE for licensing information dnl See LICENSE for licensing information
AC_INIT AC_INIT
AM_INIT_AUTOMAKE(tor, 0.2.2.1-alpha) AM_INIT_AUTOMAKE(tor, 0.2.2.2-alpha)
AM_CONFIG_HEADER(orconfig.h) AM_CONFIG_HEADER(orconfig.h)
AC_CANONICAL_HOST AC_CANONICAL_HOST

View File

@ -1,65 +1,6 @@
=== AUTONAMING FOR TOR ===
Tor directory authorities may maintain a binding of server identities Tor directory authorities may maintain a binding of server identities
(their long term identity key) and nicknames. In their status documents (their long term identity key) and nicknames.
they may for each router they know tell if this is indeed the owner of
that nickname or not.
This toolset allows automatic maintaining of a binding list of nicknames The auto-naming scripts have been moved to svn in
to identity keys, implementing Tor proposal 123[1]. projects/tor-naming/auto-naming/trunk/
The rules are simple:
- A router claiming to be Bob is named (i.e. added to the binding list)
if there currently does not exist a different binding for that
nickname, the router has been around for a bit (2 weeks), no other
router has used that nickname in a while (1 month).
- A binding is removed if the server that owns it has not been seen
in a long time (6 months).
=== REQUIREMENTS ===
* ruby, and its postgres DBI interface (Debian packages: ruby, ruby1.8, libdbi-ruby1.8, libdbd-pg-ruby1.8)
* postgres (tested with >= 8.1)
* cron
=== SETUP ===
* copy this tree some place, like into a 'auto-naming' directory in your Tor's
data directory
* create a database and a user, modifying db-config.rb accordingly
* initialize the database by executing the sql statements in create-db.sql
* setup a cronjob that feeds the current consensus to the process-consensus
script regularly.
* once the database is sufficiently populated, maybe a month or so after the
previous step, setup a cronjob to regularly build the binding list using
the build-approved-routers script. You probably want to append a manually
managed list of rejections to that file and give it to tor as its
"approved-routers" file.
The Sample-Makefile and Sample-crontab demonstrate the method used at tor26.
1. https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/proposals/123-autonaming.txt
Copyright (c) 2007 Peter Palfrader
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -1,20 +0,0 @@
all: ../approved-routers
update:
wget -q -O - http://tor.noreply.org/tor/status-vote/current/consensus | \
./process-consensus
.PHONY: approved-routers-auto
approved-routers-auto:
./build-approved-routers > "$@"
.INTERMEDIATE: approved-routers
approved-routers: approved-routers-auto /etc/tor/approved-routers
cat $^ > "$@"
../approved-routers: approved-routers
if ! diff -q "$<" "$@"; then \
mv "$<" "$@" &&\
(! [ -e /var/run/tor/tor.pid ] || kill -HUP `cat /var/run/tor/tor.pid`) ; \
fi

View File

@ -1,3 +0,0 @@
MAILTO=admin
# cronjob for tor naming
23 * * * * make -s -C auto-naming update && make -s -C auto-naming

View File

@ -1,45 +0,0 @@
#!/usr/bin/ruby
# build-approved-routers - create a name-binding list for use at a Tor
# directory authority
#
# Copyright (c) 2007 Peter Palfrader
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
require "yaml"
require 'db'
require 'db-config'
verbose = ARGV.first == "-v"
db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password'])
db.transaction_begin
named = db.query2("
SELECT fingerprint, router_id, nickname_id, nick, first_seen, last_seen
FROM router NATURAL JOIN router_claims_nickname NATURAL JOIN nickname
WHERE named")
while (n=named.next) do
puts "# (r##{n['router_id']},n##{n['nickname_id']}); first_seen: #{n['first_seen']}, last_seen: #{n['last_seen']}"
fpr = n['fingerprint'].split(/(....)/).delete_if{|x| x=="" }.join(' ')
puts "#{n['nick']} #{fpr}"
end
db.transaction_commit

View File

@ -1,50 +0,0 @@
CREATE TABLE router (
router_id SERIAL PRIMARY KEY,
fingerprint CHAR(40) NOT NULL,
UNIQUE(fingerprint)
);
-- already created implicitly due to unique contraint
-- CREATE INDEX router_fingerprint ON router(fingerprint);
CREATE TABLE nickname (
nickname_id SERIAL PRIMARY KEY,
nick VARCHAR(30) NOT NULL,
UNIQUE(nick)
);
-- already created implicitly due to unique contraint
-- CREATE INDEX nickname_nick ON nickname(nick);
CREATE TABLE router_claims_nickname (
router_id INTEGER NOT NULL REFERENCES router(router_id) ON DELETE CASCADE,
nickname_id INTEGER NOT NULL REFERENCES nickname(nickname_id) ON DELETE CASCADE,
first_seen TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
named BOOLEAN NOT NULL DEFAULT 'false',
UNIQUE(router_id, nickname_id)
);
CREATE INDEX router_claims_nickname_router_id ON router_claims_nickname(router_id);
CREATE INDEX router_claims_nickname_nickname_id ON router_claims_nickname(nickname_id);
CREATE INDEX router_claims_nickname_first_seen ON router_claims_nickname(first_seen);
CREATE INDEX router_claims_nickname_last_seen ON router_claims_nickname(last_seen);
-- Copyright (c) 2007 Peter Palfrader
--
-- Permission is hereby granted, free of charge, to any person obtaining a copy
-- of this software and associated documentation files (the "Software"), to deal
-- in the Software without restriction, including without limitation the rights
-- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-- copies of the Software, and to permit persons to whom the Software is
-- furnished to do so, subject to the following conditions:
--
-- The above copyright notice and this permission notice shall be included in
-- all copies or substantial portions of the Software.
--
-- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-- SOFTWARE.

View File

@ -1,8 +0,0 @@
$CONFIG = {} unless $CONFIG
$CONFIG['database'] = {} unless $CONFIG['database']
# if you use postgres' "ident sameuser" auth set dbhost to ''
$CONFIG['database']['dbhost'] = 'localhost';
$CONFIG['database']['dbname'] = 'tornaming';
$CONFIG['database']['user'] = 'tornaming';
$CONFIG['database']['password'] = 'x';

View File

@ -1,165 +0,0 @@
#!/usr/bin/ruby
# Copyright (c) 2006, 2007 Peter Palfrader
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
require "dbi"
class WeaselDbQueryHandle
def initialize(sth)
@sth = sth
end
def next()
row = @sth.fetch_hash
if row
return row
else
@sth.finish
return nil
end
end
end
class Db
def initialize(host, database, user, password)
@dbh = DBI.connect("dbi:Pg:#{database}:#{host}", user, password);
@dbh['AutoCommit'] = false
@transaction = false
@pre_initial_transaction=true
end
def do(query,*args)
@dbh.do(query,*args)
end
def transaction_begin()
@dbh.do("BEGIN") unless @pre_initial_transaction
@transaction = true
@pre_initial_transaction=false
end
def transaction_commit()
@dbh.do("COMMIT")
@transaction = false
end
def transaction_rollback()
@dbh.do("ROLLBACK")
end
def get_primarykey_name(table);
#return 'ref';
return table+'_id';
end
def update(table, values, keys)
cols = []
vals = []
values.each_pair{ |k,v|
cols << "#{k}=?"
vals << v
}
wheres = []
keys.each_pair{ |k,v|
wheres << "#{k}=?"
vals << v
}
throw "update value set empty" unless cols.size > 0
throw "where clause empty" unless wheres.size > 0
query = "UPDATE #{table} SET #{cols.join(',')} WHERE #{wheres.join(' AND ')}"
transaction_begin unless transaction_before=@transaction
r = @dbh.do(query, *vals)
transaction_commit unless transaction_before
return r
end
def update_row(table, values)
pk_name = get_primarykey_name(table);
throw "Ref not defined" unless values[pk_name]
return update(table, values.clone.delete_if{|k,v| k == pk_name}, { pk_name => values[pk_name] });
end
def insert(table, values)
cols = values.keys
vals = values.values
qmarks = values.values.collect{ '?' }
query = "INSERT INTO #{table} (#{cols.join(',')}) VALUES (#{qmarks.join(',')})"
transaction_begin unless transaction_before=@transaction
@dbh.do(query, *vals)
transaction_commit unless transaction_before
end
def insert_row(table, values)
pk_name = get_primarykey_name(table);
if values[pk_name]
insert(table, values)
else
transaction_begin unless transaction_before=@transaction
row = query_row("SELECT nextval(pg_get_serial_sequence('#{table}', '#{pk_name}')) AS newref");
throw "No newref?" unless row['newref']
values[pk_name] = row['newref']
insert(table, values);
transaction_commit unless transaction_before
end
end
def delete_row(table, ref)
pk_name = get_primarykey_name(table);
query = "DELETE FROM #{table} WHERE #{pk_name}=?"
transaction_begin unless transaction_before=@transaction
@dbh.do(query, ref)
transaction_commit unless transaction_before
end
def query(query, *params)
sth = @dbh.execute(query, *params)
while row = sth.fetch_hash
yield row
end
sth.finish
end
# nil if no results
# hash if one match
# throw otherwise
def query_row(query, *params)
sth = @dbh.execute(query, *params)
row = sth.fetch_hash
if row == nil
sth.finish
return nil
elsif sth.fetch_hash != nil
sth.finish
throw "More than one result when querying for #{query}"
else
sth.finish
return row
end
end
def query_all(query, *params)
sth = @dbh.execute(query, *params)
rows = sth.fetch_all
return nil if rows.size == 0
return rows
end
def query2(query, *params)
sth = @dbh.execute(query, *params)
return WeaselDbQueryHandle.new(sth)
end
end

View File

@ -1,119 +0,0 @@
#!/usr/bin/ruby
# process-consensus - read a current consensus document, inserting the
# information into a database then calling
# update-named-status.rb to update the name-binding
# flags
#
# Copyright (c) 2007 Peter Palfrader
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
require "yaml"
require 'db'
require 'db-config'
require 'update-named-status'
$db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password'])
$router_cache = {}
$nickname_cache = {}
def parse_consensus consensus
ts = nil
routers = []
consensus.each do |line|
(key, value) = line.split(' ',2)
case key
when "valid-after", "published": ts = DateTime.parse(value)
when "r":
(nick, fpr, _) = value.split(' ', 3)
nick.downcase!
next if nick == 'unnamed'
routers << {
'nick' => nick,
'fingerprint' => (fpr+'=').unpack('m').first.unpack('H*').first
}
end
end
throw "Did not find a timestamp" unless ts
throw "Did not find any routers" unless routers.size > 0
return ts, routers
end
def insert_routers_into_db(router, table, field, value)
pk = table+'_id'
row = $db.query_row("SELECT #{pk} FROM #{table} WHERE #{field}=?", value)
if row
return row[pk]
else
r = { field => value }
$db.insert_row( table, r )
return r[pk]
end
end
def handle_one_consensus(c)
puts "parsing..." if $verbose
timestamp, routers = parse_consensus c
puts "storing..." if $verbose
routers.each do |router|
fpr = router['fingerprint']
nick = router['nick']
$router_cache[fpr] = router_id = ($router_cache[fpr] or insert_routers_into_db(router, 'router', 'fingerprint', router['fingerprint']))
$nickname_cache[nick] = nickname_id = ($nickname_cache[nick] or insert_routers_into_db(router, 'nickname', 'nick', router['nick']))
row = $db.update(
'router_claims_nickname',
{ 'last_seen' => timestamp.to_s },
{ 'router_id' => router_id, 'nickname_id' => nickname_id} )
case row
when 0:
$db.insert('router_claims_nickname',
{
'first_seen' => timestamp.to_s,
'last_seen' => timestamp.to_s,
'router_id' => router_id, 'nickname_id' => nickname_id} )
when 1:
else
throw "Update of router_claims_nickname returned unexpected number of affected rows(#{row})"
end
end
end
$db.transaction_begin
if ARGV.first == '-v'
$verbose = true
ARGV.shift
end
if ARGV.size == 0
handle_one_consensus STDIN.readlines
do_update $verbose
else
ARGV.each do |filename|
puts filename if $verbose
handle_one_consensus File.new(filename).readlines
puts "updating..." if $verbose
do_update $verbose
end
end
$db.transaction_commit

View File

@ -1,70 +0,0 @@
#!/usr/bin/ruby
# update-named-status.rb - update the named status of routers in our database
#
# Copyright (c) 2007 Peter Palfrader
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
require "yaml"
require 'db'
require 'db-config'
def do_update(verbose)
now = $db.query_row("SELECT max(last_seen) AS max FROM router_claims_nickname")['max']
unless now
STDERR.puts "Could not find the latest last_seen timestamp. Is the database empty still?"
return
end
now = "TIMESTAMP '" + now.to_s + "'"
denamed = $db.do("
UPDATE router_claims_nickname
SET named=false
WHERE named
AND last_seen < #{now} - INTERVAL '6 months'")
puts "de-named: #{denamed}" if verbose
named = $db.do("
UPDATE router_claims_nickname
SET named=true
WHERE NOT named
AND first_seen < #{now} - INTERVAL '2 weeks'
AND last_seen > #{now} - INTERVAL '2 days'
AND NOT EXISTS (SELECT *
FROM router_claims_nickname AS innertable
WHERE named
AND router_claims_nickname.nickname_id=innertable.nickname_id) "+ # if that nickname is already named, we lose.
" AND NOT EXISTS (SELECT *
FROM router_claims_nickname AS innertable
WHERE router_claims_nickname.nickname_id=innertable.nickname_id
AND router_claims_nickname.router_id <> innertable.router_id
AND last_seen > #{now} - INTERVAL '1 month') ") # if nobody else wanted that nickname in the last month we are set
puts "named: #{named}" if verbose
end
if __FILE__ == $0
$db = Db.new($CONFIG['database']['dbhost'], $CONFIG['database']['dbname'], $CONFIG['database']['user'], $CONFIG['database']['password'])
verbose = ARGV.first == "-v"
$db.transaction_begin
do_update verbose
$db.transaction_commit
end

View File

@ -9,7 +9,7 @@
!include "FileFunc.nsh" !include "FileFunc.nsh"
!insertmacro GetParameters !insertmacro GetParameters
!define VERSION "0.2.2.1-alpha" !define VERSION "0.2.2.2-alpha"
!define INSTALLER "tor-${VERSION}-win32.exe" !define INSTALLER "tor-${VERSION}-win32.exe"
!define WEBSITE "https://www.torproject.org/" !define WEBSITE "https://www.torproject.org/"
!define LICENSE "LICENSE" !define LICENSE "LICENSE"

View File

@ -3,57 +3,52 @@
# Wrapper script for use of the tsocks(8) transparent socksification library # Wrapper script for use of the tsocks(8) transparent socksification library
# See the tsocks(1) and torify(1) manpages. # See the tsocks(1) and torify(1) manpages.
# Copyright (c) 2004, 2006 Peter Palfrader # Copyright (c) 2004, 2006, 2009 Peter Palfrader
# Modified by Jacob Appelbaum <jacob@appelbaum.net> April 16th 2006 # Modified by Jacob Appelbaum <jacob@appelbaum.net> April 16th 2006
# May be distributed under the same terms as Tor itself # May be distributed under the same terms as Tor itself
# taken from Debian's Developer's Reference, 6.4
# Define and ensure we have tsocks pathfind() {
# XXX: what if we don't have which? OLDIFS="$IFS"
TORSOCKS="`which torsocks`" IFS=:
TSOCKS="`which tsocks`" for p in $PATH; do
PROG="" if [ -x "$p/$*" ]; then
if [ ! -x "$TSOCKS" ] IFS="$OLDIFS"
then return 0
echo "$0: Can't find tsocks in PATH. Perhaps you haven't installed it?" >&2 fi
else done
PROG=$TSOCKS IFS="$OLDIFS"
fi return 1
if [ ! -x "$TORSOCKS" ] }
then
echo "$0: Can't find torsocks in PATH. Perhaps you haven't installed it?" >&2
else
PROG=$TORSOCKS
fi
if [ ! -x "$PROG" ]
then
echo "$0: Can't find the required tor helpers in our PATH. Perhaps you haven't installed them?" >&2
exit 1;
fi
# Check for any argument list # Check for any argument list
if [ "$#" = 0 ] if [ "$#" = 0 ]; then
then
echo "Usage: $0 [-hv] <command> [<options>...]" >&2 echo "Usage: $0 [-hv] <command> [<options>...]" >&2
exit 1 exit 1
fi fi
if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] )
then if [ "$#" = 1 ] && ( [ "$1" = "-h" ] || [ "$1" = "--help" ] ); then
echo "Usage: $0 [-hv] <command> [<options>...]" echo "Usage: $0 [-hv] <command> [<options>...]"
exit 0 exit 0
fi fi
if [ "$1" = "-v" ] || [ "$1" = "--verbose" ] if [ "$1" = "-v" ] || [ "$1" = "--verbose" ]; then
then verbose=1
echo "We're armed with the following tsocks: $TSOCKS"
echo "We're armed with the following torsocks: $TORSOCKS"
echo "We're attempting to use $PROG for all tor action."
shift 1 shift 1
else
verbose=0
fi fi
if [ "$PROG" = "$TSOCKS" ] if pathfind torsocks; then
then ! [ "$verbose" -ge 1 ] || echo "Using torsocks as socksifier." >&2
exec torsocks "$@"
echo "$0: Failed to exec torsocks $@" >&2
exit 1
elif pathfind tsocks; then
! [ "$verbose" -ge 1 ] || echo "Using tsocks as socksifier." >&2
# Define our tsocks config file # Define our tsocks config file
TSOCKS_CONF_FILE="/etc/tor/tor-tsocks.conf" TSOCKS_CONF_FILE="/etc/tor/tor-tsocks.conf"
export TSOCKS_CONF_FILE export TSOCKS_CONF_FILE
@ -61,7 +56,7 @@ then
# Check that we've got a tsocks config file # Check that we've got a tsocks config file
if [ -r "$TSOCKS_CONF_FILE" ] if [ -r "$TSOCKS_CONF_FILE" ]
then then
echo "WARNING: tsocks is known to leak DNS and UDP data." >&2 echo "WARNING: tsocks is known to leak DNS and UDP data. If you had torsocks we would use that." >&2
exec tsocks "$@" exec tsocks "$@"
echo "$0: Failed to exec tsocks $@" >&2 echo "$0: Failed to exec tsocks $@" >&2
exit 1 exit 1
@ -69,8 +64,8 @@ then
echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2 echo "$0: Missing tsocks configuration file \"$TSOCKS_CONF_FILE\"." >&2
exit 1 exit 1
fi fi
fi
if [ "$PROG" = "$TORSOCKS" ] else
then echo "$0: Can't find either tsocks or torsocks in your PATH. Perhaps you haven't installed either?" >&2
exec torsocks "$@" exit 1
fi fi

6
debian/changelog vendored
View File

@ -1,3 +1,9 @@
tor (0.2.2.2-alpha-1) experimental; urgency=low
* New upstream version.
-- Peter Palfrader <weasel@debian.org> Mon, 21 Sep 2009 13:15:36 +0200
tor (0.2.2.1-alpha-1) experimental; urgency=low tor (0.2.2.1-alpha-1) experimental; urgency=low
* New upstream version. * New upstream version.

View File

@ -3,10 +3,7 @@
0.0 The buildbot. 0.0 The buildbot.
http://tor-buildbot.freehaven.net:8010/ https://buildbot.vidalia-project.net/one_line_per_build
- Down because nickm isn't running services at home any more. ioerror says
he will resurrect it.
0.1. Useful command-lines that are non-trivial to reproduce but can 0.1. Useful command-lines that are non-trivial to reproduce but can
help with tracking bugs or leaks. help with tracking bugs or leaks.

View File

@ -557,6 +557,7 @@
"status/circuit-established" "status/circuit-established"
"status/enough-dir-info" "status/enough-dir-info"
"status/good-server-descriptor" "status/good-server-descriptor"
"status/accepted-server-descriptor"
"status/..." "status/..."
These provide the current internal Tor values for various Tor These provide the current internal Tor values for various Tor
states. See Section 4.1.10 for explanations. (Only a few of the states. See Section 4.1.10 for explanations. (Only a few of the
@ -1254,20 +1255,26 @@
CLOCK_SKEW CLOCK_SKEW
SKEW="+" / "-" SECONDS SKEW="+" / "-" SECONDS
MIN_SKEW="+" / "-" SECONDS. MIN_SKEW="+" / "-" SECONDS.
SOURCE="DIRSERV:IP:Port" / "NETWORKSTATUS:IP:PORT" / "CONSENSUS" SOURCE="DIRSERV:" IP ":" Port /
"NETWORKSTATUS:" IP ":" Port /
"OR:" IP ":" Port /
"CONSENSUS"
If "SKEW" is present, it's an estimate of how far we are from the If "SKEW" is present, it's an estimate of how far we are from the
time declared in the source. (In other words, if we're an hour in time declared in the source. (In other words, if we're an hour in
the past, the value is -3600.) "MIN_SKEW" is present, it's a lower the past, the value is -3600.) "MIN_SKEW" is present, it's a lower
bound. If the source is a DIRSERV, we got the current time from a bound. If the source is a DIRSERV, we got the current time from a
connection to a dirserver. If the source is a NETWORKSTATUS, we connection to a dirserver. If the source is a NETWORKSTATUS, we
decided we're skewed because we got a v2 networkstatus from far in decided we're skewed because we got a v2 networkstatus from far in
the future. If the source is CONSENSUS, we decided we're skewed the future. If the source is OR, the skew comes from a NETINFO
because we got a networkstatus consensus from the future. cell from a connection to another relay. If the source is
CONSENSUS, we decided we're skewed because we got a networkstatus
consensus from the future.
{Controllers may want to warn the user if the skew is high, or if {Tor should send this message to controllers when it thinks the
multiple skew messages appear at severity WARN. Controllers skew is so high that it will interfere with proper Tor operation.
shouldn't blindly adjust the clock, since the more accurate source Controllers shouldn't blindly adjust the clock, since the more
of skew info (DIRSERV) is currently unauthenticated.} accurate source of skew info (DIRSERV) is currently
unauthenticated.}
BAD_LIBEVENT BAD_LIBEVENT
"METHOD=" libevent method "METHOD=" libevent method
@ -1481,18 +1488,39 @@
We successfully uploaded our server descriptor to at least one We successfully uploaded our server descriptor to at least one
of the directory authorities, with no complaints. of the directory authorities, with no complaints.
{This event could affect the controller's idea of server status, but {Originally, the goal of this event was to declare "every authority
the controller should not interrupt the user to tell them so.} has accepted the descriptor, so there will be no complaints
about it." But since some authorities might be offline, it's
harder to get certainty than we had thought. As such, this event
is equivalent to ACCEPTED_SERVER_DESCRIPTOR below. Controllers
should just look at ACCEPTED_SERVER_DESCRIPTOR and should ignore
this event for now.}
SERVER_DESCRIPTOR_STATUS
"STATUS=" "LISTED" / "UNLISTED"
We just got a new networkstatus consensus, and whether we're in
it or not in it has changed. Specifically, status is "listed"
if we're listed in it but previous to this point we didn't know
we were listed in a consensus; and status is "unlisted" if we
thought we should have been listed in it (e.g. we were listed in
the last one), but we're not.
{Moving from listed to unlisted is not necessarily cause for
alarm. The relay might have failed a few reachability tests,
or the Internet might have had some routing problems. So this
feature is mainly to let relay operators know when their relay
has successfully been listed in the consensus.}
[Not implemented yet. We should do this in 0.2.2.x. -RD]
NAMESERVER_STATUS NAMESERVER_STATUS
"NS=addr" "NS=addr"
"STATUS=" "UP" / "DOWN" "STATUS=" "UP" / "DOWN"
"ERR=" message "ERR=" message
One of our nameservers has changed status. One of our nameservers has changed status.
// actually notice
{This event could affect the controller's idea of server status, but {This event could affect the controller's idea of server status, but
the controller should not interrupt the user to tell them so.} the controller should not interrupt the user to tell them so.}
NAMESERVER_ALL_DOWN NAMESERVER_ALL_DOWN
All of our nameservers have gone down. All of our nameservers have gone down.

View File

@ -1098,6 +1098,20 @@
enough votes were counted for the consensus for an authoritative enough votes were counted for the consensus for an authoritative
opinion to have been formed about their status. opinion to have been formed about their status.
"params" SP [Parameters] NL
[At most once]
Parameter ::= Keyword '=' Int32
Int32 ::= A decimal integer between -2147483648 and 2147483647.
Parameters ::= Parameter | Parameters SP Parameter
The parameters list, if present, contains a space-separated list of
key-value pairs, sorted in lexical order by their keyword. Each
parameter has its own meaning.
(Only included when the vote is generated with consensus-method 7 or
later.)
The authority section of a vote contains the following items, followed The authority section of a vote contains the following items, followed
in turn by the authority's current key certificate: in turn by the authority's current key certificate:
@ -1406,6 +1420,10 @@
Known-flags is the union of all flags known by any voter. Known-flags is the union of all flags known by any voter.
Entries are given on the "params" line for every keyword on which any
authority voted. The values given are the low-median of all votes on
that keyword.
"client-versions" and "server-versions" are sorted in ascending "client-versions" and "server-versions" are sorted in ascending
order; A version is recommended in the consensus if it is recommended order; A version is recommended in the consensus if it is recommended
by more than half of the voting authorities that included a by more than half of the voting authorities that included a
@ -1473,6 +1491,9 @@
a router, the authorities produce a consensus containing a a router, the authorities produce a consensus containing a
Bandwidth= keyword equal to the median of the Measured= votes. Bandwidth= keyword equal to the median of the Measured= votes.
* If consensus-method 7 or later is in use, the params line is
included in the output.
The signatures at the end of a consensus document are sorted in The signatures at the end of a consensus document are sorted in
ascending order by identity digest. ascending order by identity digest.

View File

@ -87,6 +87,8 @@ Proposals by number:
164 Reporting the status of server votes [OPEN] 164 Reporting the status of server votes [OPEN]
165 Easy migration for voting authority sets [OPEN] 165 Easy migration for voting authority sets [OPEN]
166 Including Network Statistics in Extra-Info Documents [ACCEPTED] 166 Including Network Statistics in Extra-Info Documents [ACCEPTED]
167 Vote on network parameters in consensus [CLOSED]
168 Reduce default circuit window [OPEN]
Proposals by status: Proposals by status:
@ -114,6 +116,7 @@ Proposals by status:
163 Detecting whether a connection comes from a client [for 0.2.2] 163 Detecting whether a connection comes from a client [for 0.2.2]
164 Reporting the status of server votes [for 0.2.2] 164 Reporting the status of server votes [for 0.2.2]
165 Easy migration for voting authority sets 165 Easy migration for voting authority sets
168 Reduce default circuit window [for 0.2.2]
ACCEPTED: ACCEPTED:
110 Avoiding infinite length circuits [for 0.2.1.x] [in 0.2.1.3-alpha] 110 Avoiding infinite length circuits [for 0.2.1.x] [in 0.2.1.3-alpha]
117 IPv6 exits [for 0.2.1.x] 117 IPv6 exits [for 0.2.1.x]
@ -157,6 +160,7 @@ Proposals by status:
148 Stream end reasons from the client side should be uniform [in 0.2.1.9-alpha] 148 Stream end reasons from the client side should be uniform [in 0.2.1.9-alpha]
150 Exclude Exit Nodes from a circuit [in 0.2.1.3-alpha] 150 Exclude Exit Nodes from a circuit [in 0.2.1.3-alpha]
152 Optionally allow exit from single-hop circuits [in 0.2.1.6-alpha] 152 Optionally allow exit from single-hop circuits [in 0.2.1.6-alpha]
167 Vote on network parameters in consensus [in 0.2.2]
SUPERSEDED: SUPERSEDED:
112 Bring Back Pathlen Coin Weight 112 Bring Back Pathlen Coin Weight
113 Simplifying directory authority administration 113 Simplifying directory authority administration

View File

@ -2,13 +2,13 @@ Filename: 151-path-selection-improvements.txt
Title: Improving Tor Path Selection Title: Improving Tor Path Selection
Author: Fallon Chen, Mike Perry Author: Fallon Chen, Mike Perry
Created: 5-Jul-2008 Created: 5-Jul-2008
Status: Draft Status: Implemented
Overview Overview
The performance of paths selected can be improved by adjusting the The performance of paths selected can be improved by adjusting the
CircuitBuildTimeout and avoiding failing guard nodes. This proposal CircuitBuildTimeout and avoiding failing guard nodes. This proposal
describes a method of tracking buildtime statistics at the client, and describes a method of tracking buildtime statistics at the client, and
using those statistics to adjust the CircuitBuildTimeout. using those statistics to adjust the CircuitBuildTimeout.
Motivation Motivation
@ -20,121 +20,123 @@ Motivation
Implementation Implementation
Storing Build Times Gathering Build Times
Circuit build times will be stored in the circular array Circuit build times are stored in the circular array
'circuit_build_times' consisting of uint16_t elements as milliseconds. 'circuit_build_times' consisting of uint32_t elements as milliseconds.
The total size of this array will be based on the number of circuits The total size of this array is based on the number of circuits
it takes to converge on a good fit of the long term distribution of it takes to converge on a good fit of the long term distribution of
the circuit builds for a fixed link. We do not want this value to be the circuit builds for a fixed link. We do not want this value to be
too large, because it will make it difficult for clients to adapt to too large, because it will make it difficult for clients to adapt to
moving between different links. moving between different links.
From our initial observations, this value appears to be on the order From our observations, the minimum value for a reasonable fit appears
of 1000, but will be configurable in a #define NCIRCUITS_TO_OBSERVE. to be on the order of 500 (MIN_CIRCUITS_TO_OBSERVE). However, to keep
The exact value for this #define will be determined by performing a good fit over the long term, we store 5000 most recent circuits in
goodness of fit tests using measurments obtained from the shufflebt.py the array (NCIRCUITS_TO_OBSERVE).
script from TorFlow.
The Tor client will build test circuits at a rate of one per
minute (BUILD_TIMES_TEST_FREQUENCY) up to the point of
MIN_CIRCUITS_TO_OBSERVE. This allows a fresh Tor to have
a CircuitBuildTimeout estimated within 8 hours after install,
upgrade, or network change (see below).
Long Term Storage Long Term Storage
The long-term storage representation will be implemented by storing a The long-term storage representation is implemented by storing a
histogram with BUILDTIME_BIN_WIDTH millisecond buckets (default 50) when histogram with BUILDTIME_BIN_WIDTH millisecond buckets (default 50) when
writing out the statistics to disk. The format of this histogram on disk writing out the statistics to disk. The format this takes in the
is yet to be finalized, but it will likely be of the format state file is 'CircuitBuildTime <bin-ms> <count>', with the total
'CircuitBuildTime <bin> <count>', with the total specified as specified as 'TotalBuildTimes <total>'
'TotalBuildTimes <total>'
Example: Example:
TotalBuildTimes 100 TotalBuildTimes 100
CircuitBuildTimeBin 1 50 CircuitBuildTimeBin 25 50
CircuitBuildTimeBin 2 25 CircuitBuildTimeBin 75 25
CircuitBuildTimeBin 3 13 CircuitBuildTimeBin 125 13
... ...
Reading the histogram in will entail multiplying each bin by the Reading the histogram in will entail inserting <count> values
BUILDTIME_BIN_WIDTH and then inserting <count> values into the into the circuit_build_times array each with the value of
circuit_build_times array each with the value of <bin-ms> milliseconds. In order to evenly distribute the values
<bin>*BUILDTIME_BIN_WIDTH. In order to evenly distribute the in the circular array, the Fisher-Yates shuffle will be performed
values in the circular array, a form of index skipping must after reading values from the bins.
be employed. Values from bin #N with bin count C and total T
will occupy indexes specified by N+((T/C)*k)-1, where k is the
set of integers ranging from 0 to C-1.
For example, this would mean that the values from bin 1 would
occupy indexes 1+(100/50)*k-1, or 0, 2, 4, 6, 8, 10 and so on.
The values for bin 2 would occupy positions 1, 5, 9, 13. Collisions
will be inserted at the first empty position in the array greater
than the selected index (which may requiring looping around the
array back to index 0).
Learning the CircuitBuildTimeout Learning the CircuitBuildTimeout
Based on studies of build times, we found that the distribution of Based on studies of build times, we found that the distribution of
circuit buildtimes appears to be a Pareto distribution. circuit buildtimes appears to be a Frechet distribution. However,
estimators and quantile functions of the Frechet distribution are
difficult to work with and slow to converge. So instead, since we
are only interested in the accuracy of the tail, we approximate
the tail of the distribution with a Pareto curve starting at
the mode of the circuit build time sample set.
We will calculate the parameters for a Pareto distribution We will calculate the parameters for a Pareto distribution
fitting the data using the estimators at fitting the data using the estimators at
http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation. http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation.
The timeout itself will be calculated by solving the CDF for the The timeout itself is calculated by using the Quartile function (the
a percentile cutoff BUILDTIME_PERCENT_CUTOFF. This value inverted CDF) to give us the value on the CDF such that
represents the percentage of paths the Tor client will accept out of BUILDTIME_PERCENT_CUTOFF (80%) of the mass of the distribution is
the total number of paths. We have not yet determined a good below the timeout value.
cutoff for this mathematically, but 85% seems a good choice for now.
From http://en.wikipedia.org/wiki/Pareto_distribution#Definition, Thus, we expect that the Tor client will accept the fastest 80% of
the calculation we need is pow(BUILDTIME_PERCENT_CUTOFF/100.0, k)/Xm. the total number of paths on the network.
Detecting Changing Network Conditions
We attempt to detect both network connectivity loss and drastic
changes in the timeout characteristics.
We assume that we've had network connectivity loss if 3 circuits
timeout and we've received no cells or TLS handshakes since those
circuits began. We then set the timeout to 60 seconds and stop
counting timeouts.
If 3 more circuits timeout and the network still has not been
live within this new 60 second timeout window, we then discard
the previous timeouts during this period from our history.
To detect changing network conditions, we keep a history of
the timeout or non-timeout status of the past RECENT_CIRCUITS (20)
that successfully completed at least one hop. If more than 75%
of these circuits timeout, we discard all buildtimes history,
reset the timeout to 60, and then begin recomputing the timeout.
Testing Testing
After circuit build times, storage, and learning are implemented, After circuit build times, storage, and learning are implemented,
the resulting histogram should be checked for consistency by the resulting histogram should be checked for consistency by
verifying it persists across successive Tor invocations where verifying it persists across successive Tor invocations where
no circuits are built. In addition, we can also use the existing no circuits are built. In addition, we can also use the existing
buildtime scripts to record build times, and verify that the histogram buildtime scripts to record build times, and verify that the histogram
the python produces matches that which is output to the state file in Tor, the python produces matches that which is output to the state file in Tor,
and verify that the Pareto parameters and cutoff points also match. and verify that the Pareto parameters and cutoff points also match.
Soft timeout vs Hard Timeout
At some point, it may be desirable to change the cutoff from a
single hard cutoff that destroys the circuit to a soft cutoff and
a hard cutoff, where the soft cutoff merely triggers the building
of a new circuit, and the hard cutoff triggers destruction of the
circuit.
Good values for hard and soft cutoffs seem to be 85% and 65% We will also verify that there are no unexpected large deviations from
respectively, but we should eventually justify this with observation. node selection, such as nodes from distant geographical locations being
completely excluded.
When to Begin Calculation
The number of circuits to observe (NCIRCUITS_TO_CUTOFF) before
changing the CircuitBuildTimeout will be tunable via a #define. From
our measurements, a good value for NCIRCUITS_TO_CUTOFF appears to be
on the order of 100.
Dealing with Timeouts Dealing with Timeouts
Timeouts should be counted as the expectation of the region of Timeouts should be counted as the expectation of the region of
of the Pareto distribution beyond the cutoff. The proposal will of the Pareto distribution beyond the cutoff. This is done by
be updated with this value soon. generating a random sample for each timeout at points on the
curve beyond the current timeout cutoff.
Also, in the event of network failure, the observation mechanism Future Work
should stop collecting timeout data.
Client Hints At some point, it may be desirable to change the cutoff from a
single hard cutoff that destroys the circuit to a soft cutoff and
a hard cutoff, where the soft cutoff merely triggers the building
of a new circuit, and the hard cutoff triggers destruction of the
circuit.
Some research still needs to be done to provide initial values It may also be beneficial to learn separate timeouts for each
for CircuitBuildTimeout based on values learned from modem guard node, as they will have slightly different distributions.
users, DSL users, Cable Modem users, and dedicated links. A This will take longer to generate initial values though.
radiobutton in Vidalia should eventually be provided that
sets CircuitBuildTimeout to one of these values and also
provide the option of purging all learned data, should any exist.
These values can either be published in the directory, or
shipped hardcoded for a particular Tor version.
Issues Issues
Impact on anonymity Impact on anonymity

View File

@ -2,8 +2,8 @@ Filename: 167-params-in-consensus.txt
Title: Vote on network parameters in consensus Title: Vote on network parameters in consensus
Author: Roger Dingledine Author: Roger Dingledine
Created: 18-Aug-2009 Created: 18-Aug-2009
Status: Open Status: Closed
Target: 0.2.2 Implemented-In: 0.2.2
0. History 0. History

View File

@ -1,24 +1,103 @@
## Instructions for building the official dmgs for OSX. ## Instructions for building the official dmgs for OSX.
## ##
## The loose table of contents:
## Summary
## Single Architecture Binaries for PPC or X86, not both.
## Backwards compatible single-architecture binaries for OSX x86 10.4 from newer versions of OS X.
## Universal Binaries for OSX PPC and X86
## Each section is delineated by ###.
The following steps are the exact steps used to produce the "official" The following steps are the exact steps used to produce the "official"
OSX builds of tor. OSX builds of tor.
Summary: ### Summary:
1) Compile and install a static version of the latest release of 1) Compile and install a static version of the latest release of
libevent. libevent.
2) Acquire and install your preferred version of tor. Extract. 2) Acquire and install your preferred version of tor. Extract.
3) "make dist-osx" 3) "make dist-osx"
4) You now have a dmg from which you can install Tor. 4) You now have a dmg from which you can install Tor.
## Universal Binaries for OSX PPC and X86 ### Single Architecture Binaries for PPC or X86, not both.
## This method works in OSX 10.4 (Tiger) and newer OSX versions. ### This method works in all versions of OSX 10.3 through 10.6
## See far below if you don't care about cross compiling for PPC and X86.
## The single architecture process starts with "###" ## Compiling libevent ##
1) Download the latest stable libevent from
http://www.monkey.org/~provos/libevent/
2) The first step of compiling libevent is to configure it as
follows:
./configure --enable-static --disable-shared
3) Complete the "make" and "make install". You will need to be root,
or sudo -s, to complete the "make install".
## Compiling Tor ##
4) Get your preferred version of the tor source from https://www.torproject.org. Extract the
tarball.
5) In the top level, this means /path/to/tor/, not tor/contrib/osx,
do a configure with these parameters:
CONFDIR=/Library/Tor ./configure --prefix=/Library/Tor \
--bindir=/Library/Tor --sysconfdir=/Library
6) In same top level dir, do a "make dist-osx". There now exists a
.dmg file in the same directory. Install from this dmg.
### Backwards compatible single-architecture binaries for OSX x86 10.4 from newer versions of OS X.
1) Install the latest XCode updates available from http://developer.apple.com. 1) Install the latest XCode updates available from http://developer.apple.com.
## Compiling libevent ## Compiling libevent ##
2) Download latest stable libevent from
http://www.monkey.org/~provos/libevent/
3) The first step of compiling libevent is to configure it as
follows:
CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386" \
LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \
./configure --enable-static --disable-shared --disable-dependency-tracking
4) Complete the "make" and "make install". You will need to be root,
or sudo -s, to complete the "make install".
5) Check for a successful universal binary of libevent.a in, by default,
/usr/local/lib by using the following command:
"file /usr/local/lib/libevent.a"
Your output should be:
/usr/local/lib/libevent.a (for architecture i386): current ar archive random library
6) Get your preferred version of the tor source from https://www.torproject.org/download.
Extract the tarball.
7) In the top level, this means /path/to/tor/, not tor/contrib/osx,
do a configure with these parameters:
CFLAGS="-O -g -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386" \
LDFLAGS="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk" \
CONFDIR=/Library/Tor \
./configure --prefix=/Library/Tor --bindir=/Library/Tor \
--sysconfdir=/Library --disable-dependency-tracking
8) "make dist-osx"
9) Confirm you have created a universal binary by issuing the follow command:
"file src/or/tor". Its output should be as follows:
src/or/tor (for architecture i386): Mach-O executable i386
10) There should exist in the top-level directory a
Tor-$VERSION-universal-Bundle.dmg
11) Congrats. You have a backwards-compatible binary. You are now ready to install Tor.
### Universal Binaries for OSX PPC and X86
### This method works in OSX 10.4 (Tiger) and newer OSX versions.
1) Install the latest XCode updates available from http://developer.apple.com.
## Compiling libevent ##
2) Download latest stable libevent from 2) Download latest stable libevent from
http://www.monkey.org/~provos/libevent/ http://www.monkey.org/~provos/libevent/
@ -64,31 +143,3 @@ src/or/tor (for architecture ppc): Mach-O executable ppc
Tor-$VERSION-universal-Bundle.dmg Tor-$VERSION-universal-Bundle.dmg
11) Congrats. You have a universal binary. You are now ready to install Tor. 11) Congrats. You have a universal binary. You are now ready to install Tor.
### Single Architecture Binaries for PPC or X86, not both.
### This method works in all versions of OSX 10.3 through 10.5
### Compiling libevent
1) Download the latest stable libevent from
http://www.monkey.org/~provos/libevent/
2) The first step of compiling libevent is to configure it as
follows:
./configure --enable-static --disable-shared
3) Complete the "make" and "make install". You will need to be root,
or sudo -s, to complete the "make install".
### Compiling Tor
4) Get your preferred version of the tor source from https://www.torproject.org. Extract the
tarball.
5) In the top level, this means /path/to/tor/, not tor/contrib/osx,
do a configure with these parameters:
CONFDIR=/Library/Tor ./configure --prefix=/Library/Tor \
--bindir=/Library/Tor --sysconfdir=/Library
6) In same top level dir, do a "make dist-osx". There now exists a
.dmg file in the same directory. Install from this dmg.

View File

@ -1243,6 +1243,11 @@ When this is set then
\fBVersioningAuthoritativeDirectory\fP should be set too. \fBVersioningAuthoritativeDirectory\fP should be set too.
.LP .LP
.TP .TP
\fBConsensusParams \fR\fISTRING\fP
STRING is a space-separated list of key=value pairs that Tor will
include in the "params" line of its networkstatus vote.
.LP
.TP
\fBDirAllowPrivateAddresses \fR\fB0\fR|\fB1\fR\fP \fBDirAllowPrivateAddresses \fR\fB0\fR|\fB1\fR\fP
If set to 1, Tor will accept router descriptors with arbitrary "Address" If set to 1, Tor will accept router descriptors with arbitrary "Address"
elements. Otherwise, if the address is not an IP address or is a private elements. Otherwise, if the address is not an IP address or is a private
@ -1514,7 +1519,7 @@ The most recently downloaded network status document for each authority. Each f
.LP .LP
.TP .TP
.B \fIDataDirectory\fB/cached-descriptors\fR and \fBcached-descriptors.new\fR .B \fIDataDirectory\fB/cached-descriptors\fR and \fBcached-descriptors.new\fR
These files hold downloaded router statuses. Some routers may appear more than once; if so, the most recently published descriptor is used. Lines beginning with @-signs are annotations that contain more information about a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-routers file. These files hold downloaded router statuses. Some routers may appear more than once; if so, the most recently published descriptor is used. Lines beginning with @-signs are annotations that contain more information about a given router. The ".new" file is an append-only journal; when it gets too large, all entries are merged into a new cached-descriptors file.
.LP .LP
.TP .TP
.B \fIDataDirectory\fB/cached-routers\fR and \fBcached-routers.new\fR .B \fIDataDirectory\fB/cached-routers\fR and \fBcached-routers.new\fR

View File

@ -373,10 +373,11 @@ tor_addr_parse_reverse_lookup_name(tor_addr_t *result, const char *address,
return -1; /* malformed. */ return -1; /* malformed. */
/* reverse the bytes */ /* reverse the bytes */
inaddr.s_addr = (((inaddr.s_addr & 0x000000fful) << 24) inaddr.s_addr = (uint32_t)
|((inaddr.s_addr & 0x0000ff00ul) << 8) (((inaddr.s_addr & 0x000000ff) << 24)
|((inaddr.s_addr & 0x00ff0000ul) >> 8) |((inaddr.s_addr & 0x0000ff00) << 8)
|((inaddr.s_addr & 0xff000000ul) >> 24)); |((inaddr.s_addr & 0x00ff0000) >> 8)
|((inaddr.s_addr & 0xff000000) >> 24));
if (result) { if (result) {
tor_addr_from_in(result, &inaddr); tor_addr_from_in(result, &inaddr);

View File

@ -480,8 +480,8 @@ get_uint32(const char *cp)
return v; return v;
} }
/** /**
* Read a 32-bit value beginning at <b>cp</b>. Equivalent to * Read a 64-bit value beginning at <b>cp</b>. Equivalent to
* *(uint32_t*)(cp), but will not cause segfaults on platforms that forbid * *(uint64_t*)(cp), but will not cause segfaults on platforms that forbid
* unaligned memory access. * unaligned memory access.
*/ */
uint64_t uint64_t

View File

@ -1220,6 +1220,7 @@ IMPLEMENT_ORDER_FUNC(find_nth_int, int)
IMPLEMENT_ORDER_FUNC(find_nth_time, time_t) IMPLEMENT_ORDER_FUNC(find_nth_time, time_t)
IMPLEMENT_ORDER_FUNC(find_nth_double, double) IMPLEMENT_ORDER_FUNC(find_nth_double, double)
IMPLEMENT_ORDER_FUNC(find_nth_uint32, uint32_t) IMPLEMENT_ORDER_FUNC(find_nth_uint32, uint32_t)
IMPLEMENT_ORDER_FUNC(find_nth_int32, int32_t)
IMPLEMENT_ORDER_FUNC(find_nth_long, long) IMPLEMENT_ORDER_FUNC(find_nth_long, long)
/** Return a newly allocated digestset_t, optimized to hold a total of /** Return a newly allocated digestset_t, optimized to hold a total of

View File

@ -627,6 +627,7 @@ void digestset_free(digestset_t* set);
int find_nth_int(int *array, int n_elements, int nth); int find_nth_int(int *array, int n_elements, int nth);
time_t find_nth_time(time_t *array, int n_elements, int nth); time_t find_nth_time(time_t *array, int n_elements, int nth);
double find_nth_double(double *array, int n_elements, int nth); double find_nth_double(double *array, int n_elements, int nth);
int32_t find_nth_int32(int32_t *array, int n_elements, int nth);
uint32_t find_nth_uint32(uint32_t *array, int n_elements, int nth); uint32_t find_nth_uint32(uint32_t *array, int n_elements, int nth);
long find_nth_long(long *array, int n_elements, int nth); long find_nth_long(long *array, int n_elements, int nth);
static INLINE int static INLINE int
@ -649,6 +650,11 @@ median_uint32(uint32_t *array, int n_elements)
{ {
return find_nth_uint32(array, n_elements, (n_elements-1)/2); return find_nth_uint32(array, n_elements, (n_elements-1)/2);
} }
static INLINE int32_t
median_int32(int32_t *array, int n_elements)
{
return find_nth_int32(array, n_elements, (n_elements-1)/2);
}
static INLINE long static INLINE long
median_long(long *array, int n_elements) median_long(long *array, int n_elements)
{ {

View File

@ -92,7 +92,8 @@ should_log_function_name(log_domain_mask_t domain, int severity)
} }
/** A mutex to guard changes to logfiles and logging. */ /** A mutex to guard changes to logfiles and logging. */
static tor_mutex_t *log_mutex = NULL; static tor_mutex_t log_mutex;
static int log_mutex_initialized = 0;
/** Linked list of logfile_t. */ /** Linked list of logfile_t. */
static logfile_t *logfiles = NULL; static logfile_t *logfiles = NULL;
@ -103,9 +104,9 @@ static int syslog_count = 0;
#endif #endif
#define LOCK_LOGS() STMT_BEGIN \ #define LOCK_LOGS() STMT_BEGIN \
tor_mutex_acquire(log_mutex); \ tor_mutex_acquire(&log_mutex); \
STMT_END STMT_END
#define UNLOCK_LOGS() STMT_BEGIN tor_mutex_release(log_mutex); STMT_END #define UNLOCK_LOGS() STMT_BEGIN tor_mutex_release(&log_mutex); STMT_END
/** What's the lowest log level anybody cares about? Checking this lets us /** What's the lowest log level anybody cares about? Checking this lets us
* bail out early from log_debug if we aren't debugging. */ * bail out early from log_debug if we aren't debugging. */
@ -146,8 +147,8 @@ _log_prefix(char *buf, size_t buf_len, int severity)
t = (time_t)now.tv_sec; t = (time_t)now.tv_sec;
n = strftime(buf, buf_len, "%b %d %H:%M:%S", tor_localtime_r(&t, &tm)); n = strftime(buf, buf_len, "%b %d %H:%M:%S", tor_localtime_r(&t, &tm));
r = tor_snprintf(buf+n, buf_len-n, ".%.3ld [%s] ", r = tor_snprintf(buf+n, buf_len-n, ".%.3i [%s] ",
(long)now.tv_usec / 1000, sev_to_string(severity)); (int)now.tv_usec / 1000, sev_to_string(severity));
if (r<0) if (r<0)
return buf_len-1; return buf_len-1;
else else
@ -446,8 +447,9 @@ logs_free_all(void)
log_free(victim); log_free(victim);
} }
tor_free(appname); tor_free(appname);
tor_mutex_free(log_mutex);
log_mutex = NULL; /* We _could_ destroy the log mutex here, but that would screw up any logs
* that happened between here and the end of execution. */
} }
/** Remove and free the log entry <b>victim</b> from the linked-list /** Remove and free the log entry <b>victim</b> from the linked-list
@ -543,8 +545,10 @@ add_stream_log(const log_severity_list_t *severity,
void void
init_logging(void) init_logging(void)
{ {
if (!log_mutex) if (!log_mutex_initialized) {
log_mutex = tor_mutex_new(); tor_mutex_init(&log_mutex);
log_mutex_initialized = 1;
}
} }
/** Add a log handler to receive messages during startup (before the real /** Add a log handler to receive messages during startup (before the real

View File

@ -117,6 +117,9 @@ typedef unsigned int uint32_t;
#ifndef INT32_MAX #ifndef INT32_MAX
#define INT32_MAX 0x7fffffffu #define INT32_MAX 0x7fffffffu
#endif #endif
#ifndef INT32_MIN
#define INT32_MIN (-2147483647-1)
#endif
#endif #endif
#if (SIZEOF_LONG == 4) #if (SIZEOF_LONG == 4)

View File

@ -828,6 +828,9 @@ tor_tls_new(int sock, int isServer)
if (!SSL_set_cipher_list(result->ssl, if (!SSL_set_cipher_list(result->ssl,
isServer ? SERVER_CIPHER_LIST : CLIENT_CIPHER_LIST)) { isServer ? SERVER_CIPHER_LIST : CLIENT_CIPHER_LIST)) {
tls_log_errors(NULL, LOG_WARN, "setting ciphers"); tls_log_errors(NULL, LOG_WARN, "setting ciphers");
#ifdef SSL_set_tlsext_host_name
SSL_set_tlsext_host_name(result->ssl, NULL);
#endif
SSL_free(result->ssl); SSL_free(result->ssl);
tor_free(result); tor_free(result);
return NULL; return NULL;
@ -838,6 +841,9 @@ tor_tls_new(int sock, int isServer)
bio = BIO_new_socket(sock, BIO_NOCLOSE); bio = BIO_new_socket(sock, BIO_NOCLOSE);
if (! bio) { if (! bio) {
tls_log_errors(NULL, LOG_WARN, "opening BIO"); tls_log_errors(NULL, LOG_WARN, "opening BIO");
#ifdef SSL_set_tlsext_host_name
SSL_set_tlsext_host_name(result->ssl, NULL);
#endif
SSL_free(result->ssl); SSL_free(result->ssl);
tor_free(result); tor_free(result);
return NULL; return NULL;
@ -918,6 +924,9 @@ tor_tls_free(tor_tls_t *tls)
if (!removed) { if (!removed) {
log_warn(LD_BUG, "Freeing a TLS that was not in the ssl->tls map."); log_warn(LD_BUG, "Freeing a TLS that was not in the ssl->tls map.");
} }
#ifdef SSL_set_tlsext_host_name
SSL_set_tlsext_host_name(tls->ssl, NULL);
#endif
SSL_free(tls->ssl); SSL_free(tls->ssl);
tls->ssl = NULL; tls->ssl = NULL;
tls->negotiated_callback = NULL; tls->negotiated_callback = NULL;
@ -1442,8 +1451,8 @@ tor_tls_used_v1_handshake(tor_tls_t *tls)
* buffer and *<b>wbuf_bytes</b> to the amount actually used. */ * buffer and *<b>wbuf_bytes</b> to the amount actually used. */
void void
tor_tls_get_buffer_sizes(tor_tls_t *tls, tor_tls_get_buffer_sizes(tor_tls_t *tls,
int *rbuf_capacity, int *rbuf_bytes, size_t *rbuf_capacity, size_t *rbuf_bytes,
int *wbuf_capacity, int *wbuf_bytes) size_t *wbuf_capacity, size_t *wbuf_bytes)
{ {
if (tls->ssl->s3->rbuf.buf) if (tls->ssl->s3->rbuf.buf)
*rbuf_capacity = tls->ssl->s3->rbuf.len; *rbuf_capacity = tls->ssl->s3->rbuf.len;

View File

@ -73,8 +73,8 @@ void tor_tls_get_n_raw_bytes(tor_tls_t *tls,
size_t *n_read, size_t *n_written); size_t *n_read, size_t *n_written);
void tor_tls_get_buffer_sizes(tor_tls_t *tls, void tor_tls_get_buffer_sizes(tor_tls_t *tls,
int *rbuf_capacity, int *rbuf_bytes, size_t *rbuf_capacity, size_t *rbuf_bytes,
int *wbuf_capacity, int *wbuf_bytes); size_t *wbuf_capacity, size_t *wbuf_bytes);
int tor_tls_used_v1_handshake(tor_tls_t *tls); int tor_tls_used_v1_handshake(tor_tls_t *tls);

View File

@ -735,7 +735,18 @@ tor_parse_ulong(const char *s, int base, unsigned long min,
CHECK_STRTOX_RESULT(); CHECK_STRTOX_RESULT();
} }
/** As tor_parse_log, but return a unit64_t. Only base 10 is guaranteed to /** As tor_parse_long(), but return a double. */
double
tor_parse_double(const char *s, double min, double max, int *ok, char **next)
{
char *endptr;
double r;
r = strtod(s, &endptr);
CHECK_STRTOX_RESULT();
}
/** As tor_parse_long, but return a uint64_t. Only base 10 is guaranteed to
* work for now. */ * work for now. */
uint64_t uint64_t
tor_parse_uint64(const char *s, int base, uint64_t min, tor_parse_uint64(const char *s, int base, uint64_t min,
@ -1023,6 +1034,42 @@ wrap_string(smartlist_t *out, const char *string, size_t width,
* Time * Time
* ===== */ * ===== */
/**
* Converts struct timeval to a double value.
* Preserves microsecond precision, but just barely.
* Error is approx +/- 0.1 usec when dealing with epoch values.
*/
double
tv_to_double(const struct timeval *tv)
{
double conv = tv->tv_sec;
conv += tv->tv_usec/1000000.0;
return conv;
}
/**
* Converts timeval to milliseconds.
*/
int64_t
tv_to_msec(const struct timeval *tv)
{
int64_t conv = ((int64_t)tv->tv_sec)*1000L;
/* Round ghetto-style */
conv += ((int64_t)tv->tv_usec+500)/1000L;
return conv;
}
/**
* Converts timeval to microseconds.
*/
int64_t
tv_to_usec(const struct timeval *tv)
{
int64_t conv = ((int64_t)tv->tv_sec)*1000000L;
conv += tv->tv_usec;
return conv;
}
/** Return the number of microseconds elapsed between *start and *end. /** Return the number of microseconds elapsed between *start and *end.
*/ */
long long
@ -1055,7 +1102,9 @@ tv_mdiff(const struct timeval *start, const struct timeval *end)
return LONG_MAX; return LONG_MAX;
} }
mdiff = secdiff*1000L + (end->tv_usec - start->tv_usec) / 1000L; /* Subtract and round */
mdiff = secdiff*1000L +
((long)end->tv_usec - (long)start->tv_usec + 500L) / 1000L;
return mdiff; return mdiff;
} }
@ -1865,7 +1914,8 @@ write_chunks_to_file_impl(const char *fname, const smartlist_t *chunks,
int open_flags) int open_flags)
{ {
open_file_t *file = NULL; open_file_t *file = NULL;
int fd, result; int fd;
ssize_t result;
fd = start_writing_to_file(fname, open_flags, 0600, &file); fd = start_writing_to_file(fname, open_flags, 0600, &file);
if (fd<0) if (fd<0)
return -1; return -1;
@ -1950,7 +2000,7 @@ read_file_to_str(const char *filename, int flags, struct stat *stat_out)
int fd; /* router file */ int fd; /* router file */
struct stat statbuf; struct stat statbuf;
char *string; char *string;
int r; ssize_t r;
int bin = flags & RFTS_BIN; int bin = flags & RFTS_BIN;
tor_assert(filename); tor_assert(filename);
@ -2009,7 +2059,7 @@ read_file_to_str(const char *filename, int flags, struct stat *stat_out)
* match for size. */ * match for size. */
int save_errno = errno; int save_errno = errno;
log_warn(LD_FS,"Could read only %d of %ld bytes of file \"%s\".", log_warn(LD_FS,"Could read only %d of %ld bytes of file \"%s\".",
r, (long)statbuf.st_size,filename); (int)r, (long)statbuf.st_size,filename);
tor_free(string); tor_free(string);
close(fd); close(fd);
errno = save_errno; errno = save_errno;

View File

@ -182,6 +182,8 @@ long tor_parse_long(const char *s, int base, long min,
long max, int *ok, char **next); long max, int *ok, char **next);
unsigned long tor_parse_ulong(const char *s, int base, unsigned long min, unsigned long tor_parse_ulong(const char *s, int base, unsigned long min,
unsigned long max, int *ok, char **next); unsigned long max, int *ok, char **next);
double tor_parse_double(const char *s, double min, double max, int *ok,
char **next);
uint64_t tor_parse_uint64(const char *s, int base, uint64_t min, uint64_t tor_parse_uint64(const char *s, int base, uint64_t min,
uint64_t max, int *ok, char **next); uint64_t max, int *ok, char **next);
const char *hex_str(const char *from, size_t fromlen) ATTR_NONNULL((1)); const char *hex_str(const char *from, size_t fromlen) ATTR_NONNULL((1));
@ -210,6 +212,9 @@ void base16_encode(char *dest, size_t destlen, const char *src, size_t srclen);
int base16_decode(char *dest, size_t destlen, const char *src, size_t srclen); int base16_decode(char *dest, size_t destlen, const char *src, size_t srclen);
/* Time helpers */ /* Time helpers */
double tv_to_double(const struct timeval *tv);
int64_t tv_to_msec(const struct timeval *tv);
int64_t tv_to_usec(const struct timeval *tv);
long tv_udiff(const struct timeval *start, const struct timeval *end); long tv_udiff(const struct timeval *start, const struct timeval *end);
long tv_mdiff(const struct timeval *start, const struct timeval *end); long tv_mdiff(const struct timeval *start, const struct timeval *end);
time_t tor_timegm(struct tm *tm); time_t tor_timegm(struct tm *tm);

File diff suppressed because it is too large Load Diff

View File

@ -41,14 +41,14 @@ AM_CPPFLAGS = -DSHARE_DATADIR="\"$(datadir)\"" \
tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@ tor_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ @TOR_LDFLAGS_libevent@
tor_LDADD = ../common/libor.a ../common/libor-crypto.a \ tor_LDADD = ../common/libor.a ../common/libor-crypto.a \
../common/libor-event.a \ ../common/libor-event.a \
-lz -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@ -lz -lm -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@
test_SOURCES = $(COMMON_SRC) test_data.c test.c test_SOURCES = $(COMMON_SRC) test_data.c test.c
test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \ test_LDFLAGS = @TOR_LDFLAGS_zlib@ @TOR_LDFLAGS_openssl@ \
@TOR_LDFLAGS_libevent@ @TOR_LDFLAGS_libevent@
test_LDADD = ../common/libor.a ../common/libor-crypto.a \ test_LDADD = ../common/libor.a ../common/libor-crypto.a \
../common/libor-event.a \ ../common/libor-event.a \
-lz -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@ -lz -lm -levent -lssl -lcrypto @TOR_LIB_WS32@ @TOR_LIB_GDI@
noinst_HEADERS = or.h eventdns.h eventdns_tor.h micro-revision.i noinst_HEADERS = or.h eventdns.h eventdns_tor.h micro-revision.i

View File

@ -9,9 +9,49 @@
* \brief The actual details of building circuits. * \brief The actual details of building circuits.
**/ **/
#define CIRCUIT_PRIVATE
#include "or.h" #include "or.h"
#include "crypto.h"
/*
* This madness is needed because if we simply #undef log
* before including or.h or log.h, we get linker collisions
* and random segfaults due to memory corruption (and
* not even at calls to log() either!)
*/
/* XXX022 somebody should rename Tor's log() function, so we can
* remove this wart. -RD */
#undef log
/*
* Linux doesn't provide lround in math.h by default, but mac os does...
* It's best just to leave math.h out of the picture entirely.
*/
//#define log math_h_log
//#include <math.h>
//#undef log
long int lround(double x);
double ln(double x);
double log(double x);
double pow(double x, double y);
double
ln(double x)
{
return log(x);
}
#define log _log
/********* START VARIABLES **********/ /********* START VARIABLES **********/
/** Global list of circuit build times */
// FIXME: Add this as a member for entry_guard_t instead of global?
// Then we could do per-guard statistics, as guards are likely to
// vary in their own latency. The downside of this is that guards
// can change frequently, so we'd be building a lot more circuits
// most likely.
circuit_build_times_t circ_times;
/** A global list of all circuits at this hop. */ /** A global list of all circuits at this hop. */
extern circuit_t *global_circuitlist; extern circuit_t *global_circuitlist;
@ -47,6 +87,10 @@ static smartlist_t *entry_guards = NULL;
* and those changes need to be flushed to disk. */ * and those changes need to be flushed to disk. */
static int entry_guards_dirty = 0; static int entry_guards_dirty = 0;
/** If set, we're running the unit tests: we should avoid clobbering
* our state file or accessing get_options() or get_or_state() */
static int unit_tests = 0;
/********* END VARIABLES ************/ /********* END VARIABLES ************/
static int circuit_deliver_create_cell(circuit_t *circ, static int circuit_deliver_create_cell(circuit_t *circ,
@ -60,6 +104,796 @@ static int onion_append_hop(crypt_path_t **head_ptr, extend_info_t *choice);
static void entry_guards_changed(void); static void entry_guards_changed(void);
static time_t start_of_month(time_t when); static time_t start_of_month(time_t when);
/** Make a note that we're running unit tests (rather than running Tor
* itself), so we avoid clobbering our state file. */
void
circuitbuild_running_unit_tests(void)
{
unit_tests = 1;
}
/**
* Return the initial default or configured timeout in milliseconds
*/
static double
circuit_build_times_get_initial_timeout(void)
{
double timeout;
if (!unit_tests && get_options()->CircuitBuildTimeout) {
timeout = get_options()->CircuitBuildTimeout*1000;
if (timeout < BUILD_TIMEOUT_MIN_VALUE) {
log_warn(LD_CIRC, "Config CircuitBuildTimeout too low. Setting to %ds",
BUILD_TIMEOUT_MIN_VALUE/1000);
timeout = BUILD_TIMEOUT_MIN_VALUE;
}
} else {
timeout = BUILD_TIMEOUT_INITIAL_VALUE;
}
return timeout;
}
/**
* Reset the build time state.
*
* Leave estimated parameters, timeout and network liveness intact
* for future use.
*/
void
circuit_build_times_reset(circuit_build_times_t *cbt)
{
memset(cbt->circuit_build_times, 0, sizeof(cbt->circuit_build_times));
cbt->pre_timeouts = 0;
cbt->total_build_times = 0;
cbt->build_times_idx = 0;
cbt->have_computed_timeout = 0;
}
/**
* Initialize the buildtimes structure for first use.
*
* Sets the initial timeout value based to either the
* config setting or BUILD_TIMEOUT_INITIAL_VALUE.
*/
void
circuit_build_times_init(circuit_build_times_t *cbt)
{
memset(cbt, 0, sizeof(*cbt));
cbt->timeout_ms = circuit_build_times_get_initial_timeout();
}
/**
* Rewind our timeout history by n positions.
*/
static void
circuit_build_times_rewind_history(circuit_build_times_t *cbt, int n)
{
int i = 0;
if (cbt->pre_timeouts) {
if (cbt->pre_timeouts > n) {
cbt->pre_timeouts -= n;
} else {
cbt->pre_timeouts = 0;
}
log_info(LD_CIRC,
"Rewound history by %d places. Current index: %d. Total: %d. "
"Pre-timeouts: %d", n, cbt->build_times_idx,
cbt->total_build_times, cbt->pre_timeouts);
tor_assert(cbt->build_times_idx == 0);
tor_assert(cbt->total_build_times == 0);
return;
}
cbt->build_times_idx -= n;
cbt->build_times_idx %= NCIRCUITS_TO_OBSERVE;
for (i = 0; i < n; i++) {
cbt->circuit_build_times[(i+cbt->build_times_idx)%NCIRCUITS_TO_OBSERVE]=0;
}
if (cbt->total_build_times > n) {
cbt->total_build_times -= n;
} else {
cbt->total_build_times = 0;
}
log_info(LD_CIRC,
"Rewound history by %d places. Current index: %d. "
"Total: %d", n, cbt->build_times_idx, cbt->total_build_times);
}
/**
* Add a timeoutout value to the set of build times. Time units
* are milliseconds
*
* circuit_build_times is a circular array, so loop around when
* array is full.
*/
int
circuit_build_times_add_time(circuit_build_times_t *cbt, build_time_t time)
{
if (time > BUILD_TIME_MAX) {
log_notice(LD_CIRC,
"Circuit build time of %ums exceeds max. Capping at 65536ms", time);
time = BUILD_TIME_MAX;
} else if (time <= 0) {
log_err(LD_CIRC, "Circuit build time is %u!", time);
return -1;
}
// XXX: Probably want to demote this to debug for the release.
log_info(LD_CIRC, "Adding circuit build time %u", time);
cbt->circuit_build_times[cbt->build_times_idx] = time;
cbt->build_times_idx = (cbt->build_times_idx + 1) % NCIRCUITS_TO_OBSERVE;
if (cbt->total_build_times < NCIRCUITS_TO_OBSERVE)
cbt->total_build_times++;
if ((cbt->total_build_times % BUILD_TIMES_SAVE_STATE_EVERY) == 0) {
/* Save state every n circuit builds */
if (!unit_tests && !get_options()->AvoidDiskWrites)
or_state_mark_dirty(get_or_state(), 0);
}
return 0;
}
/**
* Return maximum circuit build time
*/
static build_time_t
circuit_build_times_max(circuit_build_times_t *cbt)
{
int i = 0;
build_time_t max_build_time = 0;
for (i = 0; i < NCIRCUITS_TO_OBSERVE; i++) {
if (cbt->circuit_build_times[i] > max_build_time)
max_build_time = cbt->circuit_build_times[i];
}
return max_build_time;
}
#if 0
/** Return minimum circuit build time */
build_time_t
circuit_build_times_min(circuit_build_times_t *cbt)
{
int i = 0;
build_time_t min_build_time = BUILD_TIME_MAX;
for (i = 0; i < NCIRCUITS_TO_OBSERVE; i++) {
if (cbt->circuit_build_times[i] && /* 0 <-> uninitialized */
cbt->circuit_build_times[i] < min_build_time)
min_build_time = cbt->circuit_build_times[i];
}
if (min_build_time == BUILD_TIME_MAX) {
log_warn(LD_CIRC, "No build times less than BUILD_TIME_MAX!");
}
return min_build_time;
}
#endif
/**
* Calculate and return a histogram for the set of build times.
*
* Returns an allocated array of histrogram bins representing
* the frequency of index*BUILDTIME_BIN_WIDTH millisecond
* build times. Also outputs the number of bins in nbins.
*
* The return value must be freed by the caller.
*/
static uint32_t *
circuit_build_times_create_histogram(circuit_build_times_t *cbt,
build_time_t *nbins)
{
uint32_t *histogram;
build_time_t max_build_time = circuit_build_times_max(cbt);
int i, c;
*nbins = 1 + (max_build_time / BUILDTIME_BIN_WIDTH);
histogram = tor_malloc_zero(*nbins * sizeof(build_time_t));
// calculate histogram
for (i = 0; i < NCIRCUITS_TO_OBSERVE; i++) {
if (cbt->circuit_build_times[i] == 0) continue; /* 0 <-> uninitialized */
c = (cbt->circuit_build_times[i] / BUILDTIME_BIN_WIDTH);
histogram[c]++;
}
return histogram;
}
/**
* Return the most frequent build time (rounded to BUILDTIME_BIN_WIDTH ms).
*
* Ties go in favor of the slower time.
*/
static build_time_t
circuit_build_times_mode(circuit_build_times_t *cbt)
{
build_time_t i, nbins, max_bin=0;
uint32_t *histogram = circuit_build_times_create_histogram(cbt, &nbins);
for (i = 0; i < nbins; i++) {
if (histogram[i] >= histogram[max_bin]) {
max_bin = i;
}
}
tor_free(histogram);
return max_bin*BUILDTIME_BIN_WIDTH+BUILDTIME_BIN_WIDTH/2;
}
/**
* Output a histogram of current circuit build times to
* the or_state_t state structure.
*/
void
circuit_build_times_update_state(circuit_build_times_t *cbt,
or_state_t *state)
{
uint32_t *histogram;
build_time_t i = 0;
build_time_t nbins = 0;
config_line_t **next, *line;
histogram = circuit_build_times_create_histogram(cbt, &nbins);
// write to state
config_free_lines(state->BuildtimeHistogram);
next = &state->BuildtimeHistogram;
*next = NULL;
state->TotalBuildTimes = cbt->total_build_times;
for (i = 0; i < nbins; i++) {
// compress the histogram by skipping the blanks
if (histogram[i] == 0) continue;
*next = line = tor_malloc_zero(sizeof(config_line_t));
line->key = tor_strdup("CircuitBuildTimeBin");
line->value = tor_malloc(25);
tor_snprintf(line->value, 25, "%d %d",
i*BUILDTIME_BIN_WIDTH+BUILDTIME_BIN_WIDTH/2, histogram[i]);
next = &(line->next);
}
if (!unit_tests) {
if (!get_options()->AvoidDiskWrites)
or_state_mark_dirty(get_or_state(), 0);
}
if (histogram) tor_free(histogram);
}
/**
* Shuffle the build times array.
*
* Stolen from http://en.wikipedia.org/wiki/Fisher\u2013Yates_shuffle
*/
static void
circuit_build_times_shuffle_array(circuit_build_times_t *cbt)
{
int n = cbt->total_build_times;
/* This code can only be run on a compact array */
tor_assert(cbt->total_build_times == cbt->build_times_idx);
while (n-- > 1) {
int k = crypto_rand_int(n + 1); /* 0 <= k <= n. */
build_time_t tmp = cbt->circuit_build_times[k];
cbt->circuit_build_times[k] = cbt->circuit_build_times[n];
cbt->circuit_build_times[n] = tmp;
}
}
/**
* Load histogram from <b>state</b>, shuffling the resulting array
* after we do so. Use this result to estimate parameters and
* calculate the timeout.
*
* Returns -1 and sets msg on error. Msg must be freed by the caller.
*/
int
circuit_build_times_parse_state(circuit_build_times_t *cbt,
or_state_t *state, char **msg)
{
int tot_values = 0, N = 0;
config_line_t *line;
int i;
*msg = NULL;
circuit_build_times_init(cbt);
/* We don't support decreasing the table size yet */
tor_assert(state->TotalBuildTimes <= NCIRCUITS_TO_OBSERVE);
for (line = state->BuildtimeHistogram; line; line = line->next) {
smartlist_t *args = smartlist_create();
smartlist_split_string(args, line->value, " ",
SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 0);
if (smartlist_len(args) < 2) {
*msg = tor_strdup("Unable to parse circuit build times: "
"Too few arguments to CircuitBuildTime");
SMARTLIST_FOREACH(args, char*, cp, tor_free(cp));
smartlist_free(args);
break;
} else {
const char *ms_str = smartlist_get(args,0);
const char *count_str = smartlist_get(args,1);
uint32_t count, k;
build_time_t ms;
int ok;
ms = (build_time_t)tor_parse_ulong(ms_str, 0, 0,
BUILD_TIME_MAX, &ok, NULL);
if (!ok) {
*msg = tor_strdup("Unable to parse circuit build times: "
"Unparsable bin number");
break;
}
count = (uint32_t)tor_parse_ulong(count_str, 0, 0,
UINT32_MAX, &ok, NULL);
if (!ok) {
*msg = tor_strdup("Unable to parse circuit build times: "
"Unparsable bin count");
break;
}
for (k = 0; k < count; k++) {
circuit_build_times_add_time(cbt, ms);
}
N++;
SMARTLIST_FOREACH(args, char*, cp, tor_free(cp));
smartlist_free(args);
}
}
circuit_build_times_shuffle_array(cbt);
/* Verify that we didn't overwrite any indexes */
for (i=0; i < NCIRCUITS_TO_OBSERVE; i++) {
if (!cbt->circuit_build_times[i])
break;
tot_values++;
}
log_info(LD_CIRC,
"Loaded %d/%d values from %d lines in circuit time histogram",
tot_values, cbt->total_build_times, N);
tor_assert(cbt->total_build_times == state->TotalBuildTimes);
tor_assert(tot_values == cbt->total_build_times);
circuit_build_times_set_timeout(cbt);
return *msg ? -1 : 0;
}
/**
* Estimates the Xm and Alpha parameters using
* http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation
*
* The notable difference is that we use mode instead of min to estimate Xm.
* This is because our distribution is frechet-like. We claim this is
* an acceptable approximation because we are only concerned with the
* accuracy of the CDF of the tail.
*/
void
circuit_build_times_update_alpha(circuit_build_times_t *cbt)
{
build_time_t *x=cbt->circuit_build_times;
double a = 0;
int n=0,i=0;
/* http://en.wikipedia.org/wiki/Pareto_distribution#Parameter_estimation */
/* We sort of cheat here and make our samples slightly more pareto-like
* and less frechet-like. */
cbt->Xm = circuit_build_times_mode(cbt);
for (i=0; i< NCIRCUITS_TO_OBSERVE; i++) {
if (!x[i]) {
continue;
}
if (x[i] < cbt->Xm) {
a += ln(cbt->Xm);
} else {
a += ln(x[i]);
}
n++;
}
if (n!=cbt->total_build_times) {
log_err(LD_CIRC, "Discrepancy in build times count: %d vs %d", n,
cbt->total_build_times);
}
tor_assert(n==cbt->total_build_times);
a -= n*ln(cbt->Xm);
a = n/a;
cbt->alpha = a;
}
/**
* This is the Pareto Quantile Function. It calculates the point x
* in the distribution such that F(x) = quantile (ie quantile*100%
* of the mass of the density function is below x on the curve).
*
* We use it to calculate the timeout and also to generate synthetic
* values of time for circuits that timeout before completion.
*
* See http://en.wikipedia.org/wiki/Quantile_function,
* http://en.wikipedia.org/wiki/Inverse_transform_sampling and
* http://en.wikipedia.org/wiki/Pareto_distribution#Generating_a_
* random_sample_from_Pareto_distribution
* That's right. I'll cite wikipedia all day long.
*
* Return value is in milliseconds.
*/
double
circuit_build_times_calculate_timeout(circuit_build_times_t *cbt,
double quantile)
{
double ret;
tor_assert(quantile >= 0);
tor_assert(1.0-quantile > 0);
tor_assert(cbt->Xm > 0);
ret = cbt->Xm/pow(1.0-quantile,1.0/cbt->alpha);
if (ret > INT32_MAX) {
ret = INT32_MAX;
}
tor_assert(ret > 0);
return ret;
}
/** Pareto CDF */
double
circuit_build_times_cdf(circuit_build_times_t *cbt, double x)
{
double ret;
tor_assert(cbt->Xm > 0);
ret = 1.0-pow(cbt->Xm/x,cbt->alpha);
tor_assert(0 <= ret && ret <= 1.0);
return ret;
}
/**
* Generate a synthetic time using our distribution parameters.
*
* The return value will be within the [q_lo, q_hi) quantile points
* on the CDF.
*/
build_time_t
circuit_build_times_generate_sample(circuit_build_times_t *cbt,
double q_lo, double q_hi)
{
uint64_t r = crypto_rand_uint64(UINT64_MAX-1);
build_time_t ret;
double u;
/* Generate between [q_lo, q_hi) */
q_hi -= 1.0/(INT32_MAX);
tor_assert(q_lo >= 0);
tor_assert(q_hi < 1);
tor_assert(q_lo < q_hi);
u = q_lo + ((q_hi-q_lo)*r)/(1.0*UINT64_MAX);
tor_assert(0 <= u && u < 1.0);
/* circuit_build_times_calculate_timeout returns <= INT32_MAX */
ret = (build_time_t)lround(circuit_build_times_calculate_timeout(cbt, u));
tor_assert(ret > 0);
return ret;
}
/** Generate points in [cutoff, 1.0) on the CDF. */
void
circuit_build_times_add_timeout_worker(circuit_build_times_t *cbt,
double quantile_cutoff)
{
build_time_t gentime = circuit_build_times_generate_sample(cbt,
quantile_cutoff, MAX_SYNTHETIC_QUANTILE);
if (gentime < (build_time_t)lround(cbt->timeout_ms)) {
log_warn(LD_CIRC,
"Generated a synthetic timeout LESS than the current timeout: "
"%ums vs %lfms using Xm: %d a: %lf, q: %lf",
gentime, cbt->timeout_ms, cbt->Xm, cbt->alpha, quantile_cutoff);
} else if (gentime > BUILD_TIME_MAX) {
gentime = BUILD_TIME_MAX;
log_info(LD_CIRC,
"Generated a synthetic timeout larger than the max: %u",
gentime);
} else {
log_info(LD_CIRC, "Generated synthetic circuit build time %u for timeout",
gentime);
}
circuit_build_times_add_time(cbt, gentime);
}
/**
* Estimate an initial alpha parameter by solving the quantile
* function with a quantile point and a specific timeout value.
*/
void
circuit_build_times_initial_alpha(circuit_build_times_t *cbt,
double quantile, double timeout_ms)
{
// Q(u) = Xm/((1-u)^(1/a))
// Q(0.8) = Xm/((1-0.8))^(1/a)) = CircBuildTimeout
// CircBuildTimeout = Xm/((1-0.8))^(1/a))
// CircBuildTimeout = Xm*((1-0.8))^(-1/a))
// ln(CircBuildTimeout) = ln(Xm)+ln(((1-0.8)))*(-1/a)
// -ln(1-0.8)/(ln(CircBuildTimeout)-ln(Xm))=a
tor_assert(quantile > 0);
tor_assert(cbt->Xm > 0);
cbt->alpha = ln(1.0-quantile)/(ln(cbt->Xm)-ln(timeout_ms));
tor_assert(cbt->alpha > 0);
}
/**
* Generate synthetic timeout values for the timeouts
* that have happened before we estimated our parameters.
*/
static void
circuit_build_times_count_pretimeouts(circuit_build_times_t *cbt)
{
/* Store a timeout as a random position past the current
* cutoff on the pareto curve */
if (cbt->pre_timeouts) {
double timeout_quantile = 1.0-
((double)cbt->pre_timeouts)/
(cbt->pre_timeouts+cbt->total_build_times);
cbt->Xm = circuit_build_times_mode(cbt);
tor_assert(cbt->Xm > 0);
/* Use current timeout to get an estimate on alpha */
circuit_build_times_initial_alpha(cbt, timeout_quantile,
cbt->timeout_ms);
while (cbt->pre_timeouts-- != 0) {
circuit_build_times_add_timeout_worker(cbt, timeout_quantile);
}
cbt->pre_timeouts = 0;
}
}
/**
* Returns true if we need circuits to be built
*/
int
circuit_build_times_needs_circuits(circuit_build_times_t *cbt)
{
/* Return true if < MIN_CIRCUITS_TO_OBSERVE */
if (cbt->total_build_times < MIN_CIRCUITS_TO_OBSERVE)
return 1;
return 0;
}
/**
* Returns true if we should build a timeout test circuit
* right now.
*/
int
circuit_build_times_needs_circuits_now(circuit_build_times_t *cbt)
{
return circuit_build_times_needs_circuits(cbt) &&
approx_time()-cbt->last_circ_at > BUILD_TIMES_TEST_FREQUENCY;
}
/**
* Called to indicate that the network showed some signs of liveness.
*/
void
circuit_build_times_network_is_live(circuit_build_times_t *cbt)
{
cbt->liveness.network_last_live = approx_time();
cbt->liveness.nonlive_discarded = 0;
cbt->liveness.nonlive_timeouts = 0;
}
/**
* Called to indicate that we completed a circuit. Because this circuit
* succeeded, it doesn't count as a timeout-after-the-first-hop.
*/
void
circuit_build_times_network_circ_success(circuit_build_times_t *cbt)
{
cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx] = 0;
cbt->liveness.after_firsthop_idx++;
cbt->liveness.after_firsthop_idx %= RECENT_CIRCUITS;
}
/**
* A circuit just timed out. If there has been no recent network activity
* at all, but this circuit was launched back when we thought the network
* was live, increment the number of "nonlive" circuit timeouts.
*
* Also distinguish between whether it failed before the first hop
* and record that in our history for later deciding if the network has
* changed.
*/
static void
circuit_build_times_network_timeout(circuit_build_times_t *cbt,
int did_onehop, time_t start_time)
{
time_t now = time(NULL);
/*
* Check if this is a timeout that was for a circuit that spent its
* entire existence during a time where we have had no network activity.
*
* Also double check that it is a valid timeout after we have possibly
* just recently reset cbt->timeout_ms.
*/
if (cbt->liveness.network_last_live <= start_time &&
start_time <= (now - cbt->timeout_ms/1000.0)) {
cbt->liveness.nonlive_timeouts++;
}
/* Check for one-hop timeout */
if (did_onehop) {
cbt->liveness.timeouts_after_firsthop[cbt->liveness.after_firsthop_idx]=1;
cbt->liveness.after_firsthop_idx++;
cbt->liveness.after_firsthop_idx %= RECENT_CIRCUITS;
}
}
/**
* Returns false if the network has not received a cell or tls handshake
* in the past NETWORK_NOTLIVE_TIMEOUT_COUNT circuits.
*
* Also has the side effect of rewinding the circuit time history
* in the case of recent liveness changes.
*/
int
circuit_build_times_network_check_live(circuit_build_times_t *cbt)
{
time_t now = approx_time();
if (cbt->liveness.nonlive_timeouts >= NETWORK_NONLIVE_DISCARD_COUNT) {
if (!cbt->liveness.nonlive_discarded) {
cbt->liveness.nonlive_discarded = 1;
log_notice(LD_CIRC, "Network is no longer live (too many recent "
"circuit timeouts). Dead for %ld seconds.",
(long int)(now - cbt->liveness.network_last_live));
/* Only discard NETWORK_NONLIVE_TIMEOUT_COUNT-1 because we stopped
* counting after that */
circuit_build_times_rewind_history(cbt, NETWORK_NONLIVE_TIMEOUT_COUNT-1);
}
return 0;
} else if (cbt->liveness.nonlive_timeouts >= NETWORK_NONLIVE_TIMEOUT_COUNT) {
if (cbt->timeout_ms < circuit_build_times_get_initial_timeout()) {
log_notice(LD_CIRC,
"Network is flaky. No activity for %ld seconds. "
"Temporarily raising timeout to %lds.",
(long int)(now - cbt->liveness.network_last_live),
lround(circuit_build_times_get_initial_timeout()/1000));
cbt->timeout_ms = circuit_build_times_get_initial_timeout();
}
return 0;
}
return 1;
}
/**
* Returns true if we have seen more than MAX_RECENT_TIMEOUT_COUNT of
* the past RECENT_CIRCUITS time out after the first hop. Used to detect
* if the network connection has changed significantly.
*
* Also resets the entire timeout history in this case and causes us
* to restart the process of building test circuits and estimating a
* new timeout.
*/
int
circuit_build_times_network_check_changed(circuit_build_times_t *cbt)
{
int total_build_times = cbt->total_build_times;
int timeout_count=0;
int i;
/* how many of our recent circuits made it to the first hop but then
* timed out? */
for (i = 0; i < RECENT_CIRCUITS; i++) {
timeout_count += cbt->liveness.timeouts_after_firsthop[i];
}
/* If 75% of our recent circuits are timing out after the first hop,
* we need to re-estimate a new initial alpha and timeout. */
if (timeout_count < MAX_RECENT_TIMEOUT_COUNT) {
return 0;
}
circuit_build_times_reset(cbt);
memset(cbt->liveness.timeouts_after_firsthop, 0,
sizeof(cbt->liveness.timeouts_after_firsthop));
cbt->liveness.after_firsthop_idx = 0;
/* Check to see if this has happened before. If so, double the timeout
* to give people on abysmally bad network connections a shot at access */
if (cbt->timeout_ms >= circuit_build_times_get_initial_timeout()) {
cbt->timeout_ms *= 2;
} else {
cbt->timeout_ms = circuit_build_times_get_initial_timeout();
}
log_notice(LD_CIRC,
"Network connection speed appears to have changed. Resetting "
"timeout to %lds after %d timeouts and %d buildtimes.",
lround(cbt->timeout_ms/1000), timeout_count, total_build_times);
return 1;
}
/**
* Store a timeout as a synthetic value.
*
* Returns true if the store was successful and we should possibly
* update our timeout estimate.
*/
int
circuit_build_times_add_timeout(circuit_build_times_t *cbt,
int did_onehop,
time_t start_time)
{
circuit_build_times_network_timeout(cbt, did_onehop, start_time);
/* Only count timeouts if network is live.. */
if (!circuit_build_times_network_check_live(cbt)) {
return 0;
}
/* If there are a ton of timeouts, we should reduce
* the circuit build timeout */
if (circuit_build_times_network_check_changed(cbt)) {
return 0;
}
if (!cbt->have_computed_timeout) {
/* Store a timeout before we have enough data */
cbt->pre_timeouts++;
log_info(LD_CIRC,
"Not enough circuits yet to calculate a new build timeout."
" Need %d more.",
MIN_CIRCUITS_TO_OBSERVE-cbt->total_build_times);
return 0;
}
circuit_build_times_count_pretimeouts(cbt);
circuit_build_times_add_timeout_worker(cbt, BUILDTIMEOUT_QUANTILE_CUTOFF);
return 1;
}
/**
* Estimate a new timeout based on history and set our timeout
* variable accordingly.
*/
void
circuit_build_times_set_timeout(circuit_build_times_t *cbt)
{
if (cbt->total_build_times < MIN_CIRCUITS_TO_OBSERVE) {
return;
}
circuit_build_times_count_pretimeouts(cbt);
circuit_build_times_update_alpha(cbt);
cbt->timeout_ms = circuit_build_times_calculate_timeout(cbt,
BUILDTIMEOUT_QUANTILE_CUTOFF);
cbt->have_computed_timeout = 1;
if (cbt->timeout_ms < BUILD_TIMEOUT_MIN_VALUE) {
log_warn(LD_CIRC, "Set buildtimeout to low value %lfms. Setting to %dms",
cbt->timeout_ms, BUILD_TIMEOUT_MIN_VALUE);
cbt->timeout_ms = BUILD_TIMEOUT_MIN_VALUE;
}
log_info(LD_CIRC,
"Set circuit build timeout to %lds (%lfms, Xm: %d, a: %lf) "
"based on %d circuit times", lround(cbt->timeout_ms/1000),
cbt->timeout_ms, cbt->Xm, cbt->alpha, cbt->total_build_times);
}
/** Iterate over values of circ_id, starting from conn-\>next_circ_id, /** Iterate over values of circ_id, starting from conn-\>next_circ_id,
* and with the high bit specified by conn-\>circ_id_type, until we get * and with the high bit specified by conn-\>circ_id_type, until we get
* a circ_id that is not in use by any other circuit on that conn. * a circ_id that is not in use by any other circuit on that conn.
@ -527,9 +1361,16 @@ inform_testing_reachability(void)
routerinfo_t *me = router_get_my_routerinfo(); routerinfo_t *me = router_get_my_routerinfo();
if (!me) if (!me)
return 0; return 0;
if (me->dir_port) control_event_server_status(LOG_NOTICE,
"CHECKING_REACHABILITY ORADDRESS=%s:%d",
me->address, me->or_port);
if (me->dir_port) {
tor_snprintf(dirbuf, sizeof(dirbuf), " and DirPort %s:%d", tor_snprintf(dirbuf, sizeof(dirbuf), " and DirPort %s:%d",
me->address, me->dir_port); me->address, me->dir_port);
control_event_server_status(LOG_NOTICE,
"CHECKING_REACHABILITY DIRADDRESS=%s:%d",
me->address, me->dir_port);
}
log(LOG_NOTICE, LD_OR, "Now checking whether ORPort %s:%d%s %s reachable... " log(LOG_NOTICE, LD_OR, "Now checking whether ORPort %s:%d%s %s reachable... "
"(this may take up to %d minutes -- look for log " "(this may take up to %d minutes -- look for log "
"messages indicating success)", "messages indicating success)",
@ -537,6 +1378,7 @@ inform_testing_reachability(void)
me->dir_port ? dirbuf : "", me->dir_port ? dirbuf : "",
me->dir_port ? "are" : "is", me->dir_port ? "are" : "is",
TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT/60); TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT/60);
return 1; return 1;
} }
@ -633,8 +1475,17 @@ circuit_send_next_onion_skin(origin_circuit_t *circ)
log_debug(LD_CIRC,"starting to send subsequent skin."); log_debug(LD_CIRC,"starting to send subsequent skin.");
hop = onion_next_hop_in_cpath(circ->cpath); hop = onion_next_hop_in_cpath(circ->cpath);
if (!hop) { if (!hop) {
struct timeval end;
long timediff;
tor_gettimeofday(&end);
timediff = tv_mdiff(&circ->_base.highres_created, &end);
if (timediff > INT32_MAX)
timediff = INT32_MAX;
/* done building the circuit. whew. */ /* done building the circuit. whew. */
circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_OPEN); circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_OPEN);
circuit_build_times_add_time(&circ_times, (build_time_t)timediff);
circuit_build_times_network_circ_success(&circ_times);
circuit_build_times_set_timeout(&circ_times);
log_info(LD_CIRC,"circuit built!"); log_info(LD_CIRC,"circuit built!");
circuit_reset_failure_count(0); circuit_reset_failure_count(0);
if (circ->build_state->onehop_tunnel) if (circ->build_state->onehop_tunnel)
@ -1436,13 +2287,16 @@ choose_good_exit_server(uint8_t purpose, routerlist_t *dir,
/** Log a warning if the user specified an exit for the circuit that /** Log a warning if the user specified an exit for the circuit that
* has been excluded from use by ExcludeNodes or ExcludeExitNodes. */ * has been excluded from use by ExcludeNodes or ExcludeExitNodes. */
static void static void
warn_if_last_router_excluded(uint8_t purpose, const extend_info_t *exit) warn_if_last_router_excluded(origin_circuit_t *circ, const extend_info_t *exit)
{ {
or_options_t *options = get_options(); or_options_t *options = get_options();
routerset_t *rs = options->ExcludeNodes; routerset_t *rs = options->ExcludeNodes;
const char *description; const char *description;
int severity;
int domain = LD_CIRC; int domain = LD_CIRC;
uint8_t purpose = circ->_base.purpose;
if (circ->build_state->onehop_tunnel)
return;
switch (purpose) switch (purpose)
{ {
@ -1455,48 +2309,40 @@ warn_if_last_router_excluded(uint8_t purpose, const extend_info_t *exit)
(int)purpose); (int)purpose);
return; return;
case CIRCUIT_PURPOSE_C_GENERAL: case CIRCUIT_PURPOSE_C_GENERAL:
if (circ->build_state->is_internal)
return;
description = "Requested exit node"; description = "Requested exit node";
rs = options->_ExcludeExitNodesUnion; rs = options->_ExcludeExitNodesUnion;
severity = LOG_WARN;
break; break;
case CIRCUIT_PURPOSE_C_INTRODUCING: case CIRCUIT_PURPOSE_C_INTRODUCING:
case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT: case CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT:
case CIRCUIT_PURPOSE_C_INTRODUCE_ACKED: case CIRCUIT_PURPOSE_C_INTRODUCE_ACKED:
description = "Introduction point for hidden service"; case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO:
severity = LOG_INFO; case CIRCUIT_PURPOSE_S_CONNECT_REND:
break; case CIRCUIT_PURPOSE_S_REND_JOINED:
case CIRCUIT_PURPOSE_TESTING:
return;
case CIRCUIT_PURPOSE_C_ESTABLISH_REND: case CIRCUIT_PURPOSE_C_ESTABLISH_REND:
case CIRCUIT_PURPOSE_C_REND_READY: case CIRCUIT_PURPOSE_C_REND_READY:
case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED: case CIRCUIT_PURPOSE_C_REND_READY_INTRO_ACKED:
case CIRCUIT_PURPOSE_C_REND_JOINED: case CIRCUIT_PURPOSE_C_REND_JOINED:
description = "Chosen rendezvous point"; description = "Chosen rendezvous point";
severity = LOG_WARN;
domain = LD_BUG; domain = LD_BUG;
break; break;
case CIRCUIT_PURPOSE_S_ESTABLISH_INTRO:
description = "Chosen introduction point";
severity = LOG_INFO;
break;
case CIRCUIT_PURPOSE_S_CONNECT_REND:
case CIRCUIT_PURPOSE_S_REND_JOINED:
description = "Client-selected rendezvous point";
severity = LOG_INFO;
break;
case CIRCUIT_PURPOSE_TESTING:
description = "Target for testing circuit";
severity = LOG_INFO;
break;
case CIRCUIT_PURPOSE_CONTROLLER: case CIRCUIT_PURPOSE_CONTROLLER:
rs = options->_ExcludeExitNodesUnion; rs = options->_ExcludeExitNodesUnion;
description = "Controller-selected circuit target"; description = "Controller-selected circuit target";
severity = LOG_WARN;
break; break;
} }
if (routerset_contains_extendinfo(rs, exit)) if (routerset_contains_extendinfo(rs, exit)) {
log_fn(severity, domain, "%s '%s' is in ExcludeNodes%s. Using anyway.", log_fn(LOG_WARN, domain, "%s '%s' is in ExcludeNodes%s. Using anyway "
"(circuit purpose %d).",
description,exit->nickname, description,exit->nickname,
rs==options->ExcludeNodes?"":" or ExcludeExitNodes."); rs==options->ExcludeNodes?"":" or ExcludeExitNodes",
(int)purpose);
circuit_log_path(LOG_WARN, domain, circ);
}
return; return;
} }
@ -1521,7 +2367,7 @@ onion_pick_cpath_exit(origin_circuit_t *circ, extend_info_t *exit)
} }
if (exit) { /* the circuit-builder pre-requested one */ if (exit) { /* the circuit-builder pre-requested one */
warn_if_last_router_excluded(circ->_base.purpose, exit); warn_if_last_router_excluded(circ, exit);
log_info(LD_CIRC,"Using requested exit node '%s'", exit->nickname); log_info(LD_CIRC,"Using requested exit node '%s'", exit->nickname);
exit = extend_info_dup(exit); exit = extend_info_dup(exit);
} else { /* we have to decide one */ } else { /* we have to decide one */
@ -1568,6 +2414,7 @@ int
circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *exit) circuit_extend_to_new_exit(origin_circuit_t *circ, extend_info_t *exit)
{ {
int err_reason = 0; int err_reason = 0;
warn_if_last_router_excluded(circ, exit);
circuit_append_new_exit(circ, exit); circuit_append_new_exit(circ, exit);
circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_BUILDING); circuit_set_state(TO_CIRCUIT(circ), CIRCUIT_STATE_BUILDING);
if ((err_reason = circuit_send_next_onion_skin(circ))<0) { if ((err_reason = circuit_send_next_onion_skin(circ))<0) {
@ -1825,7 +2672,7 @@ onion_append_hop(crypt_path_t **head_ptr, extend_info_t *choice)
hop->extend_info = extend_info_dup(choice); hop->extend_info = extend_info_dup(choice);
hop->package_window = CIRCWINDOW_START; hop->package_window = circuit_initial_package_window();
hop->deliver_window = CIRCWINDOW_START; hop->deliver_window = CIRCWINDOW_START;
return 0; return 0;

View File

@ -361,14 +361,27 @@ circuit_purpose_to_controller_string(uint8_t purpose)
} }
} }
/** Pick a reasonable package_window to start out for our circuits.
* Originally this was hard-coded at 1000, but now the consensus votes
* on the answer. See proposal 168. */
int32_t
circuit_initial_package_window(void)
{
networkstatus_t *consensus = networkstatus_get_latest_consensus();
if (consensus)
return networkstatus_get_param(consensus, "circwindow", CIRCWINDOW_START);
return CIRCWINDOW_START;
}
/** Initialize the common elements in a circuit_t, and add it to the global /** Initialize the common elements in a circuit_t, and add it to the global
* list. */ * list. */
static void static void
init_circuit_base(circuit_t *circ) init_circuit_base(circuit_t *circ)
{ {
circ->timestamp_created = time(NULL); circ->timestamp_created = time(NULL);
tor_gettimeofday(&circ->highres_created);
circ->package_window = CIRCWINDOW_START; circ->package_window = circuit_initial_package_window();
circ->deliver_window = CIRCWINDOW_START; circ->deliver_window = CIRCWINDOW_START;
circuit_add(circ); circuit_add(circ);
@ -395,6 +408,8 @@ origin_circuit_new(void)
init_circuit_base(TO_CIRCUIT(circ)); init_circuit_base(TO_CIRCUIT(circ));
circ_times.last_circ_at = approx_time();
return circ; return circ;
} }

View File

@ -20,6 +20,8 @@ extern circuit_t *global_circuitlist; /* from circuitlist.c */
static void circuit_expire_old_circuits(time_t now); static void circuit_expire_old_circuits(time_t now);
static void circuit_increment_failure_count(void); static void circuit_increment_failure_count(void);
long int lround(double x);
/** Return 1 if <b>circ</b> could be returned by circuit_get_best(). /** Return 1 if <b>circ</b> could be returned by circuit_get_best().
* Else return 0. * Else return 0.
*/ */
@ -263,16 +265,18 @@ circuit_conforms_to_options(const origin_circuit_t *circ,
void void
circuit_expire_building(time_t now) circuit_expire_building(time_t now)
{ {
circuit_t *victim, *circ = global_circuitlist; circuit_t *victim, *next_circ = global_circuitlist;
time_t general_cutoff = now - get_options()->CircuitBuildTimeout; /* circ_times.timeout is BUILD_TIMEOUT_INITIAL_VALUE if we haven't
time_t begindir_cutoff = now - get_options()->CircuitBuildTimeout/2; * decided on a customized one yet */
time_t general_cutoff = now - lround(circ_times.timeout_ms/1000);
time_t begindir_cutoff = now - lround(circ_times.timeout_ms/2000);
time_t introcirc_cutoff = begindir_cutoff; time_t introcirc_cutoff = begindir_cutoff;
cpath_build_state_t *build_state; cpath_build_state_t *build_state;
while (circ) { while (next_circ) {
time_t cutoff; time_t cutoff;
victim = circ; victim = next_circ;
circ = circ->next; next_circ = next_circ->next;
if (!CIRCUIT_IS_ORIGIN(victim) || /* didn't originate here */ if (!CIRCUIT_IS_ORIGIN(victim) || /* didn't originate here */
victim->marked_for_close) /* don't mess with marked circs */ victim->marked_for_close) /* don't mess with marked circs */
continue; continue;
@ -343,6 +347,12 @@ circuit_expire_building(time_t now)
continue; continue;
break; break;
} }
} else { /* circuit not open, consider recording failure as timeout */
int first_hop_succeeded = TO_ORIGIN_CIRCUIT(victim)->cpath &&
TO_ORIGIN_CIRCUIT(victim)->cpath->state == CPATH_STATE_OPEN;
if (circuit_build_times_add_timeout(&circ_times, first_hop_succeeded,
victim->timestamp_created))
circuit_build_times_set_timeout(&circ_times);
} }
if (victim->n_conn) if (victim->n_conn)
@ -431,11 +441,11 @@ circuit_stream_is_being_handled(edge_connection_t *conn,
} }
/** Don't keep more than this many unused open circuits around. */ /** Don't keep more than this many unused open circuits around. */
#define MAX_UNUSED_OPEN_CIRCUITS 12 #define MAX_UNUSED_OPEN_CIRCUITS 14
/** Figure out how many circuits we have open that are clean. Make /** Figure out how many circuits we have open that are clean. Make
* sure it's enough for all the upcoming behaviors we predict we'll have. * sure it's enough for all the upcoming behaviors we predict we'll have.
* But if we have too many, close the not-so-useful ones. * But put an upper bound on the total number of circuits.
*/ */
static void static void
circuit_predict_and_launch_new(void) circuit_predict_and_launch_new(void)
@ -517,6 +527,19 @@ circuit_predict_and_launch_new(void)
circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags); circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags);
return; return;
} }
/* Finally, check to see if we still need more circuits to learn
* a good build timeout. But if we're close to our max number we
* want, don't do another -- we want to leave a few slots open so
* we can still build circuits preemptively as needed. */
if (num < MAX_UNUSED_OPEN_CIRCUITS-2 &&
circuit_build_times_needs_circuits_now(&circ_times)) {
flags = CIRCLAUNCH_NEED_CAPACITY;
log_info(LD_CIRC,
"Have %d clean circs need another buildtime test circ.", num);
circuit_launch_by_router(CIRCUIT_PURPOSE_C_GENERAL, NULL, flags);
return;
}
} }
/** Build a new test circuit every 5 minutes */ /** Build a new test circuit every 5 minutes */
@ -624,6 +647,11 @@ circuit_detach_stream(circuit_t *circ, edge_connection_t *conn)
tor_fragile_assert(); tor_fragile_assert();
} }
/** If we haven't yet decided on a good timeout value for circuit
* building, we close idles circuits aggressively so we can get more
* data points. */
#define IDLE_TIMEOUT_WHILE_LEARNING (10*60)
/** Find each circuit that has been unused for too long, or dirty /** Find each circuit that has been unused for too long, or dirty
* for too long and has no streams on it: mark it for close. * for too long and has no streams on it: mark it for close.
*/ */
@ -631,7 +659,15 @@ static void
circuit_expire_old_circuits(time_t now) circuit_expire_old_circuits(time_t now)
{ {
circuit_t *circ; circuit_t *circ;
time_t cutoff = now - get_options()->CircuitIdleTimeout; time_t cutoff;
if (circuit_build_times_needs_circuits(&circ_times)) {
/* Circuits should be shorter lived if we need more of them
* for learning a good build timeout */
cutoff = now - IDLE_TIMEOUT_WHILE_LEARNING;
} else {
cutoff = now - get_options()->CircuitIdleTimeout;
}
for (circ = global_circuitlist; circ; circ = circ->next) { for (circ = global_circuitlist; circ; circ = circ->next) {
if (circ->marked_for_close || ! CIRCUIT_IS_ORIGIN(circ)) if (circ->marked_for_close || ! CIRCUIT_IS_ORIGIN(circ))
@ -724,17 +760,12 @@ circuit_testing_opened(origin_circuit_t *circ)
static void static void
circuit_testing_failed(origin_circuit_t *circ, int at_last_hop) circuit_testing_failed(origin_circuit_t *circ, int at_last_hop)
{ {
routerinfo_t *me = router_get_my_routerinfo();
if (server_mode(get_options()) && check_whether_orport_reachable()) if (server_mode(get_options()) && check_whether_orport_reachable())
return; return;
if (!me)
return;
log_info(LD_GENERAL, log_info(LD_GENERAL,
"Our testing circuit (to see if your ORPort is reachable) " "Our testing circuit (to see if your ORPort is reachable) "
"has failed. I'll try again later."); "has failed. I'll try again later.");
control_event_server_status(LOG_WARN, "REACHABILITY_FAILED ORADDRESS=%s:%d",
me->address, me->or_port);
/* These aren't used yet. */ /* These aren't used yet. */
(void)circ; (void)circ;
@ -811,6 +842,9 @@ circuit_build_failed(origin_circuit_t *circ)
"(%s:%d). I'm going to try to rotate to a better connection.", "(%s:%d). I'm going to try to rotate to a better connection.",
n_conn->_base.address, n_conn->_base.port); n_conn->_base.address, n_conn->_base.port);
n_conn->is_bad_for_new_circs = 1; n_conn->is_bad_for_new_circs = 1;
} else {
log_info(LD_OR,
"Our circuit died before the first hop with no connection");
} }
if (n_conn_id) { if (n_conn_id) {
entry_guard_register_connect_status(n_conn_id, 0, 1, time(NULL)); entry_guard_register_connect_status(n_conn_id, 0, 1, time(NULL));

View File

@ -575,7 +575,7 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn)
/* Consider all the other addresses; if any matches, this connection is /* Consider all the other addresses; if any matches, this connection is
* "canonical." */ * "canonical." */
tor_addr_t addr; tor_addr_t addr;
const char *next = decode_address_from_payload(&addr, cp, end-cp); const char *next = decode_address_from_payload(&addr, cp, (int)(end-cp));
if (next == NULL) { if (next == NULL) {
log_fn(LOG_PROTOCOL_WARN, LD_OR, log_fn(LOG_PROTOCOL_WARN, LD_OR,
"Bad address in netinfo cell; closing connection."); "Bad address in netinfo cell; closing connection.");
@ -610,9 +610,11 @@ command_process_netinfo_cell(cell_t *cell, or_connection_t *conn)
conn->_base.address, (int)conn->_base.port, conn->_base.address, (int)conn->_base.port,
apparent_skew>0 ? "ahead" : "behind", dbuf, apparent_skew>0 ? "ahead" : "behind", dbuf,
apparent_skew>0 ? "behind" : "ahead"); apparent_skew>0 ? "behind" : "ahead");
control_event_general_status(LOG_WARN, if (severity == LOG_WARN) /* only tell the controller if an authority */
"CLOCK_SKEW SKEW=%ld SOURCE=OR:%s:%d", control_event_general_status(LOG_WARN,
apparent_skew, conn->_base.address, conn->_base.port); "CLOCK_SKEW SKEW=%ld SOURCE=OR:%s:%d",
apparent_skew,
conn->_base.address, conn->_base.port);
} }
/* XXX maybe act on my_apparent_addr, if the source is sufficiently /* XXX maybe act on my_apparent_addr, if the source is sufficiently

View File

@ -164,10 +164,11 @@ static config_var_t _option_vars[] = {
V(BridgeRecordUsageByCountry, BOOL, "1"), V(BridgeRecordUsageByCountry, BOOL, "1"),
V(BridgeRelay, BOOL, "0"), V(BridgeRelay, BOOL, "0"),
V(CellStatistics, BOOL, "0"), V(CellStatistics, BOOL, "0"),
V(CircuitBuildTimeout, INTERVAL, "1 minute"), V(CircuitBuildTimeout, INTERVAL, "0"),
V(CircuitIdleTimeout, INTERVAL, "1 hour"), V(CircuitIdleTimeout, INTERVAL, "1 hour"),
V(ClientDNSRejectInternalAddresses, BOOL,"1"), V(ClientDNSRejectInternalAddresses, BOOL,"1"),
V(ClientOnly, BOOL, "0"), V(ClientOnly, BOOL, "0"),
V(ConsensusParams, STRING, NULL),
V(ConnLimit, UINT, "1000"), V(ConnLimit, UINT, "1000"),
V(ConstrainedSockets, BOOL, "0"), V(ConstrainedSockets, BOOL, "0"),
V(ConstrainedSockSize, MEMUNIT, "8192"), V(ConstrainedSockSize, MEMUNIT, "8192"),
@ -408,6 +409,10 @@ static config_var_t _state_vars[] = {
V(LastRotatedOnionKey, ISOTIME, NULL), V(LastRotatedOnionKey, ISOTIME, NULL),
V(LastWritten, ISOTIME, NULL), V(LastWritten, ISOTIME, NULL),
V(TotalBuildTimes, UINT, NULL),
VAR("CircuitBuildTimeBin", LINELIST_S, BuildtimeHistogram, NULL),
VAR("BuildtimeHistogram", LINELIST_V, BuildtimeHistogram, NULL),
{ NULL, CONFIG_TYPE_OBSOLETE, 0, NULL } { NULL, CONFIG_TYPE_OBSOLETE, 0, NULL }
}; };
@ -596,6 +601,10 @@ static config_var_description_t options_description[] = {
/* Hidden service options: HiddenService: dir,excludenodes, nodes, /* Hidden service options: HiddenService: dir,excludenodes, nodes,
* options, port. PublishHidServDescriptor */ * options, port. PublishHidServDescriptor */
/* Circuit build time histogram options */
{ "CircuitBuildTimeBin", "Histogram of recent circuit build times"},
{ "TotalBuildTimes", "Total number of buildtimes in histogram"},
/* Nonpersistent options: __LeaveStreamsUnattached, __AllDirActionsPrivate */ /* Nonpersistent options: __LeaveStreamsUnattached, __AllDirActionsPrivate */
{ NULL, NULL }, { NULL, NULL },
}; };
@ -1506,7 +1515,10 @@ expand_abbrev(config_format_t *fmt, const char *option, int command_line,
fmt->abbrevs[i].abbreviated, fmt->abbrevs[i].abbreviated,
fmt->abbrevs[i].full); fmt->abbrevs[i].full);
} }
return fmt->abbrevs[i].full; /* Keep going through the list in case we want to rewrite it more.
* (We could imagine recursing here, but I don't want to get the
* user into an infinite loop if we craft our list wrong.) */
option = fmt->abbrevs[i].full;
} }
} }
return option; return option;
@ -2521,7 +2533,8 @@ is_local_addr(const tor_addr_t *addr)
* the same /24 as last_resolved_addr will be the same as checking whether * the same /24 as last_resolved_addr will be the same as checking whether
* it was on net 0, which is already done by is_internal_IP. * it was on net 0, which is already done by is_internal_IP.
*/ */
if ((last_resolved_addr & 0xffffff00ul) == (ip & 0xffffff00ul)) if ((last_resolved_addr & (uint32_t)0xffffff00ul)
== (ip & (uint32_t)0xffffff00ul))
return 1; return 1;
} }
return 0; return 0;
@ -2909,11 +2922,6 @@ compute_publishserverdescriptor(or_options_t *options)
/** Highest allowable value for RendPostPeriod. */ /** Highest allowable value for RendPostPeriod. */
#define MAX_DIR_PERIOD (MIN_ONION_KEY_LIFETIME/2) #define MAX_DIR_PERIOD (MIN_ONION_KEY_LIFETIME/2)
/** Lowest allowable value for CircuitBuildTimeout; values too low will
* increase network load because of failing connections being retried, and
* might prevent users from connecting to the network at all. */
#define MIN_CIRCUIT_BUILD_TIMEOUT 30
/** Lowest allowable value for MaxCircuitDirtiness; if this is too low, Tor /** Lowest allowable value for MaxCircuitDirtiness; if this is too low, Tor
* will generate too many circuits and potentially overload the network. */ * will generate too many circuits and potentially overload the network. */
#define MIN_MAX_CIRCUIT_DIRTINESS 10 #define MIN_MAX_CIRCUIT_DIRTINESS 10
@ -3360,12 +3368,6 @@ options_validate(or_options_t *old_options, or_options_t *options,
options->RendPostPeriod = MAX_DIR_PERIOD; options->RendPostPeriod = MAX_DIR_PERIOD;
} }
if (options->CircuitBuildTimeout < MIN_CIRCUIT_BUILD_TIMEOUT) {
log(LOG_WARN, LD_CONFIG, "CircuitBuildTimeout option is too short; "
"raising to %d seconds.", MIN_CIRCUIT_BUILD_TIMEOUT);
options->CircuitBuildTimeout = MIN_CIRCUIT_BUILD_TIMEOUT;
}
if (options->MaxCircuitDirtiness < MIN_MAX_CIRCUIT_DIRTINESS) { if (options->MaxCircuitDirtiness < MIN_MAX_CIRCUIT_DIRTINESS) {
log(LOG_WARN, LD_CONFIG, "MaxCircuitDirtiness option is too short; " log(LOG_WARN, LD_CONFIG, "MaxCircuitDirtiness option is too short; "
"raising to %d seconds.", MIN_MAX_CIRCUIT_DIRTINESS); "raising to %d seconds.", MIN_MAX_CIRCUIT_DIRTINESS);
@ -4280,7 +4282,7 @@ options_init_from_string(const char *cf,
err: err:
config_free(&options_format, newoptions); config_free(&options_format, newoptions);
if (*msg) { if (*msg) {
int len = strlen(*msg)+256; int len = (int)strlen(*msg)+256;
char *newmsg = tor_malloc(len); char *newmsg = tor_malloc(len);
tor_snprintf(newmsg, len, "Failed to parse/validate config: %s", *msg); tor_snprintf(newmsg, len, "Failed to parse/validate config: %s", *msg);
@ -4860,35 +4862,28 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok)
uint64_t v = 0; uint64_t v = 0;
double d = 0; double d = 0;
int use_float = 0; int use_float = 0;
char *cp;
smartlist_t *sl;
tor_assert(ok); tor_assert(ok);
sl = smartlist_create();
smartlist_split_string(sl, val, NULL,
SPLIT_SKIP_SPACE|SPLIT_IGNORE_BLANK, 3);
if (smartlist_len(sl) < 1 || smartlist_len(sl) > 2) { v = tor_parse_uint64(val, 10, 0, UINT64_MAX, ok, &cp);
*ok = 0; if (!*ok || (cp && *cp == '.')) {
goto done; d = tor_parse_double(val, 0, UINT64_MAX, ok, &cp);
} if (!*ok)
v = tor_parse_uint64(smartlist_get(sl,0), 10, 0, UINT64_MAX, ok, NULL);
if (!*ok) {
int r = sscanf(smartlist_get(sl,0), "%lf", &d);
if (r == 0 || d < 0)
goto done; goto done;
use_float = 1; use_float = 1;
} }
if (smartlist_len(sl) == 1) { if (!cp) {
*ok = 1; *ok = 1;
v = use_float ? DBL_TO_U64(d) : v; v = use_float ? DBL_TO_U64(d) : v;
goto done; goto done;
} }
cp = (char*) eat_whitespace(cp);
for ( ;u->unit;++u) { for ( ;u->unit;++u) {
if (!strcasecmp(u->unit, smartlist_get(sl,1))) { if (!strcasecmp(u->unit, cp)) {
if (use_float) if (use_float)
v = u->multiplier * d; v = u->multiplier * d;
else else
@ -4897,11 +4892,9 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok)
goto done; goto done;
} }
} }
log_warn(LD_CONFIG, "Unknown unit '%s'.", (char*)smartlist_get(sl,1)); log_warn(LD_CONFIG, "Unknown unit '%s'.", cp);
*ok = 0; *ok = 0;
done: done:
SMARTLIST_FOREACH(sl, char *, cp, tor_free(cp));
smartlist_free(sl);
if (*ok) if (*ok)
return v; return v;
@ -4916,7 +4909,8 @@ config_parse_units(const char *val, struct unit_table_t *u, int *ok)
static uint64_t static uint64_t
config_parse_memunit(const char *s, int *ok) config_parse_memunit(const char *s, int *ok)
{ {
return config_parse_units(s, memory_units, ok); uint64_t u = config_parse_units(s, memory_units, ok);
return u;
} }
/** Parse a string in the format "number unit", where unit is a unit of time. /** Parse a string in the format "number unit", where unit is a unit of time.
@ -5066,6 +5060,10 @@ or_state_set(or_state_t *new_state)
log_warn(LD_GENERAL,"Unparseable bandwidth history state: %s",err); log_warn(LD_GENERAL,"Unparseable bandwidth history state: %s",err);
tor_free(err); tor_free(err);
} }
if (circuit_build_times_parse_state(&circ_times, global_state, &err) < 0) {
log_warn(LD_GENERAL,"%s",err);
tor_free(err);
}
} }
/** Reload the persistent state from disk, generating a new state as needed. /** Reload the persistent state from disk, generating a new state as needed.
@ -5198,6 +5196,7 @@ or_state_save(time_t now)
* to avoid redundant writes. */ * to avoid redundant writes. */
entry_guards_update_state(global_state); entry_guards_update_state(global_state);
rep_hist_update_state(global_state); rep_hist_update_state(global_state);
circuit_build_times_update_state(&circ_times, global_state);
if (accounting_is_enabled(get_options())) if (accounting_is_enabled(get_options()))
accounting_run_housekeeping(now); accounting_run_housekeeping(now);

View File

@ -2346,7 +2346,7 @@ loop_again:
return -1; return -1;
} }
if (conn->linked_conn) { if (conn->linked_conn) {
/* The other side's handle_write will never actually get called, so /* The other side's handle_write() will never actually get called, so
* we need to invoke the appropriate callbacks ourself. */ * we need to invoke the appropriate callbacks ourself. */
connection_t *linked = conn->linked_conn; connection_t *linked = conn->linked_conn;
@ -2363,7 +2363,7 @@ loop_again:
if (!buf_datalen(linked->outbuf) && conn->active_on_link) if (!buf_datalen(linked->outbuf) && conn->active_on_link)
connection_stop_reading_from_linked_conn(conn); connection_stop_reading_from_linked_conn(conn);
} }
/* If we hit the EOF, call connection_reached_eof. */ /* If we hit the EOF, call connection_reached_eof(). */
if (!conn->marked_for_close && if (!conn->marked_for_close &&
conn->inbuf_reached_eof && conn->inbuf_reached_eof &&
connection_reached_eof(conn) < 0) { connection_reached_eof(conn) < 0) {
@ -2589,7 +2589,7 @@ connection_handle_write(connection_t *conn, int force)
return 0; /* do nothing */ return 0; /* do nothing */
if (conn->in_flushed_some) { if (conn->in_flushed_some) {
log_warn(LD_BUG, "called recursively from inside conn->in_flushed_some()"); log_warn(LD_BUG, "called recursively from inside conn->in_flushed_some");
return 0; return 0;
} }
@ -2678,8 +2678,8 @@ connection_handle_write(connection_t *conn, int force)
if (!connection_is_reading(conn)) { if (!connection_is_reading(conn)) {
connection_stop_writing(conn); connection_stop_writing(conn);
conn->write_blocked_on_bw = 1; conn->write_blocked_on_bw = 1;
/* we'll start reading again when the next second arrives, /* we'll start reading again when we get more tokens in our
* and then also start writing again. * read bucket; then we'll start writing again too.
*/ */
} }
/* else no problem, we're already reading */ /* else no problem, we're already reading */
@ -3067,7 +3067,7 @@ client_check_address_changed(int sock)
return; return;
} }
/* Okay. If we've used this address previously, we're okay. */ /* If we've used this address previously, we're okay. */
ip_out = ntohl(out_addr.sin_addr.s_addr); ip_out = ntohl(out_addr.sin_addr.s_addr);
SMARTLIST_FOREACH(outgoing_addrs, uint32_t*, ip_ptr, SMARTLIST_FOREACH(outgoing_addrs, uint32_t*, ip_ptr,
if (*ip_ptr == ip_out) return; if (*ip_ptr == ip_out) return;

View File

@ -1036,6 +1036,8 @@ connection_tls_finish_handshake(or_connection_t *conn)
digest_rcvd) < 0) digest_rcvd) < 0)
return -1; return -1;
circuit_build_times_network_is_live(&circ_times);
if (tor_tls_used_v1_handshake(conn->tls)) { if (tor_tls_used_v1_handshake(conn->tls)) {
conn->link_proto = 1; conn->link_proto = 1;
if (!started_here) { if (!started_here) {
@ -1087,6 +1089,7 @@ connection_or_set_state_open(or_connection_t *conn)
control_event_or_conn_status(conn, OR_CONN_EVENT_CONNECTED, 0); control_event_or_conn_status(conn, OR_CONN_EVENT_CONNECTED, 0);
if (started_here) { if (started_here) {
circuit_build_times_network_is_live(&circ_times);
rep_hist_note_connect_succeeded(conn->identity_digest, now); rep_hist_note_connect_succeeded(conn->identity_digest, now);
if (entry_guard_register_connect_status(conn->identity_digest, if (entry_guard_register_connect_status(conn->identity_digest,
1, 0, now) < 0) { 1, 0, now) < 0) {
@ -1187,6 +1190,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn)
if (connection_fetch_var_cell_from_buf(conn, &var_cell)) { if (connection_fetch_var_cell_from_buf(conn, &var_cell)) {
if (!var_cell) if (!var_cell)
return 0; /* not yet. */ return 0; /* not yet. */
circuit_build_times_network_is_live(&circ_times);
command_process_var_cell(var_cell, conn); command_process_var_cell(var_cell, conn);
var_cell_free(var_cell); var_cell_free(var_cell);
} else { } else {
@ -1196,6 +1200,7 @@ connection_or_process_cells_from_inbuf(or_connection_t *conn)
available? */ available? */
return 0; /* not yet */ return 0; /* not yet */
circuit_build_times_network_is_live(&circ_times);
connection_fetch_from_buf(buf, CELL_NETWORK_SIZE, TO_CONN(conn)); connection_fetch_from_buf(buf, CELL_NETWORK_SIZE, TO_CONN(conn));
/* retrieve cell info from buf (create the host-order struct from the /* retrieve cell info from buf (create the host-order struct from the

View File

@ -1696,7 +1696,11 @@ getinfo_helper_events(control_connection_t *control_conn,
*answer = tor_strdup(has_completed_circuit ? "1" : "0"); *answer = tor_strdup(has_completed_circuit ? "1" : "0");
} else if (!strcmp(question, "status/enough-dir-info")) { } else if (!strcmp(question, "status/enough-dir-info")) {
*answer = tor_strdup(router_have_minimum_dir_info() ? "1" : "0"); *answer = tor_strdup(router_have_minimum_dir_info() ? "1" : "0");
} else if (!strcmp(question, "status/good-server-descriptor")) { } else if (!strcmp(question, "status/good-server-descriptor") ||
!strcmp(question, "status/accepted-server-descriptor")) {
/* They're equivalent for now, until we can figure out how to make
* good-server-descriptor be what we want. See comment in
* control-spec.txt. */
*answer = tor_strdup(directories_have_accepted_server_descriptor() *answer = tor_strdup(directories_have_accepted_server_descriptor()
? "1" : "0"); ? "1" : "0");
} else if (!strcmp(question, "status/reachability-succeeded/or")) { } else if (!strcmp(question, "status/reachability-succeeded/or")) {
@ -2495,7 +2499,7 @@ handle_control_resolve(control_connection_t *conn, uint32_t len,
int is_reverse = 0; int is_reverse = 0;
(void) len; /* body is nul-terminated; it's safe to ignore the length */ (void) len; /* body is nul-terminated; it's safe to ignore the length */
if (!(conn->event_mask & (1L<<EVENT_ADDRMAP))) { if (!(conn->event_mask & ((uint32_t)1L<<EVENT_ADDRMAP))) {
log_warn(LD_CONTROL, "Controller asked us to resolve an address, but " log_warn(LD_CONTROL, "Controller asked us to resolve an address, but "
"isn't listening for ADDRMAP events. It probably won't see " "isn't listening for ADDRMAP events. It probably won't see "
"the answer."); "the answer.");

View File

@ -554,11 +554,6 @@ void
connection_dir_request_failed(dir_connection_t *conn) connection_dir_request_failed(dir_connection_t *conn)
{ {
if (directory_conn_is_self_reachability_test(conn)) { if (directory_conn_is_self_reachability_test(conn)) {
routerinfo_t *me = router_get_my_routerinfo();
if (me)
control_event_server_status(LOG_WARN,
"REACHABILITY_FAILED DIRADDRESS=%s:%d",
me->address, me->dir_port);
return; /* this was a test fetch. don't retry. */ return; /* this was a test fetch. don't retry. */
} }
if (entry_list_can_grow(get_options())) if (entry_list_can_grow(get_options()))
@ -886,7 +881,7 @@ static char *
directory_get_consensus_url(int supports_conditional_consensus) directory_get_consensus_url(int supports_conditional_consensus)
{ {
char *url; char *url;
int len; size_t len;
if (supports_conditional_consensus) { if (supports_conditional_consensus) {
char *authority_id_list; char *authority_id_list;
@ -2337,7 +2332,7 @@ client_likes_consensus(networkstatus_t *v, const char *want_url)
need_at_least = smartlist_len(want_authorities)/2+1; need_at_least = smartlist_len(want_authorities)/2+1;
SMARTLIST_FOREACH(want_authorities, const char *, d, { SMARTLIST_FOREACH(want_authorities, const char *, d, {
char want_digest[DIGEST_LEN]; char want_digest[DIGEST_LEN];
int want_len = strlen(d)/2; size_t want_len = strlen(d)/2;
if (want_len > DIGEST_LEN) if (want_len > DIGEST_LEN)
want_len = DIGEST_LEN; want_len = DIGEST_LEN;

View File

@ -1864,7 +1864,7 @@ version_from_platform(const char *platform)
* NS_V2 - Output an entry suitable for a V2 NS opinion document * NS_V2 - Output an entry suitable for a V2 NS opinion document
* NS_V3_CONSENSUS - Output the first portion of a V3 NS consensus entry * NS_V3_CONSENSUS - Output the first portion of a V3 NS consensus entry
* NS_V3_VOTE - Output a complete V3 NS vote * NS_V3_VOTE - Output a complete V3 NS vote
* NS_CONTROL_PORT - Output a NS docunent for the control port * NS_CONTROL_PORT - Output a NS document for the control port
*/ */
int int
routerstatus_format_entry(char *buf, size_t buf_len, routerstatus_format_entry(char *buf, size_t buf_len,
@ -2324,7 +2324,7 @@ measured_bw_line_apply(measured_bw_line_t *parsed_line,
if (rs) { if (rs) {
rs->has_measured_bw = 1; rs->has_measured_bw = 1;
rs->measured_bw = parsed_line->bw; rs->measured_bw = (uint32_t)parsed_line->bw;
} else { } else {
log_info(LD_DIRSERV, "Node ID %s not found in routerstatus list", log_info(LD_DIRSERV, "Node ID %s not found in routerstatus list",
parsed_line->node_hex); parsed_line->node_hex);
@ -2553,6 +2553,13 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key,
} }
smartlist_sort_strings(v3_out->known_flags); smartlist_sort_strings(v3_out->known_flags);
if (options->ConsensusParams) {
v3_out->net_params = smartlist_create();
smartlist_split_string(v3_out->net_params,
options->ConsensusParams, NULL, 0, 0);
smartlist_sort_strings(v3_out->net_params);
}
voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t)); voter = tor_malloc_zero(sizeof(networkstatus_voter_info_t));
voter->nickname = tor_strdup(options->Nickname); voter->nickname = tor_strdup(options->Nickname);
memcpy(voter->identity_digest, identity_digest, DIGEST_LEN); memcpy(voter->identity_digest, identity_digest, DIGEST_LEN);

View File

@ -24,7 +24,9 @@ static int dirvote_publish_consensus(void);
static char *make_consensus_method_list(int low, int high); static char *make_consensus_method_list(int low, int high);
/** The highest consensus method that we currently support. */ /** The highest consensus method that we currently support. */
#define MAX_SUPPORTED_CONSENSUS_METHOD 6 #define MAX_SUPPORTED_CONSENSUS_METHOD 7
#define MIN_METHOD_FOR_PARAMS 7
/* ===== /* =====
* Voting * Voting
@ -97,6 +99,7 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
char fu[ISO_TIME_LEN+1]; char fu[ISO_TIME_LEN+1];
char vu[ISO_TIME_LEN+1]; char vu[ISO_TIME_LEN+1];
char *flags = smartlist_join_strings(v3_ns->known_flags, " ", 0, NULL); char *flags = smartlist_join_strings(v3_ns->known_flags, " ", 0, NULL);
char *params;
authority_cert_t *cert = v3_ns->cert; authority_cert_t *cert = v3_ns->cert;
char *methods = char *methods =
make_consensus_method_list(1, MAX_SUPPORTED_CONSENSUS_METHOD); make_consensus_method_list(1, MAX_SUPPORTED_CONSENSUS_METHOD);
@ -105,6 +108,11 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
format_iso_time(fu, v3_ns->fresh_until); format_iso_time(fu, v3_ns->fresh_until);
format_iso_time(vu, v3_ns->valid_until); format_iso_time(vu, v3_ns->valid_until);
if (v3_ns->net_params)
params = smartlist_join_strings(v3_ns->net_params, " ", 0, NULL);
else
params = tor_strdup("");
tor_assert(cert); tor_assert(cert);
tor_snprintf(status, len, tor_snprintf(status, len,
"network-status-version 3\n" "network-status-version 3\n"
@ -117,6 +125,7 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
"voting-delay %d %d\n" "voting-delay %d %d\n"
"%s" /* versions */ "%s" /* versions */
"known-flags %s\n" "known-flags %s\n"
"params %s\n"
"dir-source %s %s %s %s %d %d\n" "dir-source %s %s %s %s %d %d\n"
"contact %s\n", "contact %s\n",
v3_ns->type == NS_TYPE_VOTE ? "vote" : "opinion", v3_ns->type == NS_TYPE_VOTE ? "vote" : "opinion",
@ -125,9 +134,11 @@ format_networkstatus_vote(crypto_pk_env_t *private_signing_key,
v3_ns->vote_seconds, v3_ns->dist_seconds, v3_ns->vote_seconds, v3_ns->dist_seconds,
version_lines, version_lines,
flags, flags,
params,
voter->nickname, fingerprint, voter->address, voter->nickname, fingerprint, voter->address,
ipaddr, voter->dir_port, voter->or_port, voter->contact); ipaddr, voter->dir_port, voter->or_port, voter->contact);
tor_free(params);
tor_free(flags); tor_free(flags);
tor_free(methods); tor_free(methods);
outp = status + strlen(status); outp = status + strlen(status);
@ -507,6 +518,89 @@ compute_consensus_versions_list(smartlist_t *lst, int n_versioning)
return result; return result;
} }
/** Helper: given a list of valid networkstatus_t, return a new string
* containing the contents of the consensus network parameter set.
*/
/* private */ char *
dirvote_compute_params(smartlist_t *votes)
{
int i;
int32_t *vals;
int cur_param_len;
const char *cur_param;
const char *eq;
char *result;
const int n_votes = smartlist_len(votes);
smartlist_t *output;
smartlist_t *param_list = smartlist_create();
/* We require that the parameter lists in the votes are well-formed: that
is, that their keywords are unique and sorted, and that their values are
between INT32_MIN and INT32_MAX inclusive. This should be guaranteed by
the parsing code. */
vals = tor_malloc(sizeof(int)*n_votes);
SMARTLIST_FOREACH_BEGIN(votes, networkstatus_t *, v) {
if (!v->net_params)
continue;
smartlist_add_all(param_list, v->net_params);
} SMARTLIST_FOREACH_END(v);
if (smartlist_len(param_list) == 0) {
tor_free(vals);
smartlist_free(param_list);
return NULL;
}
smartlist_sort_strings(param_list);
i = 0;
cur_param = smartlist_get(param_list, 0);
eq = strchr(cur_param, '=');
tor_assert(eq);
cur_param_len = (int)(eq+1 - cur_param);
output = smartlist_create();
SMARTLIST_FOREACH_BEGIN(param_list, const char *, param) {
const char *next_param;
int ok=0;
eq = strchr(param, '=');
tor_assert(i<n_votes);
vals[i++] = (int32_t)
tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL);
tor_assert(ok);
if (param_sl_idx+1 == smartlist_len(param_list))
next_param = NULL;
else
next_param = smartlist_get(param_list, param_sl_idx+1);
if (!next_param || strncmp(next_param, param, cur_param_len)) {
/* We've reached the end of a series. */
int32_t median = median_int32(vals, i);
char *out_string = tor_malloc(64+cur_param_len);
memcpy(out_string, param, cur_param_len);
tor_snprintf(out_string+cur_param_len,64, "%ld", (long)median);
smartlist_add(output, out_string);
i = 0;
if (next_param) {
eq = strchr(next_param, '=');
cur_param_len = (int)(eq+1 - next_param);
}
}
} SMARTLIST_FOREACH_END(param);
result = smartlist_join_strings(output, " ", 0, NULL);
SMARTLIST_FOREACH(output, char *, cp, tor_free(cp));
smartlist_free(output);
smartlist_free(param_list);
tor_free(vals);
return result;
}
/** Given a list of vote networkstatus_t in <b>votes</b>, our public /** Given a list of vote networkstatus_t in <b>votes</b>, our public
* authority <b>identity_key</b>, our private authority <b>signing_key</b>, * authority <b>identity_key</b>, our private authority <b>signing_key</b>,
* and the number of <b>total_authorities</b> that we believe exist in our * and the number of <b>total_authorities</b> that we believe exist in our
@ -659,6 +753,15 @@ networkstatus_compute_consensus(smartlist_t *votes,
tor_free(flaglist); tor_free(flaglist);
} }
if (consensus_method >= MIN_METHOD_FOR_PARAMS) {
char *params = dirvote_compute_params(votes);
if (params) {
smartlist_add(chunks, tor_strdup("params "));
smartlist_add(chunks, params);
smartlist_add(chunks, tor_strdup("\n"));
}
}
/* Sort the votes. */ /* Sort the votes. */
smartlist_sort(votes, _compare_votes_by_authority_id); smartlist_sort(votes, _compare_votes_by_authority_id);
/* Add the authority sections. */ /* Add the authority sections. */

View File

@ -2332,7 +2332,7 @@ out1:
/* exported function */ /* exported function */
int int
evdns_nameserver_add(unsigned long int address) { evdns_nameserver_add(uint32_t address) {
struct sockaddr_in sin; struct sockaddr_in sin;
memset(&sin, 0, sizeof(sin)); memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET; sin.sin_family = AF_INET;
@ -2363,13 +2363,13 @@ evdns_nameserver_ip_add(const char *ip_as_string) {
cp = strchr(ip_as_string, ':'); cp = strchr(ip_as_string, ':');
if (*ip_as_string == '[') { if (*ip_as_string == '[') {
int len; size_t len;
if (!(cp = strchr(ip_as_string, ']'))) { if (!(cp = strchr(ip_as_string, ']'))) {
log(EVDNS_LOG_DEBUG, "Nameserver missing closing ]"); log(EVDNS_LOG_DEBUG, "Nameserver missing closing ]");
return 4; return 4;
} }
len = cp-(ip_as_string + 1); len = cp-(ip_as_string + 1);
if (len > (int)sizeof(buf)-1) { if (len > sizeof(buf)-1) {
log(EVDNS_LOG_DEBUG, "[Nameserver] does not fit in buffer."); log(EVDNS_LOG_DEBUG, "[Nameserver] does not fit in buffer.");
return 4; return 4;
} }

View File

@ -112,7 +112,7 @@
* *
* API reference: * API reference:
* *
* int evdns_nameserver_add(unsigned long int address) * int evdns_nameserver_add(uint32_t address)
* Add a nameserver. The address should be an IP address in * Add a nameserver. The address should be an IP address in
* network byte order. The type of address is chosen so that * network byte order. The type of address is chosen so that
* it matches in_addr.s_addr. * it matches in_addr.s_addr.
@ -258,7 +258,7 @@ typedef void (*evdns_callback_type) (int result, char type, int count, int ttl,
int evdns_init(void); int evdns_init(void);
void evdns_shutdown(int fail_requests); void evdns_shutdown(int fail_requests);
const char *evdns_err_to_string(int err); const char *evdns_err_to_string(int err);
int evdns_nameserver_add(unsigned long int address); int evdns_nameserver_add(uint32_t address);
int evdns_count_nameservers(void); int evdns_count_nameservers(void);
int evdns_clear_nameservers_and_suspend(void); int evdns_clear_nameservers_and_suspend(void);
int evdns_resume(void); int evdns_resume(void);

View File

@ -340,7 +340,7 @@ geoip_determine_shares(time_t now)
((double) (now - last_time_determined_shares)); ((double) (now - last_time_determined_shares));
v3_share_times_seconds += v3_share * v3_share_times_seconds += v3_share *
((double) (now - last_time_determined_shares)); ((double) (now - last_time_determined_shares));
share_seconds += now - last_time_determined_shares; share_seconds += (int)(now - last_time_determined_shares);
} }
last_time_determined_shares = now; last_time_determined_shares = now;
} }
@ -768,7 +768,7 @@ geoip_get_dirreq_history(geoip_client_action_t action,
time_diff = 1; /* Avoid DIV/0; "instant" answers are impossible time_diff = 1; /* Avoid DIV/0; "instant" answers are impossible
* by law of nature or something, but a milisecond * by law of nature or something, but a milisecond
* is a bit greater than "instantly" */ * is a bit greater than "instantly" */
bytes_per_second = 1000 * ent->response_size / time_diff; bytes_per_second = (uint32_t)(1000 * ent->response_size / time_diff);
dltimes[ent_sl_idx] = bytes_per_second; dltimes[ent_sl_idx] = bytes_per_second;
} SMARTLIST_FOREACH_END(ent); } SMARTLIST_FOREACH_END(ent);
median_uint32(dltimes, complete); /* sorts as a side effect. */ median_uint32(dltimes, complete); /* sorts as a side effect. */

View File

@ -1240,17 +1240,26 @@ second_elapsed_callback(int fd, short event, void *args)
TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) {
/* every 20 minutes, check and complain if necessary */ /* every 20 minutes, check and complain if necessary */
routerinfo_t *me = router_get_my_routerinfo(); routerinfo_t *me = router_get_my_routerinfo();
if (me && !check_whether_orport_reachable()) if (me && !check_whether_orport_reachable()) {
log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that " log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that "
"its ORPort is reachable. Please check your firewalls, ports, " "its ORPort is reachable. Please check your firewalls, ports, "
"address, /etc/hosts file, etc.", "address, /etc/hosts file, etc.",
me->address, me->or_port); me->address, me->or_port);
if (me && !check_whether_dirport_reachable()) control_event_server_status(LOG_WARN,
"REACHABILITY_FAILED ORADDRESS=%s:%d",
me->address, me->or_port);
}
if (me && !check_whether_dirport_reachable()) {
log_warn(LD_CONFIG, log_warn(LD_CONFIG,
"Your server (%s:%d) has not managed to confirm that its " "Your server (%s:%d) has not managed to confirm that its "
"DirPort is reachable. Please check your firewalls, ports, " "DirPort is reachable. Please check your firewalls, ports, "
"address, /etc/hosts file, etc.", "address, /etc/hosts file, etc.",
me->address, me->dir_port); me->address, me->dir_port);
control_event_server_status(LOG_WARN,
"REACHABILITY_FAILED DIRADDRESS=%s:%d",
me->address, me->dir_port);
}
} }
/** If more than this many seconds have elapsed, probably the clock /** If more than this many seconds have elapsed, probably the clock
@ -1653,7 +1662,7 @@ dumpstats(int severity)
{ {
time_t now = time(NULL); time_t now = time(NULL);
time_t elapsed; time_t elapsed;
int rbuf_cap, wbuf_cap, rbuf_len, wbuf_len; size_t rbuf_cap, wbuf_cap, rbuf_len, wbuf_len;
log(severity, LD_GENERAL, "Dumping stats:"); log(severity, LD_GENERAL, "Dumping stats:");
@ -1689,7 +1698,7 @@ dumpstats(int severity)
log(severity, LD_GENERAL, log(severity, LD_GENERAL,
"Conn %d: %d/%d bytes used on OpenSSL read buffer; " "Conn %d: %d/%d bytes used on OpenSSL read buffer; "
"%d/%d bytes used on write buffer.", "%d/%d bytes used on write buffer.",
i, rbuf_len, rbuf_cap, wbuf_len, wbuf_cap); i, (int)rbuf_len, (int)rbuf_cap, (int)wbuf_len, (int)wbuf_cap);
} }
} }
} }

View File

@ -286,6 +286,10 @@ networkstatus_vote_free(networkstatus_t *ns)
SMARTLIST_FOREACH(ns->known_flags, char *, c, tor_free(c)); SMARTLIST_FOREACH(ns->known_flags, char *, c, tor_free(c));
smartlist_free(ns->known_flags); smartlist_free(ns->known_flags);
} }
if (ns->net_params) {
SMARTLIST_FOREACH(ns->net_params, char *, c, tor_free(c));
smartlist_free(ns->net_params);
}
if (ns->supported_methods) { if (ns->supported_methods) {
SMARTLIST_FOREACH(ns->supported_methods, char *, c, tor_free(c)); SMARTLIST_FOREACH(ns->supported_methods, char *, c, tor_free(c));
smartlist_free(ns->supported_methods); smartlist_free(ns->supported_methods);
@ -1889,6 +1893,33 @@ networkstatus_dump_bridge_status_to_file(time_t now)
tor_free(status); tor_free(status);
} }
/** Return the value of a integer parameter from the networkstatus <b>ns</b>
* whose name is <b>param_name</b>. Return <b>default_val</b> if ns is NULL,
* or if it has no parameter called <b>param_name</b>. */
int32_t
networkstatus_get_param(networkstatus_t *ns, const char *param_name,
int32_t default_val)
{
size_t name_len;
if (!ns || !ns->net_params)
return default_val;
name_len = strlen(param_name);
SMARTLIST_FOREACH_BEGIN(ns->net_params, const char *, p) {
if (!strcmpstart(p, param_name) && p[name_len] == '=') {
int ok=0;
long v = tor_parse_long(p+name_len+1, 10, INT32_MIN,
INT32_MAX, &ok, NULL);
if (ok)
return (int32_t) v;
}
} SMARTLIST_FOREACH_END(p);
return default_val;
}
/** If <b>question</b> is a string beginning with "ns/" in a format the /** If <b>question</b> is a string beginning with "ns/" in a format the
* control interface expects for a GETINFO question, set *<b>answer</b> to a * control interface expects for a GETINFO question, set *<b>answer</b> to a
* newly-allocated string containing networkstatus lines for the appropriate * newly-allocated string containing networkstatus lines for the appropriate

View File

@ -1672,6 +1672,10 @@ typedef struct networkstatus_t {
* not listed here, the voter has no opinion on what its value should be. */ * not listed here, the voter has no opinion on what its value should be. */
smartlist_t *known_flags; smartlist_t *known_flags;
/** List of key=value strings for the parameters in this vote or
* consensus, sorted by key. */
smartlist_t *net_params;
/** List of networkstatus_voter_info_t. For a vote, only one element /** List of networkstatus_voter_info_t. For a vote, only one element
* is included. For a consensus, one element is included for every voter * is included. For a consensus, one element is included for every voter
* whose vote contributed to the consensus. */ * whose vote contributed to the consensus. */
@ -1866,9 +1870,9 @@ typedef struct crypt_path_t {
struct crypt_path_t *prev; /**< Link to previous crypt_path_t in the struct crypt_path_t *prev; /**< Link to previous crypt_path_t in the
* circuit. */ * circuit. */
int package_window; /**< How many bytes are we allowed to originate ending int package_window; /**< How many cells are we allowed to originate ending
* at this step? */ * at this step? */
int deliver_window; /**< How many bytes are we willing to deliver originating int deliver_window; /**< How many cells are we willing to deliver originating
* at this step? */ * at this step? */
} crypt_path_t; } crypt_path_t;
@ -1973,6 +1977,7 @@ typedef struct circuit_t {
time_t timestamp_created; /**< When was this circuit created? */ time_t timestamp_created; /**< When was this circuit created? */
time_t timestamp_dirty; /**< When the circuit was first used, or 0 if the time_t timestamp_dirty; /**< When the circuit was first used, or 0 if the
* circuit is clean. */ * circuit is clean. */
struct timeval highres_created; /**< When exactly was the circuit created? */
uint16_t marked_for_close; /**< Should we close this circuit at the end of uint16_t marked_for_close; /**< Should we close this circuit at the end of
* the main loop? (If true, holds the line number * the main loop? (If true, holds the line number
@ -2583,6 +2588,10 @@ typedef struct {
/** Location of bandwidth measurement file */ /** Location of bandwidth measurement file */
char *V3BandwidthsFile; char *V3BandwidthsFile;
/** Authority only: key=value pairs that we add to our networkstatus
* consensus vote on the 'params' line. */
char *ConsensusParams;
/** The length of time that we think an initial consensus should be fresh. /** The length of time that we think an initial consensus should be fresh.
* Only altered on testing networks. */ * Only altered on testing networks. */
int TestingV3AuthInitialVotingInterval; int TestingV3AuthInitialVotingInterval;
@ -2675,6 +2684,10 @@ typedef struct {
int BWHistoryWriteInterval; int BWHistoryWriteInterval;
smartlist_t *BWHistoryWriteValues; smartlist_t *BWHistoryWriteValues;
/** Build time histogram */
config_line_t * BuildtimeHistogram;
uint16_t TotalBuildTimes;
/** What version of Tor wrote this state file? */ /** What version of Tor wrote this state file? */
char *TorVersion; char *TorVersion;
@ -2844,6 +2857,155 @@ void bridges_retry_all(void);
void entry_guards_free_all(void); void entry_guards_free_all(void);
/* Circuit Build Timeout "public" functions and structures. */
/** Maximum quantile to use to generate synthetic timeouts.
* We want to stay a bit short of 1.0, because longtail is
* loooooooooooooooooooooooooooooooooooooooooooooooooooong. */
#define MAX_SYNTHETIC_QUANTILE 0.985
/** Minimum circuits before estimating a timeout */
#define MIN_CIRCUITS_TO_OBSERVE 500
/** Total size of the circuit timeout history to accumulate.
* 5000 is approx 1.5 weeks worth of continual-use circuits. */
#define NCIRCUITS_TO_OBSERVE 5000
/** Width of the histogram bins in milliseconds */
#define BUILDTIME_BIN_WIDTH ((build_time_t)50)
/** Cutoff point on the CDF for our timeout estimation.
* TODO: This should be moved to the consensus */
#define BUILDTIMEOUT_QUANTILE_CUTOFF 0.8
/** A build_time_t is milliseconds */
typedef uint32_t build_time_t;
#define BUILD_TIME_MAX ((build_time_t)(INT32_MAX))
/** Lowest allowable value for CircuitBuildTimeout in milliseconds */
#define BUILD_TIMEOUT_MIN_VALUE (3*1000)
/** Initial circuit build timeout in milliseconds */
#define BUILD_TIMEOUT_INITIAL_VALUE (60*1000)
/** How often in seconds should we build a test circuit */
#define BUILD_TIMES_TEST_FREQUENCY 60
/** Save state every 10 circuits */
#define BUILD_TIMES_SAVE_STATE_EVERY 10
/* Circuit Build Timeout network liveness constants */
/**
* How many circuits count as recent when considering if the
* connection has gone gimpy or changed.
*/
#define RECENT_CIRCUITS 20
/**
* Have we received a cell in the last N circ attempts?
*
* This tells us when to temporarily switch back to
* BUILD_TIMEOUT_INITIAL_VALUE until we start getting cells,
* at which point we switch back to computing the timeout from
* our saved history.
*/
#define NETWORK_NONLIVE_TIMEOUT_COUNT (lround(RECENT_CIRCUITS*0.15))
/**
* This tells us when to toss out the last streak of N timeouts.
*
* If instead we start getting cells, we switch back to computing the timeout
* from our saved history.
*/
#define NETWORK_NONLIVE_DISCARD_COUNT (lround(NETWORK_NONLIVE_TIMEOUT_COUNT*2))
/**
* Maximum count of timeouts that finish the first hop in the past
* RECENT_CIRCUITS before calculating a new timeout.
*
* This tells us to abandon timeout history and set
* the timeout back to BUILD_TIMEOUT_INITIAL_VALUE.
*/
#define MAX_RECENT_TIMEOUT_COUNT (lround(RECENT_CIRCUITS*0.75))
/** Information about the state of our local network connection */
typedef struct {
/** The timestamp we last completed a TLS handshake or received a cell */
time_t network_last_live;
/** If the network is not live, how many timeouts has this caused? */
int nonlive_timeouts;
/** If the network is not live, have we yet discarded our history? */
int nonlive_discarded;
/** Circular array of circuits that have made it to the first hop. Slot is
* 1 if circuit timed out, 0 if circuit succeeded */
int8_t timeouts_after_firsthop[RECENT_CIRCUITS];
/** Index into circular array. */
int after_firsthop_idx;
} network_liveness_t;
/** Structure for circuit build times history */
typedef struct {
/** The circular array of recorded build times in milliseconds */
build_time_t circuit_build_times[NCIRCUITS_TO_OBSERVE];
/** Current index in the circuit_build_times circular array */
int build_times_idx;
/** Total number of build times accumulated. Maxes at NCIRCUITS_TO_OBSERVE */
int total_build_times;
/** Information about the state of our local network connection */
network_liveness_t liveness;
/** Last time we built a circuit. Used to decide to build new test circs */
time_t last_circ_at;
/** Number of timeouts that have happened before estimating pareto
* parameters */
int pre_timeouts;
/** "Minimum" value of our pareto distribution (actually mode) */
build_time_t Xm;
/** alpha exponent for pareto dist. */
double alpha;
/** Have we computed a timeout? */
int have_computed_timeout;
/** The exact value for that timeout in milliseconds */
double timeout_ms;
} circuit_build_times_t;
extern circuit_build_times_t circ_times;
void circuit_build_times_update_state(circuit_build_times_t *cbt,
or_state_t *state);
int circuit_build_times_parse_state(circuit_build_times_t *cbt,
or_state_t *state, char **msg);
int circuit_build_times_add_timeout(circuit_build_times_t *cbt,
int did_onehop, time_t start_time);
void circuit_build_times_set_timeout(circuit_build_times_t *cbt);
int circuit_build_times_add_time(circuit_build_times_t *cbt,
build_time_t time);
int circuit_build_times_needs_circuits(circuit_build_times_t *cbt);
int circuit_build_times_needs_circuits_now(circuit_build_times_t *cbt);
void circuit_build_times_init(circuit_build_times_t *cbt);
#ifdef CIRCUIT_PRIVATE
double circuit_build_times_calculate_timeout(circuit_build_times_t *cbt,
double quantile);
build_time_t circuit_build_times_generate_sample(circuit_build_times_t *cbt,
double q_lo, double q_hi);
void circuit_build_times_initial_alpha(circuit_build_times_t *cbt,
double quantile, double time_ms);
void circuit_build_times_update_alpha(circuit_build_times_t *cbt);
double circuit_build_times_cdf(circuit_build_times_t *cbt, double x);
void circuit_build_times_add_timeout_worker(circuit_build_times_t *cbt,
double quantile_cutoff);
void circuitbuild_running_unit_tests(void);
void circuit_build_times_reset(circuit_build_times_t *cbt);
/* Network liveness functions */
int circuit_build_times_network_check_changed(circuit_build_times_t *cbt);
#endif
/* Network liveness functions */
void circuit_build_times_network_is_live(circuit_build_times_t *cbt);
int circuit_build_times_network_check_live(circuit_build_times_t *cbt);
void circuit_build_times_network_circ_success(circuit_build_times_t *cbt);
/********************************* circuitlist.c ***********************/ /********************************* circuitlist.c ***********************/
circuit_t * _circuit_get_global_list(void); circuit_t * _circuit_get_global_list(void);
@ -2856,6 +3018,7 @@ void circuit_set_n_circid_orconn(circuit_t *circ, circid_t id,
or_connection_t *conn); or_connection_t *conn);
void circuit_set_state(circuit_t *circ, uint8_t state); void circuit_set_state(circuit_t *circ, uint8_t state);
void circuit_close_all_marked(void); void circuit_close_all_marked(void);
int32_t circuit_initial_package_window(void);
origin_circuit_t *origin_circuit_new(void); origin_circuit_t *origin_circuit_new(void);
or_circuit_t *or_circuit_new(circid_t p_circ_id, or_connection_t *p_conn); or_circuit_t *or_circuit_new(circid_t p_circ_id, or_connection_t *p_conn);
circuit_t *circuit_get_by_circid_orconn(circid_t circ_id, circuit_t *circuit_get_by_circid_orconn(circid_t circ_id,
@ -3661,9 +3824,9 @@ dirserv_generate_networkstatus_vote_obj(crypto_pk_env_t *private_key,
authority_cert_t *cert); authority_cert_t *cert);
#ifdef DIRVOTE_PRIVATE #ifdef DIRVOTE_PRIVATE
char * char *format_networkstatus_vote(crypto_pk_env_t *private_key,
format_networkstatus_vote(crypto_pk_env_t *private_key, networkstatus_t *v3_ns);
networkstatus_t *v3_ns); char *dirvote_compute_params(smartlist_t *votes);
#endif #endif
/********************************* dns.c ***************************/ /********************************* dns.c ***************************/
@ -3956,6 +4119,8 @@ void signed_descs_update_status_from_consensus_networkstatus(
char *networkstatus_getinfo_helper_single(routerstatus_t *rs); char *networkstatus_getinfo_helper_single(routerstatus_t *rs);
char *networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now); char *networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now);
void networkstatus_dump_bridge_status_to_file(time_t now); void networkstatus_dump_bridge_status_to_file(time_t now);
int32_t networkstatus_get_param(networkstatus_t *ns, const char *param_name,
int32_t default_val);
int getinfo_helper_networkstatus(control_connection_t *conn, int getinfo_helper_networkstatus(control_connection_t *conn,
const char *question, char **answer); const char *question, char **answer);
void networkstatus_free_all(void); void networkstatus_free_all(void);

View File

@ -1635,7 +1635,8 @@ cell_queue_append_packed_copy(cell_queue_t *queue, const cell_t *cell)
it_pool = mp_pool_new(sizeof(insertion_time_elem_t), 1024); it_pool = mp_pool_new(sizeof(insertion_time_elem_t), 1024);
tor_gettimeofday(&now); tor_gettimeofday(&now);
#define SECONDS_IN_A_DAY 86400L #define SECONDS_IN_A_DAY 86400L
added = (now.tv_sec % SECONDS_IN_A_DAY) * 100L + now.tv_usec / 10000L; added = (uint32_t)(((now.tv_sec % SECONDS_IN_A_DAY) * 100L)
+ ((uint32_t)now.tv_usec / (uint32_t)10000L));
if (!it_queue) { if (!it_queue) {
it_queue = tor_malloc_zero(sizeof(insertion_time_queue_t)); it_queue = tor_malloc_zero(sizeof(insertion_time_queue_t));
queue->insertion_times = it_queue; queue->insertion_times = it_queue;
@ -1879,15 +1880,17 @@ connection_or_flush_from_first_active_circuit(or_connection_t *conn, int max,
uint32_t cell_waiting_time; uint32_t cell_waiting_time;
insertion_time_queue_t *it_queue = queue->insertion_times; insertion_time_queue_t *it_queue = queue->insertion_times;
tor_gettimeofday(&now); tor_gettimeofday(&now);
flushed = (now.tv_sec % SECONDS_IN_A_DAY) * 100L + flushed = (uint32_t)((now.tv_sec % SECONDS_IN_A_DAY) * 100L +
now.tv_usec / 10000L; (uint32_t)now.tv_usec / (uint32_t)10000L);
if (!it_queue || !it_queue->first) { if (!it_queue || !it_queue->first) {
log_warn(LD_BUG, "Cannot determine insertion time of cell."); log_warn(LD_BUG, "Cannot determine insertion time of cell.");
} else { } else {
or_circuit_t *orcirc = TO_OR_CIRCUIT(circ); or_circuit_t *orcirc = TO_OR_CIRCUIT(circ);
insertion_time_elem_t *elem = it_queue->first; insertion_time_elem_t *elem = it_queue->first;
cell_waiting_time = (flushed * 10L + SECONDS_IN_A_DAY * 1000L - cell_waiting_time =
elem->insertion_time * 10L) % (SECONDS_IN_A_DAY * 1000L); (uint32_t)((flushed * 10L + SECONDS_IN_A_DAY * 1000L -
elem->insertion_time * 10L) %
(SECONDS_IN_A_DAY * 1000L));
#undef SECONDS_IN_A_DAY #undef SECONDS_IN_A_DAY
elem->counter--; elem->counter--;
if (elem->counter < 1) { if (elem->counter < 1) {

View File

@ -91,8 +91,9 @@ rend_client_send_introduction(origin_circuit_t *introcirc,
} }
}); });
if (!intro_key) { if (!intro_key) {
log_warn(LD_BUG, "Internal error: could not find intro key; we " log_info(LD_REND, "Our introduction point knowledge changed in "
"only have a v2 rend desc with %d intro points.", "mid-connect! Could not find intro key; we only have a "
"v2 rend desc with %d intro points. Giving up.",
smartlist_len(entry->parsed->intro_nodes)); smartlist_len(entry->parsed->intro_nodes));
goto err; goto err;
} }
@ -128,7 +129,7 @@ rend_client_send_introduction(origin_circuit_t *introcirc,
REND_DESC_COOKIE_LEN); REND_DESC_COOKIE_LEN);
v3_shift += 2+REND_DESC_COOKIE_LEN; v3_shift += 2+REND_DESC_COOKIE_LEN;
} }
set_uint32(tmp+v3_shift+1, htonl(time(NULL))); set_uint32(tmp+v3_shift+1, htonl((uint32_t)time(NULL)));
v3_shift += 4; v3_shift += 4;
} /* if version 2 only write version number */ } /* if version 2 only write version number */
else if (entry->parsed->protocols & (1<<2)) { else if (entry->parsed->protocols & (1<<2)) {
@ -644,7 +645,7 @@ rend_client_receive_rendezvous(origin_circuit_t *circ, const char *request,
/* set the windows to default. these are the windows /* set the windows to default. these are the windows
* that alice thinks bob has. * that alice thinks bob has.
*/ */
hop->package_window = CIRCWINDOW_START; hop->package_window = circuit_initial_package_window();
hop->deliver_window = CIRCWINDOW_START; hop->deliver_window = CIRCWINDOW_START;
onion_append_to_cpath(&circ->cpath, hop); onion_append_to_cpath(&circ->cpath, hop);

View File

@ -264,7 +264,7 @@ rend_config_services(or_options_t *options, int validate_only)
for (line = options->RendConfigLines; line; line = line->next) { for (line = options->RendConfigLines; line; line = line->next) {
if (!strcasecmp(line->key, "HiddenServiceDir")) { if (!strcasecmp(line->key, "HiddenServiceDir")) {
if (service) { if (service) { /* register the one we just finished parsing */
if (validate_only) if (validate_only)
rend_service_free(service); rend_service_free(service);
else else
@ -921,7 +921,7 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
len = r; len = r;
if (*buf == 3) { if (*buf == 3) {
/* Version 3 INTRODUCE2 cell. */ /* Version 3 INTRODUCE2 cell. */
time_t ts = 0, now = time(NULL); time_t ts = 0;
v3_shift = 1; v3_shift = 1;
auth_type = buf[1]; auth_type = buf[1];
switch (auth_type) { switch (auth_type) {
@ -944,13 +944,12 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
} }
/* Check timestamp. */ /* Check timestamp. */
memcpy((char*)&ts, buf+1+v3_shift, sizeof(uint32_t)); ts = ntohl(get_uint32(buf+1+v3_shift));
v3_shift += 4; v3_shift += 4;
ts = ntohl(ts);
if ((now - ts) < -1 * REND_REPLAY_TIME_INTERVAL / 2 || if ((now - ts) < -1 * REND_REPLAY_TIME_INTERVAL / 2 ||
(now - ts) > REND_REPLAY_TIME_INTERVAL / 2) { (now - ts) > REND_REPLAY_TIME_INTERVAL / 2) {
log_warn(LD_REND, "INTRODUCE2 cell is too %s. Discarding.", log_warn(LD_REND, "INTRODUCE2 cell is too %s. Discarding.",
(now - ts) < 0 ? "old" : "new"); (now - ts) < 0 ? "old" : "new");
return -1; return -1;
} }
} }
@ -1101,7 +1100,7 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
circ_needs_uptime = rend_service_requires_uptime(service); circ_needs_uptime = rend_service_requires_uptime(service);
/* help predict this next time */ /* help predict this next time */
rep_hist_note_used_internal(time(NULL), circ_needs_uptime, 1); rep_hist_note_used_internal(now, circ_needs_uptime, 1);
/* Launch a circuit to alice's chosen rendezvous point. /* Launch a circuit to alice's chosen rendezvous point.
*/ */
@ -1137,7 +1136,7 @@ rend_service_introduce(origin_circuit_t *circuit, const char *request,
launched->build_state->pending_final_cpath = cpath = launched->build_state->pending_final_cpath = cpath =
tor_malloc_zero(sizeof(crypt_path_t)); tor_malloc_zero(sizeof(crypt_path_t));
cpath->magic = CRYPT_PATH_MAGIC; cpath->magic = CRYPT_PATH_MAGIC;
launched->build_state->expiry_time = time(NULL) + MAX_REND_TIMEOUT; launched->build_state->expiry_time = now + MAX_REND_TIMEOUT;
cpath->dh_handshake_state = dh; cpath->dh_handshake_state = dh;
dh = NULL; dh = NULL;
@ -1477,7 +1476,7 @@ rend_service_rendezvous_has_opened(origin_circuit_t *circuit)
/* set the windows to default. these are the windows /* set the windows to default. these are the windows
* that bob thinks alice has. * that bob thinks alice has.
*/ */
hop->package_window = CIRCWINDOW_START; hop->package_window = circuit_initial_package_window();
hop->deliver_window = CIRCWINDOW_START; hop->deliver_window = CIRCWINDOW_START;
onion_append_to_cpath(&circuit->cpath, hop); onion_append_to_cpath(&circuit->cpath, hop);

View File

@ -772,9 +772,6 @@ consider_testing_reachability(int test_or, int test_dir)
me->address, me->or_port); me->address, me->or_port);
circuit_launch_by_router(CIRCUIT_PURPOSE_TESTING, me, circuit_launch_by_router(CIRCUIT_PURPOSE_TESTING, me,
CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL); CIRCLAUNCH_NEED_CAPACITY|CIRCLAUNCH_IS_INTERNAL);
control_event_server_status(LOG_NOTICE,
"CHECKING_REACHABILITY ORADDRESS=%s:%d",
me->address, me->or_port);
} }
tor_addr_from_ipv4h(&addr, me->addr); tor_addr_from_ipv4h(&addr, me->addr);
@ -790,10 +787,6 @@ consider_testing_reachability(int test_or, int test_dir)
DIR_PURPOSE_FETCH_SERVERDESC, DIR_PURPOSE_FETCH_SERVERDESC,
ROUTER_PURPOSE_GENERAL, ROUTER_PURPOSE_GENERAL,
1, "authority.z", NULL, 0, 0); 1, "authority.z", NULL, 0, 0);
control_event_server_status(LOG_NOTICE,
"CHECKING_REACHABILITY DIRADDRESS=%s:%d",
me->address, me->dir_port);
} }
} }
@ -809,8 +802,11 @@ router_orport_found_reachable(void)
" Publishing server descriptor." : ""); " Publishing server descriptor." : "");
can_reach_or_port = 1; can_reach_or_port = 1;
mark_my_descriptor_dirty(); mark_my_descriptor_dirty();
if (!me) if (!me) { /* should never happen */
log_warn(LD_BUG, "ORPort found reachable, but I have no routerinfo "
"yet. Failing to inform controller of success.");
return; return;
}
control_event_server_status(LOG_NOTICE, control_event_server_status(LOG_NOTICE,
"REACHABILITY_SUCCEEDED ORADDRESS=%s:%d", "REACHABILITY_SUCCEEDED ORADDRESS=%s:%d",
me->address, me->or_port); me->address, me->or_port);
@ -828,8 +824,11 @@ router_dirport_found_reachable(void)
can_reach_dir_port = 1; can_reach_dir_port = 1;
if (!me || decide_to_advertise_dirport(get_options(), me->dir_port)) if (!me || decide_to_advertise_dirport(get_options(), me->dir_port))
mark_my_descriptor_dirty(); mark_my_descriptor_dirty();
if (!me) if (!me) { /* should never happen */
log_warn(LD_BUG, "DirPort found reachable, but I have no routerinfo "
"yet. Failing to inform controller of success.");
return; return;
}
control_event_server_status(LOG_NOTICE, control_event_server_status(LOG_NOTICE,
"REACHABILITY_SUCCEEDED DIRADDRESS=%s:%d", "REACHABILITY_SUCCEEDED DIRADDRESS=%s:%d",
me->address, me->dir_port); me->address, me->dir_port);
@ -1909,7 +1908,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
if (options->DirReqStatistics && if (options->DirReqStatistics &&
load_stats_file("stats"PATH_SEPARATOR"dirreq-stats", load_stats_file("stats"PATH_SEPARATOR"dirreq-stats",
"dirreq-stats-end", since, &contents) > 0) { "dirreq-stats-end", since, &contents) > 0) {
int pos = strlen(s); size_t pos = strlen(s);
if (strlcpy(s + pos, contents, maxlen - strlen(s)) != if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
strlen(contents)) { strlen(contents)) {
log_warn(LD_DIR, "Could not write dirreq-stats to extra-info " log_warn(LD_DIR, "Could not write dirreq-stats to extra-info "
@ -1921,7 +1920,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
if (options->EntryStatistics && if (options->EntryStatistics &&
load_stats_file("stats"PATH_SEPARATOR"entry-stats", load_stats_file("stats"PATH_SEPARATOR"entry-stats",
"entry-stats-end", since, &contents) > 0) { "entry-stats-end", since, &contents) > 0) {
int pos = strlen(s); size_t pos = strlen(s);
if (strlcpy(s + pos, contents, maxlen - strlen(s)) != if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
strlen(contents)) { strlen(contents)) {
log_warn(LD_DIR, "Could not write entry-stats to extra-info " log_warn(LD_DIR, "Could not write entry-stats to extra-info "
@ -1933,7 +1932,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
if (options->CellStatistics && if (options->CellStatistics &&
load_stats_file("stats"PATH_SEPARATOR"buffer-stats", load_stats_file("stats"PATH_SEPARATOR"buffer-stats",
"cell-stats-end", since, &contents) > 0) { "cell-stats-end", since, &contents) > 0) {
int pos = strlen(s); size_t pos = strlen(s);
if (strlcpy(s + pos, contents, maxlen - strlen(s)) != if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
strlen(contents)) { strlen(contents)) {
log_warn(LD_DIR, "Could not write buffer-stats to extra-info " log_warn(LD_DIR, "Could not write buffer-stats to extra-info "
@ -1945,7 +1944,7 @@ extrainfo_dump_to_string(char *s, size_t maxlen, extrainfo_t *extrainfo,
if (options->ExitPortStatistics && if (options->ExitPortStatistics &&
load_stats_file("stats"PATH_SEPARATOR"exit-stats", load_stats_file("stats"PATH_SEPARATOR"exit-stats",
"exit-stats-end", since, &contents) > 0) { "exit-stats-end", since, &contents) > 0) {
int pos = strlen(s); size_t pos = strlen(s);
if (strlcpy(s + pos, contents, maxlen - strlen(s)) != if (strlcpy(s + pos, contents, maxlen - strlen(s)) !=
strlen(contents)) { strlen(contents)) {
log_warn(LD_DIR, "Could not write exit-stats to extra-info " log_warn(LD_DIR, "Could not write exit-stats to extra-info "

View File

@ -102,6 +102,7 @@ typedef enum {
K_VOTING_DELAY, K_VOTING_DELAY,
K_KNOWN_FLAGS, K_KNOWN_FLAGS,
K_PARAMS,
K_VOTE_DIGEST, K_VOTE_DIGEST,
K_CONSENSUS_DIGEST, K_CONSENSUS_DIGEST,
K_CONSENSUS_METHODS, K_CONSENSUS_METHODS,
@ -433,6 +434,7 @@ static token_rule_t networkstatus_token_table[] = {
T1("valid-until", K_VALID_UNTIL, CONCAT_ARGS, NO_OBJ ), T1("valid-until", K_VALID_UNTIL, CONCAT_ARGS, NO_OBJ ),
T1("voting-delay", K_VOTING_DELAY, GE(2), NO_OBJ ), T1("voting-delay", K_VOTING_DELAY, GE(2), NO_OBJ ),
T1("known-flags", K_KNOWN_FLAGS, ARGS, NO_OBJ ), T1("known-flags", K_KNOWN_FLAGS, ARGS, NO_OBJ ),
T01("params", K_PARAMS, ARGS, NO_OBJ ),
T( "fingerprint", K_FINGERPRINT, CONCAT_ARGS, NO_OBJ ), T( "fingerprint", K_FINGERPRINT, CONCAT_ARGS, NO_OBJ ),
CERTIFICATE_MEMBERS CERTIFICATE_MEMBERS
@ -470,6 +472,7 @@ static token_rule_t networkstatus_consensus_token_table[] = {
T01("client-versions", K_CLIENT_VERSIONS, CONCAT_ARGS, NO_OBJ ), T01("client-versions", K_CLIENT_VERSIONS, CONCAT_ARGS, NO_OBJ ),
T01("server-versions", K_SERVER_VERSIONS, CONCAT_ARGS, NO_OBJ ), T01("server-versions", K_SERVER_VERSIONS, CONCAT_ARGS, NO_OBJ ),
T01("consensus-method", K_CONSENSUS_METHOD, EQ(1), NO_OBJ), T01("consensus-method", K_CONSENSUS_METHOD, EQ(1), NO_OBJ),
T01("params", K_PARAMS, ARGS, NO_OBJ ),
END_OF_TABLE END_OF_TABLE
}; };
@ -2002,8 +2005,9 @@ routerstatus_parse_entry_from_string(memarea_t *area,
for (i=0; i < tok->n_args; ++i) { for (i=0; i < tok->n_args; ++i) {
if (!strcmpstart(tok->args[i], "Bandwidth=")) { if (!strcmpstart(tok->args[i], "Bandwidth=")) {
int ok; int ok;
rs->bandwidth = tor_parse_ulong(strchr(tok->args[i], '=')+1, 10, rs->bandwidth = (uint32_t)tor_parse_ulong(strchr(tok->args[i], '=')+1,
0, UINT32_MAX, &ok, NULL); 10, 0, UINT32_MAX,
&ok, NULL);
if (!ok) { if (!ok) {
log_warn(LD_DIR, "Invalid Bandwidth %s", escaped(tok->args[i])); log_warn(LD_DIR, "Invalid Bandwidth %s", escaped(tok->args[i]));
goto err; goto err;
@ -2011,8 +2015,9 @@ routerstatus_parse_entry_from_string(memarea_t *area,
rs->has_bandwidth = 1; rs->has_bandwidth = 1;
} else if (!strcmpstart(tok->args[i], "Measured=")) { } else if (!strcmpstart(tok->args[i], "Measured=")) {
int ok; int ok;
rs->measured_bw = tor_parse_ulong(strchr(tok->args[i], '=')+1, 10, rs->measured_bw =
0, UINT32_MAX, &ok, NULL); (uint32_t)tor_parse_ulong(strchr(tok->args[i], '=')+1,
10, 0, UINT32_MAX, &ok, NULL);
if (!ok) { if (!ok) {
log_warn(LD_DIR, "Invalid Measured Bandwidth %s", log_warn(LD_DIR, "Invalid Measured Bandwidth %s",
escaped(tok->args[i])); escaped(tok->args[i]));
@ -2406,6 +2411,34 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out,
goto err; goto err;
} }
tok = find_opt_by_keyword(tokens, K_PARAMS);
if (tok) {
inorder = 1;
ns->net_params = smartlist_create();
for (i = 0; i < tok->n_args; ++i) {
int ok=0;
char *eq = strchr(tok->args[i], '=');
if (!eq) {
log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i]));
goto err;
}
tor_parse_long(eq+1, 10, INT32_MIN, INT32_MAX, &ok, NULL);
if (!ok) {
log_warn(LD_DIR, "Bad element '%s' in params", escaped(tok->args[i]));
goto err;
}
if (i > 0 && strcmp(tok->args[i-1], tok->args[i]) >= 0) {
log_warn(LD_DIR, "%s >= %s", tok->args[i-1], tok->args[i]);
inorder = 0;
}
smartlist_add(ns->net_params, tor_strdup(tok->args[i]));
}
if (!inorder) {
log_warn(LD_DIR, "params not in order");
goto err;
}
}
ns->voters = smartlist_create(); ns->voters = smartlist_create();
SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, _tok) { SMARTLIST_FOREACH_BEGIN(tokens, directory_token_t *, _tok) {
@ -2605,6 +2638,14 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out,
} else { } else {
if (tok->object_size >= INT_MAX) if (tok->object_size >= INT_MAX)
goto err; goto err;
/* We already parsed a vote from this voter. Use the first one. */
if (v->signature) {
log_fn(LOG_PROTOCOL_WARN, LD_DIR, "We received a networkstatus "
"that contains two votes from the same voter. Ignoring "
"the second vote.");
continue;
}
v->signature = tor_memdup(tok->object_body, tok->object_size); v->signature = tor_memdup(tok->object_body, tok->object_size);
v->signature_len = (int) tok->object_size; v->signature_len = (int) tok->object_size;
} }
@ -2614,6 +2655,10 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out,
if (! n_signatures) { if (! n_signatures) {
log_warn(LD_DIR, "No signatures on networkstatus vote."); log_warn(LD_DIR, "No signatures on networkstatus vote.");
goto err; goto err;
} else if (ns->type == NS_TYPE_VOTE && n_signatures != 1) {
log_warn(LD_DIR, "Received more than one signature on a "
"network-status vote.");
goto err;
} }
if (eos_out) if (eos_out)
@ -3516,9 +3561,11 @@ tor_version_parse(const char *s, tor_version_t *out)
if (! close_paren) if (! close_paren)
return -1; return -1;
cp += 5; cp += 5;
hexlen = (close_paren-cp); if (close_paren-cp > HEX_DIGEST_LEN)
return -1;
hexlen = (int)(close_paren-cp);
memset(digest, 0, sizeof(digest)); memset(digest, 0, sizeof(digest));
if (hexlen > HEX_DIGEST_LEN || hexlen == 0 || (hexlen % 2) == 1) if ( hexlen == 0 || (hexlen % 2) == 1)
return -1; return -1;
if (base16_decode(digest, hexlen/2, cp, hexlen)) if (base16_decode(digest, hexlen/2, cp, hexlen))
return -1; return -1;

View File

@ -37,6 +37,15 @@ const char tor_git_revision[] = "";
#define GEOIP_PRIVATE #define GEOIP_PRIVATE
#define MEMPOOL_PRIVATE #define MEMPOOL_PRIVATE
#define ROUTER_PRIVATE #define ROUTER_PRIVATE
#define CIRCUIT_PRIVATE
/*
* Linux doesn't provide lround in math.h by default, but mac os does...
* It's best just to leave math.h out of the picture entirely.
*/
//#include <math.h>
long int lround(double x);
double fabs(double x);
#include "or.h" #include "or.h"
#include "test.h" #include "test.h"
@ -410,7 +419,7 @@ test_crypto_dh(void)
char p2[DH_BYTES]; char p2[DH_BYTES];
char s1[DH_BYTES]; char s1[DH_BYTES];
char s2[DH_BYTES]; char s2[DH_BYTES];
int s1len, s2len; ssize_t s1len, s2len;
test_eq(crypto_dh_get_bytes(dh1), DH_BYTES); test_eq(crypto_dh_get_bytes(dh1), DH_BYTES);
test_eq(crypto_dh_get_bytes(dh2), DH_BYTES); test_eq(crypto_dh_get_bytes(dh2), DH_BYTES);
@ -1114,6 +1123,24 @@ test_util(void)
tor_parse_uint64("12345678901",10,500,INT32_MAX, &i, &cp)); tor_parse_uint64("12345678901",10,500,INT32_MAX, &i, &cp));
test_assert(i == 0); test_assert(i == 0);
{
/* Test tor_parse_double. */
double d = tor_parse_double("10", 0, UINT64_MAX,&i,NULL);
test_assert(i == 1);
test_assert(DBL_TO_U64(d) == 10);
d = tor_parse_double("0", 0, UINT64_MAX,&i,NULL);
test_assert(i == 1);
test_assert(DBL_TO_U64(d) == 0);
d = tor_parse_double(" ", 0, UINT64_MAX,&i,NULL);
test_assert(i == 0);
d = tor_parse_double(".0a", 0, UINT64_MAX,&i,NULL);
test_assert(i == 0);
d = tor_parse_double(".0a", 0, UINT64_MAX,&i,&cp);
test_assert(i == 1);
d = tor_parse_double("-.0", 0, UINT64_MAX,&i,NULL);
test_assert(i == 1);
}
/* Test failing snprintf cases */ /* Test failing snprintf cases */
test_eq(-1, tor_snprintf(buf, 0, "Foo")); test_eq(-1, tor_snprintf(buf, 0, "Foo"));
test_eq(-1, tor_snprintf(buf, 2, "Foo")); test_eq(-1, tor_snprintf(buf, 2, "Foo"));
@ -3337,6 +3364,220 @@ done:
return; return;
} }
static void
test_dirutil_param_voting(void)
{
networkstatus_t vote1, vote2, vote3, vote4;
smartlist_t *votes = smartlist_create();
char *res = NULL;
/* dirvote_compute_params only looks at the net_params field of the votes,
so that's all we need to set.
*/
memset(&vote1, 0, sizeof(vote1));
memset(&vote2, 0, sizeof(vote2));
memset(&vote3, 0, sizeof(vote3));
memset(&vote4, 0, sizeof(vote4));
vote1.net_params = smartlist_create();
vote2.net_params = smartlist_create();
vote3.net_params = smartlist_create();
vote4.net_params = smartlist_create();
smartlist_split_string(vote1.net_params,
"ab=90 abcd=20 cw=50 x-yz=-99", NULL, 0, 0);
smartlist_split_string(vote2.net_params,
"ab=27 cw=5 x-yz=88", NULL, 0, 0);
smartlist_split_string(vote3.net_params,
"abcd=20 c=60 cw=500 x-yz=-9 zzzzz=101", NULL, 0, 0);
smartlist_split_string(vote4.net_params,
"ab=900 abcd=200 c=1 cw=51 x-yz=100", NULL, 0, 0);
test_eq(100, networkstatus_get_param(&vote4, "x-yz", 50));
test_eq(222, networkstatus_get_param(&vote4, "foobar", 222));
smartlist_add(votes, &vote1);
smartlist_add(votes, &vote2);
smartlist_add(votes, &vote3);
smartlist_add(votes, &vote4);
res = dirvote_compute_params(votes);
test_streq(res,
"ab=90 abcd=20 c=1 cw=50 x-yz=-9 zzzzz=101");
done:
tor_free(res);
SMARTLIST_FOREACH(vote1.net_params, char *, cp, tor_free(cp));
SMARTLIST_FOREACH(vote2.net_params, char *, cp, tor_free(cp));
SMARTLIST_FOREACH(vote3.net_params, char *, cp, tor_free(cp));
SMARTLIST_FOREACH(vote4.net_params, char *, cp, tor_free(cp));
smartlist_free(vote1.net_params);
smartlist_free(vote2.net_params);
smartlist_free(vote3.net_params);
smartlist_free(vote4.net_params);
return;
}
static void
test_circuit_timeout(void)
{
/* Plan:
* 1. Generate 1000 samples
* 2. Estimate parameters
* 3. If difference, repeat
* 4. Save state
* 5. load state
* 6. Estimate parameters
* 7. compare differences
*/
circuit_build_times_t initial;
circuit_build_times_t estimate;
circuit_build_times_t final;
double timeout1, timeout2;
or_state_t state;
char *msg;
int i, runs;
circuit_build_times_init(&initial);
circuit_build_times_init(&estimate);
circuit_build_times_init(&final);
memset(&state, 0, sizeof(or_state_t));
circuitbuild_running_unit_tests();
#define timeout0 (build_time_t)(30*1000.0)
initial.Xm = 750;
circuit_build_times_initial_alpha(&initial, BUILDTIMEOUT_QUANTILE_CUTOFF,
timeout0);
do {
int n = 0;
for (i=0; i < MIN_CIRCUITS_TO_OBSERVE; i++) {
if (circuit_build_times_add_time(&estimate,
circuit_build_times_generate_sample(&initial, 0, 1)) == 0) {
n++;
}
}
circuit_build_times_update_alpha(&estimate);
timeout1 = circuit_build_times_calculate_timeout(&estimate,
BUILDTIMEOUT_QUANTILE_CUTOFF);
circuit_build_times_set_timeout(&estimate);
log_warn(LD_CIRC, "Timeout is %lf, Xm is %d", timeout1, estimate.Xm);
/* XXX: 5% distribution error may not be the right metric */
} while (fabs(circuit_build_times_cdf(&initial, timeout0) -
circuit_build_times_cdf(&initial, timeout1)) > 0.05
/* 5% error */
&& estimate.total_build_times < NCIRCUITS_TO_OBSERVE);
test_assert(estimate.total_build_times < NCIRCUITS_TO_OBSERVE);
circuit_build_times_update_state(&estimate, &state);
test_assert(circuit_build_times_parse_state(&final, &state, &msg) == 0);
circuit_build_times_update_alpha(&final);
timeout2 = circuit_build_times_calculate_timeout(&final,
BUILDTIMEOUT_QUANTILE_CUTOFF);
circuit_build_times_set_timeout(&final);
log_warn(LD_CIRC, "Timeout is %lf, Xm is %d", timeout2, final.Xm);
test_assert(fabs(circuit_build_times_cdf(&initial, timeout0) -
circuit_build_times_cdf(&initial, timeout2)) < 0.05);
for (runs = 0; runs < 50; runs++) {
int build_times_idx = 0;
int total_build_times = 0;
final.timeout_ms = BUILD_TIMEOUT_INITIAL_VALUE;
estimate.timeout_ms = BUILD_TIMEOUT_INITIAL_VALUE;
for (i = 0; i < RECENT_CIRCUITS*2; i++) {
circuit_build_times_network_circ_success(&estimate);
circuit_build_times_add_time(&estimate,
circuit_build_times_generate_sample(&estimate, 0,
BUILDTIMEOUT_QUANTILE_CUTOFF));
estimate.have_computed_timeout = 1;
circuit_build_times_network_circ_success(&estimate);
circuit_build_times_add_time(&final,
circuit_build_times_generate_sample(&final, 0,
BUILDTIMEOUT_QUANTILE_CUTOFF));
final.have_computed_timeout = 1;
}
test_assert(!circuit_build_times_network_check_changed(&estimate));
test_assert(!circuit_build_times_network_check_changed(&final));
/* Reset liveness to be non-live */
final.liveness.network_last_live = 0;
estimate.liveness.network_last_live = 0;
build_times_idx = estimate.build_times_idx;
total_build_times = estimate.total_build_times;
for (i = 0; i < NETWORK_NONLIVE_TIMEOUT_COUNT; i++) {
test_assert(circuit_build_times_network_check_live(&estimate));
test_assert(circuit_build_times_network_check_live(&final));
if (circuit_build_times_add_timeout(&estimate, 0,
(time_t)(approx_time()-estimate.timeout_ms/1000.0-1)))
estimate.have_computed_timeout = 1;
if (circuit_build_times_add_timeout(&final, 0,
(time_t)(approx_time()-final.timeout_ms/1000.0-1)))
final.have_computed_timeout = 1;
}
test_assert(!circuit_build_times_network_check_live(&estimate));
test_assert(!circuit_build_times_network_check_live(&final));
for ( ; i < NETWORK_NONLIVE_DISCARD_COUNT; i++) {
if (circuit_build_times_add_timeout(&estimate, 0,
(time_t)(approx_time()-estimate.timeout_ms/1000.0-1)))
estimate.have_computed_timeout = 1;
if (i < NETWORK_NONLIVE_DISCARD_COUNT-1) {
if (circuit_build_times_add_timeout(&final, 0,
(time_t)(approx_time()-final.timeout_ms/1000.0-1)))
final.have_computed_timeout = 1;
}
}
test_assert(!circuit_build_times_network_check_live(&estimate));
test_assert(!circuit_build_times_network_check_live(&final));
log_info(LD_CIRC, "idx: %d %d, tot: %d %d",
build_times_idx, estimate.build_times_idx,
total_build_times, estimate.total_build_times);
/* Check rollback index. Should match top of loop. */
test_assert(build_times_idx == estimate.build_times_idx);
test_assert(total_build_times == estimate.total_build_times);
/* Now simulate that the network has become live and we need
* a change */
circuit_build_times_network_is_live(&estimate);
circuit_build_times_network_is_live(&final);
for (i = 0; i < MAX_RECENT_TIMEOUT_COUNT; i++) {
if (circuit_build_times_add_timeout(&estimate, 1, approx_time()-1))
estimate.have_computed_timeout = 1;
if (i < MAX_RECENT_TIMEOUT_COUNT-1) {
if (circuit_build_times_add_timeout(&final, 1, approx_time()-1))
final.have_computed_timeout = 1;
}
}
test_assert(estimate.liveness.after_firsthop_idx == 0);
test_assert(final.liveness.after_firsthop_idx ==
MAX_RECENT_TIMEOUT_COUNT-1);
test_assert(circuit_build_times_network_check_live(&estimate));
test_assert(circuit_build_times_network_check_live(&final));
if (circuit_build_times_add_timeout(&final, 1, approx_time()-1))
final.have_computed_timeout = 1;
}
done:
return;
}
extern const char AUTHORITY_CERT_1[]; extern const char AUTHORITY_CERT_1[];
extern const char AUTHORITY_SIGNKEY_1[]; extern const char AUTHORITY_SIGNKEY_1[];
extern const char AUTHORITY_CERT_2[]; extern const char AUTHORITY_CERT_2[];
@ -3494,6 +3735,9 @@ test_v3_networkstatus(void)
crypto_pk_get_digest(cert1->identity_key, voter->identity_digest); crypto_pk_get_digest(cert1->identity_key, voter->identity_digest);
smartlist_add(vote->voters, voter); smartlist_add(vote->voters, voter);
vote->cert = authority_cert_dup(cert1); vote->cert = authority_cert_dup(cert1);
vote->net_params = smartlist_create();
smartlist_split_string(vote->net_params, "circuitwindow=101 foo=990",
NULL, 0, 0);
vote->routerstatus_list = smartlist_create(); vote->routerstatus_list = smartlist_create();
/* add the first routerstatus. */ /* add the first routerstatus. */
vrs = tor_malloc_zero(sizeof(vote_routerstatus_t)); vrs = tor_malloc_zero(sizeof(vote_routerstatus_t));
@ -3635,6 +3879,9 @@ test_v3_networkstatus(void)
vote->dist_seconds = 300; vote->dist_seconds = 300;
authority_cert_free(vote->cert); authority_cert_free(vote->cert);
vote->cert = authority_cert_dup(cert2); vote->cert = authority_cert_dup(cert2);
vote->net_params = smartlist_create();
smartlist_split_string(vote->net_params, "bar=2000000000 circuitwindow=20",
NULL, 0, 0);
tor_free(vote->client_versions); tor_free(vote->client_versions);
tor_free(vote->server_versions); tor_free(vote->server_versions);
voter = smartlist_get(vote->voters, 0); voter = smartlist_get(vote->voters, 0);
@ -3673,6 +3920,9 @@ test_v3_networkstatus(void)
vote->dist_seconds = 250; vote->dist_seconds = 250;
authority_cert_free(vote->cert); authority_cert_free(vote->cert);
vote->cert = authority_cert_dup(cert3); vote->cert = authority_cert_dup(cert3);
vote->net_params = smartlist_create();
smartlist_split_string(vote->net_params, "circuitwindow=80 foo=660",
NULL, 0, 0);
smartlist_add(vote->supported_methods, tor_strdup("4")); smartlist_add(vote->supported_methods, tor_strdup("4"));
vote->client_versions = tor_strdup("0.1.2.14,0.1.2.17"); vote->client_versions = tor_strdup("0.1.2.14,0.1.2.17");
vote->server_versions = tor_strdup("0.1.2.10,0.1.2.15,0.1.2.16"); vote->server_versions = tor_strdup("0.1.2.10,0.1.2.15,0.1.2.16");
@ -3729,6 +3979,10 @@ test_v3_networkstatus(void)
test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:" test_streq(cp, "Authority:Exit:Fast:Guard:MadeOfCheese:MadeOfTin:"
"Running:Stable:V2Dir:Valid"); "Running:Stable:V2Dir:Valid");
tor_free(cp); tor_free(cp);
cp = smartlist_join_strings(con->net_params, ":", 0, NULL);
test_streq(cp, "bar=2000000000:circuitwindow=80:foo=660");
tor_free(cp);
test_eq(4, smartlist_len(con->voters)); /*3 voters, 1 legacy key.*/ test_eq(4, smartlist_len(con->voters)); /*3 voters, 1 legacy key.*/
/* The voter id digests should be in this order. */ /* The voter id digests should be in this order. */
test_assert(memcmp(cert2->cache_info.identity_digest, test_assert(memcmp(cert2->cache_info.identity_digest,
@ -4848,6 +5102,8 @@ static struct {
ENT(dir_format), ENT(dir_format),
ENT(dirutil), ENT(dirutil),
SUBENT(dirutil, measured_bw), SUBENT(dirutil, measured_bw),
SUBENT(dirutil, param_voting),
ENT(circuit_timeout),
ENT(v3_networkstatus), ENT(v3_networkstatus),
ENT(policies), ENT(policies),
ENT(rend_fns), ENT(rend_fns),

View File

@ -70,7 +70,7 @@ show_help(void)
static void static void
crypto_log_errors(int severity, const char *doing) crypto_log_errors(int severity, const char *doing)
{ {
unsigned int err; unsigned long err;
const char *msg, *lib, *func; const char *msg, *lib, *func;
while ((err = ERR_get_error()) != 0) { while ((err = ERR_get_error()) != 0) {
msg = (const char*)ERR_reason_error_string(err); msg = (const char*)ERR_reason_error_string(err);
@ -94,7 +94,7 @@ load_passphrase(void)
{ {
char *cp; char *cp;
char buf[1024]; /* "Ought to be enough for anybody." */ char buf[1024]; /* "Ought to be enough for anybody." */
int n = read_all(passphrase_fd, buf, sizeof(buf), 0); ssize_t n = read_all(passphrase_fd, buf, sizeof(buf), 0);
if (n < 0) { if (n < 0) {
log_err(LD_GENERAL, "Couldn't read from passphrase fd: %s", log_err(LD_GENERAL, "Couldn't read from passphrase fd: %s",
strerror(errno)); strerror(errno));

View File

@ -51,7 +51,7 @@ static void usage(void) ATTR_NORETURN;
/** Set *<b>out</b> to a newly allocated SOCKS4a resolve request with /** Set *<b>out</b> to a newly allocated SOCKS4a resolve request with
* <b>username</b> and <b>hostname</b> as provided. Return the number * <b>username</b> and <b>hostname</b> as provided. Return the number
* of bytes in the request. */ * of bytes in the request. */
static int static ssize_t
build_socks_resolve_request(char **out, build_socks_resolve_request(char **out,
const char *username, const char *username,
const char *hostname, const char *hostname,
@ -184,7 +184,7 @@ do_resolve(const char *hostname, uint32_t sockshost, uint16_t socksport,
int s; int s;
struct sockaddr_in socksaddr; struct sockaddr_in socksaddr;
char *req = NULL; char *req = NULL;
int len = 0; ssize_t len = 0;
tor_assert(hostname); tor_assert(hostname);
tor_assert(result_addr); tor_assert(result_addr);

View File

@ -226,5 +226,5 @@
#define USING_TWOS_COMPLEMENT #define USING_TWOS_COMPLEMENT
/* Version number of package */ /* Version number of package */
#define VERSION "0.2.2.1-alpha" #define VERSION "0.2.2.2-alpha"