mirror of
https://gitlab.torproject.org/tpo/core/tor.git
synced 2025-02-26 15:42:34 +01:00
90 lines
3.5 KiB
Text
90 lines
3.5 KiB
Text
README.geoip -- information on the IP-to-country-code file shipped with tor
|
|
===========================================================================
|
|
|
|
The IP-to-country-code file in src/config/geoip is based on MaxMind's
|
|
GeoLite Country database with the following modifications:
|
|
|
|
- Those "A1" ("Anonymous Proxy") entries lying inbetween two entries with
|
|
the same country code are automatically changed to that country code.
|
|
These changes can be overriden by specifying a different country code
|
|
in src/config/geoip-manual.
|
|
|
|
- Other "A1" entries are replaced with country codes specified in
|
|
src/config/geoip-manual, or are left as is if there is no corresponding
|
|
entry in that file. Even non-"A1" entries can be modified by adding a
|
|
replacement entry to src/config/geoip-manual. Handle with care.
|
|
|
|
|
|
1. Updating the geoip file from a MaxMind database file
|
|
-------------------------------------------------------
|
|
|
|
Download the most recent MaxMind GeoLite Country database:
|
|
http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
|
|
|
|
Run `python deanonymind.py` in the local directory. Review the output to
|
|
learn about applied automatic/manual changes and watch out for any
|
|
warnings.
|
|
|
|
Possibly edit geoip-manual to make more/fewer/different manual changes and
|
|
re-run `python deanonymind.py`.
|
|
|
|
When done, prepend the new geoip file with a comment like this:
|
|
|
|
# Last updated based on $DATE Maxmind GeoLite Country
|
|
# See README.geoip for details on the conversion.
|
|
|
|
|
|
2. Verifying automatic and manual changes using diff
|
|
----------------------------------------------------
|
|
|
|
To unzip the original MaxMind file and look at the automatic changes, run:
|
|
|
|
unzip GeoIPCountryCSV.zip
|
|
diff -U1 GeoIPCountryWhois.csv AutomaticGeoIPCountryWhois.csv
|
|
|
|
To look at subsequent manual changes, run:
|
|
|
|
diff -U1 AutomaticGeoIPCountryWhois.csv ManualGeoIPCountryWhois.csv
|
|
|
|
To manually generate the geoip file and compare it to the automatically
|
|
created one, run:
|
|
|
|
cut -d, -f3-5 < ManualGeoIPCountryWhois.csv | sed 's/"//g' > mygeoip
|
|
diff -U1 geoip mygeoip
|
|
|
|
|
|
3. Verifying automatic and manual changes using blockfinder
|
|
-----------------------------------------------------------
|
|
|
|
Blockfinder is a powerful tool to handle multiple IP-to-country data
|
|
sources. Blockfinder has a function to specify a country code and compare
|
|
conflicting country code assignments in different data sources.
|
|
|
|
We can use blockfinder to compare A1 entries in the original MaxMind file
|
|
with the same or overlapping blocks in the file generated above and in the
|
|
RIR delegation files:
|
|
|
|
git clone https://github.com/ioerror/blockfinder
|
|
cd blockfinder/
|
|
python blockfinder -i
|
|
python blockfinder -r ../GeoIPCountryWhois.csv
|
|
python blockfinder -r ../ManualGeoIPCountryWhois.csv
|
|
python blockfinder -p A1 > A1-comparison.txt
|
|
|
|
The output marks conflicts between assignments using either '*' in case of
|
|
two different opinions or '#' for three or more different opinions about
|
|
the country code for a given block.
|
|
|
|
The '*' conflicts are most likely harmless, because there will always be
|
|
at least two opinions with the original MaxMind file saying A1 and the
|
|
other two sources saying something more meaningful.
|
|
|
|
However, watch out for '#' conflicts. In these cases, the original
|
|
MaxMind file ("A1"), the updated MaxMind file (hopefully the correct
|
|
country code), and the RIR delegation files (some other country code) all
|
|
disagree.
|
|
|
|
There are perfectly valid cases where the updated MaxMind file and the RIR
|
|
delegation files don't agree. But each of those cases must be verified
|
|
manually.
|
|
|