Merge bitcoin/bitcoin#30695: seeds: Add additional seed source and bump uptime requirements for Onion and I2P nodes

b061b35105 seeds: Regenerate mainnet seeds (virtu)
02dc45c506 seeds: Pull nodes from Luke's seeder (virtu)
7a2068a0ff seeds: Pull nodes from virtu's crawler (virtu)

Pull request description:

  This builds on #30008 and adds data [exported](https://github.com/virtu/seed-exporter) by [my crawler](https://github.com/virtu/p2p-crawler) an additional source for seed nodes. Data covers all supported network types.

  [edit: Added Luke's seeder as input as well.]

  ### Motivation
  - Further decentralizes the seed node selection process (in the long term potentially enabling an _n_-source threshold for nodes to prevent a single source from entering malicious nodes)
  - No longer need to manually curate seed node list for any network type: See last paragraph of OP in #30008. My crawler has been [discovering the handful of available cjdns nodes](https://21.ninja/reachable-nodes/nodes-by-net-type/) for around two months, all but one of which meet the reliability criteria.
  - Alignment of uptime requirements for Onion and I2P nodes with those of clearnet nodes to 50%: If I'm reading the code correctly, seeders appear to optimize for up-to-dateness by using [lower connection timeouts](3c1a63c672/src/crawl.rs (L349)) than [Bitcoin Core](bc87ad9854/src/netbase.cpp (L40C27-L40C48)) to maximize throughput. Since my crawler does not have the same timeliness requirements, it opts for accuracy by using generous timeouts. As a result, its data contains additional eligible Onion (and other darknet nodes), as is shown in the histogram below. Around 4500 Onion nodes are discovered so far (blue); my data adds ~6400 more (orange); ~ 1500 nodes take longer than the default 20-second Bitcoin Core timeout and won't qualify as "good".

  ![Connection time histogram for Onion nodes](https://github.com/user-attachments/assets/c3513604-aa48-4c75-b51d-13421eaed9eb)

  Here's the current results with 512 nodes for all networks except cjdns:
  <details>
  <summary>Using the extra data</summary>

  ```
  IPv4   IPv6  Onion  I2P    CJDNS Pass
  10335   2531  11545   1589     10 Initial
  10335   2531  11545   1589     10 Skip entries with invalid address
  5639   1431  11163   1589      8 After removing duplicates
  5606   1417  11163   1589      8 Enforce minimal number of blocks
  5606   1417  11163   1589      8 Require service bit 1
  4873   1228  11163   1589      8 Require minimum uptime
  4846   1225  11161   1588      8 Require a known and recent user agent
  4846   1225  11161   1588      8 Filter out hosts with multiple bitcoin ports
  512    512    512    512      8 Look up ASNs and limit results per ASN and per net
  ```
  </details>
  <details>
  <summary>Before</summary>

  ```
  IPv4   IPv6  Onion  I2P    CJDNS Pass
  5772   1323    443      0      2 Initial
  5772   1323    443      0      2 Skip entries with invalid address
  4758   1110    443      0      2 After removing duplicates
  4723   1094    443      0      2 Enforce minimal number of blocks
  4723   1094    443      0      2 Require service bit 1
  3732    867    443      0      2 Require minimum uptime
  3718    864    443      0      2 Require a known and recent user agent
  3718    864    443      0      2 Filter out hosts with multiple bitcoin ports
   512    409    443      0      2 Look up ASNs and limit results per ASN and per net
  ```
  </details>

  ### To dos
  - [x] Remove manual nodes and update README
  - [x] Mark nodes with connection times exceeding Bitcoin Core's default as bad in [exporter](https://github.com/virtu/seed-exporter): [done](https://github.com/virtu/seed-exporter/pull/12)
  - [x] Regenerate mainnet seeds
  - [x] Rebase, then remove WIP label once #30008 gets merged

ACKs for top commit:
  achow101:
    ACK b061b35105
  fjahr:
    utACK b061b35105

Tree-SHA512: 63e86220787251c7e8d2d5957bad69352e19ae17d7b9b2d27d8acddfec5bdafe588edb68d77d19c57f25f149de723e2eeadded0c8cf13eaca22dc33bd8cf92a0

contrib/seeds/README.md

@@ -8,16 +8,18 @@ and remove old versions as necessary (at a minimum when SeedsServiceFlags()
 changes its default return value, as those are the services which seeds are added
 to addrman with).
 
-The seeds compiled into the release are created from sipa's and achow101's DNS seed and AS map
-data. Run the following commands from the `/contrib/seeds` directory:
+The seeds compiled into the release are created from sipa's, achow101's and luke-jr's
+DNS seed, virtu's crawler, and fjahr's community AS map data. Run the following commands
+from the `/contrib/seeds` directory:
 
 ```
 curl https://bitcoin.sipa.be/seeds.txt.gz | gzip -dc > seeds_main.txt
 curl https://mainnet.achownodes.xyz/seeds.txt.gz | gzip -dc >> seeds_main.txt
+curl https://21.ninja/seeds.txt.gz | gzip -dc >> seeds_main.txt
+curl https://luke.dashjr.org/programs/bitcoin/files/charts/seeds.txt >> seeds_main.txt
 curl https://testnet.achownodes.xyz/seeds.txt.gz | gzip -dc > seeds_test.txt
 curl https://raw.githubusercontent.com/fjahr/asmap-data/main/latest_asmap.dat > asmap-filled.dat
 python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt
-cat nodes_main_manual.txt >> nodes_main.txt
 python3 makeseeds.py -a asmap-filled.dat -s seeds_test.txt > nodes_test.txt
 # TODO: Uncomment when a seeder publishes seeds.txt.gz for testnet4
 # python3 makeseeds.py -a asmap-filled.dat -s seeds_testnet4.txt -m 30000 > nodes_testnet4.txt

contrib/seeds/makeseeds.py

@@ -230,12 +230,12 @@ def main():
     # Require service bit 1.
     ips = [ip for ip in ips if (ip['service'] & 1) == 1]
     print(f'{ip_stats(ips):s} Require service bit 1', file=sys.stderr)
-    # Require at least 50% 30-day uptime for clearnet, 10% for onion and i2p.
+    # Require at least 50% 30-day uptime for clearnet, onion and i2p; 10% for cjdns
     req_uptime = {
         'ipv4': 50,
         'ipv6': 50,
-        'onion': 10,
-        'i2p' : 10,
+        'onion': 50,
+        'i2p': 50,
         'cjdns': 10,
     }
     ips = [ip for ip in ips if ip['uptime'] > req_uptime[ip['net']]]

contrib/seeds/nodes_main_manual.txt

@@ -1,4 +0,0 @@
-# manually updated 2023-04 for minimal cjdns bootstrap support
-[fc32:17ea:e415:c3bf:9808:149d:b5a2:c9aa]:8333
-[fcc7:be49:ccd1:dc91:3125:f0da:457d:8ce]:8333
-[fcdc:73ae:b1a9:1bf8:d4c2:811:a4c7:c34e]:8333