Commit graph

29 commits

Author SHA1 Message Date
Michael Schmoock
04ea37d88f pygossmap: rename GossipStoreHeader to GossipStoreMsgHeader
Changelog-Added: pyln-client: Improvements on the gossmap implementation
2023-04-05 06:13:08 +09:30
Michael Schmoock
6e46a63c57 pygossmap: adds statistic and filter module
Includes a lot of useful filters and statistical methods.

To see a gossip_store summary:
```
s = GossmapStats(g)
s.print_stats()
```
2023-04-05 06:13:08 +09:30
Michael Schmoock
f1b6047d69 pygossmap: store features for nodes and channels
also makes them acessible using bitmask functions
2023-04-05 06:13:08 +09:30
Michael Schmoock
3130f4ec27 pygossmap: read .disabled from channel_flags 2023-04-05 06:13:08 +09:30
Michael Schmoock
6a16a31a98 pygossmap: parse node addresses and other data 2023-04-05 06:13:08 +09:30
Michael Schmoock
9409f2f1ea pygossmap: adds get_neighbors and get_neighbors_hc flodding method 2023-04-05 06:13:08 +09:30
Michael Schmoock
5a9a3d83c9 pygossmap: adds get_halfchannel 2023-04-05 06:13:08 +09:30
Michael Schmoock
eb9cb5ef31 pygossmap: adds missing __str__, __eq__ and __hash__
Also caches certain __hash__ and __str__ operations,
This way graph operations can be done quicker.
2023-04-05 06:13:08 +09:30
Michael Schmoock
3f651b08d5 pygossmap: cleanups and optimizations
- moves offset into GossipHeader hdr which is passed to all constuctors
 - reads .flags as u16 instead of extracting it from the .length, see 0274d88ba
 - adds zombie and ratelimit flag to GossipHeader
 - bytes_read start at 0 instead of 1 which is more correct,
   the one byte is then corrected for when setting the offset of new header.
 - bytes_read is increased in pull_bytes as this is the only place where
   something is read
 - use new style for various format-strings
2023-04-05 06:13:08 +09:30
Rusty Russell
6338758018 gossmap: make API more robust against future changes.
Many changes to gossmap (including the pending ones!) don't actually
concern readers, as long as they obey certain rules:

1. Ignore unknown messages.
2. Treat all 16 upper bits of length as flags, ignore unknown ones.

So now we split the version byte into MAJOR and MINOR, and you can
ignore MINOR changes.

We don't expose the internal version (for creating the map)
programmatically: you should really hardcode what major version you
understand!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-09-24 15:22:27 +09:30
Alex Myers
9dc794dba8 gossipd: make use of new ratelimit bit in gossip_store length mask
routing.c now flags rate-limited gossip as it enters the gossip_store but
makes use of it in updating the routing graph. Flagged gossip is not
rebroadcast to gossip peers.

Changelog-Changed: gossipd: now accepts spam gossip, but squelches it for
peers.
2022-07-06 14:31:19 +09:30
Alex Myers
cbafc0fa33 gossip_store: add flag for spam gossip, update to v10
This will be used to decouple internal use of gossip from what is
passed to gossip peers. Updates GOSSIP_STORE_VERION to 10.

Changelog-Changed: gossip_store updated to version 10.
2022-07-06 14:31:19 +09:30
Rusty Russell
e8e7c4719f pyln-client: fix mypy warnings, fix and test deletion of a channel.
This only happens when a deletion is added by a running gossipd, so
we put a deletion at the end of the store to test it.

mypy noticed that this code was nonsensical, so clearly untested.

The testing noticed that making a nodeid from a string was also buggy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Michael Schmoock
125752118a pyln-client/gossmap: init GossmapNode and Id also with hexstring
also improves test coverage
Changelog-Added: pyln-client: routines for direct access to the gossip store as Gossmap
2021-09-08 09:34:14 +09:30
Michael Schmoock
fa8e74a2ad pyln-client/gossmap: make node and node_id comparable with __lt__ and __eq__ 2021-09-08 09:34:14 +09:30
Michael Schmoock
47efea92c6 pyln-client/gossmap: let half channel have source and destination 2021-09-08 09:34:14 +09:30
Michael Schmoock
2bca76b6e9 pyln-client/gossmap: start internal function names with underscore _ 2021-09-08 09:34:14 +09:30
Michael Schmoock
de6c593272 pyln-client/gossmap: have channels link to their nodes instead of id
This is likely easier for programmers and does not use more mem as we
already load all this.
2021-09-08 09:34:14 +09:30
Rusty Russell
487facf1f0 pyln-client/gossmap: more fixes, make mypy happier.
Mainly fixing type annotations, but some real fixes:

1. GossmapHalfchannel.from_str() should be a classmethod.
2. update_channel had weird, unusable default values (fields can't be NULL,
   since we use it below).

[ There was one more occurence where isinstance should be used above
type() == xyz comparison. -- MS ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Michael Schmoock
ba2bcac530 pyln-client/gossmap: Don't mix bytes and GossmapNodeId
Do not mix bytes and GossmapNodeId when accessing Gossmap.nodes dicts.

Therefore the definion got GossmapNodeId also needed to be pulled to the
beginning of the file.
2021-09-08 09:34:14 +09:30
Michael Schmoock
ac27217114 pyln-client/gossmap: adds __repr__ functions 2021-09-08 09:34:14 +09:30
Michael Schmoock
b6ed405d8c pyln-client/gossmap: adds channel satoshi capacity
This reads the `gossip_store_channel_amount` that always follows the
`channel_announcement` messages. Therefore it uses an internal variable
_last_scid to know what channel has been added last time.
2021-09-08 09:34:14 +09:30
Michael Schmoock
c5c909acd3 pyln-client/gossmap: adds helpers to get channels and nodes 2021-09-08 09:34:14 +09:30
Michael Schmoock
fd16535f68 pyln-client/gossmap: adds GossmapHalfchannel objects 2021-09-08 09:34:14 +09:30
Rusty Russell
75126a0997 pyln-client/gossmap: add NodeId class.
This is more efficient than converting them all to Pubkeys: about 3.8
seconds vs 5.4 seconds.  Usually treating them as raw bytes is what we
want anyway.

[ Fixup by Michael Schmoock <michael@schmoock.net> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell
887fb38e06 pyln-client/gossmap: use ShortChannelId class from pyln.proto, if available.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell
4753ca4d35 pyln-client/gossmap: add a little documentation.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell
e495462c50 pyln-client/gossmap: save deconstructed fields instead of raw msg bytes.
We have to parse them anyway, so why not make them accessible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30
Rusty Russell
1a5a26d659 pyln.client: new functionality to access Gossmap.
It doesn't do much work, but it does parse the gossmap file and extract
nodes and channels.

[ Fixup by Michael Schmoock <michael@schmoock.net> ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 09:34:14 +09:30