autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
package autopilot
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
prand "math/rand"
|
|
|
|
|
"time"
|
|
|
|
|
|
2018-06-05 03:34:16 +02:00
|
|
|
|
"github.com/btcsuite/btcd/btcec"
|
|
|
|
|
"github.com/btcsuite/btcutil"
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
)
|
|
|
|
|
|
2019-03-27 15:04:15 +01:00
|
|
|
|
// minMedianChanSizeFraction determines the minimum size a channel must have to
|
|
|
|
|
// count positively when calculating the scores using preferential attachment.
|
|
|
|
|
// The minimum channel size is calculated as median/minMedianChanSizeFraction,
|
|
|
|
|
// where median is the median channel size of the entire graph.
|
|
|
|
|
const minMedianChanSizeFraction = 4
|
|
|
|
|
|
2018-12-19 14:54:54 +01:00
|
|
|
|
// PrefAttachment is an implementation of the AttachmentHeuristic interface
|
|
|
|
|
// that implement a non-linear preferential attachment heuristic. This means
|
|
|
|
|
// that given a threshold to allocate to automatic channel establishment, the
|
|
|
|
|
// heuristic will attempt to favor connecting to nodes which already have a set
|
|
|
|
|
// amount of links, selected by sampling from a power law distribution. The
|
|
|
|
|
// attachment is non-linear in that it favors nodes with a higher in-degree but
|
|
|
|
|
// less so than regular linear preferential attachment. As a result, this
|
|
|
|
|
// creates smaller and less clusters than regular linear preferential
|
|
|
|
|
// attachment.
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
//
|
|
|
|
|
// TODO(roasbeef): BA, with k=-3
|
2018-12-19 14:54:54 +01:00
|
|
|
|
type PrefAttachment struct {
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
}
|
|
|
|
|
|
2018-12-19 14:54:54 +01:00
|
|
|
|
// NewPrefAttachment creates a new instance of a PrefAttachment heuristic.
|
|
|
|
|
func NewPrefAttachment() *PrefAttachment {
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
prand.Seed(time.Now().Unix())
|
2018-12-19 14:54:54 +01:00
|
|
|
|
return &PrefAttachment{}
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
}
|
|
|
|
|
|
2018-12-19 14:54:54 +01:00
|
|
|
|
// A compile time assertion to ensure PrefAttachment meets the
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
// AttachmentHeuristic interface.
|
2018-12-19 14:54:54 +01:00
|
|
|
|
var _ AttachmentHeuristic = (*PrefAttachment)(nil)
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
|
2018-04-18 04:02:04 +02:00
|
|
|
|
// NodeID is a simple type that holds an EC public key serialized in compressed
|
autopilot: "Look ma no hands!", introducing autopilot mode
This commit introduces the initial implementation of the autopilot
mode. Autopilot is new mode within lnd that enables automatic channel
management. This means that if enabled lnd will attempt to
automatically manage channels according to a set of heuristic defined
within the main configuration for autopilot.Agent instance.
The autopilot.Agent implements a simple closed control loop. It takes
in external signals such as wallet balance updates, new open channel,
and channels that are now closed the updates its internal state. With
each external trigger it will consult the registered
AttachmentHeuristic to decide: if it needs to open any more channels,
and if so how much it should use to open the channels, ultimately
returning a set of recommended AttachmentDirectives. The
autopilot.Agent loop will then take those attempt to establish
connection, and go back in waiting for a new external signal.
With this first implementation the default heuristic is the
ConstrainedPrefAttachment implementation of AttachmentHeuristic. Given
a min and max channel size, a limit on the number of channels, and the
percentage of wallet funds to allocate to channels, it will attempt to
execute a heuristic drive by the Barabási–Albert model model in order
to attempt to drive the global graph towards a scale free topology.
This is commit implements a foundational layer for future simulations,
optimization, and additional heuristics.
2017-08-11 06:14:41 +02:00
|
|
|
|
// format.
|
|
|
|
|
type NodeID [33]byte
|
|
|
|
|
|
|
|
|
|
// NewNodeID creates a new nodeID from a passed public key.
|
|
|
|
|
func NewNodeID(pub *btcec.PublicKey) NodeID {
|
|
|
|
|
var n NodeID
|
|
|
|
|
copy(n[:], pub.SerializeCompressed())
|
|
|
|
|
return n
|
|
|
|
|
}
|
|
|
|
|
|
2018-12-19 14:54:54 +01:00
|
|
|
|
// Name returns the name of this heuristic.
|
|
|
|
|
//
|
|
|
|
|
// NOTE: This is a part of the AttachmentHeuristic interface.
|
|
|
|
|
func (p *PrefAttachment) Name() string {
|
|
|
|
|
return "preferential"
|
|
|
|
|
}
|
|
|
|
|
|
2019-01-09 09:14:45 +01:00
|
|
|
|
// NodeScores is a method that given the current channel graph and current set
|
|
|
|
|
// of local channels, scores the given nodes according to the preference of
|
|
|
|
|
// opening a channel of the given size with them. The returned channel
|
|
|
|
|
// candidates maps the NodeID to a NodeScore for the node.
|
2018-11-22 23:18:09 +01:00
|
|
|
|
//
|
|
|
|
|
// The heuristic employed by this method is one that attempts to promote a
|
|
|
|
|
// scale-free network globally, via local attachment preferences for new nodes
|
|
|
|
|
// joining the network with an amount of available funds to be allocated to
|
|
|
|
|
// channels. Specifically, we consider the degree of each node (and the flow
|
|
|
|
|
// in/out of the node available via its open channels) and utilize the
|
|
|
|
|
// Barabási–Albert model to drive our recommended attachment heuristics. If
|
|
|
|
|
// implemented globally for each new participant, this results in a channel
|
|
|
|
|
// graph that is scale-free and follows a power law distribution with k=-3.
|
|
|
|
|
//
|
2019-03-27 15:04:15 +01:00
|
|
|
|
// To avoid assigning a high score to nodes with a large number of small
|
|
|
|
|
// channels, we only count channels at least as large as a given fraction of
|
|
|
|
|
// the graph's median channel size.
|
|
|
|
|
//
|
2018-11-22 23:18:09 +01:00
|
|
|
|
// The returned scores will be in the range [0.0, 1.0], where higher scores are
|
|
|
|
|
// given to nodes already having high connectivity in the graph.
|
|
|
|
|
//
|
|
|
|
|
// NOTE: This is a part of the AttachmentHeuristic interface.
|
2018-12-19 14:54:54 +01:00
|
|
|
|
func (p *PrefAttachment) NodeScores(g ChannelGraph, chans []Channel,
|
2018-12-19 14:54:53 +01:00
|
|
|
|
chanSize btcutil.Amount, nodes map[NodeID]struct{}) (
|
2018-12-19 15:24:17 +01:00
|
|
|
|
map[NodeID]*NodeScore, error) {
|
2018-11-22 23:18:09 +01:00
|
|
|
|
|
2019-03-27 15:04:15 +01:00
|
|
|
|
// We first run though the graph once in order to find the median
|
|
|
|
|
// channel size.
|
|
|
|
|
var (
|
|
|
|
|
allChans []btcutil.Amount
|
|
|
|
|
seenChans = make(map[uint64]struct{})
|
|
|
|
|
)
|
|
|
|
|
if err := g.ForEachNode(func(n Node) error {
|
|
|
|
|
err := n.ForEachChannel(func(e ChannelEdge) error {
|
|
|
|
|
if _, ok := seenChans[e.ChanID.ToUint64()]; ok {
|
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
seenChans[e.ChanID.ToUint64()] = struct{}{}
|
|
|
|
|
allChans = append(allChans, e.Capacity)
|
|
|
|
|
return nil
|
|
|
|
|
})
|
|
|
|
|
if err != nil {
|
|
|
|
|
return err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return nil
|
|
|
|
|
}); err != nil {
|
|
|
|
|
return nil, err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
medianChanSize := Median(allChans)
|
2019-08-27 09:55:08 +02:00
|
|
|
|
log.Tracef("Found channel median %v for preferential score heuristic",
|
|
|
|
|
medianChanSize)
|
2019-03-27 15:04:15 +01:00
|
|
|
|
|
|
|
|
|
// Count the number of large-ish channels for each particular node in
|
|
|
|
|
// the graph.
|
2019-01-09 09:14:45 +01:00
|
|
|
|
var maxChans int
|
2018-11-22 23:18:09 +01:00
|
|
|
|
nodeChanNum := make(map[NodeID]int)
|
|
|
|
|
if err := g.ForEachNode(func(n Node) error {
|
|
|
|
|
var nodeChans int
|
2019-03-27 15:04:15 +01:00
|
|
|
|
err := n.ForEachChannel(func(e ChannelEdge) error {
|
|
|
|
|
// Since connecting to nodes with a lot of small
|
|
|
|
|
// channels actually worsens our connectivity in the
|
|
|
|
|
// graph (we will potentially waste time trying to use
|
|
|
|
|
// these useless channels in path finding), we decrease
|
|
|
|
|
// the counter for such channels.
|
|
|
|
|
if e.Capacity < medianChanSize/minMedianChanSizeFraction {
|
|
|
|
|
nodeChans--
|
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Larger channels we count.
|
2018-11-22 23:18:09 +01:00
|
|
|
|
nodeChans++
|
|
|
|
|
return nil
|
|
|
|
|
})
|
|
|
|
|
if err != nil {
|
|
|
|
|
return err
|
|
|
|
|
}
|
|
|
|
|
|
2019-01-09 09:14:45 +01:00
|
|
|
|
// We keep track of the highest-degree node we've seen, as this
|
|
|
|
|
// will be given the max score.
|
|
|
|
|
if nodeChans > maxChans {
|
|
|
|
|
maxChans = nodeChans
|
|
|
|
|
}
|
|
|
|
|
|
2018-11-22 23:18:09 +01:00
|
|
|
|
// If this node is not among our nodes to score, we can return
|
|
|
|
|
// early.
|
|
|
|
|
nID := NodeID(n.PubKey())
|
|
|
|
|
if _, ok := nodes[nID]; !ok {
|
2019-08-27 09:55:08 +02:00
|
|
|
|
log.Tracef("Node %x not among nodes to score, "+
|
|
|
|
|
"ignoring", nID[:])
|
2018-11-22 23:18:09 +01:00
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
|
2018-12-19 14:54:54 +01:00
|
|
|
|
// Otherwise we'll record the number of channels.
|
2018-11-22 23:18:09 +01:00
|
|
|
|
nodeChanNum[nID] = nodeChans
|
2019-08-27 09:55:08 +02:00
|
|
|
|
log.Tracef("Counted %v channels for node %x", nodeChans, nID[:])
|
2018-11-22 23:18:09 +01:00
|
|
|
|
|
|
|
|
|
return nil
|
|
|
|
|
}); err != nil {
|
|
|
|
|
return nil, err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// If there are no channels in the graph we cannot determine any
|
|
|
|
|
// preferences, so we return, indicating all candidates get a score of
|
|
|
|
|
// zero.
|
2019-01-09 09:14:45 +01:00
|
|
|
|
if maxChans == 0 {
|
2019-08-27 09:55:08 +02:00
|
|
|
|
log.Tracef("No channels in the graph")
|
2018-11-22 23:18:09 +01:00
|
|
|
|
return nil, nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
existingPeers := make(map[NodeID]struct{})
|
|
|
|
|
for _, c := range chans {
|
|
|
|
|
existingPeers[c.Node] = struct{}{}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// For each node in the set of nodes, count their fraction of channels
|
|
|
|
|
// in the graph, and use that as the score.
|
2018-12-19 15:24:17 +01:00
|
|
|
|
candidates := make(map[NodeID]*NodeScore)
|
2018-11-22 23:18:09 +01:00
|
|
|
|
for nID, nodeChans := range nodeChanNum {
|
|
|
|
|
|
|
|
|
|
// If the node is among or existing channel peers, we don't
|
|
|
|
|
// need another channel.
|
2019-08-27 09:55:08 +02:00
|
|
|
|
if _, ok := existingPeers[nID]; ok {
|
|
|
|
|
log.Tracef("Node %x among existing peers for pref "+
|
|
|
|
|
"attach heuristic, giving zero score", nID[:])
|
2018-11-22 23:18:09 +01:00
|
|
|
|
continue
|
2019-08-27 09:55:08 +02:00
|
|
|
|
}
|
2018-11-22 23:18:09 +01:00
|
|
|
|
|
2019-03-27 15:04:15 +01:00
|
|
|
|
// If the node had no large channels, we skip it, since it
|
|
|
|
|
// would have gotten a zero score anyway.
|
2019-08-27 09:55:08 +02:00
|
|
|
|
if nodeChans <= 0 {
|
|
|
|
|
log.Tracef("Skipping node %x with channel count %v",
|
|
|
|
|
nID[:], nodeChans)
|
2018-12-10 14:56:54 +01:00
|
|
|
|
continue
|
2018-11-22 23:18:09 +01:00
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Otherwise we score the node according to its fraction of
|
2019-01-09 09:14:45 +01:00
|
|
|
|
// channels in the graph, scaled such that the highest-degree
|
|
|
|
|
// node will be given a score of 1.0.
|
|
|
|
|
score := float64(nodeChans) / float64(maxChans)
|
2019-08-27 09:55:08 +02:00
|
|
|
|
log.Tracef("Giving node %x a pref attach score of %v",
|
|
|
|
|
nID[:], score)
|
|
|
|
|
|
2018-12-19 15:24:17 +01:00
|
|
|
|
candidates[nID] = &NodeScore{
|
|
|
|
|
NodeID: nID,
|
|
|
|
|
Score: score,
|
2018-11-22 23:18:09 +01:00
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return candidates, nil
|
|
|
|
|
}
|