The gRPC interceptor was not using the correct method/ratemeter
map key, and failing to find a rate meter for the server call.
- Fix the rate meter key lookup bug.
- Disable most strict, default call rate meters in tests.
Made an adjustment to the test harness' scaffold setup so an interceptor
testing or disabling config file is always picked up by bob and alice
daemons.
- Set arbitration daemon's registerDisputeAgent rate @ 10/second, so
it does not interfere with the test harness. (There is no pre-existing
arb node appDataDir before that daemon starts.)
Note: The test harness cannot install the custom rate metering file in
an arb daemon's appDataDir before it starts because there is no dao-setup
file for that node type.
TODO: Adjust api simulation scripts to interceptor bug fix.
This change gives CLI users a 'stop' command to shutdown a daemon.
- Server side gRPC boilerpate was added to :proto and :daemon.
- When the new GrpcShutdownService accepts a 'stop' request from the CLI,
it calls UserThread.runAfter(BisqHeadlessApp.getShutDownHandler(), 500, MILLISECONDS);
- Method help file 'stop-help.txt' was added to :core.
- Client side gRPC boiler plate and 'stop' implementation was added to :cli.
The general rule is limit calls that change p2p data to 1/minute,
others to 1/second. An exception is made to set/remove wallet password
methods (1/5s), due to the latency of writing wallet changes to disk.
This change may affect api testing in the future. If that happens,
further changes to the call rate metering interceptor may be made to loosen
the constraints when running in regtest/dev mode.
Migrate RpcService over to the new block notification daemon and RPC
client based on jsonrpc4j. Drop in own DTO classes in place of the ones
defined by btcd-cli4j and rename requestBtcBlock & addNewBtcBlockHandler
to requestDtoBlock & addNewDtoBlockHandler respectively.
Also remove now redundant filtering from the logback config and update
grade-witness.
Adds all the gRPC server boilerplate, and a simple help service
that serves method help in man page format. Help text is maintained
in text files located in core/src/main/resources/help.
Only some of the method help text files are defined in this
change, more to be added.
This is a feature that will not be included in api v1, but partial
support is added in this change to the server. CLI will pass a
default triggerPrice of 0 (unused) with the createoffer command.
When fully implemented, an optional trigger-price param will be
added to the CLI's createoffer method, and the value will only be
visible to offer owners. New enableoffer and disableoffer
methods will also need to be added.
Similar to 'getoffers' and 'getoffer', but filters out offers not
created by the user. The new methods are so similar some offer list
filtering and sorting was refactored in CoreOffersService.
Also fixed some createoffer apitest cases in anticipation of a new OfferFilter,
which will filter out offers not matching any user payment account.
We need to be able to define call rate meters for time spans not limited
to TimeUnit intervals of 1 SECOND, 1 HOUR, or 1 DAY. This change allows
more flexibility, e.g., 10 per 5 seconds, 10 per 5 hrs, 100 per 7 days.
This change demonstrates how a method call rate metering interceptor
is configured for a gRPC service. GrpcVersionService uses a custom
rate metering interceptor, or none. A commented out, 'default'
interceptor is defined as a usage example.
This adds a GrpcServiceRateMeteringConfig class that can read and
write rate metering interceptor config files, and configure
a gRPC rate metering service interceptor at startup.
This seems excessive, but we need to be able to test and tune
method rate metering without having to change hard coded, default
interceptor rate meters.
This change provides a gRPC CallRateMeteringInterceptor to help protect
the server and network against being overloaded by CLI scripting mistakes.
An interceptor instance can be configured on a gRPC service to set
individual method call rate limits on one or more of the the service's
methods. For example, the GrpcOffersService could be configured with
this interceptor to set the createoffer rate limit to 5/hour, and
the takeoffer call rate limit could be set to 20/day. Whenever a
call rate limit is exceeded, the gRPC call is aborted and the client
recieves a "rate limit exceeded" error.
Below is a simple example showing how to set rate limits for one method
in GrpcVersionService.
final ServerInterceptor[] interceptors() {
return new ServerInterceptor[]{
new CallRateMeteringInterceptor(new HashMap<>() {{
put("getVersion", new GrpcCallRateMeter(2, SECONDS));
}})
};
}
It specifies a CLI can execute getversion 2 times / second.
This is not a throttling mechansim, there is no blocking nor locking
to slow call rates. When call rates are exceeded, calls are
simply aborted.
This change reduces gRPC service error handling duplication by moving
it into a @Singleton encapsulating everything needed to wrap
an expected or unexpected core api exception into a gRPC
StatusRuntimeException before sending it to the client. It also
fixes some boilerpate classes were gRPC error handling was missing.
This change was prompted by the recent changes in the main branch to
allow a tx memo field to be set from the UI and API.
This and the prior PR address the API's need to be able to fetch a
tx (with a memo). The API can now get a completed trade's withdrawal
txid and pass it as a gettransaction parameter.
See previous PR "Append nullable withdrawalTxId field to Trade".
https://github.com/bisq-network/bisq/pull/4937
A summary of changes by file:
grpc.proto
- Added withdrawalTxId field to existing TradeInfo proto & wrapper.
- Reordered fields in TradeInfo proto.
- Added new fields to be displayed by TxInfo proto in CLI.
- Fixed typo: unsetTxFeeRatePreference -> UnsetTxFeeRatePreference.
- Added new GetTransaction rpc.
GrpcWalletsService - Added new getTransaction gRPC boilerplate.
CoreWalletsService - Added new getTransaction implementation.
TxInfo - Added the new fields for displaying a tx summary from CLI.
This is not intended to be more than a brief summary; a block explorer
or bitcoin-core client should be used to see the complete definition.
TradeInfo - Added the new withdrawalTxId field defined in grpc.proto.
CliMain - Added new 'case gettransaction'.
TransactionFormat - Formats a TxInfo sent from the server to CLI.
ColumnHeaderConstants - Added console headers used by TransactionFormat.
TradeFormat - Displays a completed trade's WithdrawalTxId if present.
Apitest - Adjusted affected tests: assert tx memo is persisted and
test gettransaction.
- Added optional memo parameter to the api's sendbtc and
withdrawfunds commands.
- Removed the @Nullable annotation was removed because protobuf
does not support null.
- Visibility in two wallet check methods were changed from private
to pkg protected so the CoreTradeService could use them.
- Adjusted affected tests. (Asserting the memo field was set on a
transaction cannot be checked from apitest yet.)
Takes an address, amount, and optional txfeerate param,
returns a lightweight TxInfo proto.
Also overloaded two BtcWalletService methods to allow sendbtc
to pass in the tx fee rate -- overriding the fee service and
custom fee rate setting.
If present in the sendbsq command, the parameter will override the fee
service and custom fee rate setting for the BSQ transaction.
Also changed the sendbsq grpc return type to a lightweight TX proto wrapper.
Besides some small refactoring in the CLI, all the changes are
adjustments for this new sendbsq parameter and its new grpc return value.
This change fixes the blocking problem in the fee rate request api.
Also redefined the TxFeeRateInfo.
- Redefined grpc.proto message TxFeeRateInfo, added
lastFeeServiceRequestTs field. (CLI user may want to know
TS of last fee request.)
- Adjusted TxFeeRateInfo proto wrapper.
- Adjusted CurrencyFormat and BtcTxFeeRateTest to new TxFeeRateInfo.
- Added @Getter annotation to FeeService. (CLI user may want to know
TS of last fee request).
- Pass resultHandler from GrpcWalletsService through CoreApi, to
CoreWalletsService's tx fee rate api methods.
- Add missing boilerplate error handling to GrpcPaymentAccountsService.
- Edit some PaymentAccountForm & PaymentAccountTypeAdapter exception
messages that would be passed to CLI; they should be in the same
style as existing CLI err msgs.
In the CLI, this method now takes a single argument, the path to a json
file (a payment account form). The json text is read from the file
and sent to the server, where it is serialized to a new PaymentAccount
instance, saved, and sent back to the CLI as a protobuf.PaymentAccount.
- Modified existing gRPC boilerplate in grpc.proto and GrpcPaymentAccountsService.
- Modified existing method signatures in CoreApi and CorePaymentAccountsService.
- Replaced existing method impl in CorePaymentAccountsService and
removed dead code.
- Replaced the CLI's existing 'createpaymentacct' method impl.
- Modified existing tests.
Changed the core getPaymentMethods() -> getFiatPaymentMethods() to avoid
"Unnecessary use of fully qualified name 'PaymentMethod.getPaymentMethods'
due to existing static import 'bisq.core.payment.payload.PaymentMethod.*'"
Had to change the getPaymentMethods() names to getPaymentMethodIds() to
avoid this codacy issue:
"Unnecessary use of fully qualified name 'PaymentMethod.getPaymentMethods'
due to existing static import 'bisq.core.payment.payload.PaymentMethod.*'"
If 'PaymentMethod.getPaymentMethods' was changed to 'getPaymentMethods',
a recursive loop would result, ending in an out of stack heap crash.
This renaming of the method on the server is correct, but the CLI's
'getpaymentmethods' name was not changed.
Returns a list of supported payment method IDs. This gives CLI users
the correct payment method id for creating a new payment account.
- Added gRPC service GetPaymentMethods to grpc.proto.
- Added gRPC boilerplate method to GrpcPaymentAccountsService.
- Added implimentation to CoreApi -> CorePaymentAccountsService.
- Refactored PaymentAccountTest hierarchy.
- Add api method to CLI.
This change fixes the recently bloated wallet balances api. Now there
is one CLI getbalance [bsq|btc] method that calls a getbalances(currency-code)
on the server. The server returns full wallet balance information for
BSQ, BTC, or both if the CLI's currency argument is absent.
- grpc.proto: Reduced number of getbalance(s) proto services from
four to one.
- GrpcWalletsService: Reduced number of getbalance(s) gRPC service
boilerplate methods from four to one.
- CoreApi, CoreWalletsService: Reduced number of getbalance(s) gRPC
service implementation methods from four to one.
- CliMain: Reduced number of getbalance(s) commands from four to one.
- BalancesInfo: Changed BsqBalanceInfo & BtcBalanceInfo field names
to bsq and btc, respectively, to make calling their accessors
more readable, less verbose.
- BtcBalanceInfo, BsqBalanceInfo: Defined static final EMPTY instances
as place holders in a BalancesInfo returned by the gRPC server, when
one or the other balance types is not requested. Would be nice
to serve Optional.empty() instead, but protobuf does not support
it or null.
- Adjusted affected api tests and build doc.
- Add optional makerFeeCurrencyCode argument to CLI's 'createoffer'.
- Add optional takerFeeCurrencyCode argument to CLI's 'takeoffer'.
- Add isCurrencyForMakerFeeBtc field to OfferInfo proto wrapper.
- Pass fee currency argument down to CoreOffersService and CoreTradesService's
create and take offer methods.
- Add maybeSetFeePaymentCurrencyPreference method to OfferUtil to
conditionally set the 'payFeeInBtc' preference before creating
or taking an offer.
- Adjust affected tests.
All the boilerplate for this method is included in this change, but
the implementation in CoreWalletsService#sendBsq is commented out
until the needed logic to send bsq is refactored out of desktop's
BsqSendView class -- to be included in the next PR.
- Added new method to CLI.
- Added the gRPC server boilerplate.
- Added the core implementation, commented out.
- Enabled tests.
- Added three new methods to CLI:
getbalances ... returns complete bsq and btc balance info
getbsqbalance ... returns complete bsq balance info
getbtcbalance ... returns complete btc balance info
The old getbalance method is deprecated and will be removed
if there is agreement to do that.
- Made the needed changes in the CLI's output formatting classes.
- Added new tests to existing BsqWalletTest, added new BtcWalletTest
and WalletBalancesTest.
- Added disabled tests for funding a bsq wallet (todo in next PR).
- Added new method to CLI, split some long msg strings into two lines,
and added a white space after a braceless else statement.
- Added the gRPC server boilerplate.
- Added the core implementation.
- Added a test, and moved method wallet tests into their own package.
This PR adds trade closing method stubs to keep funds in the
Bisq wallet or send them to an external BTC wallet.
- Add grpc protos
- Add new methods to GrpcTradesService, CoreApi
- Stub out implementations in CoreTradesService
- Add methods to CLI
API users will need to see their role as maker/taker when looking at trade details.
- Add getRole(trade) to TradeUtil.
- Add getTradeRole(tradeId) to CoreApi, CoreTradesService.
- Add role field to TradeInfo proto and its wrapper class.
- Add new core.offer.takeoffer.TakeOfferModel
Would have been nice to move more logic from
bisq.desktop.main.offer.takeoffer.TakeOfferDataModel,
but it has JFX dependencies that cannot be use in :core.
- Add grpc protos to support takeoffer, confirmpaymentsent, confirmpaymentreceived
Only takeoffer is implemented in this commit.
- Refactor OfferInfo grpc proto wrapper, and add offer state field
- Add new TradeInfo grpc proto wrapper
- Implement takeoffer on server and cli side
- Refactor offer/trade tests, add test cases
There are a number of use cases where a user may want to see a single
offer instead of every offer for a currency pair on the buy or sell side.
The changes are:
- Add getoffer to grpc.proto
- Add new method to GrpcOffersService, CoreApi, CoreOffersService
- Add new method to CLI
- Adjust create offer tests to use this new convenience
Separates offer placement from offer creation to fix tx result
handling problem in GrpcOffersService, and readies the core api
for a new CLI 'placeoffer' implementation. Offer placement still
happens in the api's 'createoffer', but we may want to change it
to show the created offer to a CLI user for review, prior to manual
placement via a new 'placeoffer offer-id' (of 'confirmoffer offer-id'?)
api method.
This commit contains most of the changes suggested by @chimp1984 in
his api-suggestions branch. See
961703ecea
A new 'registerdisputeagent' method was also added to MainCli, finishing
the work to support registration of mediators and refund agents on
arbitration daemons running in regtest mode. This method cannot be
used to register dispute agents on mainnet; users will see an error
msg if they try.
This change moves gRPC boilerplate classes from the :core.grpc pkg
into a new :daemon.grpc pkg.
* The :core.grpc pkg was renamed :core.api, and no longer has any
dependencies on gRPC libraries.
* All core service classes in the :core.api pkg are now package
protected, excepting CoreApi, making CoreApi the only possible
entry point for all Grpc*Service -> -Core*Service calls.
* All grpc service classes in the :daemon.grpc pkg are now package
protected, excepting GrpcServer; the only class depending on
Grpc*Service class is GrpcServer.
* gRPC dependencies were moved from the gradle.build file's :core
subproject to :daemon.
Previously, each wallet-related method was implemented with its own grpc
service. There is no need to do this, as a grpc service may declare
multiple rpc methods. This commit refactors everything wallet-related
into a single GrpcWalletService and also extracts a CoreWalletService
from CoreApi in order to avoid the latter becoming overly large.
Ideally, there would be no need for an abstraction in bisq.grpc called
CoreWalletService; we would ideally use such a service implemented in
bisq.core. The closest we have is WalletsManager, but it is not designed
to be used the way we are using it here in the grpc context. Rather than
making changes directly to core (which can be very risky), we will
rather make them here in this layer, designing exactly the "core wallet
service" we need, and can then later see about folding it into the
actual core.
* Renamed CliCommand to RpcCommand, differentiating it from cmd
string token(s)
* Added new CliConfig, based on :common Config
* Injected Config into CoreApi to make bisq.properties and cmd line
args available to server
* Added cleartext username:password BisqCallCredentials to :cli, and
an AuthenticationInterceptor to :core (server)
* Moved :daemon resources folder to expected location under src/main
* Duplicated CompositeOptionSet, ConfigException and BisqException
in :cli because they are not visible from :common. (TODO refactor?)
CompositeOptionSet will be used to let command line parameters
override params set in config file.
* Removed outdated references to :cli in a couple of comments in
:common Config
* gRPC parameters & command names are lowercase to mimic bitcoind/rpc
TBD
* Decide what rpc auth param names to use, to differentiate them from
bitcoind rpc user/passord param names
Protobuf definition files were moved from common and core to a new
protodefinition subproject.
The two main reasons for doing this are to speed up builds by not
having to regenerate common and core protobuf classes
every time a change is made in those subprojects, and to remove
the grpc cli's direct dependency on core, and the transitive dependency
on common.
In order to accomplish this, cli's BisqCliMain was stripped of
its dependencies on common and core. Cli can only get the version
and balance now.
gRPC stub boilerplate was moved from BisqCliMain to a CliCommand
class to avoid some of the bloat that is going to happen as the
read-response loop supports more rpc commands.
In previous commits, BisqEnvironment functionality has been fully ported
to the new, simpler and more type-safe Config class. This change removes
BisqEnvironment and all dependencies on the Spring Framework Environment
interface that it implements.
The one exception is the pricenode module, which is separate and apart
from the rest of the codebase in that it is a standalone, Spring-based
HTTP service.
Prior to this commit, BisqExecutable has been responsible for parsing
command line and config file options and BisqEnvironment has been
responsible for assigning default values to those options and providing
access to option values to callers throughout the codebase.
This approach has worked, but at considerable costs in complexity,
verbosity, and lack of any type-safety in option values. BisqEnvironment
is based on the Spring Framework's Environment abstraction, which
provides a great deal of flexibility in handling command line options,
environment variables, and more, but also operates on the assumption
that such inputs have String-based values.
After having this infrastructure in place for years now, it has become
evident that using Spring's Environment abstraction was both overkill
for what we needed and limited us from getting the kind of concision and
type saftey that we want. The Environment abstraction is by default
actually too flexible. For example, Bisq does not want or need to have
environment variables potentially overriding configuration file values,
as this increases our attack surface and makes our threat model more
complex. This is why we explicitly removed support for handling
environment variables quite some time ago.
The BisqEnvironment class has also organically evolved toward becoming a
kind of "God object", responsible for more than just option handling. It
is also, for example, responsible for tracking the status of the user's
local Bitcoin node, if any. It is also responsible for writing values to
the bisq.properties config file when certain ban filters arrive via the
p2p network. In the commits that follow, these unrelated functions will
be factored out appropriately in order to separate concerns.
As a solution to these problems, this commit begins the process of
eliminating BisqEnvironment in favor of a new, bespoke Config class
custom-tailored to Bisq's needs. Config removes the responsibility for
option parsing from BisqExecutable, and in the end provides "one-stop
shopping" for all option parsing and access needs.
The changes included in this commit represent a proof of concept for the
Config class, where handling of a number of options has been moved from
BisqEnvironment and BisqExecutable over to Config. Because the migration
is only partial, both Config and BisqEnvironment are injected
side-by-side into calling code that needs access to options. As the
migration is completed, BisqEnvironment will be removed entirely, and
only the Config object will remain.
An additional benefit of the elimination of BisqEnvironment is that it
will allow us to remove our dependency on the Spring Framework (with the
exception of the standalone pricenode application, which is Spring-based
by design).
Note that while this change and those that follow it are principally a
refactoring effort, certain functional changes have been introduced. For
example, Bisq now supports a `--configFile` argument at the command line
that functions very similarly to Bitcoin Core's `-conf` option.
Problem: a stack trace was being thrown during daemon startup from
BisqDaemonMain.onSetupComplete when it attempted to start a
second BisqGrpcServer and failed to bind to the already-bound port.
The first BisqGrpcServer is started in
BisqDaemonMain.onApplicationStarted much earlier in the startup process.
Solution: remove the second attempt to start the server by removing
BisqDaemonMain's implementation of onSetupComplete, and in turn remove
the now-obsolete bisqGrpcServer field as well.
This change also eliminates the BisqGrpcServer.blockUntilShutdown
method, which in turn called the underlying grpc server's
awaitTermination method. As the comment there explained, this was
thought to be necessary because grpc "does not use daemon threads by
default", but this is actually incorrect. According to the grpc Javadoc
at [1], "Grpc uses non-daemon Threads by default and thus a Server will
continue to run even after the main thread has terminated."
[1]: https://git.io/JePjn
- Rename package bisq.grpc => bisq.daemon.app
- Rename BisqGrpcApp => BisqDaemon
- Rename BisqGrpcServerMain => BisqDaemonMain
The script `bisq-grpc` has been renamed to `bisq-daemon` accordingly
(and will later be customized to `bisqd`). To see everything in action,
issue the following commands:
$ gradle build
$ ./bisq-daemon --appName=Bisq-throwaway --daoActivated=false
$ echo getVersion | ./bisq-cli # in a second terminal
1.2.3
