2017-06-27 01:30:22 +02:00
# Protobuffer FAQ
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
NOTE: this doc is out of date and should probably be deleted. Protobuffer stuff is all managed in bisq-core now, not bisq-desktop.
2017-07-16 15:24:40 +02:00
## Installation
2017-07-03 13:09:17 +02:00
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
Protobuffer is installed automatically via the Gradle build.
2017-07-03 13:09:17 +02:00
2018-05-07 21:23:00 +02:00
## Why Protobuffer?
2017-06-27 01:30:22 +02:00
There are a number of reasons why protobuffer was chosen, here are some of them:
* avoids java serialisation security issues
* smaller in size than java serialisation (less network usage)
2018-05-07 21:23:00 +02:00
* All P2P network messages are described in a clear Protobuffer schema
2017-06-27 01:30:22 +02:00
* allows to evolve your schema in a backward compatible way
* can generate code in many languages, making alternative bisq clients or monitoring tools easier
2018-05-07 21:23:00 +02:00
## Which classes are transformed to Protobuffer?
2017-06-27 01:30:22 +02:00
All classes in the 'wire' module. This module contains the following classes:
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
* classes sent over the wire (P2P network)
2017-06-27 01:30:22 +02:00
* classes serialized to disk
2018-05-07 21:23:00 +02:00
## Where are the Protobuffer related files?
2017-06-27 01:30:22 +02:00
2018-05-07 21:23:00 +02:00
The Protobuffer schema file(s), generated classes and domain classes are in the 'wire' module.
2017-06-27 01:30:22 +02:00
Look for *.proto for the schema files.
## How is serialisation done (Java -> Protobuffer)
Some interfaces have a 'toProtobuf' method to force all extending classes to implement that method.
## How is deserialisation done (Java -> Protobuffer)
Some interfaces have a 'toProtobuf' method to force all extending classes to implement that method.
2018-05-07 21:23:00 +02:00
## If fields are not filled in, what are Protobuffer's default values?
2017-06-27 01:30:22 +02:00
Read this very carefully:
https://developers.google.com/protocol-buffers/docs/proto3#default
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
## How to handle Enums
2017-06-27 01:30:22 +02:00
For Java -> Protobuffer, you should extract the name from the Java enum:
.setContext(PB.AddressEntry.Context.valueOf(context.name()))
For Protobuffer -> Java, use the ProtoUtil helper method 'enumFromProto' to avoid crashes:
ProtoUtil.enumFromProto(OfferPayload.Direction.class, direction.name());
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
## How to handle Date
2017-06-27 01:30:22 +02:00
For Java -> Protobuffer, you should extract the name from the Java enum:
.setDate(date.getTime())
For Protobuffer -> Java, do the opposite:
Date date = new Date(PB.bla.getDate());
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
2017-06-27 01:30:22 +02:00
# Other Stuff
## Frameworks
- Drop-in replacement for serialization, fixes size concerns but not backward compat:
https://ruedigermoeller.github.io/fast-serialization/
-
## Checklist and conventions
### Code style
* Use line breaks after each param in constructor and PB methods
* Use line breaks at PB builders for best readability
* Use after the constructor a comment separator with the PB stuff and after the PB stuff a comment separator with API
* Use same order of fields as in main constructor and follow that order in the PB methods to make it easier to spot missing fields
### Conventions
* Try to use value objects for data which gets serialized with PB
* Use toProto and a static fromProto in the class which gets serialized
* Use proto as name for the PB param in the fromProto method
2018-05-07 21:23:00 +02:00
* If a constructor is used only for PB make it private and put it to the PB section, so it's clear it is used only in that context
2017-06-27 01:30:22 +02:00
* Use same name for classes and fields in PB definition as in the java code base
* Use final
* Use Lombok Annotations
* Annotate all nullable fields with @Nullable
* If nullable fields must not be null at time of serialisation use checkNotNull with comment inside
* Use ProtoUtil convenience methods (e.g. stringOrNullFromProto, byteArrayOrNullFromProto, collectionToProto, enumFromProto,..)
* Use ProtoUtil.enumFromProto for all enums
* Enum in PB definition file needs an additional first entry with PB_ERROR in case of multiple enums in one message add postfix of enum (PB_ERROR_REASON = 0;)
* When serializing a custom type use the toProto and fromProto methods in the class of that type.
* Use the ProtoResolver classes for switching between different message cases. Pass the reference to the resolver if needed in fromProto
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
* For abstract super classes use a getBuilder method for handling the fields in that super class (e.g. PaymentAccountPayload)
2017-06-27 01:30:22 +02:00
* Use Payload as postfix for objects which are used in Envelopes or other Payloads if it helps to distinguish between the domain object and the value object (e.g. UserPayload)
* Separate network and persistable domains if possible
### Architecture
* Envelope is the base interface for all objects carrying PB data (messages, persisted objects)
* Payload is used inside Envelopes or other Payloads
* Interface structure:
Proto: base has Message toProtoMessage();
Envelope extends Proto: Marker interface for objects carrying PB data
NetworkEnvelope extends Envelope: Marker interface, has getDefaultBuilder for P2PMessageVersion
PersistableEnvelope extends Envelope: Marker interface
Payload extends Proto: Marker interface for objects used inside other Payload or Envelope objects
NetworkPayload extends Payload: Marker interface
PersistablePayload extends Payload: Marker interface
ProtoResolver: Base for resolver (switch message cases)
NetworkProtoResolver extends ProtoResolver: Marker interface
CoreNetworkProtoResolver implements NetworkProtoResolver: Implements switches for network messages
PersistableProtoResolver extends ProtoResolver: Marker interface
CorePersistableProtoResolver implements NetworkProtoResolver: Implements switches for persistable messages
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
2017-06-27 01:30:22 +02:00
### Check list
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
* Treat nullable fields correctly in the toProto and fromProto methods.
2017-06-27 01:30:22 +02:00
PB does not support null values but use default implementation for not set fields.
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
Depending on the type there is: isEmpty (string, collections) or has[Propertyname]
2017-06-27 01:30:22 +02:00
* If using @EqualsAndHashCode or @Data/@Value make sure to use callSuper=true if the class is extending another class
* If collections are modifiable take care to wrap the result of PB to a modifiable collection. PB delivers unmodifiable collections
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
* For network envelopes use NetworkEnvelope.getDefaultBuilder() which includes the P2PMessageVersion.
2017-06-27 01:30:22 +02:00
Store the messageVersion in all NetworkEnvelope instances
// TODO update, outdated...
Replace Maven with Gradle in docs and scripts
This commit takes a pass through all documentation and scripts,
replacing any remaining `mvn` commands with their `gradle` equivalents,
replacing `target/*` paths with their `build/*` equivalents, and so on.
Note that the instruction to install Maven has been removed entirely
from documentation and scripts, as opposed to instructing users to
install Gradle, because with the Gradle wrapper (the `gradlew` script in
the root of this repository), it is unnecessary to install Gradle at
all. Users may still do this with, e.g. `brew install gradle` if they
like, but otherwise can get everything they need done with `./gradlew`
commands.
This commit also replaces lowercase 'bisq' with capitalized 'Bisq' where
appropriate in documentation.
Like the previous commit, this is a best-effort approach, and not all
scripts are perfectly up-to-date and tested. Indeed, many of the docs
and scripts we have now are essentially legacy documents and should
probably be scrapped or reworked completely.
2018-03-21 06:03:27 +01:00
## Actually transformed subtypes of Message
2017-06-27 01:30:22 +02:00
```
I Message (io.bisq.p2p)
I AnonymousMessage (io.bisq.p2p.network.messages)
+ PreliminaryGetDataRequest (io.bisq.p2p.peers.getdata.messages)
C BroadcastMessage (io.bisq.p2p.storage.messages)
+ AddDataMessage (io.bisq.p2p.storage.messages)
+ RefreshTTLMessage (io.bisq.p2p.storage.messages)
+ RemoveDataMessage (io.bisq.p2p.storage.messages)
+ RemoveMailboxDataMessage (io.bisq.p2p.storage.messages)
+ CloseConnectionMessage (io.bisq.p2p.network.messages)
I DirectMessage (io.bisq.p2p.messaging)
I MailboxMessage (io.bisq.p2p.messaging)
+ DepositTxPublishedMessage (io.bisq.trade.protocol.trade.messages)
C DisputeMessage (io.bisq.arbitration.messages)
+ DisputeCommunicationMessage (io.bisq.arbitration.messages)
+ DisputeResultMessage (io.bisq.arbitration.messages)
+ OpenNewDisputeMessage (io.bisq.arbitration.messages)
+ PeerOpenedDisputeMessage (io.bisq.arbitration.messages)
+ PeerPublishedPayoutTxMessage (io.bisq.arbitration.messages)
+ FiatTransferStartedMessage (io.bisq.trade.protocol.trade.messages)
+ FinalizePayoutTxRequest (io.bisq.trade.protocol.trade.messages)
+ MockMailboxPayload (io.bisq.p2p.mocks)
+ PayDepositRequest (io.bisq.trade.protocol.trade.messages)
+ PayoutTxFinalizedMessage (io.bisq.trade.protocol.trade.messages)
+ PrefixedSealedAndSignedMessage (io.bisq.p2p.messaging)
+ PrivateNotificationMessage (io.bisq.alert)
T StressTestMailboxMessage (io.bisq.p2p.network)
T TestMessage (io.bisq.crypto)
C OfferMessage (io.bisq.trade.protocol.availability.messages)
+ OfferAvailabilityRequest (io.bisq.trade.protocol.availability.messages)
+ OfferAvailabilityResponse (io.bisq.trade.protocol.availability.messages)
T StressTestDirectMessage (io.bisq.p2p.network)
C TradeMessage (io.bisq.trade.protocol.trade.messages)
+ DepositTxPublishedMessage (io.bisq.trade.protocol.trade.messages)
D FiatTransferStartedMessage (io.bisq.trade.protocol.trade.messages)
D FinalizePayoutTxRequest (io.bisq.trade.protocol.trade.messages)
D PayDepositRequest (io.bisq.trade.protocol.trade.messages)
D PayoutTxFinalizedMessage (io.bisq.trade.protocol.trade.messages)
+ PublishDepositTxRequest (io.bisq.trade.protocol.trade.messages)
I GetDataRequest (io.bisq.p2p.peers.getdata.messages)
+ GetUpdatedDataRequest (io.bisq.p2p.peers.getdata.messages)
D PreliminaryGetDataRequest (io.bisq.p2p.peers.getdata.messages)
C KeepAliveMessage (io.bisq.p2p.peers.keepalive.messages)
+ Ping (io.bisq.p2p.peers.keepalive.messages)
+ Pong (io.bisq.p2p.peers.keepalive.messages)
T MockPayload (io.bisq.p2p.mocks)
C PeerExchangeMessage (io.bisq.p2p.peers.peerexchange.messages)
+ GetPeersRequest (io.bisq.p2p.peers.peerexchange.messages)
+ GetPeersResponse (io.bisq.p2p.peers.peerexchange.messages)
I SendersNodeAddressMessage (io.bisq.p2p.network.messages)
D GetPeersRequest (io.bisq.p2p.peers.peerexchange.messages)
D GetUpdatedDataRequest (io.bisq.p2p.peers.getdata.messages)
D PrefixedSealedAndSignedMessage (io.bisq.p2p.messaging)
I SupportedCapabilitiesMessage (io.bisq.p2p.messaging)
+ GetDataResponse (io.bisq.p2p.peers.getdata.messages)
D GetPeersRequest (io.bisq.p2p.peers.peerexchange.messages)
D GetPeersResponse (io.bisq.p2p.peers.peerexchange.messages)
D OfferAvailabilityRequest (io.bisq.trade.protocol.availability.messages)
D OfferAvailabilityResponse (io.bisq.trade.protocol.availability.messages)
D PreliminaryGetDataRequest (io.bisq.p2p.peers.getdata.messages)
```
== Actually transformed subtypes of Payload
```
+ Attachment (io.bisq.arbitration.payload)
I CapabilityRequiringPayload (io.bisq.p2p.storage.payload)
+ TradeStatistics (io.bisq.trade.statistics)
+ Contract (io.bisq.trade)
+ Dispute (io.bisq.arbitration)
+ DisputeResult (io.bisq.arbitration)
I ExpirablePayload (io.bisq.p2p.storage.payload)
I StoragePayload (io.bisq.p2p.storage.payload)
+ Alert (io.bisq.alert)
+ Arbitrator (io.bisq.arbitration)
+ Filter (io.bisq.filter)
I LazyProcessedStoragePayload (io.bisq.p2p.storage.payload)
+ CompensationRequestPayload (io.bisq.dao.compensation)
D TradeStatistics (io.bisq.trade.statistics)
+ MailboxStoragePayload (io.bisq.p2p.storage.payload)
+ Offer (io.bisq.trade.offerPayload)
I PersistedStoragePayload (io.bisq.p2p.storage.payload)
D CompensationRequestPayload (io.bisq.dao.compensation)
D TradeStatistics (io.bisq.trade.statistics)
+ NodeAddress (io.bisq.p2p)
C+PaymentAccountContractData (io.bisq.payment)
+ AliPayAccountContractData (io.bisq.payment)
2018-04-10 14:28:52 +02:00
+ WeChatPayAccountContractData (io.bisq.payment)
2017-06-27 01:30:22 +02:00
+ ChaseQuickPayAccountContractData (io.bisq.payment)
+ ClearXchangeAccountContractData (io.bisq.payment)
C+ CountryBasedPaymentAccountContractData (io.bisq.payment)
C+ BankAccountContractData (io.bisq.payment)
+ NationalBankAccountContractData (io.bisq.payment)
+ SameBankAccountContractData (io.bisq.payment)
+ SpecificBanksAccountContractData (io.bisq.payment)
+ CashDepositAccountContractData (io.bisq.payment)
+ SepaAccountContractData (io.bisq.payment)
+ CryptoCurrencyAccountContractData (io.bisq.payment)
+ FasterPaymentsAccountContractData (io.bisq.payment)
+ InteracETransferAccountContractData (io.bisq.payment)
+ OKPayAccountContractData (io.bisq.payment)
+ PerfectMoneyAccountContractData (io.bisq.payment)
+ SwishAccountContractData (io.bisq.payment)
+ USPostalMoneyOrderAccountContractData (io.bisq.payment)
+ Peer (io.bisq.p2p.peers.peerexchange)
+ PrivateNotification (io.bisq.alert)
+ ProtectedStorageEntry (io.bisq.p2p.storage.storageentry)
+ ProtectedMailboxStorageEntry (io.bisq.p2p.storage.storageentry)
+ PubKeyRing (io.bisq.common.crypto)
+ RawTransactionInput (io.bisq.btc.data)
I RequiresOwnerIsOnlinePayload (io.bisq.p2p.storage.payload)
D Offer (io.bisq.trade.offerPayload)
+ SealedAndSigned (io.bisq.common.crypto)
```
