The BIP 39 wordlist contained two significant technical errors:
- Byte Order Marker (BOM) U+FEFF at the beginning of the first line,
preceding the word "abaisser".
- No newline '\n' char terminating the last line, after "zoologie".
The former may cause user loss of funds. An implementation which
generates a mnemonic phrase and also turns it into a BIP 39 seed value
may feed the string "<U+FEFF>abaisser" to the KDF, while displaying the
word "abaisser" to the user. Of course, it cannot be expected that the
user would enter "<U+FEFF>abaisser" upon attempt to restore a wallet.
In the face of a buggy wordlist, whitespace handling and normalization
cannot be absolutely relied on to remove a notoriously mischievous
character. Those who provide technical support may be well advised to
ask French users with unrestorable wallets, "Did your mnemonic phrase
contain the word 'abaisser'?"
The latter broke the shell script I use to massage wordlists into C
sources when building https://github.com/nym-zone/easyseed .
I know of only one commonplace platform where software regularly
prepends UTF-8 files with a spurious U+FEFF, and oftentimes omits a line
terminator on the last line even when asked to create a Unix ('\n') text
file. It is RECOMMENDED that new wordlists be examined for correctness
using standard shell tools on a sane platform.
I left it unclear / open to interpretation on whether to use ideograpic
spaces, but realized that without being specific on its necessity,
developers may implement something that would cause trouble with the
Japanese user. (two words looking like one word, or phrase verification
failing because it can't handle ideographic spaces, etc.)
