Commit Graph

133 Commits

Author SHA1 Message Date
Vlad Stan
c9093715b7
[FEAT] Auth, Login, OAuth, create account with username and password #1653 (#2092)
no more superuser url!
delete cookie on logout
add usr login feature
fix node management
* Cleaned up login form
* CreateUser
* information leak
* cleaner parsing usr from url
* rename decorators
* login secret
* fix: add back `superuser` command
* chore: remove `fastapi_login`
* fix: extract `token` from cookie
* chore: prepare to extract user
* feat: check user
* chore: code clean-up
* feat: happy flow working
* fix: usr only login
* fix: user already logged in
* feat: check user in URL
* fix: verify password at DB level
* fix: do not show `Login` controls if user already logged in
* fix: separate login endpoints
* fix: remove `usr` param
* chore: update error message
* refactor: register method
* feat: logout
* chore: move comments
* fix: remove user auth check from API
* fix: user check unnecessary
* fix: redirect after logout
* chore: remove garbage files
* refactor: simplify constructor call
* fix: hide user icon if not authorized
* refactor: rename auth env vars
* chore: code clean-up
* fix: add types for `python-jose`
* fix: add types for `passlib`
* fix: return type
* feat: set default value for `auth_secret_key` to hash of super user
* fix: default value
* feat: rework login page
* feat: ui polishing
* feat: google auth
* feat: add google auth
* chore: remove `authlib` dependency
* refactor: extract `_handle_sso_login` method
* refactor: convert methods to `properties`
* refactor: rename: `user_api` to `auth_api`
* feat: store user info from SSO
* chore: re-arange the buttons
* feat: conditional rendering of login options
* feat: correctly render buttons
* fix: re-add `Claim Bitcoin` from the main page
* fix: create wallet must send new user
* fix:  no `username-password` auth method
* refactor: rename auth method
* fix: do not force API level UUID4 validation
* feat: add validation for username
* feat: add account page
* feat: update account
* feat: add `has_password` for user
* fix: email not editable
* feat: validate email for existing account
* fix: register check
* feat: reset password
* chore: code clean-up
* feat: handle token expired
* fix: only redirect if `text/html`
* refactor: remove `OAuth2PasswordRequestForm`
* chore: remove `python-multipart` dependency
* fix: handle no headers for exception
* feat: add back button on error screen
* feat: show user profile image
* fix: check account creation permissions
* fix: auth for internal api call
* chore: add some docs
* chore: code clean-up
* fix: rebase stuff
* fix: default value types
* refactor: customize error messages
* fix: move types libs to dev dependencies
* doc: specify the `Authorization callback URL`
* fix: pass missing superuser id in node ui test
* fix: keep usr param on wallet redirect
removing usr param causes an issue if the browser doesnt yet have an access token.
* fix: do not redirect if `wal` query param not present
* fix: add nativeBuildInputs and buildInputs overrides to flake.nix
* bump fastapi-sso to 0.9.0 which fixes some security issues
* refactor: move the `lnbits_admin_extensions` to decorators
* chore: bring package config from `dev`
* chore: re-add dependencies
* chore: re-add cev dependencies
* chore: re-add mypy ignores
* feat: i18n
* refactor: move admin ext check to decorator (fix after rebase)
* fix: label mapping
* fix: re-fetch user after first wallet was created
* fix: unlikely case that `user` is not found
* refactor translations (move '*' to code)
* reorganize deps in pyproject.toml, add comment
* update flake.lock and simplify flake.nix after upstreaming
overrides for fastapi-sso, types-passlib, types-pyasn1, types-python-jose
were upstreamed in https://github.com/nix-community/poetry2nix/pull/1463
* fix: more relaxed email verification (by @prusnak)
* fix: remove `\b` (boundaries) since we re using `fullmatch`
* chore: `make bundle`

---------

Co-authored-by: dni  <office@dnilabs.com>
Co-authored-by: Arc <ben@arc.wales>
Co-authored-by: jackstar12 <jkranawetter05@gmail.com>
Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
2023-12-12 11:38:19 +01:00
dni ⚡
9fcf566540
fix: time margin for deleting failed payments (#2147)
this pr adds a 15 minutes margin before deleting failed outgoing payments.
2023-12-02 00:41:09 +01:00
callebtc
6a27b91fcb
Add service fee to specific wallet (#2050)
* add service fee to specific wallet

* add to .env.example

* Added service fee wallet to manage server

* cleaned

* prettier

* Added badge for service fee

* Added tooltip

* Added service fee max

* allow ignoring service fee for internal transactions

* add fee_reserve_total helper funciton that includes service_fee

* html for admin ui

* typo

* Update .env.example

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>

* fix .env.template comment

* bundle

* WIP: expose fee reserve endpoint

---------

Co-authored-by: Arc <ben@arc.wales>
Co-authored-by: dni  <office@dnilabs.com>
Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
2023-11-21 11:11:21 +00:00
dni ⚡
dbf687c47a
feat: add useful query helpers for insert and update (#2020)
* feat: add useful query helpers for insert and update

i saw this bits of code all over the codebase, this `helpers.py`
tries to abstract that away and make it more readable.
2023-10-16 13:27:59 +02:00
dni ⚡
5b16f54857
[FEAT] cleanup GET /wallet endpoint, add wallet api routes (#1932)
* [FEAT] cleanup GET /wallet endpoint, add wallet api route
this removes the functionalitiy to create accounts and wallets via
the GET /wallet endpoint in generic.py

it add endpoints inside the api.py for it and the frontend is modified to use the api endpoints

this also simplifies for the `feat/login` for the route.

* remove stale generic tests and add api tests

* bug wrong endpoint create account

* vlad nitpick

* added checkif deleted is 404

* reload after renaming wallet

* another iteration with vlad

* create new wallet if it none exist

* fix delete refresh

* formatting
2023-09-25 14:06:00 +01:00
dni ⚡
eb73daffe9
[FEAT] Node Managment (#1895)
* [FEAT] Node Managment

feat: node dashboard channels and transactions
fix: update channel variables
better types
refactor ui
add onchain balances and backend_name
mock values for fake wallet
remove app tab
start implementing peers and channel management
peer and channel management
implement channel closing
add channel states, better errors
seperate payments and invoices on transactions tab
display total channel balance
feat: optional public page
feat: show node address
fix: port conversion
feat: details dialog on transactions
fix: peer info without alias
fix: rename channel balances
small improvements to channels tab
feat: pagination on transactions tab
test caching transactions
refactor: move WALLET into wallets module
fix: backwards compatibility
refactor: move get_node_class to nodes modules
post merge bundle fundle
feat: disconnect peer
feat: initial lnd support
only use filtered channels for total balance
adjust closing logic
add basic node tests
add setting for disabling transactions tab
revert unnecessary changes
add tests for invoices and payments
improve payment and invoice implementations
the previously used invoice fixture has a session scope, but a new invoice is required
tests and bug fixes for channels api
use query instead of body in channel delete
delete requests should generally not use a body
take node id through path instead of body for delete endpoint
add peer management tests
more tests for errors
improve error handling
rename id and pubkey to peer_id for consistency
remove dead code
fix http status codes
make cache keys safer
cache node public info
comments for node settings
rename node prop in frontend
adjust tests to new status codes
cln: use amount_msat instead of value for onchain balance
turn transactions tab off by default
enable transactions in tests
only allow super user to create or delete
fix prop name in admin navbar

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-09-25 15:04:44 +02:00
dni ⚡
1646b087cf
adding bolt11 lib and removing bolt11.py from the codebase (#1817)
* add latest bolt11 lib

decode exception handling for create_payment
fix json response for decode
bugfix hexing description hash
improvement on bolt11 lib
update to bolt11 2.0.1
fix clnrest
* bolt 2.0.4
* refactor core/crud.py

* catch bolt11 erxception clnrest
2023-09-25 12:06:54 +02:00
jackstar12
4c16675b3b
Fix payments chart (#1851)
* feat: payment history endpoint

* test payment history

* use new endpoint in frontend

* refactor tests
2023-09-12 13:38:30 +01:00
dni ⚡
fee40d7321
[REFACTOR] core/__init__ to not have circular import issues (#1876)
* F541 fix

remove unused workflow and combine linters into one

add lnbits/static to ruff ignore
remote setupnode

ignore upgrades for mypy

ignore F401 for __init__ files

unused noqa

ignore upgrades for black

F821: undefine name

disabled and logged webhook_listener for opennode and lnpay because they are obvisouly not working

E402: module level import not at top of file

fixup

revert breaking changes wait for PR #1876

https://github.com/lnbits/lnbits/pull/1876

E721 fixes, only popped up for python3.9 not 3.10

[REFACTOR] core/__init__ to not have circular import issues

WIP

add db for backwards compat

fix pyright

make mypy happy again

pyright did not catch those, i think mypy got confused with relative imports. maybe we should use absolute ones everywhere

E402: module level import not at top of file

dont forget to add core_app

rebase on ruff pr

f

remo

format

* fix clnrest

* ignore E402 in conftest

* refactoring issues

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-09-12 11:25:05 +01:00
dni ⚡
f19d59d429
[BUG] self payments for the same wallet (#1914)
* [BUG] self payments for fakewallet

make it possible to pay to yourself on fakewallet

f

* bugfix selfpayments for fakewallet

* delete by wallet and fix previous_payment check (#1919)

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-09-12 11:23:23 +01:00
jackstar12
a97f298586
[TEST] use test data as mockdata (#1929)
* add db group to cli

* delete mock_data.zip

* generate migration data through tests
2023-09-12 11:21:05 +01:00
jackstar12
c29e7a1759
[TEST] use clean db in postgres tests (#1928) 2023-09-12 11:02:01 +01:00
dni ⚡
bda054415a
[FEAT] improve update_admin_settings (#1903)
* [FEAT] improve update_admin_settings

while working on the push notification pr i found it very hard just to update
2 settings inside the db, so i improved upon update_admin_settings.
now you just need to provide a dict with key/values you want to update inside db.

also debugging the endpoints for update_settings i found despite the type of `EditableSettings`
fastapi did in fact pass a dict.

* t

* use `EditableSettings` as param in update_settings

* fix settings model validation

we previously overrode the pydantic validation with our own method

* make `LnbitsSettings` a `BaseModel` and only add `BaseSettings` later

this allows us to instantiate `EditableSettings` without the environment values being loaded in

* add test

* forbid extra fields in update api

* fixup

* add test

* test datadir

* move UpdateSettings

* fix compat

* fixup webpush

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-09-12 10:59:32 +01:00
Tiago Vasconcelos
576e20d0cd
Add deleted flag wallet (#1826)
* add deleted flag on wallets

set deleted on delete wallet

2 twelves

format

fail on create invoice

make deleted check on SQL query

nazi flake8

add_test

boom... it works and passes!!

* add app fixture

vlad's recommendations

add deleted

* Add deleted flag to Wallet

* restore crud

* do not check for wallet in services.py

* add deleted flag on wallets

set deleted on delete wallet

2 twelves

format

fail on create invoice

make deleted check on SQL query

nazi flake8

add_test

boom... it works and passes!!

* add app fixture

vlad's recommendations

* add deleted

* error checks

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-09-11 14:06:31 +01:00
dni ⚡
62435b3723
[TEST] workflows, added 10s timeout to lncli subprocess and logging (#1910)
* [FIX] workflows

testing why workflwos unstable instable

* debug run_command

* add timeout to process run

* log lncli output

* try catch json exception in clnrest

* settle does not return a json

* add 10 minutes timeout to regtests if they ever get stuck

* better subprocess handling

* fix race condition

* test 3

* remove

* run nr 4

* increase waits for race conditions

* revert clnrest

* better fiat amount test
2023-08-30 11:01:32 +01:00
jackstar12
2623e9247a
track fiat value of payments (#1789)
* save fiat_amounts on payment creation

* show fiat amount in frontend

* add lnbits_default_accounting_currency

* extract fiat calculation logic into service

* move all currency conversions to calc_fiat_amounts

move all conversion logic into create_invoice so extensions can benefit aswell

* show user-defined and wallet-defined currency in frontend

* remove sat from fiat units

* make bundle

* improve tests

* debug log
2023-08-28 11:00:59 +01:00
dni ⚡
7a37e72915
[REFACTOR] replace async_wrap with run_sync (#1858)
* replace async_wrap with run_sync

formatting

remove unused fn

properly raise exception, not for timeout though

* [TEST] proper startup and shutdown (#1860)

* add proper startup / shutdown in tests

* fix event loop issues

because uvloop was installed in server.py which is not the main entry point when tests are ran.
this caused the loops referenced by the locks and queues to be a different one than the one used to run the tests
(only an issue in python 3.9)

* give openapi more time, does not matter anyway, regtest takes way longer

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>

remove uvloop

* fix test

* dont touch pyproject

* fix: install uvloop in conftest

not using uvloop at all causes tests to take way longer

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-08-28 10:59:56 +01:00
dni ⚡
48f25488df
[CHORE] cleanup cache and exception on make test (#1899)
* [CHORE] cleanup cache and exception on `make test`

moved the invalidate_forever task to proper location and use `create_permanent_task`.
and there was a Task never awaited Exception when running tests

* imrpove expiration test

* move cache into utils

* fix issue with pytest_asyncio

s

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-08-28 10:56:59 +01:00
dni ⚡
6077edad8f
[CHORE] remove redundant mocks (#1902)
* [CHORE] rmeove redundant mocks

* remove mock dependency
2023-08-28 10:56:20 +01:00
jackstar12
e50a7fb2d1
refactor exchange rates (#1847)
* simplify and cache exchange rate

note that exir was removed as a provider

* add binance as provider

* log exception

* add test

* add blockchain.com provider
2023-08-24 11:59:57 +01:00
dni ⚡
4e6f229db2
[CHORE] string formatting default length 88 (#1887)
* [CHORE] string formatting default length 88

uses blacks default off 88 and enabled autostringformatting

* formatting

* nitpicks jackstar

fix
2023-08-24 10:26:09 +01:00
callebtc
3a653630f1
Wallets: add cln-rest (#1775)
* receive and pay works

* fix linter issues

* import Paymentstatus from core.models

* fix test real payment

* fix get_payment_status check in lnbits

* fix tests?

* simplify

* refactor AsyncClient

* inline import of get_wallet_class

fixes the previous cyclic import

* invoice stream working

* add notes as a reminder to get rid of labels when cln-rest supports payment_hash

* create Payment dummy classmethod

* remove unnecessary fields from dummy

* fixes tests?

* fix model

* fix cln bug (#1814)

* auth header

* rename cln to corelightning

* add clnrest to admin_ui

* add to clnrest allowed sources

* add allowed sources to .env.example

* allow macaroon files

* add corelightning rest to workflow

* proper env names

* cleanup routine

* log wallet connection errors and fix macaroon clnrest

* print error on connection fails

* clnrest: handle disconnects faster

* fix test use of get_payment_status

* make format

* clnrest: add unhashed_description

* add unhashed_description to test

* description_hash test

* unhashed_description not supported by clnrest

* fix checking_id return in api_payments_create_invoice

* refactor test to use client instead of api_payments

* formatting, some errorlogging

* fix test 1

* fix other tests, paid statuses was missing

* error handling

* revert unnecessary changes (#1854)

* apply review of motorina0

---------

Co-authored-by: jackstar12 <jkranawetter05@gmail.com>
Co-authored-by: jackstar12 <62219658+jackstar12@users.noreply.github.com>
Co-authored-by: dni  <office@dnilabs.com>
2023-08-23 08:59:39 +02:00
dni ⚡
eb0b06f98c
[DEV] add ruff for linting and combine linters into one workflow (#1875)
* remove unused and update black + precommit
* makefile add ruff remove unused
* F541 fix
* complete ruff ignore
* remove unused workflow and combine linters into one
add lnbits/static to ruff ignore
save preview and linelength for later
define target version for black
* ignore upgrades for mypy
* remove flake8
* ignore F401 for __init__ files
* unused pylint comment
* unused noqa
* ignore upgrades for black
* run linting on py3.9 and py3.10
* dont assume python
2023-08-19 07:38:04 +02:00
dni ⚡
59acd3a2ef
[REFACTOR] cleanup views/api.py (#1865)
pull out models
line lengths
change models for tests
2023-08-18 10:22:22 +01:00
dni ⚡
c8414829d7
[TEST] dont install default extensions for tests (#1861) 2023-08-16 19:19:21 +02:00
dni ⚡
f0a66e41fb
[CHORE] flake8 issues E402, E721 and F821 (#1874)
* F821: undefine name
disabled and logged webhook_listener for opennode and lnpay because they are obviously not working
* E402: module level import not at top of file
* E721 fixes, only popped up for python3.9 not 3.10
2023-08-16 12:22:14 +02:00
dni ⚡
f096b51f70
[TEST] proper credit_wallet function from services (#1862) 2023-08-07 21:49:37 +02:00
jackstar12
2577ce7f81
add cache utility (#1790)
* add simple caching utility

* test cache

* remove prefix, default on get

* check expiry in pop aswell

* remove unnecessary type

* improve invalidation task

increase default interval to 10 seconds - doesnt have to check that often.
instead of recreating the dict everytime mutate the existing one
2023-08-02 14:13:31 +02:00
callebtc
9f40b3cdbd
Tests: hold invoice tests pending during payment (#1839)
* tests: hold invoice tests pending during payment

* sigh...

* ok

* hash preimage

* preimage_hash == payment_hash

* should error

* check that payment is gone

* cancel task while in-flight

* lnbits endpoing return error

* return error

* add both options

* fix list

* respond to error

* return false if payment status errors

* outgoing invoice should be deleted after one status check

* test api_payments_create error
2023-07-31 20:25:22 +02:00
jackstar12
1ab81f6d7e
Bug: invalid type converter (#1842)
* temporary test case

* remove buggy converter
2023-07-31 10:21:45 +02:00
dni ⚡
7f0c7138af
TEST: add hold invoices to test helpers and a basic test (#1838)
* add hold invoices and basic test

* run request in parallel

* create_task

* dont cancel it

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-07-26 14:18:14 +02:00
jackstar12
dda6c1b3c1
websocket internal payment notifications (#1831)
* add send_payment_notification service
payment notifications are sent from multiple places with inconsistent and incomplete data
* adopt new send_payment_notification service
* add tests
2023-07-26 12:08:22 +02:00
callebtc
11fec7a889
[Wallets] CLN: fix pending state check (#1770)
* better checking

* flake8 fix

* make format

* invoices scope are function for tests

* invoice back to sessionbut keep real_invoice for now

* make format

* comment

* get payment by checking id and test
2023-07-24 12:00:41 +02:00
jackstar12
bc55d52ea2
Refactor get walletclass (#1776)
* move `get_wallet_class` to wallets module

* adjust imports, fix type issues

flake8
2023-06-27 16:11:00 +02:00
Arc
7e1f43933d
Adds security tools, such as a rate limiter, IP block/allow, server logs (#1606)
* added ratelimiter
* Adds server logs to admin ui
* Added IP allow/ban list
* fixed remove ips
* Split rate limit number and unit
* security tab and background tasks for killswitch
* fix test for auditor api

---------

Co-authored-by: dni  <office@dnilabs.com>
2023-06-20 11:26:33 +02:00
jackstar12
e6499104c0
Wallets refactor (#1729)
* feat: cleanup function for wallet

* update eclair implementation

* update lnd implementation

* update lnbits implementation

* update lnpay implementation

* update lnbits implementation

* update opennode implementation

* update spark implementation

* use base_url for clients

* fix lnpay

* fix opennode

* fix lntips

* test real invoice creation

* add small delay to test

* test paid invoice stream

* fix lnbits

* fix lndrest

* fix spark

fix spark

* check node balance in test

* increase balance check delay

* check balance in pay test aswell

* make sure get_payment_status is called

* fix lndrest

* revert unnecessary changes
2023-06-19 12:12:00 +02:00
dni ⚡
c48e2329c5
TEST: fix tests for real wallets (#1761)
* fix tests for real wallets
* change realinvoice provider to lnd-1
* use lnd-3 for tests
* fix eclair tests
* use bullseye in dockerimage
2023-06-15 12:36:28 +02:00
jackstar12
c0f66989cb
Serverside Pagination for payments (#1613)
* initial backend support

* implement payments pagination on frontend

* implement search for payments api

* fix pyright issues

* sqlite support for searching

* backwards compatability

* formatting, small fixes

* small optimization

* fix sorting issue, add error handling

* GET payments test

* filter by dates, use List instead of list

* fix sqlite

* update bundle

* test old payments endpoint aswell

* refactor for easier review

* optimise test

* revert unnecessary change

---------

Co-authored-by: dni  <office@dnilabs.com>
2023-05-09 09:18:53 +01:00
Pavol Rusnak
b2bc378a6f
small followup to #1597 2023-04-03 17:54:55 +02:00
Pavol Rusnak
689c443d72
increase readability by using f-strings (#1597)
* increase readability by using f-strings

* readd merge conflict stuff

---------

Co-authored-by: calle <93376500+callebtc@users.noreply.github.com>
2023-04-03 15:34:17 +02:00
dni ⚡
fe9e821af5
BUG: proper exception for already paid internal invoices (#1593)
* BUG: proper exception for paid interal invoices

* test should now fail, because we pay invoice twice, and should also work in regtest

* sorting

* unpack check_internal

* introduce another crud fn for checking internal paid payment

* rename

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-04-03 14:44:17 +02:00
dni ⚡
ca75b328a1
remove invoices (#1515)
* remove invoices

* remove invoices tests

* remove unused tests/extensions dir

* fix issue

* formatting

* test passing locally. and i dont know why

* fix _extension_dir loading

* gitkeep extension dir

* formatting

* remove bundle from check

* do not prettier compliled css

* do not prettier compliled css

* create extension dir, if it does not exists, just to be sure

* Update lnbits/extension_manager.py

---------

Co-authored-by: calle <93376500+callebtc@users.noreply.github.com>
2023-02-20 15:34:18 +01:00
dni ⚡
4211998959
remove lnaddress (#1525)
* remove lnaddress
2023-02-20 09:08:23 +01:00
dni ⚡
451c8f7a9e
remove bleskomat (#1521) 2023-02-17 16:06:04 +01:00
Arc
1026b242ba
Removed lnurlp (#1511)
* Removed lnurlp
* comment out extension checks for lnurlp

---------

Co-authored-by: dni  <office@dnilabs.com>
2023-02-17 15:21:53 +01:00
dni ⚡
7baa248204
remove boltz 2023-02-15 10:06:21 +01:00
dni ⚡
db97ef5543
reintroduce extension migration check (#1490)
* reintroduce extension migration check

* introduce checkversions again and remove extensions from dbversions table
2023-02-14 13:06:01 +01:00
dni ⚡
3276347d5d
remove all extensions from migration data at once (#1489)
* remove all extnesions from migration data at once

* nerf workflow for now

* check dbversions
2023-02-14 12:42:46 +01:00
ben
82add2189a Removes events 2023-02-11 08:53:24 +00:00
dni ⚡
0dffb2c1a0
moving build file into tools, keep rootdir clean (#1434)
* moving build file into tools, keep rootdir clean

* formatting

* make flake happy
2023-02-01 17:19:56 +01:00