Commit Graph

127 Commits

Author SHA1 Message Date
dni ⚡
1646b087cf
adding bolt11 lib and removing bolt11.py from the codebase (#1817)
* add latest bolt11 lib

decode exception handling for create_payment
fix json response for decode
bugfix hexing description hash
improvement on bolt11 lib
update to bolt11 2.0.1
fix clnrest
* bolt 2.0.4
* refactor core/crud.py

* catch bolt11 erxception clnrest
2023-09-25 12:06:54 +02:00
jackstar12
4c16675b3b
Fix payments chart (#1851)
* feat: payment history endpoint

* test payment history

* use new endpoint in frontend

* refactor tests
2023-09-12 13:38:30 +01:00
dni ⚡
fee40d7321
[REFACTOR] core/__init__ to not have circular import issues (#1876)
* F541 fix

remove unused workflow and combine linters into one

add lnbits/static to ruff ignore
remote setupnode

ignore upgrades for mypy

ignore F401 for __init__ files

unused noqa

ignore upgrades for black

F821: undefine name

disabled and logged webhook_listener for opennode and lnpay because they are obvisouly not working

E402: module level import not at top of file

fixup

revert breaking changes wait for PR #1876

https://github.com/lnbits/lnbits/pull/1876

E721 fixes, only popped up for python3.9 not 3.10

[REFACTOR] core/__init__ to not have circular import issues

WIP

add db for backwards compat

fix pyright

make mypy happy again

pyright did not catch those, i think mypy got confused with relative imports. maybe we should use absolute ones everywhere

E402: module level import not at top of file

dont forget to add core_app

rebase on ruff pr

f

remo

format

* fix clnrest

* ignore E402 in conftest

* refactoring issues

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-09-12 11:25:05 +01:00
dni ⚡
f19d59d429
[BUG] self payments for the same wallet (#1914)
* [BUG] self payments for fakewallet

make it possible to pay to yourself on fakewallet

f

* bugfix selfpayments for fakewallet

* delete by wallet and fix previous_payment check (#1919)

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-09-12 11:23:23 +01:00
jackstar12
a97f298586
[TEST] use test data as mockdata (#1929)
* add db group to cli

* delete mock_data.zip

* generate migration data through tests
2023-09-12 11:21:05 +01:00
jackstar12
c29e7a1759
[TEST] use clean db in postgres tests (#1928) 2023-09-12 11:02:01 +01:00
dni ⚡
bda054415a
[FEAT] improve update_admin_settings (#1903)
* [FEAT] improve update_admin_settings

while working on the push notification pr i found it very hard just to update
2 settings inside the db, so i improved upon update_admin_settings.
now you just need to provide a dict with key/values you want to update inside db.

also debugging the endpoints for update_settings i found despite the type of `EditableSettings`
fastapi did in fact pass a dict.

* t

* use `EditableSettings` as param in update_settings

* fix settings model validation

we previously overrode the pydantic validation with our own method

* make `LnbitsSettings` a `BaseModel` and only add `BaseSettings` later

this allows us to instantiate `EditableSettings` without the environment values being loaded in

* add test

* forbid extra fields in update api

* fixup

* add test

* test datadir

* move UpdateSettings

* fix compat

* fixup webpush

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-09-12 10:59:32 +01:00
Tiago Vasconcelos
576e20d0cd
Add deleted flag wallet (#1826)
* add deleted flag on wallets

set deleted on delete wallet

2 twelves

format

fail on create invoice

make deleted check on SQL query

nazi flake8

add_test

boom... it works and passes!!

* add app fixture

vlad's recommendations

add deleted

* Add deleted flag to Wallet

* restore crud

* do not check for wallet in services.py

* add deleted flag on wallets

set deleted on delete wallet

2 twelves

format

fail on create invoice

make deleted check on SQL query

nazi flake8

add_test

boom... it works and passes!!

* add app fixture

vlad's recommendations

* add deleted

* error checks

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-09-11 14:06:31 +01:00
dni ⚡
62435b3723
[TEST] workflows, added 10s timeout to lncli subprocess and logging (#1910)
* [FIX] workflows

testing why workflwos unstable instable

* debug run_command

* add timeout to process run

* log lncli output

* try catch json exception in clnrest

* settle does not return a json

* add 10 minutes timeout to regtests if they ever get stuck

* better subprocess handling

* fix race condition

* test 3

* remove

* run nr 4

* increase waits for race conditions

* revert clnrest

* better fiat amount test
2023-08-30 11:01:32 +01:00
jackstar12
2623e9247a
track fiat value of payments (#1789)
* save fiat_amounts on payment creation

* show fiat amount in frontend

* add lnbits_default_accounting_currency

* extract fiat calculation logic into service

* move all currency conversions to calc_fiat_amounts

move all conversion logic into create_invoice so extensions can benefit aswell

* show user-defined and wallet-defined currency in frontend

* remove sat from fiat units

* make bundle

* improve tests

* debug log
2023-08-28 11:00:59 +01:00
dni ⚡
7a37e72915
[REFACTOR] replace async_wrap with run_sync (#1858)
* replace async_wrap with run_sync

formatting

remove unused fn

properly raise exception, not for timeout though

* [TEST] proper startup and shutdown (#1860)

* add proper startup / shutdown in tests

* fix event loop issues

because uvloop was installed in server.py which is not the main entry point when tests are ran.
this caused the loops referenced by the locks and queues to be a different one than the one used to run the tests
(only an issue in python 3.9)

* give openapi more time, does not matter anyway, regtest takes way longer

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>

remove uvloop

* fix test

* dont touch pyproject

* fix: install uvloop in conftest

not using uvloop at all causes tests to take way longer

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-08-28 10:59:56 +01:00
dni ⚡
48f25488df
[CHORE] cleanup cache and exception on make test (#1899)
* [CHORE] cleanup cache and exception on `make test`

moved the invalidate_forever task to proper location and use `create_permanent_task`.
and there was a Task never awaited Exception when running tests

* imrpove expiration test

* move cache into utils

* fix issue with pytest_asyncio

s

---------

Co-authored-by: jacksn <jkranawetter05@gmail.com>
2023-08-28 10:56:59 +01:00
dni ⚡
6077edad8f
[CHORE] remove redundant mocks (#1902)
* [CHORE] rmeove redundant mocks

* remove mock dependency
2023-08-28 10:56:20 +01:00
jackstar12
e50a7fb2d1
refactor exchange rates (#1847)
* simplify and cache exchange rate

note that exir was removed as a provider

* add binance as provider

* log exception

* add test

* add blockchain.com provider
2023-08-24 11:59:57 +01:00
dni ⚡
4e6f229db2
[CHORE] string formatting default length 88 (#1887)
* [CHORE] string formatting default length 88

uses blacks default off 88 and enabled autostringformatting

* formatting

* nitpicks jackstar

fix
2023-08-24 10:26:09 +01:00
callebtc
3a653630f1
Wallets: add cln-rest (#1775)
* receive and pay works

* fix linter issues

* import Paymentstatus from core.models

* fix test real payment

* fix get_payment_status check in lnbits

* fix tests?

* simplify

* refactor AsyncClient

* inline import of get_wallet_class

fixes the previous cyclic import

* invoice stream working

* add notes as a reminder to get rid of labels when cln-rest supports payment_hash

* create Payment dummy classmethod

* remove unnecessary fields from dummy

* fixes tests?

* fix model

* fix cln bug (#1814)

* auth header

* rename cln to corelightning

* add clnrest to admin_ui

* add to clnrest allowed sources

* add allowed sources to .env.example

* allow macaroon files

* add corelightning rest to workflow

* proper env names

* cleanup routine

* log wallet connection errors and fix macaroon clnrest

* print error on connection fails

* clnrest: handle disconnects faster

* fix test use of get_payment_status

* make format

* clnrest: add unhashed_description

* add unhashed_description to test

* description_hash test

* unhashed_description not supported by clnrest

* fix checking_id return in api_payments_create_invoice

* refactor test to use client instead of api_payments

* formatting, some errorlogging

* fix test 1

* fix other tests, paid statuses was missing

* error handling

* revert unnecessary changes (#1854)

* apply review of motorina0

---------

Co-authored-by: jackstar12 <jkranawetter05@gmail.com>
Co-authored-by: jackstar12 <62219658+jackstar12@users.noreply.github.com>
Co-authored-by: dni  <office@dnilabs.com>
2023-08-23 08:59:39 +02:00
dni ⚡
eb0b06f98c
[DEV] add ruff for linting and combine linters into one workflow (#1875)
* remove unused and update black + precommit
* makefile add ruff remove unused
* F541 fix
* complete ruff ignore
* remove unused workflow and combine linters into one
add lnbits/static to ruff ignore
save preview and linelength for later
define target version for black
* ignore upgrades for mypy
* remove flake8
* ignore F401 for __init__ files
* unused pylint comment
* unused noqa
* ignore upgrades for black
* run linting on py3.9 and py3.10
* dont assume python
2023-08-19 07:38:04 +02:00
dni ⚡
59acd3a2ef
[REFACTOR] cleanup views/api.py (#1865)
pull out models
line lengths
change models for tests
2023-08-18 10:22:22 +01:00
dni ⚡
c8414829d7
[TEST] dont install default extensions for tests (#1861) 2023-08-16 19:19:21 +02:00
dni ⚡
f0a66e41fb
[CHORE] flake8 issues E402, E721 and F821 (#1874)
* F821: undefine name
disabled and logged webhook_listener for opennode and lnpay because they are obviously not working
* E402: module level import not at top of file
* E721 fixes, only popped up for python3.9 not 3.10
2023-08-16 12:22:14 +02:00
dni ⚡
f096b51f70
[TEST] proper credit_wallet function from services (#1862) 2023-08-07 21:49:37 +02:00
jackstar12
2577ce7f81
add cache utility (#1790)
* add simple caching utility

* test cache

* remove prefix, default on get

* check expiry in pop aswell

* remove unnecessary type

* improve invalidation task

increase default interval to 10 seconds - doesnt have to check that often.
instead of recreating the dict everytime mutate the existing one
2023-08-02 14:13:31 +02:00
callebtc
9f40b3cdbd
Tests: hold invoice tests pending during payment (#1839)
* tests: hold invoice tests pending during payment

* sigh...

* ok

* hash preimage

* preimage_hash == payment_hash

* should error

* check that payment is gone

* cancel task while in-flight

* lnbits endpoing return error

* return error

* add both options

* fix list

* respond to error

* return false if payment status errors

* outgoing invoice should be deleted after one status check

* test api_payments_create error
2023-07-31 20:25:22 +02:00
jackstar12
1ab81f6d7e
Bug: invalid type converter (#1842)
* temporary test case

* remove buggy converter
2023-07-31 10:21:45 +02:00
dni ⚡
7f0c7138af
TEST: add hold invoices to test helpers and a basic test (#1838)
* add hold invoices and basic test

* run request in parallel

* create_task

* dont cancel it

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-07-26 14:18:14 +02:00
jackstar12
dda6c1b3c1
websocket internal payment notifications (#1831)
* add send_payment_notification service
payment notifications are sent from multiple places with inconsistent and incomplete data
* adopt new send_payment_notification service
* add tests
2023-07-26 12:08:22 +02:00
callebtc
11fec7a889
[Wallets] CLN: fix pending state check (#1770)
* better checking

* flake8 fix

* make format

* invoices scope are function for tests

* invoice back to sessionbut keep real_invoice for now

* make format

* comment

* get payment by checking id and test
2023-07-24 12:00:41 +02:00
jackstar12
bc55d52ea2
Refactor get walletclass (#1776)
* move `get_wallet_class` to wallets module

* adjust imports, fix type issues

flake8
2023-06-27 16:11:00 +02:00
Arc
7e1f43933d
Adds security tools, such as a rate limiter, IP block/allow, server logs (#1606)
* added ratelimiter
* Adds server logs to admin ui
* Added IP allow/ban list
* fixed remove ips
* Split rate limit number and unit
* security tab and background tasks for killswitch
* fix test for auditor api

---------

Co-authored-by: dni  <office@dnilabs.com>
2023-06-20 11:26:33 +02:00
jackstar12
e6499104c0
Wallets refactor (#1729)
* feat: cleanup function for wallet

* update eclair implementation

* update lnd implementation

* update lnbits implementation

* update lnpay implementation

* update lnbits implementation

* update opennode implementation

* update spark implementation

* use base_url for clients

* fix lnpay

* fix opennode

* fix lntips

* test real invoice creation

* add small delay to test

* test paid invoice stream

* fix lnbits

* fix lndrest

* fix spark

fix spark

* check node balance in test

* increase balance check delay

* check balance in pay test aswell

* make sure get_payment_status is called

* fix lndrest

* revert unnecessary changes
2023-06-19 12:12:00 +02:00
dni ⚡
c48e2329c5
TEST: fix tests for real wallets (#1761)
* fix tests for real wallets
* change realinvoice provider to lnd-1
* use lnd-3 for tests
* fix eclair tests
* use bullseye in dockerimage
2023-06-15 12:36:28 +02:00
jackstar12
c0f66989cb
Serverside Pagination for payments (#1613)
* initial backend support

* implement payments pagination on frontend

* implement search for payments api

* fix pyright issues

* sqlite support for searching

* backwards compatability

* formatting, small fixes

* small optimization

* fix sorting issue, add error handling

* GET payments test

* filter by dates, use List instead of list

* fix sqlite

* update bundle

* test old payments endpoint aswell

* refactor for easier review

* optimise test

* revert unnecessary change

---------

Co-authored-by: dni  <office@dnilabs.com>
2023-05-09 09:18:53 +01:00
Pavol Rusnak
b2bc378a6f
small followup to #1597 2023-04-03 17:54:55 +02:00
Pavol Rusnak
689c443d72
increase readability by using f-strings (#1597)
* increase readability by using f-strings

* readd merge conflict stuff

---------

Co-authored-by: calle <93376500+callebtc@users.noreply.github.com>
2023-04-03 15:34:17 +02:00
dni ⚡
fe9e821af5
BUG: proper exception for already paid internal invoices (#1593)
* BUG: proper exception for paid interal invoices

* test should now fail, because we pay invoice twice, and should also work in regtest

* sorting

* unpack check_internal

* introduce another crud fn for checking internal paid payment

* rename

---------

Co-authored-by: callebtc <93376500+callebtc@users.noreply.github.com>
2023-04-03 14:44:17 +02:00
dni ⚡
ca75b328a1
remove invoices (#1515)
* remove invoices

* remove invoices tests

* remove unused tests/extensions dir

* fix issue

* formatting

* test passing locally. and i dont know why

* fix _extension_dir loading

* gitkeep extension dir

* formatting

* remove bundle from check

* do not prettier compliled css

* do not prettier compliled css

* create extension dir, if it does not exists, just to be sure

* Update lnbits/extension_manager.py

---------

Co-authored-by: calle <93376500+callebtc@users.noreply.github.com>
2023-02-20 15:34:18 +01:00
dni ⚡
4211998959
remove lnaddress (#1525)
* remove lnaddress
2023-02-20 09:08:23 +01:00
dni ⚡
451c8f7a9e
remove bleskomat (#1521) 2023-02-17 16:06:04 +01:00
Arc
1026b242ba
Removed lnurlp (#1511)
* Removed lnurlp
* comment out extension checks for lnurlp

---------

Co-authored-by: dni  <office@dnilabs.com>
2023-02-17 15:21:53 +01:00
dni ⚡
7baa248204
remove boltz 2023-02-15 10:06:21 +01:00
dni ⚡
db97ef5543
reintroduce extension migration check (#1490)
* reintroduce extension migration check

* introduce checkversions again and remove extensions from dbversions table
2023-02-14 13:06:01 +01:00
dni ⚡
3276347d5d
remove all extensions from migration data at once (#1489)
* remove all extnesions from migration data at once

* nerf workflow for now

* check dbversions
2023-02-14 12:42:46 +01:00
ben
82add2189a Removes events 2023-02-11 08:53:24 +00:00
dni ⚡
0dffb2c1a0
moving build file into tools, keep rootdir clean (#1434)
* moving build file into tools, keep rootdir clean

* formatting

* make flake happy
2023-02-01 17:19:56 +01:00
dni ⚡
3ac6a98adc
TEST: add helpers and a test to pay external invoice (#1414)
* add helpers and a test to pay external invoice

* skip real test in fakewallet

* formatting

* fix test

* formatting
2023-02-01 16:54:07 +01:00
ben
9b21e2028e Removed example from mock 2023-01-31 22:39:10 +00:00
Pavol Rusnak
58a7098be5
fix unused variable 2023-01-30 11:46:45 +00:00
Pavol Rusnak
24fbb9ef5d
fix flake8 F811 (redefinition of unused) 2023-01-30 11:46:45 +00:00
Pavol Rusnak
a49a654137
fix flake8 F401+F403 (module imported but unused + unable to detect undefined names) 2023-01-30 11:46:45 +00:00
Pavol Rusnak
a02f147482
fix flake8 E302 (expected 2 blank lines) 2023-01-30 11:46:44 +00:00