Commit graph

469 commits

Author SHA1 Message Date
fusion44
63b136d5ca
fix(apps): report electrs installed/configured/online in advanced status
_do_electrs_status_advanced only handled the negative cases: each guard
set installed/configured/status to its false-y value and returned early.
On the happy path (electrs installed, configured and running) it fell
through, populated the ports and sync details, but never set
s.installed/s.configured/s.status - so they kept their AppStatus
defaults (False/False/offline).

The /apps/status_advanced/electrs endpoint therefore reported electrs as
not installed and offline while simultaneously returning its ports and
sync details, disagreeing with the app_state_update_message status
(fusion44/blitz_api#286). Set the positive values when each guard passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 13:07:52 +02:00
fusion44
a9bbc9a27c
fix(api): release Redis connections in channel listeners/notifiers
Some checks failed
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
BaseChannelListener.listen() only called pubsub.unsubscribe() in its
finally block: the pubsub connection and the listener's own Redis client
were never released, leaking a connection for every install/uninstall
(_watcher) and every recreation of the app-status listener. The Celery
task notifiers were likewise never closed, and used the deprecated
Redis.close() instead of aclose().

- add aclose() to BaseChannelNotifier and BaseChannelListener
- listen() now closes the pubsub and its Redis client in finally
- close the notifiers in the app_manage / app_status_update tasks and
  switch to aclose()
- back off in the app-status watch loop so a Redis outage no longer
  spins, and drop a leftover debug print

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:40:14 +02:00
fusion44
868457ffa5
fix(api): replace deprecated asyncio.get_event_loop()
get_event_loop() is deprecated on Python 3.11+ when there is no running
loop and is slated to change behaviour further.

- SSEManager.setup() ran at import time (app.api.utils) via
  get_event_loop(); this only worked because uvicorn imports the app
  inside its loop and would break when imported without a running loop
  (e.g. a Celery worker). Start the broadcast consumer lazily from
  within a running loop instead.
- everywhere else the pattern was get_event_loop().create_task(x)
  inside a coroutine; replace with asyncio.create_task(x).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:34:44 +02:00
fusion44
df62098cd1
refactor(lightning): migrate SendCoinsInput validator to Pydantic v2
Replace the deprecated Pydantic v1 @validator('amount', pre=True,
always=True) with a v2 @model_validator(mode='after'). The model
validator always runs and can see both amount and send_all, preserving
the cross-field rule (and the always=True semantics that reject the
empty/default case). Removes the last Pydantic v1 deprecation warning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 22:22:05 +02:00
fusion44
cf423f52c1
fix(system): stop native_python impl swallowing errors into None
@logger.catch defaults to reraise=False, so change_password and
get_debug_logs_raw - which raise NotImplementedError - silently
returned None. The service layer's 'except NotImplementedError -> 501'
never fired, yielding '200 null' or a response-model 500 instead.
login likewise turned unexpected errors into a None result.

Drop the pointless decorator from the two methods that only raise, and
let login reraise so failures surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:44:25 +02:00
fusion44
c4afca50e9
fix(system): stop leaking passwords via change-password endpoint
The change-password endpoint accepted old_password/new_password as bare
str parameters, i.e. query parameters, so the passwords ended up in
access logs, proxy logs and browser history. Accept them in a
ChangePasswordInput request body instead.

Also mark the RaspiBlitz blitz.passwords.sh check/set invocations
sensitive=True so the plaintext passwords are not written to the debug
log, and guard against a missing password type (was an AttributeError
-> 500).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:42:25 +02:00
fusion44
9912ff0bfa
fix(apps): honour keep_data on uninstall and fix install log name
- uninstall_app dropped input.keep_data; _manage_app always ran the
  bonus script with a bare 'off', so the RaspiBlitz script fell back to
  an interactive whiptail prompt that hangs the non-interactive API.
  Thread keep_data through and pass the explicit
  --keep-data/--delete-data flag the scripts expect.
- install.{app_id}.log rendered as install.AppId.MEMPOOL.log because
  str-enum formatting includes the class name on Python 3.11+; use
  app_id.value here and in the CLN-incompatibility message.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:58:35 +02:00
fusion44
d5a568da3e
fix(lightning): include payments in CLN list_all_tx
Two bugs in the CLN JSON-RPC list_all_tx:
- 'if pay is not Payment' compared each item to the Payment class rather
  than its type, so it was always true and every payment was skipped -
  payments never showed up in the transaction list.
- the successful_only filter appended the item inside the match branch
  and then again unconditionally, so it never filtered anything.

Use isinstance for the type check and skip non-matching items when
successful_only is set. Also guard the source lists against None so a
failed sub-query no longer crashes the loop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:45:21 +02:00
fusion44
dccaa17a8e
fix(auth): correct JWT expiry unit and use standard exp claim
sign_jwt added JWT_EXPIRY_TIME (seconds) to a milliseconds epoch and
stored it in a custom 'expires' claim, while register_cookie_updater
slept JWT_EXPIRY_TIME as seconds. With the code default (300) tokens
effectively expired almost immediately; with the sampled 3600000 the
cookie-refresh loop slept ~41 days, so the local .cookie held an
expired token nearly always. The custom claim also meant PyJWT never
validated expiry itself.

- issue standard 'iat'/'exp' claims (seconds) and let PyJWT validate,
  requiring 'exp' on decode
- derive the cookie refresh interval from the same unit, guarded
  against tiny/negative values
- default BAPI_JWT_EXPIRY_TIME to 3600s and fix .env_sample (was
  3600000 'milliseconds')

Existing tokens and the local .cookie are invalidated by this change;
clients re-login and the cookie regenerates at startup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:35:34 +02:00
fusion44
5aa79acb10
fix(apps): don't release another task's app-manage lock
app_manage_task_impl released APP_MANAGE_LOCK_KEY and broadcast a
FINISHED message from its finally block unconditionally, including on
the early-return path where acquire_lock reported the lock as already
held by a running install. A duplicate install/uninstall request would
therefore delete the running task's lock (allowing concurrent
management of the same app) and send FINISHED, which stopped that
task's AppManageListener before it was done.

Track whether this task actually acquired the lock and only
release + finish when it did.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:01:54 +02:00
fusion44
ff54f93397
fix(api): catch subprocess timeouts in exec_bash_command
'from redis.asyncio import ... TimeoutError' shadowed the builtin
TimeoutError. redis's TimeoutError is a RedisError subclass, not a
builtin subclass, so the 'except TimeoutError' guarding
asyncio.wait_for never matched: timed-out commands fell through to the
generic handler, the child process was never terminated (leak), and
the caller got a misleading 'unable to execute' error instead of a
timeout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 19:59:57 +02:00
fusion44
4355c8eabd
fix(lightning): prevent shell injection in CLN local calls
Two authenticated code paths interpolated user-controlled input into a
shell command:

- decode_pay_request passed the bolt11 string into _make_local_call,
  which ran it via create_subprocess_shell; a crafted /lightning/
  decode-pay-req request could execute arbitrary commands. Switch
  _make_local_call to create_subprocess_exec with a discrete argv list.
- blitz_cln_unlock interpolated the wallet password into a
  cl.hsmtool.sh invocation run through a shell, and logged it in the
  clear. shlex.quote the interpolated values and mark the call
  sensitive=True.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 19:58:41 +02:00
fusion44
ba78b10aa2
ci: modernize the CodeQL workflow
Some checks are pending
CodeQL / Analyze (actions) (push) Waiting to run
CodeQL / Analyze (python) (push) Waiting to run
- run on the actual default branch: the workflow only triggered on
  main, but the default branch is dev, so pushes were never analyzed
- bump github/codeql-action from the deprecated v1/v2 to v4 and
  actions/checkout from v2 (node12, disabled by GitHub) to v7
- fix the stale paths-ignore: the generated protos moved from
  app/repositories/ln_impl/protos to app/lightning/impl/protos long
  ago, and exclude them from the analysis itself via the init config
  (trigger-level paths-ignore only affects when the workflow runs)
- also scan the GitHub Actions workflows (language: actions)
- drop the Autobuild step (a no-op for Python) and the template
  boilerplate comments
- allow manual CodeQL runs via workflow_dispatch

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:01:57 +02:00
fusion44
65588d6c7c
chore(deps): bump dependencies with known security vulnerabilities
Direct dependencies:
- fastapi 0.128.0 -> 0.139.0 (needed for starlette 1.x support)
- starlette 0.49.1 -> 1.3.1
- pyjwt 2.10.1 -> 2.13.0
- aiohttp 3.13.3 -> 3.14.1
- deepdiff 8.6.1 -> 9.1.0
- requests 2.32.5 -> 2.33.0

Transitive dependencies:
- urllib3 2.6.3 -> 2.7.0
- idna 3.11 -> 3.18
- python-multipart 0.0.22 -> 0.0.32

Dev dependencies:
- pytest 9.0.2 -> 9.0.3

The update-requirements-file make target now passes --upgrade to
uv pip compile; without it, stale (vulnerable) pins from the existing
requirements.txt are silently preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:46:36 +02:00
fusion44
c2e3e88e66
chore: remove unused Docker regtest setup
Removes Dockerfile.regtest and the docker/ directory along with the
remaining references: the docker-regtest-image Makefile target, its
help text, and the docker/.env gitignore entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:08:23 +02:00
fusion44
b9e18ff677
chore: update devenv.lock (devenv 2.x) 2026-07-03 16:27:23 +02:00
fusion44
c9f27c3f47
chore(nix): migrate flake from poetry2nix to uv2nix
The project switched from poetry to uv in 44385ca, but the flake still
built everything through poetry2nix (an undeclared input resolved via
the flake registry) and no longer evaluated against the uv-based
pyproject.toml.

- build packages.blitz-api as a uv2nix virtualenv from uv.lock
  (wheels preferred), keeping the bin/api entry point the NixOS
  module expects
- replace the poetry dev shell with a uv-based tooling shell that
  mirrors devenv.nix
- move click to the main dependencies: the api console script
  imports it at runtime, so a production install without the dev
  group would crash on startup
- update the nixpkgs pin so the dev shell ships a uv that
  understands the current lockfile revision

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 16:00:42 +02:00
fusion44
49d5fcb9bf
fix: send bitcoin-only warmup app status as app_state_update_message
The WebUI only listens for the app_state_update_message SSE event to
populate the Apps tab. In bitcoin-only mode the warmup data was sent
under installed_app_status, which no client listens to, so the Apps
tab was stuck on the loading screen whenever the app status cache was
warm. Installing LND made it work again because the lightning warmup
path already used the correct event (raspiblitz#3608, raspiblitz#5141).

Also hardens the warmup path:
- reset the warmup_running flag on errors so a single failure no
  longer starves all future SSE clients of warmup data
- convert per-source exceptions in the bitcoin-only warmup gather
  instead of discarding the whole data set
- don't fall through to the partial-data branches when the API is
  fully initialized with lightning disabled
- remove the now-unused INSTALLED_APP_STATUS event and the dead
  cached_status_raw variable

Adds regression tests plus a conftest.py providing test env defaults
so the suite runs without a developer .env file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:30:17 +02:00
fusion44
6edd69a9fc
docs: update README for uv-based tooling
Replace stale poetry instructions with uv, and correct the required
Python version (3.7 -> 3.11/3.12) to match pyproject.toml. Follow-up
to 44385ca which deprecated poetry for uv in the build tooling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:44:05 +02:00
fusion44
e0f4d7e2fc
fix(deps): ensure async-timeout is included for older Python 3.11
- Add async-timeout as a direct dependency in pyproject.toml to fix
  ModuleNotFoundError on systems with Python < 3.11.3 (e.g. Raspberry Pi).
- Switch to universal requirements generation in Makefile and
  requirements.txt to include platform-specific markers and conditional
  dependencies.
- Update uv.lock to reflect dependency changes.

This resolves a crash where redis-py attempted to import async_timeout,
which was missing because it was pruned during requirements generation
on a newer Python version.
2026-02-21 08:04:34 +01:00
dependabot[bot]
5c1669aed5 chore(deps): bump urllib3 from 2.5.0 to 2.6.3
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.5.0 to 2.6.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.5.0...2.6.3)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.6.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-05 15:02:39 +01:00
dependabot[bot]
ae7f8f54fe chore(deps): bump python-multipart from 0.0.20 to 0.0.22
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.20 to 0.0.22.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.20...0.0.22)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.22
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-05 15:02:21 +01:00
fusion44
74059b9256
tests: add better integration tests 2026-02-03 11:06:06 +01:00
fusion44
e564d18d2b
chore: clean up imports; add better LND error logging 2026-02-03 11:05:19 +01:00
fusion44
07d2bfbc76
feat: update Python and Devenv deps
- Updated all dependencies to their latest versions
- Vendored fastapi-plugins[redis] as it looks unmaintained
2026-02-03 11:04:41 +01:00
fusion44
44385ca40c
chore: deprecate poetry for uv; update dependencies 2025-11-10 12:30:03 +01:00
Justin Sharp
6b3acf0239 fix: handle missing fields in CLN invoice data gracefully
Fixes #129, #128, and addresses part of raspiblitz/raspiblitz#3182

BOLT12 offers, keysend payments, and certain CLN invoice types don't
always include all fields that the API expects, causing KeyError
exceptions that crash the web interface.

Changes:
- Modified Invoice.from_cln_json() to use .get() with safe defaults
  for all potentially missing fields (bolt11, amount_msat, payment_hash,
  description, label, status, etc.)
- Added fallback logic for amount_msat to use amount_received_msat
  when the primary field is missing
- Enhanced InvoiceState.from_cln_json() to handle unknown/missing
  statuses gracefully with logging instead of raising exceptions

This allows the web interface to display all CLN invoices including
BOLT12 payments from services like OCEAN mining pool, while preserving
all existing payment data for standard BOLT11 invoices.

Tested on RaspiBlitz v1.12.0 with Core Lightning and OCEAN mining
pool BOLT12 payouts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 10:00:05 +01:00
dependabot[bot]
acad234651 chore(deps): bump urllib3 from 2.4.0 to 2.5.0
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.4.0...2.5.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.5.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-08 18:38:14 +02:00
fusion44
b555466053
fix: increase timeout for fetching app statuses 2025-08-08 18:27:48 +02:00
fusion44
b5786399bf
fix: target python 3.11 2025-08-08 09:10:41 +02:00
fusion44
d63e63f4e6
chore: remove deprecated async-timeout dep use 2025-05-13 19:22:47 +02:00
fusion44
6d1aca7854
chore: pin dependencies to an exact version 2025-05-09 12:45:31 +02:00
fusion44
382f8303f3 chore: update deps and fake blitz scripts 2025-05-06 09:16:46 +02:00
fusion44
bda5d565ee fix: don't stop app status listener 2025-05-06 09:16:46 +02:00
fusion44
f1aa2107fd refactor: code cleanup
- rename various keys
- update app status updater code to work like the app management code
- remove some unused code

refs #123
2025-05-06 09:16:46 +02:00
fusion44
1f491ed0a0 feat: implement (un)install celery task
refs #123
2025-05-06 09:16:46 +02:00
fusion44
de1594f7e4 refactor: nicer layout for app status cache code 2025-05-06 09:16:46 +02:00
fusion44
6bd875f16b refactor: move app update code into its own file 2025-05-06 09:16:46 +02:00
fusion44
824fa62c4d chore: small fixes and updates
- formatting
- small fix in a bitcoind model
- update the fake blitz scripts
- remove some dead code (app status sub)
2025-05-06 09:16:46 +02:00
fusion44
4e038e8fc7 feat: fetch app status via a celery task
This is a feature that allows to fetch the app status via a celery
task which stores the result in the Redis database and notifies the API
of the change. The API sends a notification to connected clients via the
SSE mechanism.

Using a background task allows to avoid crashing the whole API if the
script call fails.

The by default the cache is refreshed every 30 minutes. This can be
changed by setting the `BAPI_APP_STATUS_UPDATE_INTERVAL_MIN` environment
variable.

refs #123
2025-05-06 09:16:46 +02:00
fusion44
4a70d2ca36 feat: add error handling to get_redis helper func
refs #123
2025-05-06 09:16:46 +02:00
fusion44
2b98912796 feat: replace call_script func with better func
use exec_bash_command instead

refs #123
2025-05-06 09:16:46 +02:00
fusion44
e1f87a3e8b feat: update fake blitz environment 2025-05-06 09:16:46 +02:00
fusion44
751687e60b chore: run ruff format and isort 2025-05-06 09:16:46 +02:00
fusion44
e5ba965818 feat: update test script to include fake_blitz env 2025-05-06 09:16:46 +02:00
fusion44
8ee5a49892 feat: update get app status error handling code
refs #123
2025-05-06 09:16:46 +02:00
fusion44
691d5da6b9 feat: update formatting of errors sent to clients
refs #123
2025-05-06 09:16:46 +02:00
fusion44
ef9bfbefe4 feat: add supporting types for error handling
- add a result type similar to the Rust Result type from
https://github.com/rustedpy/result
- add a Report class to propagate error information back on the stack
with return types instead of exceptions

refs #123
2025-05-06 09:16:46 +02:00
fusion44
8b372ecb47 feat: make typos available to the devenv 2025-05-06 09:16:46 +02:00
dependabot[bot]
b4e9112b1f chore(deps): bump h11 from 0.14.0 to 0.16.0
Bumps [h11](https://github.com/python-hyper/h11) from 0.14.0 to 0.16.0.
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

---
updated-dependencies:
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-25 20:22:28 +02:00