Feature: Service integration - Swan (#1517)

* fix create_order issue
* fix url and better error-handling
* basic balances tab
* activating services
* Service management
* Improve settings page and default to it if token unset
* Fix history tab
* Fix trade error handling
* Add withdraw
* calling specter-cloud for creating vaultoro orders
* Improve trade screen and fixes
* sidebar fix
* black
* refactor Service integration
* refactor to have Service Classes like manifests
* maturity
* dynamic initialisation of service-classes and blueprints
* fix sidebar_services
* migrated templates and static into vaultoro folder
* refactor config to manifest
* some minor things
* swan initial
* rename and fix test
* adding ServiceApiKeyStorageUserAware
* fix
* store the access token
* directory indirection to shield templates from each others blueprint
* first attempts with automatic withdrawals
* proper tab highlighting
* Update service_apikey_storage.py
* Update oauth2_success.jinja
* Awaiting refresh_token support
* Service logo display on Addresses
* Associate addr with a Service
* address-data component reorg
Separates the presentation html from the data as much as possible.
* Services data/icon added to tx History
* Now hitting the updated Swan endpoint to save deposit addrs
* Simplified injecting Services data into JS
* Reducing js calls back to server in tx-data; templatizing utxo in/outs
* renaming "reserving" to "associating" an Address with a Service.
* rename `manifest.py` files to `service.py`.
* rename "api_data" to "service_data" to make the storage a bit more generalized. `ServiceApiKeyStorage` is now `ServiceEncryptedStorage` to match.
* Beginning of factoring out Swan api to its own `api.py` file; need to rectify with `swan_client.py`.
* Removed tx-table/row/data changes and address-table/row/data
Kept only the bare minimum changes required to display the Services icon, plus optimizations.
* deleted no longer used CustomElement
* Adding services docs to mkdocs
* Configuration for Services
* more clever configuration
* fix test
* deleted swan_client
* Changing the address abbreviation format to 7...7 and little big fix for not vertically aligned addresses in Firefox.
* Better state management if Auth method changes
* Cleanup, better user messaging; pulling Service methods out of controller and User
* Redirect to services endpoint after setting up authentication.
* PR cleanup, bug fixes, test suite updates
* Fixed test case problem
* First fix for delete API key button.
* Service hooks; Option to fully remove Swan Integration; Logout clears plaintext_user_secret
* Restoring bugfix from @moneymanolis
* cleanup and black
* Make service-decovery in AppImage work
* import hashlib, maybe fix cypress
* Update service_encrypted_storage.py
* further bugfix on update
* TODO: remove debugging in client.py before first release
* fix controller-test
* fix singleton testing issues

Co-authored-by: Kim Neunert <k9ert@gmx.de>
Co-authored-by: benk10 <ben.kaufman10@gmail.com>
Co-authored-by: Kim Neunert <kneunert@gmail.com>
Co-authored-by: moneymanolis <moneymanolis@protonmail.com>

docs/archblog.md

@@ -1,7 +1,7 @@
 # Purpose
 Let's find a place to docoment non straightforward design decisions.
 
-## 02nd Oct 2020 - k9ert
+## 02nd Oct 2020 - Kim
 It's already mentioned in Development.md. I spend far too much time figuring out that we have created a nasty workaround in server.py.
 So the problem looks like this: The fixtures are creating the app anew for each test in test_controller. For some reason hwi-view-endpoints are somehow treated differently then the normal endpoints. As a result, the second test gets a app-object which, for some reason doesn't have the normal endpoints, but just the hwi-endpoint. A healthy app.view_functions looks like this:
 ```

docs/services/services.md

@@ -0,0 +1,130 @@
+# Third-Party Service Integrations
+
+A developer's guide for Specter Desktop `Service` integrations.
+
+
+## Basic Code Philosophy
+As much as possible, each `Service` implementation should be entirely self-contained with little or no custom code altering existing/core Specter functionality.
+
+Effort has been taken to provide `Service` data storage that is separate from existing data stores in order to keep those areas clean and simple. Where touchpoints are unavoidable, they are kept to the absolute bare minimum (e.g. `User.services` list, `Address.service_id` field).
+
+
+## `Address`-Level Integration
+An `Address` can be associated with a `Service` (e.g. addr X received a smash buy from `Service` Foo) via the `Address.service_id` field.
+
+A `Service` can also "reserve" and `Address` for future use by setting `Address.service_id`. The normal "Receive" UI will automatically skip any reserved `Address`es when generating a new receive addr. The reserved addresses are interleaved with ready-to-use addresses so that we don't create any potentially confusing wallet gaps (e.g. addrs 4, 6, and 8 are reserved but addrs 3, 5, and 7 are available).
+
+Users can also manually associate an existing `Address` with a `Service` (this is useful when the user has info that the particular `Service` api can't provide for whatever reason).
+
+_Note: TODO: manually un-reserve an `Address` from a `Service`._
+
+
+## Basic Code Structure
+All `Service`-related code should be contained within `cryptoadvance.specter.services`. The base components are:
+
+
+### `Service` Base Class
+Defines the base `Service` class that all service integrations must inherit from. This is wired to enable `Service` auto-discovery. Any feature that is common to most or all `Service` integrations should be implemented here.
+
+Each `Service` must specify a unique `Service.id` that is just a short string (e.g. "swan"). This is the main identifier throughout the code.
+
+Includes methods to "reserve" addresses for the `Service` to basically make those not-yet-used addresses somewhat off-limits to the rest of the UI (can still be manually overridden though).
+
+
+### `Service` Configuration
+In order to separate the service-configuration from the main-configuration, you can specify your config in a file called `config.py`. It's structure is similiar to the specter-wide `config.py`, e.g.:
+```
+class BaseConfig():
+    SWAN_API_URL="https://dev-api.swanbitcoin.com"
+
+class ProductionConfig(BaseConfig):
+    SWAN_API_URL="https://api.swanbitcoin.com"
+```
+In your code, you can access the correct value as in any other flask-code, like `api_url = app.config.get("SWAN_API_URL")`. If the instance is running a config (e.g. `DevelopmentConfig`) which is not available in your service-specific config (as above), the inheritance-hirarchy from the mainconfig will get traversed and the first hit will get get configured. In this example, it would be `BaseConfig`.
+
+
+### `ServiceManager`
+Simple manager that contains all `Service`s. Performs the `Service` auto-discovery at startup and filters availability by each `Service`'s release level (i.e. alpha, beta, etc).
+
+
+### `ServiceEncryptedStorage`
+Most `Service`s will require user secrets (e.g. API key and secret). Each Specter `User` will have their own on-disk encrypted `ServiceEncryptedStorage` with filename `<username>_services.json`. Note that the user's secrets for all `Service`s will be stored in this one file.
+
+This is built upon the `GenericDataManager` class which supports optional encrypted fields. In this case all fields are encrypted. The `GenericDataManager` encryption can only be unlocked by each `User`'s individual `user_secret` that itself is stored encrypted on-disk; it is decrypted to memory when the `User` logs in.
+
+For this reason `Service`s cannot be activated unless the user is signing in with a password-protected account (the default no-password `admin` account will not work).
+
+_Note: during development if the Flask server is restarted or auto-reloads, the user's decrypted `user_secret` will no longer be in memory. The Flask context will still consider the user logged in after restart, but code that relies on having access to the `ServiceEncryptedStorage` will throw an error and/or prompt the user to log in again._
+
+It is up to each `Service` implementation to decide what data is stored; the `ServiceEncryptedStorage` simply takes arbitrary json in and delivers it back out.
+
+This is also where `Service`-wide configuration or other information should be stored, _**even if it is not secret**_ (see above intro about not polluting other existing data stores).
+
+
+### `ServiceEncryptedStorageManager`
+Because the `ServiceEncryptedStorage` is specific to each individual user, this manager provides convenient access to automatically retrieve the `current_user` from the Flask context and provide the correct user's `ServiceEncryptedStorage`. It is implemented as a `Singleton` which can be retrieved simply by importing the class and calling `get_instance()`.
+
+This simplifies code to just asking for:
+```
+from .service_encrypted_storage import ServiceEncryptedStorageManager
+
+ServiceEncryptedStorageManager.get_instance().get_current_user_service_data(service_id=some_service_id)
+```
+
+As a further convenience, the `Service` base class itself encapsulates `Service`-aware access to this per-`User` encrypted storage:
+```
+@classmethod
+def get_current_user_service_data(cls) -> dict:
+    return ServiceEncryptedStorageManager.get_instance().get_current_user_service_data(service_id=cls.id)
+```
+
+Whenever possible, external code should not directly access these `Service`-related support classes but rather should ask for them through the `Service` class.
+
+
+### `ServiceAnnotationsStorage`
+Annotations are any address-specific or transaction-specific data from a `Service` that we might want to present to the user. Example: a `Service` that integrates with a onchain storefront would have product/order data associated with a utxo. That additional data could be imported by the `Service` and stored as an annotation. This annotation data could then be displayed to the user when viewing the details for that particular address or tx.
+
+Annotations are stored on a per-wallet and per-`Service` basis as _unencrypted_ on-disk data (filename: `<wallet_alias>_<service>.json`).
+
+_Note: current `Service` implementations have not yet needed this feature so displaying annotations is not yet implemented._
+
+
+### `controller.py`
+The minimal url routes for `Service` selection and management.
+
+
+## Implementation Class Structure
+Child implementation classes (e.g. `SwanService`) should be self-contained within their own subdirectory in `services`. e.g.:
+```
+cryptoadvance.specter.services.swan
+```
+
+Each implementation must have the following required components:
+```
+/static
+/templates/<service_id>
+controller.py
+service.py
+```
+
+This makes each implementation its own Flask `Blueprint`.
+
+### `/static`
+Because of Flask `Blueprint` imports, you can just add static files here and reference them (e.g. "static/img/blah.png") as if they were in the main `/static` files root dir.
+
+### `/templates/<service_id>`
+Again, Flask `Blueprint`s import the `/templates` directory as-is, but to avoid namespace collisions on the template files (e.g. `/templates/index.html`) they should be contained within a subdirectory named with the `Service.id` (e.g. `/templates/swan/index.html`)
+
+### `Service` Implementation Class
+Must inherit from `Service` and provide any additional functionality needed. The `Service` implementation class is meant to be the main hub for all things related to that particular `Service`. In general, external code would ideally only interact with the `Service` implementation class (e.g. )
+
+### `controller.py`
+Flask `Blueprint` for any endpoints required by this `Service`.
+
+The coding philosophy should be to keep this code as simple as possible and keep most or all of the actual logic in the `Service` implementation class.
+
+### Additional Files
+The `SwanService` also includes an `api.py` to separate its back-end API calls from the user-facing `controller.py` endpoints. In general this is recommended to provide a clear separation.
+
+An individual `Service` implementation may add whatever additional files or classes it needs.
+

mkdocs.yml

@@ -25,6 +25,7 @@ nav:
       - build-instructions.md
       - 'Continuous Integration': continuous-integration.md
       - cypress-testing.md
+      - services/services.md
       - 'Some Random Dev Thoughts': archblog.md
     - API:
       - api/README.md

pyinstaller/electron/styles.css

@@ -11,7 +11,7 @@ html, body{
     --cmap-red:           #A12737;
     --cmap-red-darker:    #951E2D;
     --cmap-bg:            #192432;
-    --cmap-bg-ligher:     #263044;
+    --cmap-bg-lighter:     #263044;
     --cmap-bg-lightest:   #313E50;
     --cmap-border:        #506072;
     --cmap-border-darker: #405062;
@@ -75,7 +75,7 @@ ul, li{
     align-items: center;
 }
 .settings-bar-btn:hover {
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
 }
 .logout{
     position: relative;
@@ -90,7 +90,7 @@ ul, li{
     border-radius: 0 0 0 5px;
 }
 .logout:hover{
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
 }
 .logout img{
     margin-right: 10px;
@@ -177,7 +177,7 @@ nav.side{
     width: 250px;
 }
 #side-content{
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
     border-right: 1px solid var(--cmap-bg-lightest);
     padding-top: 10px;
     display: flex;
@@ -651,7 +651,7 @@ td, th{
     border-bottom: 1px solid var(--cmap-bg-lightest);
 }
 tr, thead{
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
 }
 th{
     text-align: left;
@@ -952,7 +952,7 @@ a:focus + .tool-tip .tool-tip__info {
     bottom: 0;
     right: 0;
     width: 20px;
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
   }
   td.xpub{
     max-width: 230px;

pytest.ini

@@ -6,3 +6,6 @@ markers =
     slow: mark test as slow.
     elm: mark test as elementsd dependent
 #log_cli = 1
+
+filterwarnings =
+    ignore::DeprecationWarning:bitbox02[.*]

src/cryptoadvance/specter/addresslist.py

@@ -23,6 +23,7 @@ class Address(dict):
         "change",  # bool, change or receive
         "label",  # str, address label
         "used",  # bool, does this address have a transaction?
+        "service_id",  # str, reserved for or used by Service.id
     ]
     type_converter = [
         str,
@@ -30,6 +31,7 @@ class Address(dict):
         to_bool,
         str,
         to_bool,
+        str,
     ]
 
     def __init__(self, rpc, **kwargs):
@@ -49,6 +51,16 @@ class Address(dict):
         self["label"] = label
         self.rpc.setlabel(self.address, label)
 
+    def set_service_id(self, service_id: str):
+        if service_id:
+            # Declares that this Address is associated with a Service
+            self["service_id"] = service_id
+        else:
+            # Frees a reserved Address; reset its label
+            self["service_id"] = None
+            if not self.used and self["label"]:
+                self.set_label("")
+
     @property
     def is_external(self):
         return self.index is None
@@ -95,6 +107,14 @@ class Address(dict):
     def is_labeled(self):
         return bool(self["label"])
 
+    @property
+    def is_reserved(self):
+        return bool(self["service_id"])
+
+    @property
+    def service_id(self):
+        return self["service_id"]
+
     def __str__(self):
         return self.address
 
@@ -179,10 +199,14 @@ class AddressList(dict):
                 )
         self.save()
 
-    def set_label(self, address, label):
+    def get_address(self, address: str) -> Address:
         if address not in self:
-            self[address] = self.AddressCls(self.rpc, address=address, label=label)
-        self[address].set_label(label)
+            self[address] = self.AddressCls(self.rpc, address=address)
+        return self[address]
+
+    def set_label(self, address, label):
+        addr_obj = self.get_address(address)
+        addr_obj.set_label(label)
         self.save()
 
     def get_labels(self):
@@ -193,6 +217,29 @@ class AddressList(dict):
                 labels[lbl] = labels.get(lbl, []) + [addr.address]
         return labels
 
+    def associate_with_service(
+        self, address: str, service_id: str, label: str, autosave: bool = True
+    ):
+        """
+        Associates the Address (i.e. sets Address.service_id) with the specified
+        Service.id.
+        """
+        addr_obj = self.get_address(address)
+        addr_obj.set_service_id(service_id)
+        addr_obj.set_label(label)
+        if autosave:
+            self.save()
+
+    def deassociate(self, address: str, autosave: bool = True):
+        """
+        Removes the Address's association with a Service (i.e. sets
+        Address.service_id to None and resets Address.label).
+        """
+        addr_obj = self.get_address(address)
+        addr_obj.set_service_id(None)
+        if autosave:
+            self.save()
+
     def set_used(self, addresses):
         need_save = False
         for address in addresses:

src/cryptoadvance/specter/config.py

@@ -135,6 +135,12 @@ class BaseConfig(object):
     # Babel integration. List of languages written from right to left for RTL support in the UI
     RTL_LANGUAGES = ["he"]
 
+    # One of "prod", "beta" or "alpha". Every Service below will be not available
+    SERVICES_DEVSTATUS_THRESHOLD = os.getenv("SERVICES_DEVSTATUS_THRESHOLD", "prod")
+
+    # This is just a placeholder in order to be aware that you cannot set this
+    # It'll be filled up with the fully qualified Classname the Config is derived from
+    SPECTER_CONFIGURATION_CLASS_FULLNAME = None
     # The user will get a warning if a request takes longer than this threshold
     REQUEST_TIME_WARNING_THRESHOLD = int(
         os.getenv("REQUEST_TIME_WARNING_THRESHOLD", "20")
@@ -144,12 +150,18 @@ class BaseConfig(object):
 class DevelopmentConfig(BaseConfig):
     # https://stackoverflow.com/questions/22463939/demystify-flask-app-secret-key
     SECRET_KEY = "development key"
+
+    # EXPLAIN_TEMPLATE_LOADING = os.getenv("EXPLAIN_TEMPLATE_LOADING", "False")
+
     SPECTER_DATA_FOLDER = os.path.expanduser(
         os.getenv("SPECTER_DATA_FOLDER", "~/.specter_dev")
     )
     # API active by default in dev-mode
     SPECTER_API_ACTIVE = _get_bool_env_var("SPECTER_API_ACTIVE", "True")
 
+    # One of "prod", "beta" or "alpha". Every Service below will be not available
+    SERVICES_DEVSTATUS_THRESHOLD = os.getenv("SERVICES_DEVSTATUS_THRESHOLD", "beta")
+
 
 class TestConfig(BaseConfig):
     SECRET_KEY = "test key"

src/cryptoadvance/specter/liquid/wallet.py

@@ -1,14 +1,18 @@
-from ..wallet import *
-from ..addresslist import Address
+import hashlib
+from typing import List
+
 from embit import ec
+from embit.descriptor.checksum import add_checksum
+from embit.liquid.addresses import to_unconfidential
+from embit.liquid.descriptor import LDescriptor
 from embit.liquid.pset import PSET
 from embit.liquid.transaction import LTransaction
-from embit.liquid.descriptor import LDescriptor
-from embit.descriptor.checksum import add_checksum
-from .txlist import LTxList
-from .addresslist import LAddressList
-from embit.liquid.addresses import to_unconfidential
+
+from ..addresslist import Address
 from ..specter_error import SpecterError
+from ..wallet import *
+from .addresslist import LAddressList
+from .txlist import LTxList
 from .util.pset import SpecterPSET
 
 
@@ -136,8 +140,8 @@ class LWallet(Wallet):
 
     def createpsbt(
         self,
-        addresses: [str],
-        amounts: [float],
+        addresses: List[str],
+        amounts: List[float],
         subtract: bool = False,
         subtract_from: int = 0,
         fee_rate: float = 1.0,

src/cryptoadvance/specter/managers/singleton.py

@@ -0,0 +1,40 @@
+class Singleton:
+    _instance = None
+
+    def __init__(self):
+        # Singleton pattern must prevent normal instantiation
+        raise Exception(
+            "Cannot directly instantiate a Singleton. Access via get_instance()"
+        )
+
+    @classmethod
+    def get_instance(cls):
+        # This is the only way to access the one and only instance
+        if cls._instance is None:
+            cls._instance = cls.__new__(cls)
+        return cls._instance
+
+
+class ConfigurableSingletonException(Exception):
+    pass
+
+
+class ConfigurableSingleton(Singleton):
+    @classmethod
+    def get_instance(cls):
+        # This is the only way to access the one and only instance
+        if cls._instance:
+            return cls._instance
+        else:
+            raise ConfigurableSingletonException(
+                f"Must call {cls.__name__}.configure_instance(config) first"
+            )
+
+    @classmethod
+    def configure_instance(cls, **kwargs):
+        # Must be called before the first get_instance() call
+        if cls._instance:
+            raise ConfigurableSingletonException(f"{cls.__name__} already configured")
+
+        # Instantiate the one and only instance
+        cls._instance = cls.__new__(cls)

src/cryptoadvance/specter/managers/wallet_manager.py

@@ -5,33 +5,17 @@ import pathlib
 import shutil
 import threading
 import traceback
-import hashlib
-from collections import OrderedDict
-from io import BytesIO
 
 from ..helpers import alias, load_jsons, is_liquid, add_dicts
-from ..persistence import delete_file, delete_folder
+from ..persistence import delete_folder
 from ..rpc import RpcError, get_default_datadir
 from ..specter_error import SpecterError
-from ..util.descriptor import AddChecksum
-from ..wallet import Wallet, purposes
+from ..wallet import (
+    Wallet,
+    purposes,
+)  # TODO: `purposes` unused here, but other files rely on this import
 from ..liquid.wallet import LWallet
 
-from embit import ec
-from embit.descriptor import Descriptor
-from embit.liquid.descriptor import LDescriptor
-from embit.descriptor.checksum import add_checksum
-
-from embit import ec
-from embit.descriptor import Descriptor
-from embit.liquid.descriptor import LDescriptor
-from embit.descriptor.checksum import add_checksum
-
-from embit import ec
-from embit.descriptor import Descriptor
-from embit.liquid.descriptor import LDescriptor
-from embit.descriptor.checksum import add_checksum
-
 logger = logging.getLogger(__name__)
 
 
@@ -128,7 +112,7 @@ class WalletManager:
         try:
             if self.wallets_update_list:
                 loaded_wallets = self.rpc.listwallets()
-                logger.info("Getting loaded wallets list from Bitcoin Core")
+                # logger.info("Getting loaded wallets list from Bitcoin Core")
                 for wallet in self.wallets_update_list:
                     wallet_alias = self.wallets_update_list[wallet]["alias"]
                     wallet_name = self.wallets_update_list[wallet]["name"]
@@ -179,10 +163,10 @@ class WalletManager:
                                     ],
                                 )
                             self.wallets[wallet_name] = loaded_wallet
-                            logger.info(
-                                "Finished loading wallet into Bitcoin Core and Specter: %s"
-                                % self.wallets_update_list[wallet]["alias"]
-                            )
+                            # logger.info(
+                            #     "Finished loading wallet into Bitcoin Core and Specter: %s"
+                            #     % self.wallets_update_list[wallet]["alias"]
+                            # )
                         except RpcError as e:
                             logger.warning(
                                 f"Couldn't load wallet {wallet_alias} into core. Silently ignored! RPC error: {e}"
@@ -208,10 +192,10 @@ class WalletManager:
                             # ok wallet is already there
                             # we only need to update
                             try:
-                                logger.info(
-                                    "Wallet already loaded in Bitcoin Core. Initializing %s Wallet object"
-                                    % self.wallets_update_list[wallet]["alias"]
-                                )
+                                # logger.info(
+                                #     "Wallet already loaded in Bitcoin Core. Initializing %s Wallet object"
+                                #     % self.wallets_update_list[wallet]["alias"]
+                                # )
                                 loaded_wallet = self.WalletClass.from_json(
                                     self.wallets_update_list[wallet],
                                     self.device_manager,
@@ -219,10 +203,10 @@ class WalletManager:
                                 )
                                 if loaded_wallet:
                                     self.wallets[wallet_name] = loaded_wallet
-                                    logger.info(
-                                        "Finished loading wallet into Specter: %s"
-                                        % self.wallets_update_list[wallet]["alias"]
-                                    )
+                                    # logger.info(
+                                    #     "Finished loading wallet into Specter: %s"
+                                    #     % self.wallets_update_list[wallet]["alias"]
+                                    # )
                                 else:
                                     raise Exception("Failed to load wallet")
                             except Exception as e:
@@ -238,20 +222,20 @@ class WalletManager:
                                 )
                         else:
                             # wallet is loaded and should stay
-                            logger.info(
-                                "Wallet already in Specter, updating wallet: %s"
-                                % self.wallets_update_list[wallet]["alias"]
-                            )
+                            # logger.info(
+                            #     "Wallet already in Specter, updating wallet: %s"
+                            #     % self.wallets_update_list[wallet]["alias"]
+                            # )
                             self.wallets[wallet_name].update()
-                            logger.info(
-                                "Finished updating wallet:  %s"
-                                % self.wallets_update_list[wallet]["alias"]
-                            )
+                            # logger.info(
+                            #     "Finished updating wallet:  %s"
+                            #     % self.wallets_update_list[wallet]["alias"]
+                            # )
                             # TODO: check wallet file didn't change
         # only ignore rpc errors
         except RpcError as e:
             logger.error(f"Failed updating wallet manager. RPC error: {e}")
-        logger.info("Done updating wallet manager")
+        # logger.info("Done updating wallet manager")
         self.wallets_update_list = {}
         self.is_loading = False
 
@@ -368,22 +352,29 @@ class WalletManager:
         fetch_transactions=True,
         validate_merkle_proofs=False,
         current_blockheight=None,
+        service_id=None,
     ):
         """Returns a list of all transactions in all wallets loaded in the wallet_manager.
         #Parameters:
         #    fetch_transactions (bool): Update the TxList CSV caching by fetching transactions from the Bitcoin RPC
         #    validate_merkle_proofs (bool): Return transactions with validated_blockhash
         #    current_blockheight (int): Current blockheight for calculating confirmations number (None will fetch the block count from the RPC)
+        #    service_id (str): Filters results for just the specified Service
         """
+        # Nested comprehensions:
         txlists = [
             [
+                # Inner comprehension: Return each tx and all its attrs as a list of dicts...
+                # TODO: Simplify this by adding an `as_dict` option to `Wallet.txlist()`?
                 {**tx, "wallet_alias": wallet.alias}
                 for tx in wallet.txlist(
                     fetch_transactions=fetch_transactions,
                     validate_merkle_proofs=validate_merkle_proofs,
                     current_blockheight=current_blockheight,
+                    service_id=service_id,
                 )
             ]
+            # Outer comprehension: ...from each wallet, each returning their own tx list.
             for wallet in self.wallets.values()
         ]
         result = []
@@ -411,6 +402,23 @@ class WalletManager:
                 result.append(tx)
         return list(reversed(sorted(result, key=lambda tx: tx["time"])))
 
+    def full_addresses_info(self, is_change: bool = False, service_id: str = None):
+        """Mimics full_txlist in concept, but is really only expected to be used for
+        retrieving all addresses across all Wallets that are associated with a
+        Service.
+
+        Not currently used yet."""
+        addresses_info = []
+        for wallet_alias, wallet in self.wallets.items():
+            addresses_info.extend(
+                wallet.addresses_info(
+                    is_change=is_change,
+                    service_id=service_id,
+                    include_wallet_alias=True,
+                )
+            )
+        return addresses_info
+
     def delete(self, specter):
         """Deletes all the wallets"""
         for w in list(self.wallets.keys()):

src/cryptoadvance/specter/persistence.py

@@ -195,7 +195,7 @@ def storage_callback(mode="write", path=None):
     """Call this whenever anything in the .specter directory changes. Be aware that we might store node-data in the specter-folder"""
     # Might be usefull to figure out why the callback has been triggered:
     # traceback.print_stack()
-    logger.info(f"Storage Callback called mode {mode} with path {path}")
+    # logger.debug(f"Storage Callback called mode {mode} with path {path}")
     if os.getenv("SPECTER_PERSISTENCE_CALLBACK_ASYNC"):
         cmd_list = os.getenv("SPECTER_PERSISTENCE_CALLBACK_ASYNC").split(" ")
         cmd_list.append(mode)

src/cryptoadvance/specter/server.py

@@ -93,6 +93,8 @@ def create_app(config=None):
     app.jinja_env.autoescape = select_autoescape(default_for_string=True, default=True)
     logger.info(f"Configuration: {config}")
     app.config.from_object(config)
+    # Might be convenient to know later where it came from (see Service configuration)
+    app.config["SPECTER_CONFIGURATION_CLASS_FULLNAME"] = config
     app.wsgi_app = ProxyFix(
         app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_port=1, x_prefix=1
     )
@@ -121,6 +123,7 @@ def init_app(app, hwibridge=False, specter=None):
             data_folder=app.config["SPECTER_DATA_FOLDER"],
             config=app.config["DEFAULT_SPECTER_CONFIG"],
             internal_bitcoind_version=app.config["INTERNAL_BITCOIND_VERSION"],
+            service_devstatus_threshold=app.config["SERVICES_DEVSTATUS_THRESHOLD"],
         )
 
     login_manager = LoginManager()
@@ -150,12 +153,14 @@ def init_app(app, hwibridge=False, specter=None):
         app.config["LOGIN_DISABLED"] = True
     else:
         app.logger.info("Login enabled")
+        app.config["LOGIN_DISABLED"] = False
     app.logger.info("Initializing Controller ...")
     app.register_blueprint(hwi_server, url_prefix="/hwi")
     csrf.exempt(hwi_server)
     if not hwibridge:
         with app.app_context():
             from cryptoadvance.specter.server_endpoints import controller
+            from cryptoadvance.specter.services import controller as serviceController
 
             if app.config.get("TESTING") and len(app.view_functions) <= 20:
                 # Need to force a reload as otherwise the import is skipped
@@ -165,7 +170,9 @@ def init_app(app, hwibridge=False, specter=None):
                 # see archblog for more about this nasty workaround
                 import importlib
 
+                logger.info("Reloading controllers")
                 importlib.reload(controller)
+                importlib.reload(serviceController)
     else:
 
         @app.route("/", methods=["GET"])

src/cryptoadvance/specter/server_endpoints/auth.py

@@ -1,4 +1,6 @@
-import random, time
+import random
+import time
+
 from flask import (
     Flask,
     Blueprint,
@@ -175,6 +177,9 @@ please request a new link from the node operator."
 
 @auth_endpoint.route("/logout", methods=["GET", "POST"])
 def logout():
+    # Clear the decrypted user_secret from memory
+    current_user.plaintext_user_secret = None
+
     logout_user()
     if "timeout" in request.args:
         flash(_("You were automatically logged out"), "info")
@@ -196,6 +201,14 @@ def redirect_login(request):
         response = redirect(request.form["next"])
     else:
         response = redirect(url_for("index"))
+
+    for service_id in app.specter.user_manager.get_user().services:
+        try:
+            service_cls = app.specter.service_manager.get_service(service_id)
+            service_cls.on_user_login()
+        except Exception as e:
+            app.logger.exception(e)
+
     return response
 
 

src/cryptoadvance/specter/server_endpoints/controller.py

@@ -3,7 +3,7 @@ from time import time
 from binascii import unhexlify
 from flask import make_response
 from flask_wtf.csrf import CSRFError
-from werkzeug.exceptions import MethodNotAllowed
+from werkzeug.exceptions import MethodNotAllowed, NotFound
 from flask import render_template, request, redirect, url_for, flash, g
 from flask_babel import lazy_gettext as _
 from flask_login import login_required, current_user
@@ -35,10 +35,14 @@ from .wallets import wallets_endpoint
 from .wallets_api import wallets_endpoint_api
 from ..rpc import RpcError
 
+# Services live in their own separate path
+from cryptoadvance.specter.services.controller import services_endpoint
+
 app.register_blueprint(auth_endpoint, url_prefix="/auth")
 app.register_blueprint(devices_endpoint, url_prefix="/devices")
 app.register_blueprint(nodes_endpoint, url_prefix="/nodes")
 app.register_blueprint(price_endpoint, url_prefix="/price")
+app.register_blueprint(services_endpoint, url_prefix="/services")
 app.register_blueprint(settings_endpoint, url_prefix="/settings")
 app.register_blueprint(setup_endpoint, url_prefix="/setup")
 app.register_blueprint(wallets_endpoint, url_prefix="/wallets")
@@ -79,6 +83,14 @@ def server_specter_error(se):
         return redirect(url_for("about"))
 
 
+@app.errorhandler(NotFound)
+def server_notFound_error(e):
+    """Unspecific Exceptions get a 404 Error-Page"""
+    # if rpc is not available
+    app.logger.error("Could not find Resource (404): %s" % request.url)
+    return render_template("500.jinja", error=e), 500
+
+
 @app.errorhandler(Exception)
 def server_error(e):
     """Unspecific Exceptions get a 500 Error-Page"""

src/cryptoadvance/specter/server_endpoints/settings.py

@@ -1,3 +1,4 @@
+import cryptography
 import json
 import logging
 import os
@@ -18,6 +19,9 @@ from flask import current_app as app
 from flask import flash, jsonify, redirect, render_template, request, send_file, url_for
 from flask_babel import lazy_gettext as _
 from flask_login import current_user, login_required
+from cryptoadvance.specter.services.service import Service
+
+from cryptoadvance.specter.user import User, UserSecretException
 
 from ..helpers import (
     get_loglevel,
@@ -50,11 +54,11 @@ def settings():
 def general():
     current_version = notify_upgrade(app, flash)
     explorer_id = app.specter.explorer_id
-    explorer = ""
     fee_estimator = app.specter.fee_estimator
     fee_estimator_custom_url = app.specter.config.get("fee_estimator_custom_url", "")
     loglevel = get_loglevel(app)
     unit = app.specter.unit
+    services = app.specter.service_manager.services
     if request.method == "POST":
         action = request.form["action"]
 
@@ -90,8 +94,12 @@ def general():
         fee_estimator_custom_url = request.form["fee_estimator_custom_url"]
         unit = request.form["unit"]
         validate_merkleproof_bool = request.form.get("validatemerkleproof") == "on"
-
         if current_user.is_admin:
+            active_services = []
+            for service_name in services:
+                if request.form.get(f"service_{service_name}"):
+                    active_services.append(service_name)
+
             loglevel = request.form["loglevel"]
 
         if action == "save":
@@ -110,6 +118,7 @@ def general():
             app.specter.update_merkleproof_settings(
                 validate_bool=validate_merkleproof_bool
             )
+            app.specter.service_manager.set_active_services(active_services)
             app.specter.update_fee_estimator(
                 fee_estimator=fee_estimator,
                 custom_url=fee_estimator_custom_url,
@@ -198,6 +207,7 @@ This may take a few hours to complete."
 
     return render_template(
         "settings/general_settings.jinja",
+        services=services,
         fee_estimator=fee_estimator,
         fee_estimator_custom_url=fee_estimator_custom_url,
         loglevel=loglevel,
@@ -377,6 +387,9 @@ def tor():
 @settings_endpoint.route("/auth", methods=["GET", "POST"])
 @login_required
 def auth():
+    # TODO: Simplify this endpoint. Separate out setting the Authentication mode from all
+    # the other options here: updating admin username/password, adding users, deleting
+    # users, etc. Do those in simple separate screens with their own endpoints.
     current_version = notify_upgrade(app, flash)
     auth = app.specter.config["auth"]
     method = auth["method"]
@@ -403,6 +416,7 @@ def auth():
 
             min_chars = int(auth["password_min_chars"])
             if specter_username:
+
                 if current_user.username != specter_username:
                     if app.specter.user_manager.get_user_by_username(specter_username):
                         flash(
@@ -418,8 +432,12 @@ def auth():
                             specter=app.specter,
                             current_version=current_version,
                             rand=rand,
+                            has_service_encrypted_storage=app.specter.service_manager.user_has_encrypted_storage(
+                                current_user
+                            ),
                         )
-                current_user.username = specter_username
+                    current_user.username = specter_username
+
                 if specter_password:
                     if len(specter_password) < min_chars:
                         flash(
@@ -437,15 +455,20 @@ def auth():
                             specter=app.specter,
                             current_version=current_version,
                             rand=rand,
+                            has_service_encrypted_storage=app.specter.service_manager.user_has_encrypted_storage(
+                                current_user
+                            ),
                         )
                     current_user.set_password(specter_password)
+
                 current_user.save_info()
+
             if current_user.is_admin:
                 app.specter.update_auth(method, rate_limit, registration_link_timeout)
                 if method in ["rpcpasswordaspin", "passwordonly", "usernamepassword"]:
                     if method == "passwordonly":
-                        new_password = request.form.get("specter_password_only")
-                        if new_password and len(new_password) < min_chars:
+                        specter_password = request.form.get("specter_password_only")
+                        if specter_password and len(specter_password) < min_chars:
                             flash(
                                 _(
                                     "Please enter a password of a least {} characters"
@@ -461,14 +484,48 @@ def auth():
                                 specter=app.specter,
                                 current_version=current_version,
                                 rand=rand,
+                                has_service_encrypted_storage=app.specter.service_manager.user_has_encrypted_storage(
+                                    current_user
+                                ),
                             )
-                        elif not new_password:
+                        elif not specter_password:
                             # Set to the default
-                            new_password = "admin"
+                            specter_password = "admin"
+
+                        try:
+                            current_user.set_password(specter_password)
+                        except UserSecretException as e:
+                            # Most likely the admin User is logged in but the
+                            # plaintext_user_secret is not available in memory (happens
+                            # when the server restarts).
+                            logger.warn(e)
+                            flash(
+                                _(
+                                    "Error re-encrypting Service data. Log out and log back in before trying again."
+                                ),
+                                "error",
+                            )
+                            return render_template(
+                                "settings/auth_settings.jinja",
+                                method=method,
+                                rate_limit=rate_limit,
+                                registration_link_timeout=registration_link_timeout,
+                                users=users,
+                                specter=app.specter,
+                                current_version=current_version,
+                                rand=rand,
+                                has_service_encrypted_storage=app.specter.service_manager.user_has_encrypted_storage(
+                                    current_user
+                                ),
+                            )
 
-                        current_user.set_password(new_password)
                         current_user.save_info()
 
+                        flash(
+                            _("Admin password successfully updated"),
+                            "info",
+                        )
+
                     if method == "usernamepassword":
                         users = [
                             user
@@ -477,11 +534,22 @@ def auth():
                         ]
                     else:
                         users = None
+
                     app.config["LOGIN_DISABLED"] = False
                 else:
                     users = None
                     app.config["LOGIN_DISABLED"] = True
 
+                    # Cannot support Services if there's no password (admin was already
+                    # warned about this in the UI). Remove User.services, clear the
+                    # `user_secret`, and wipe the ServiceEncryptedStorage.
+                    app.specter.service_manager.remove_all_services_from_user(
+                        current_user
+                    )
+
+            # Redirect if a URL was given via the next variable
+            if request.form.get("next") and request.form.get("next") != "":
+                return redirect(request.form.get("next"))
             app.specter.check()
 
         elif action == "adduser":
@@ -536,6 +604,10 @@ def auth():
         specter=app.specter,
         current_version=current_version,
         rand=rand,
+        has_service_encrypted_storage=app.specter.service_manager.user_has_encrypted_storage(
+            current_user
+        ),
+        next=request.args.get("next", ""),
     )
 
 

src/cryptoadvance/specter/server_endpoints/wallets.py

@@ -6,13 +6,15 @@ from functools import wraps
 import requests
 from cryptoadvance.specter.util.psbt_creator import PsbtCreator
 from cryptoadvance.specter.util.wallet_importer import WalletImporter
+from cryptoadvance.specter.wallet import Wallet
 from flask import Blueprint
 from flask import current_app as app
 from flask import flash, jsonify, redirect, render_template, request, url_for
 from flask_babel import lazy_gettext as _
 from flask_login import login_required
 
-from ..helpers import get_devices_with_keys_by_type
+from ..helpers import bcur2base64, get_devices_with_keys_by_type, get_txid
+from ..key import Key
 from ..managers.wallet_manager import purposes
 from ..persistence import delete_file
 from ..specter_error import SpecterError, handle_exception
@@ -61,6 +63,7 @@ def wallets_overview():
         "wallet/wallets_overview.jinja",
         specter=app.specter,
         rand=rand,
+        services=app.specter.service_manager.services,
     )
 
 
@@ -395,6 +398,7 @@ def history(wallet_alias):
         tx_list_type=tx_list_type,
         specter=app.specter,
         rand=rand,
+        services=app.specter.service_manager.services,
     )
 
 
@@ -405,7 +409,7 @@ def history(wallet_alias):
 @login_required
 @check_wallet
 def receive(wallet_alias):
-    wallet = app.specter.wallet_manager.get_by_alias(wallet_alias)
+    wallet: Wallet = app.specter.wallet_manager.get_by_alias(wallet_alias)
     if request.method == "POST":
         action = request.form["action"]
         if action == "newaddress":
@@ -456,6 +460,7 @@ def send_new(wallet_alias):
     err = None
     ui_option = "ui"
     recipients_txt = ""
+    fillform = False
     subtract = False
     subtract_from = 1
     fee_options = "dynamic"
@@ -579,6 +584,13 @@ def send_new(wallet_alias):
                 specter=app.specter,
                 rand=rand,
             )
+        elif action == "fillform":
+            # TODO: Not yet used. Remove if the use case doesn't happen.
+            # can be used to recommend a transaction from a service (goind to an exchange or so)
+            addresses = request.form.getlist("addresses[]")
+            labels = request.form.getlist("labels[]")
+            amounts = request.form.getlist("amounts[]")
+            fillform = True
 
     if rbf_tx_id:
         try:
@@ -604,6 +616,10 @@ def send_new(wallet_alias):
         psbt=psbt,
         ui_option=ui_option,
         recipients_txt=recipients_txt,
+        addresses=addresses,
+        labels=labels,
+        amounts=amounts,
+        fillform=fillform,
         recipients=list(zip(addresses, amounts, amount_units, labels)),
         subtract=subtract,
         subtract_from=subtract_from,
@@ -713,6 +729,7 @@ def addresses(wallet_alias):
         wallet=wallet,
         specter=app.specter,
         rand=rand,
+        services=app.specter.service_manager.services,
     )
 
 

src/cryptoadvance/specter/services/bitcoinreserve/controller.py

@@ -0,0 +1,24 @@
+import logging
+from flask import Flask, Response, redirect, render_template, request, url_for, flash
+from flask_login import login_required
+
+from ..controller import user_secret_decrypted_required
+from .service import BitcoinReserveService
+
+
+"""
+    Empty placeholder just so the dummyservice/static folder can be wired up to retrieve its img
+"""
+
+logger = logging.getLogger(__name__)
+
+bitcoinreserve_endpoint = BitcoinReserveService.blueprint
+
+
+@bitcoinreserve_endpoint.route("/")
+@login_required
+@user_secret_decrypted_required
+def index():
+    return render_template(
+        "bitcoinreserve/index.jinja",
+    )

src/cryptoadvance/specter/services/bitcoinreserve/service.py

@@ -0,0 +1,14 @@
+from ..service import Service, devstatus_alpha
+
+
+class BitcoinReserveService(Service):
+    id = "bitcoinreserve"
+    name = "Bitcoin Reserve"
+    icon = "img/bitcoinreserve_icon.svg"
+    logo = "img/bitcoinreserve_icon.svg"
+    desc = "Where Europe buys Bitcoin."
+    has_blueprint = True
+    devstatus = devstatus_alpha
+
+    # TODO: As more Services are integrated, we'll want more robust categorization and sorting logic
+    sort_priority = 2

src/cryptoadvance/specter/services/bitcoinreserve/static/img/bitcoinreserve_icon.svg

@@ -0,0 +1,12 @@
+
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 43.9 43.9" style="enable-background:new 0 0 43.9 43.9;" xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#FD5329;}
+	.st1{fill:#231F20;}
+</style>
+<path class="st0" d="M43.9,0L0,0.1v43.9h43.9L43.9,0z M20.4,40.8H3.1V3.1h17.2v4.7H7.8v28.2h12.6L20.4,40.8z M28.2,20.4H11V11h17.3
+	c2.6,0,4.7,2.2,4.6,4.8C32.8,18.3,30.8,20.3,28.2,20.4z M32.9,28.2v4.7H11v-9.4h17.3C30.8,23.5,32.9,25.6,32.9,28.2
+	C32.9,28.2,32.9,28.2,32.9,28.2L32.9,28.2z M40.8,40.8H23.5v-4.7h12.6v-7.8c0-2.5-1.2-4.8-3.1-6.3l0,0c3.5-2.6,4.2-7.5,1.6-11
+	c-1.5-2-3.8-3.1-6.3-3.1h-4.7V3.1h17.2L40.8,40.8z"/>
+</svg>

src/cryptoadvance/specter/services/controller.py

@@ -0,0 +1,84 @@
+import logging
+
+from flask import Blueprint, render_template, redirect, url_for, flash
+from flask import current_app as app, request
+from flask_babel import lazy_gettext as _
+from flask_login import current_user, login_required
+from functools import wraps
+
+logger = logging.getLogger(__name__)
+
+
+# This endpoint is just there to share templates between services.
+services_endpoint = Blueprint(
+    "services_endpoint", __name__, template_folder="templates"
+)
+
+# All blueprint from Services are no longer loaded statically but dynamically when the service-class in initialized
+# check cryptoadvance.specter.services.service_manager.Service for doing that and
+# check cryptoadvance.specter.services/**/manifest for instances of Service-classes and
+# check cryptoadvance.specter.services.service_manager.ServiceManager.services for initialisation of ServiceClasses
+
+
+def user_secret_decrypted_required(func):
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        if app.config["LOGIN_DISABLED"]:
+            # No logins means no password so no user_secret is possible
+            flash(
+                _(
+                    "Service integration requires an authentication method that includes a password"
+                )
+            )
+            return redirect(url_for(f"settings_endpoint.auth"))
+        elif not current_user.is_user_secret_decrypted:
+            flash(_("Must login again to enable protected Services-related data"))
+            # Force re-login; automatically redirects back to calling page
+            return app.login_manager.unauthorized()
+        else:
+            return func(*args, **kwargs)
+
+    return wrapper
+
+
+@services_endpoint.route("/choose", methods=["GET"])
+def choose():
+    return render_template(
+        "services/choose.jinja",
+        is_login_disabled=app.config["LOGIN_DISABLED"],
+        specter=app.specter,
+        services=app.specter.service_manager.services_sorted,
+    )
+
+
+@services_endpoint.route(
+    "/associate_addr/<wallet_alias>/<address>", methods=["GET", "POST"]
+)
+@login_required
+@user_secret_decrypted_required
+def associate_addr(wallet_alias, address):
+    wallet = app.specter.wallet_manager.get_by_alias(wallet_alias)
+
+    if request.method == "POST":
+        service_id = request.form["service_id"]
+        wallet = app.specter.wallet_manager.get_by_alias(wallet_alias)
+        service_cls = app.specter.service_manager.get_service(service_id)
+        service_cls.reserve_address(wallet=wallet, address=address)
+        return redirect(
+            url_for("wallets_endpoint.addresses", wallet_alias=wallet_alias)
+        )
+
+    addr_obj = wallet.get_address_obj(address=address)
+
+    # Inject the User's active Services
+    services = []
+    for service_id in current_user.services:
+        services.append(app.specter.service_manager.get_service(service_id=service_id))
+
+    return render_template(
+        "services/associate_addr.jinja",
+        specter=app.specter,
+        services=services,
+        wallet=wallet,
+        addr_obj=addr_obj,
+    )

src/cryptoadvance/specter/services/service.py

@@ -0,0 +1,181 @@
+import logging
+import os
+from importlib import import_module
+
+from flask import current_app as app
+from flask.blueprints import Blueprint
+from cryptoadvance.specter.wallet import Wallet
+from flask_babel import lazy_gettext as _
+from typing import List
+
+from .service_encrypted_storage import ServiceEncryptedStorageManager
+from .service_annotations_storage import ServiceAnnotationsStorage
+
+from cryptoadvance.specter.addresslist import Address
+
+
+logger = logging.getLogger(__name__)
+
+
+devstatus_alpha = "alpha"
+devstatus_beta = "beta"
+devstatus_prod = "prod"
+
+
+class Service:
+    """A base class for Services"""
+
+    # These should be overrided in implementation classes
+    id = None
+    name = None
+    icon = None
+    logo = None
+    desc = None  # TODO: rename to "description" to be explicit
+    has_blueprint = True  # the default
+    devstatus = devstatus_alpha
+
+    def __init__(self, active, specter):
+        if not hasattr(self, "id"):
+            raise Exception(f"Service {self.__class__} needs ID")
+        if not hasattr(self, "name"):
+            raise Exception(f"Service {self.__class__} needs name")
+        self.active = active
+        self.specter = specter
+        if self.has_blueprint:
+            self.__class__.blueprint = Blueprint(
+                f"{self.id}_endpoint",
+                f"cryptoadvance.specter.services.{self.id}.service",  # To Do: move to subfolder
+                template_folder="templates",
+                static_folder="static",
+            )
+
+            def inject_stuff():
+                """Can be used in all jinja2 templates"""
+                return dict(specter=app.specter, service=self)
+
+            self.__class__.blueprint.context_processor(inject_stuff)
+            # Import the controller for this service
+            import_module(f"cryptoadvance.specter.services.{self.id}.controller")
+            app.register_blueprint(
+                self.__class__.blueprint, url_prefix=f"/svc/{self.id}"
+            )
+
+    @classmethod
+    def set_current_user_service_data(cls, service_data: dict):
+        ServiceEncryptedStorageManager.get_instance().set_current_user_service_data(
+            service_id=cls.id, service_data=service_data
+        )
+
+    @classmethod
+    def update_current_user_service_data(cls, service_data: dict):
+        ServiceEncryptedStorageManager.get_instance().update_current_user_service_data(
+            service_id=cls.id, service_data=service_data
+        )
+
+    @classmethod
+    def get_current_user_service_data(cls) -> dict:
+        return (
+            ServiceEncryptedStorageManager.get_instance().get_current_user_service_data(
+                service_id=cls.id
+            )
+        )
+
+    @classmethod
+    def get_blueprint_name(cls):
+        return f"{cls.id}_endpoint"
+
+    @classmethod
+    def default_address_label(cls):
+        # Have to str() it; can't pass a LazyString to json serializer
+        return str(_("Reserved for {}").format(cls.name))
+
+    @classmethod
+    def reserve_address(cls, wallet: Wallet, address: str, label: str = None):
+        # Mark an Address in a persistent way as being reserved by a Service
+        if not label:
+            label = cls.default_address_label()
+        wallet.associate_address_with_service(
+            address=address, service_id=cls.id, label=label
+        )
+
+    @classmethod
+    def reserve_addresses(
+        cls,
+        wallet: Wallet,
+        label: str = None,
+        num_addresses: int = 10,
+        annotations: dict = None,
+    ) -> List[str]:
+        """
+        Reserve n unused addresses but leave a gap between each one so that this reserved
+        range never causes an address gap in the wallet (e.g. if you reserve ten in a
+        row it's possible that some wallet software will miss a new tx on the 11th
+        address).
+
+        If `label` is not provided, we use Service.default_address_label().
+
+        Optional `annotations` data can be attached to each Address being reserved.
+        """
+        # Track Service-related addresses in ServiceAnnotationsStorage
+        if annotations:
+            annotations_storage = ServiceAnnotationsStorage(
+                service_id=cls.id, wallet=wallet
+            )
+
+        addresses = []
+        start_index = wallet.address_index + 1
+        for i in range(start_index, start_index + (2 * num_addresses), 2):
+            address = wallet.get_address(i)
+
+            # Mark an Address in a persistent way as being reserved by a Service
+            cls.reserve_address(wallet=wallet, address=address)
+
+            addresses.append(address)
+
+            if annotations:
+                annotations_storage.set_addr_annotations(
+                    addr=address, annotations=annotations, autosave=False
+                )
+        if annotations:
+            annotations_storage.save()
+
+        return addresses
+
+    @classmethod
+    def unreserve_addresses(cls, wallet: Wallet):
+        """
+        Clear out Services-related data from any unused Addresses, but leave already-used
+        Addresses as-is.
+        """
+        annotations_storage = ServiceAnnotationsStorage(
+            service_id=cls.id, wallet=wallet
+        )
+        addrs = wallet.get_associated_addresses(service_id=cls.id, unused_only=True)
+        for addr_obj in addrs:
+            wallet.deassociate_address(addr_obj["address"])
+            annotations_storage.remove_addr_annotations(
+                addr_obj.address, autosave=False
+            )
+        annotations_storage.save()
+
+    # def is_active(self):
+    #     return self.active
+
+    # def set_active(self, value):
+    #     self.active = value
+
+    @classmethod
+    def update(self):
+        """
+        Called by backend periodic process to keep Service in sync with any remote
+        data (e.g. fetching the latest data from an external API).
+        """
+        logger.info(f"update() not implemented / not necessary for Service {self.id}")
+
+    """ ***********************************************************************
+                                    Update hooks
+    *********************************************************************** """
+
+    @classmethod
+    def on_user_login(cls):
+        pass

src/cryptoadvance/specter/services/service_annotations_storage.py

@@ -0,0 +1,72 @@
+import os
+
+from cryptoadvance.specter.managers.genericdata_manager import GenericDataManager
+from cryptoadvance.specter.wallet import Wallet
+
+
+class ServiceAnnotationsStorage(GenericDataManager):
+    """
+    Stores Service-specific annotations for addresses and txs. Each wallet + service
+    pairing will get its own <wallet_alias>_<service>.json file.
+
+    Annotations must be in a json-serializable dict, keyed on addr or tx:
+    {
+        "someaddress": {
+            "foo": 1,
+            "bar": "hello",
+        }
+    }
+    """
+
+    def __init__(self, service_id: str, wallet: Wallet):
+        # Must set these before calling parent's __init__() so that the data_file
+        #   property will have the proper member vars.
+        self.service_id = service_id
+        self.wallet = wallet
+
+        super().__init__(wallet.manager.data_folder)
+
+        if "addrs" not in self.data:
+            self.data["addrs"] = {}
+
+        if "txs" not in self.data:
+            self.data["txs"] = {}
+
+    @property
+    def data_file(self):
+        # TODO: currently saving in the wallets/ dir but not in the main vs regtest subdir
+        return os.path.join(
+            self.wallet.manager.data_folder,
+            f"{self.wallet.alias}_{self.service_id}.json",
+        )
+
+    def save(self):
+        # Expose for external use
+        self._save()
+
+    def set_addr_annotations(self, addr: str, annotations: dict, autosave: bool = True):
+        self.data["addrs"][addr] = annotations
+        if autosave:
+            self._save()
+
+    def remove_addr_annotations(self, addr: str, autosave: bool = True):
+        self.data["addrs"].pop(addr, None)
+        if autosave:
+            self._save()
+
+    def get_addr_annotations(self, addr: str):
+        return self.data["addrs"].get(addr, None)
+
+    def get_all_addr_annotations(self):
+        return self.data["addrs"]
+
+    def set_tx_annotations(self, txid: str, annotations: dict, autosave: bool = True):
+        self.data["txs"][txid] = annotations
+        if autosave:
+            self._save()
+
+    def get_tx_annotations(self, txid: str):
+        return self.data["txs"].get(txid, None)
+
+    def get_all_tx_annotations(self):
+        return self.data["txs"]

src/cryptoadvance/specter/services/service_encrypted_storage.py

@@ -0,0 +1,162 @@
+import json
+import logging
+import os
+
+from cryptoadvance.specter.managers.genericdata_manager import GenericDataManager
+from cryptoadvance.specter.managers.singleton import ConfigurableSingleton
+from cryptoadvance.specter.specter_error import SpecterError
+from cryptoadvance.specter.user import User
+
+logger = logging.getLogger("__name__")
+
+
+class ServiceEncryptedStorageError(Exception):
+    pass
+
+
+class ServiceEncryptedStorage(GenericDataManager):
+    """
+    Encrypted storage class for *ALL* Services-related secrets and config for a given user.
+    The json file is stored as: <username>_services.json
+
+    Each Service may specify its own json data format.
+
+    Data is stored internally as a json string for easier encryption handling to disk.
+
+    Note that this is storage is separate from the un-encrypted ServiceAnnotationsStorage.
+    This storage is meant for secrets that must be encrypted and any Service-related config
+    that the User might need to store (e.g. which Wallet is attached to the Service). This
+    config data doesn't need to be encrypted, but we may as well store it here rather than
+    adding data bloat to the User json.
+
+    * `disable_decrypt`: Allows the ServiceEncryptedStorageManager to see which Services
+        have data but without attempting to decrypt the values. This allows us to check
+        if there is encrypted Service data even when we don't have the
+        plaintext_user_secret.
+    """
+
+    def __init__(self, data_folder: str, user: User, disable_decrypt: bool = False):
+        if not user.plaintext_user_secret and not disable_decrypt:
+            raise ServiceEncryptedStorageError(
+                f"User {user} must be authenticated with password before encrypted service data can be loaded"
+            )
+
+        # Must set the user before calling the parent's __init__(); it calls load()
+        #   which then calls data_file below.
+        self.user = user
+
+        if disable_decrypt:
+            super().__init__(data_folder, encryption_key=None)
+        else:
+            super().__init__(data_folder, encryption_key=user.plaintext_user_secret)
+
+    @property
+    def encrypted_fields(self):
+        """We override the class member to force ALL data fields to be considered
+        encrypted."""
+        fields = list(self.data.keys())
+        if "encrypted_storage_version" in fields:
+            fields.pop(fields.index("encrypted_storage_version"))
+        return fields
+
+    @property
+    def data_file(self):
+        return os.path.join(self.data_folder, f"{self.user.username}_services.json")
+
+    def set_service_data(self, service_id: str, data: dict, autosave: bool = True):
+        """Store the api_data json blob as a string; completely overwrites previous state"""
+        if data == {} and service_id in self.data:
+            del self.data[service_id]
+        else:
+            self.data[service_id] = json.dumps(data)
+        if autosave:
+            self._save()
+
+    def update_service_data(self, service_id: str, data: dict, autosave: bool = True):
+        if data == {}:
+            logger.debug("This is a nonsense no-op")
+            return
+
+        # Add or update fields; does not remove existing fields
+        if service_id not in self.data:
+            # Initialize a blank entry
+            cur_data = {}
+        else:
+            cur_data = json.loads(self.data[service_id])
+        cur_data.update(data)
+        self.data[service_id] = json.dumps(cur_data)
+        if autosave:
+            self._save()
+
+    def get_service_data(self, service_id: str) -> dict:
+        service_data = self.data.get(service_id, None)
+        if service_data:
+            # Convert string back into json blob
+            service_data = json.loads(service_data)
+        else:
+            service_data = {}
+        return service_data
+
+
+class ServiceEncryptedStorageManager(ConfigurableSingleton):
+    """Singleton that manages access to users' ServiceApiKeyStorage; context-aware so it
+    knows who the current_user is for the given request context.
+
+    Requires a one-time configuration call on startup in the ServiceManager.
+    """
+
+    @classmethod
+    def configure_instance(cls, data_folder, user_manager):
+        super().configure_instance()
+        cls._instance.data_folder = data_folder
+        cls._instance.user_manager = user_manager
+        cls._instance.storage_by_user = {}
+
+    def get_raw_encrypted_data(self, user: User) -> dict:
+        """Doesn't attempt to decrypt the ServiceEncryptedStorage, just returns the
+        user's full encrypted Service data json as-is."""
+        return ServiceEncryptedStorage(
+            self.data_folder, user, disable_decrypt=True
+        ).data
+
+    def _get_current_user_service_storage(self) -> ServiceEncryptedStorage:
+        """Returns the storage-class for the current_user. Lazy_init if necessary"""
+        user = self.user_manager.get_user()
+
+        if user not in self.storage_by_user:
+            self.storage_by_user[user] = ServiceEncryptedStorage(self.data_folder, user)
+        return self.storage_by_user[user]
+
+    def set_current_user_service_data(self, service_id: str, service_data: dict):
+        self._get_current_user_service_storage().set_service_data(
+            service_id, service_data
+        )
+
+    def update_current_user_service_data(self, service_id: str, service_data: dict):
+        # Add or update fields; does not remove existing fields
+        self._get_current_user_service_storage().update_service_data(
+            service_id, service_data
+        )
+
+    def get_current_user_service_data(self, service_id: str) -> dict:
+        service_storage = self._get_current_user_service_storage()
+        if service_storage:
+            return service_storage.get_service_data(service_id)
+
+    def unload_current_user(self):
+        """Clear user's ServiceEncryptedStorage from memory (but it remains safely on disk)"""
+        user = self.user_manager.get_user()
+        if user and user in self.storage_by_user:
+            self.storage_by_user[user] = None
+
+    def delete_all_service_data(self, user: User):
+        """Completely removes all data in the User's ServiceEncryptedStorage from memory and on-disk."""
+        # Clear it from memory...
+        self.storage_by_user.pop(user, None)
+
+        # ...and wipe the on-disk storage
+        encrypted_storage = ServiceEncryptedStorage(
+            self.data_folder, user, disable_decrypt=True
+        )
+        encrypted_storage.data = {}
+        encrypted_storage._save()

src/cryptoadvance/specter/services/service_manager.py

@@ -0,0 +1,162 @@
+import json
+import logging
+import os
+
+from flask import current_app as app, url_for
+from flask.blueprints import Blueprint
+from importlib import import_module
+from inspect import isclass
+from pathlib import Path
+from pkgutil import iter_modules
+from typing import Dict, List
+from cryptoadvance.specter.user import User
+
+from cryptoadvance.specter.managers.singleton import ConfigurableSingletonException
+from ..util.reflection import get_subclasses_for_class
+
+from .service import Service
+from .service_encrypted_storage import ServiceEncryptedStorageManager
+
+logger = logging.getLogger(__name__)
+
+
+class ServiceManager:
+    """Loads support for all Services it auto-discovers."""
+
+    def __init__(self, specter, devstatus_threshold):
+        self.specter = specter
+        self.devstatus_threshold = devstatus_threshold
+
+        # Each Service class is stored here, keyed on its Service.id str
+        self._services: Dict[str, Service] = {}
+        logger.info("----> starting service discovery <----")
+        for clazz in get_subclasses_for_class(Service):
+            compare_map = {"alpha": 1, "beta": 2, "prod": 3}
+            if compare_map[self.devstatus_threshold] <= compare_map[clazz.devstatus]:
+                # First configure the service
+                self.configure_service_for_module(clazz.id)
+                # Now activate it
+                self._services[clazz.id] = clazz(
+                    active=clazz.id in self.specter.config.get("services", []),
+                    specter=self.specter,
+                )
+                logger.info(f"Service {clazz.__name__} activated ({clazz.devstatus})")
+            else:
+                logger.info(
+                    f"Service {clazz.__name__} not activated due to devstatus ( {self.devstatus_threshold} > {clazz.devstatus} )"
+                )
+
+        # Configure and instantiate the one and only ServiceEncryptedStorageManager
+        try:
+            ServiceEncryptedStorageManager.configure_instance(
+                specter.data_folder, specter.user_manager
+            )
+        except ConfigurableSingletonException as e:
+            # Test suite triggers multiple calls; ignore for now.
+            pass
+        logger.info("----> finished service discovery <----")
+
+    @classmethod
+    def configure_service_for_module(cls, service_id):
+        """searches for ConfigClasses in the module-Directory and merges its config in the global config"""
+        try:
+            module = import_module(
+                f"cryptoadvance.specter.services.{service_id}.config"
+            )
+        except ModuleNotFoundError:
+            logger.warning(
+                f"Service {service_id} does not have a service Configuration! Skipping!"
+            )
+            return
+        main_config_clazz_name = app.config.get("SPECTER_CONFIGURATION_CLASS_FULLNAME")
+        main_config_clazz_slug = main_config_clazz_name.split(".")[-1]
+        potential_config_classes = []
+        for attribute_name in dir(module):
+            attribute = getattr(module, attribute_name)
+            if isclass(attribute):
+                clazz = attribute
+                potential_config_classes.append(clazz)
+                if clazz.__name__.endswith(
+                    main_config_clazz_slug
+                ):  # e.g. BaseConfig or DevelopmentConfig
+                    cls.import_config(clazz)
+                    return
+
+        logger.warning(
+            f"Could not find a configuration for Service {module} ... trying parent-classes of main-config"
+        )
+        config_module = import_module(".".join(main_config_clazz_name.split(".")[0:-1]))
+
+        config_clazz = getattr(config_module, main_config_clazz_slug)
+        config_candidate_class = config_clazz.__bases__[0]
+        while config_candidate_class != object:
+            for clazz in potential_config_classes:
+                if clazz.__name__.endswith(config_candidate_class.__name__):
+                    cls.import_config(clazz)
+                    return
+            config_candidate_class = config_candidate_class.__bases__[0]
+
+    @classmethod
+    def import_config(cls, clazz):
+        logger.info(f"Loading Service-specific configuration from {clazz}")
+        for key in dir(clazz):
+            if key.isupper():
+                if app.config.get(key):
+                    raise Exception(
+                        f"Config {clazz} tries to override existing key {key}"
+                    )
+                app.config[key] = getattr(clazz, key)
+                logger.debug(f"setting {key} = {app.config[key]}")
+
+    @property
+    def services(self) -> Dict[str, Service]:
+        return self._services
+
+    @property
+    def services_sorted(self):
+        service_names = sorted(
+            self._services, key=lambda s: self._services[s].sort_priority
+        )
+        return [self._services[s] for s in service_names]
+
+    def user_has_encrypted_storage(self, user: User) -> bool:
+        """Looks for any data for any service in the User's ServiceEncryptedStorage.
+        This check works even if the user doesn't have their plaintext_user_secret
+        available."""
+        encrypted_data = (
+            ServiceEncryptedStorageManager.get_instance().get_raw_encrypted_data(user)
+        )
+        print(f"encrypted_data: {encrypted_data} for {user}")
+        return encrypted_data != {}
+
+    def set_active_services(self, service_names_active):
+        logger.debug(f"Setting these services active: {service_names_active}")
+        self.specter.update_services(service_names_active)
+        for _, service in self.services.items():
+            logger.debug(
+                f"Setting service '{service.id}' active to {service.id in service_names_active}"
+            )
+            service.active = service.id in service_names_active
+
+    def get_service(self, service_id: str) -> Service:
+        if service_id not in self._services:
+            # TODO: better error handling?
+            raise Exception(f"No such Service: '{service_id}'")
+        return self._services[service_id]
+
+    def remove_all_services_from_user(self, user: User):
+        """
+        Clears User.services and `user_secret`; wipes the User's
+        ServiceEncryptedStorage.
+        """
+        # Don't show any Services on the sidebar for the admin user
+        user.services.clear()
+
+        # Reset as if we never had any encrypted storage
+        user.delete_user_secret(autosave=False)
+        user.save_info()
+
+        if self.user_has_encrypted_storage(user=user):
+            # Encrypted Service data is now orphaned since there is no
+            # password. So wipe it from the disk.
+            ServiceEncryptedStorageManager.get_instance().delete_all_service_data(user)

src/cryptoadvance/specter/services/swan/client.py

@@ -0,0 +1,514 @@
+import base64
+import datetime
+import hashlib
+import json
+import logging
+import pytz
+import requests
+import secrets
+
+from decimal import Decimal
+from flask import current_app as app
+from flask_babel import lazy_gettext as _
+from typing import List
+
+from cryptoadvance.specter.wallet import Wallet
+
+from .service import SwanService
+
+
+logger = logging.getLogger(__name__)
+
+
+# TODO: Update with prod values
+client_id = "specter-dev"
+client_secret = "BcetcVcmueWf5P3UPJnHhCBMQ49p38fhzYwM7t3DJGzsXSjm89dDR5URE46SY69j"
+code_verifier = "64fRjTuy6SKqdC1wSoInUNxX65dQUhVVKTqZXuQ7dqw"
+api_url = app.config.get("SWAN_API_URL")
+
+
+class SwanApiException(Exception):
+    pass
+
+
+class SwanApiRefreshTokenException(SwanApiException):
+    pass
+
+
+def get_oauth2_start_url():
+    """
+    Set up the Swan API integration by requesting our initial access_token and
+    refresh_token.
+    """
+    # Let's start the PKCE-flow
+    global code_verifier
+
+    if code_verifier is None:
+        code_verifier = secrets.token_urlsafe(43)
+    # see specification: https://datatracker.ietf.org/doc/html/rfc7636#section-4.2
+    # and example impl: https://github.com/RomeoDespres/pkce/blob/master/pkce/__init__.py#L94-L96
+    hashed = hashlib.sha256(code_verifier.encode("ascii")).digest()
+    encoded = base64.urlsafe_b64encode(hashed)
+    code_challenge = encoded.decode("ascii")[:-1]
+
+    flow_url = f"{api_url}/oidc/auth?"
+    query_params = [
+        "client_id=specter-dev",
+        "redirect_uri=http://localhost:25441/svc/swan/oauth2/callback",  # TODO: Will localhost work in all usage contexts (e.g. standalone app)?
+        "response_type=code",
+        "response_mode=query",
+        f"code_challenge={code_challenge}",
+        "code_challenge_method=S256",
+        "state=kjkmdskdmsmmsmdslmdlsm",
+        "scope=offline_access v1 write:vendor_wallet read:vendor_wallet write:automatic_withdrawal read:automatic_withdrawal",
+        "prompt=consent",
+    ]
+    flow_url += "&".join(query_params)
+
+    return flow_url
+
+
+def get_access_token(code: str = None, code_verifier: str = None):
+    """
+    If code and code_verifier are specified, this is our initial request for an
+    access_token and, more importantly, the refresh_token.
+
+    If code is None, use the refresh_token to get a new short-lived access_token.
+
+    If we don't have the refresh_token, raise SwanApiRefreshTokenException.
+    """
+    if code:
+        # Requesting initial refresh_token and access_token
+        payload = {
+            "client_id": "specter-dev",
+            "client_secret": client_secret,
+            "code_verifier": code_verifier,
+            "grant_type": "authorization_code",
+            "code": code,
+        }
+        auth_header = None
+    else:
+        service_data = SwanService.get_current_user_service_data()
+        if SwanService.is_access_token_valid():
+            return service_data[SwanService.ACCESS_TOKEN]
+
+        # Use the refresh_token to get a new access_token
+        if SwanService.REFRESH_TOKEN not in service_data:
+            raise SwanApiRefreshTokenException(
+                "access_token is expired but we don't have a refresh_token"
+            )
+
+        payload = {
+            "grant_type": "refresh_token",
+            # "redirect_uri":   # Necessary?
+            "refresh_token": service_data[SwanService.REFRESH_TOKEN],
+            "scope": "offline_access v1 write:vendor_wallet read:vendor_wallet write:automatic_withdrawal read:automatic_withdrawal",
+        }
+
+        auth_hash = base64.b64encode(f"{client_id}:{client_secret}".encode()).decode()
+        auth_header = {
+            "Authorization": f"Basic {auth_hash}",
+        }
+        logger.debug(f"auth_hash: {auth_hash}")
+        logger.debug("Using the refresh_token to request an access_token")
+        logger.debug(f"payload: {json.dumps(payload, indent=4)}")
+
+    logger.debug(payload)
+    logger.debug(auth_header)
+
+    response = requests.post(
+        f"{api_url}/oidc/token",
+        data=payload,
+        headers=auth_header,
+    )
+    logger.debug(f"api_url: {api_url}")
+    logger.debug(response)
+    logger.debug(response.text)
+    resp = json.loads(response.text)
+    """
+        {
+            "access_token": "***************",
+            "expires_in": 3600,
+            "refresh_token": "***************",
+            "scope": "offline_access v1 write:vendor_wallet read:vendor_wallet write:automatic_withdrawal read:automatic_withdrawal",
+            "token_type": "Bearer"
+        }
+    """
+    # TODO: Remove debugging
+    logger.debug(json.dumps(resp, indent=4))
+    if resp.get("access_token"):
+        new_api_data = {
+            SwanService.ACCESS_TOKEN: resp["access_token"],
+            SwanService.ACCESS_TOKEN_EXPIRES: (
+                datetime.datetime.now(tz=pytz.utc)
+                + datetime.timedelta(seconds=resp["expires_in"])
+            ).timestamp(),
+        }
+        if "refresh_token" in resp:
+            new_api_data[SwanService.REFRESH_TOKEN] = resp["refresh_token"]
+
+        SwanService.update_current_user_service_data(new_api_data)
+
+        # TODO: Remove debugging
+        logger.debug(
+            f"service_data: {json.dumps(SwanService.get_current_user_service_data(), indent=4)}"
+        )
+        return resp["access_token"]
+    else:
+        logger.warning(response)
+        raise SwanApiException(response.text)
+
+
+def handle_oauth2_auth_callback(request):
+    code = request.args.get("code")
+    logger.debug(f"request.args: {request.args}")
+    logger.debug(f"looks good, we got a code: {code}")
+    get_access_token(code=code, code_verifier=code_verifier)
+
+
+def authenticated_request(
+    endpoint: str, method: str = "GET", json_payload: dict = {}
+) -> dict:
+    logger.debug(f"{method} endpoint: {endpoint}")
+
+    access_token = get_access_token()
+
+    auth_header = {
+        "Authorization": f"Bearer {access_token}",
+    }
+    try:
+        if method == "GET":
+            response = requests.get(api_url + endpoint, headers=auth_header)
+        elif method in ["POST", "PATCH", "PUT", "DELETE"]:
+            response = requests.request(
+                method=method,
+                url=api_url + endpoint,
+                headers=auth_header,
+                json=json_payload,
+            )
+        if response.status_code != 200:
+            raise SwanApiException(f"{response.status_code}: {response.text}")
+        return response.json()
+    except Exception as e:
+        # TODO: tighten up expected Exceptions
+        logger.exception(e)
+        logger.error(
+            f"endpoint: {endpoint} | method: {method} | payload: {json.dumps(json_payload, indent=4)}"
+        )
+        logger.error(f"{response.status_code}: {response.text}")
+        raise e
+
+
+def get_autowithdrawal_addresses(swan_wallet_id: str = None) -> dict:
+    """
+    {
+        "entity": "wallet",
+        "item": {
+            "id": "c47e1e83-90a0-45da-ae25-6a0d324b9f29",
+            "isConfirmed": false,
+            "displayName": "Specter autowithdrawal to SeedSigner demo",
+            "metadata": {
+                "oidc": {
+                    "clientId": "specter-dev"
+                },
+                "specter_wallet_alias": "seedsigner_demo"
+            },
+            "btcAddresses": []
+        }
+    }
+    """
+    if not swan_wallet_id:
+        swan_wallet_id = SwanService.get_current_user_service_data().get(
+            SwanService.SWAN_WALLET_ID
+        )
+
+    resp = authenticated_request(
+        endpoint=f"/apps/v20210824/wallets/{swan_wallet_id}?full=true",
+        method="GET",
+    )
+
+    logger.debug(json.dumps(resp, indent=4))
+    return resp
+
+
+def update_autowithdrawal_addresses(
+    specter_wallet_name: str, specter_wallet_alias: str, addresses: List[str]
+) -> dict:
+    """
+    * If SWAN_WALLET_ID is known, any existing unused addresses are cleared.
+    * If there is no known SWAN_WALLET_ID, we `POST` to create an initial Swan wallet and store the resulting SWAN_WALLET_ID.
+    * Sends the list of new addresses for SWAN_WALLET_ID.
+    """
+    swan_wallet_id = SwanService.get_current_user_service_data().get(
+        SwanService.SWAN_WALLET_ID
+    )
+
+    if swan_wallet_id:
+        # We already have a Swan walletId. DELETE the existing unused addresses...
+        delete_autowithdrawal_addresses(swan_wallet_id)
+
+        # ...and then append the new ones.
+        endpoint = f"/apps/v20210824/wallets/{swan_wallet_id}/addresses"
+        method = "PATCH"
+    else:
+        # We don't yet have a Swan walletId. POST to create one.
+        endpoint = "/apps/v20210824/wallets"
+        method = "POST"
+
+    resp = authenticated_request(
+        endpoint=endpoint,
+        method=method,
+        json_payload={
+            "btcAddresses": [{"address": addr} for addr in addresses],
+            "displayName": str(
+                _('Specter Desktop "{}"').format(specter_wallet_name)
+            ),  # Can't pass a LazyString into json
+            "metadata": {
+                "specter_wallet_alias": specter_wallet_alias,
+            },
+        },
+    )
+
+    """
+    Response should include wallet ("item") details:
+        {
+            "entity": "wallet",
+            "item": {
+                "id": "c47e1e83-90a0-45da-ae25-6a0d324b9f29",
+                "isConfirmed": false,
+                "displayName": "Specter autowithdrawal to SeedSigner demo",
+                "metadata": {
+                    "oidc": {
+                        "clientId": "specter-dev"
+                    },
+                    "specter_wallet_alias": "seedsigner_demo"
+                },
+                "btcAddresses": []
+            }
+        }
+    """
+    logger.debug(json.dumps(resp, indent=4))
+
+    if "item" in resp and "id" in resp["item"]:
+        if resp["item"]["id"] != swan_wallet_id:
+            swan_wallet_id = resp["item"]["id"]
+
+            # Save the swan_wallet_id in the user's persistent Service settings
+            logger.debug(f"Updating the Swan wallet id to {swan_wallet_id}")
+            SwanService.update_current_user_service_data(
+                {SwanService.SWAN_WALLET_ID: swan_wallet_id}
+            )
+    else:
+        raise SwanApiException(
+            f"No 'id' returned for the new/updated wallet: {json.dumps(resp, indent=4)}"
+        )
+
+
+def delete_autowithdrawal_addresses(swan_wallet_id: str):
+    """
+    Deletes all unused autowithdrawal addresses from the specified SWAN_WALLET_ID
+    """
+    resp = authenticated_request(
+        endpoint=f"/apps/v20210824/wallets/{swan_wallet_id}/addresses",
+        method="DELETE",
+    )
+    logger.debug(json.dumps(resp, indent=4))
+    return resp
+
+
+def get_autowithdrawal_info() -> dict:
+    """
+    See note in set_autowithdrawal. This returns all autowithdrawal objs from the Swan
+    side.
+    """
+    resp = authenticated_request(
+        endpoint="/apps/v20210824/automatic-withdrawal",
+        method="GET",
+    )
+    logger.debug(json.dumps(resp, indent=4))
+    return resp
+
+
+def set_autowithdrawal(btc_threshold: Decimal) -> dict:
+    """
+    0 == Weekly; other float values = BTC threshold
+
+    The Swan api generates a new autowithdrawal id each time but there is no support to
+    update an existing autowithdrawal, other than activating or deactivating it.
+
+    New autowithdrawals are initialized as `isActive: false` and require the user to
+    complete a Swan-side email verification step.
+
+    We save the resulting autowithdrawal_id even though it isn't clear at the moment if
+    it's desirable to ever call the `deactivate/` or `activate/` endpoints.
+    """
+    service_data = SwanService.get_current_user_service_data()
+    swan_wallet_id = service_data.get(SwanService.SWAN_WALLET_ID)
+
+    endpoint = "/apps/v20210824/automatic-withdrawal"
+    method = "POST"
+    resp = authenticated_request(
+        endpoint=endpoint,
+        method=method,
+        json_payload={
+            "walletId": swan_wallet_id,
+            "minBtcThreshold": btc_threshold,
+        },
+    )
+    logger.debug(json.dumps(resp, indent=4))
+
+    """
+        {
+            "entity": "automaticWithdrawal",
+            "item": {
+                "id": "******************",
+                "minBtcThreshold": "0.01",
+                "isActive": false,
+                "isCanceled": false,
+                "createdAt": "2022-01-07T02:14:56.070Z",
+                "walletId": "******************",
+                "walletAddressId": null
+            }
+        }
+    """
+    if "item" in resp and "id" in resp["item"]:
+        autowithdrawal_id = resp["item"]["id"]
+        if autowithdrawal_id != service_data.get(SwanService.AUTOWITHDRAWAL_ID):
+            SwanService.update_current_user_service_data(
+                {
+                    SwanService.AUTOWITHDRAWAL_ID: autowithdrawal_id,
+                }
+            )
+    else:
+        raise SwanApiException(
+            f"No 'id' returned for the new/updated autowithdrawal: {json.dumps(resp, indent=4)}"
+        )
+
+    return resp
+
+
+def activate_autowithdrawal() -> dict:
+    """
+    Activates the autowithdrawal specified in SwanService.AUTOWITHDRAWAL_ID.
+
+    If the automatic withdrawal was just created, this will generate a 400 error:
+    "Cannot activate an automatic withdrawal before withdrawal address is confirmed".
+
+    The user must first confirm the first withdrawal addr via Swan-side email flow.
+    After they confirm, the autowithdrawal should then return `isActive: true`.
+
+    NOT CURRENTLY USED; remove if we don't ever enable disable/activate flows.
+    """
+    service_data = SwanService.get_current_user_service_data()
+    autowithdrawal_id = service_data.get(SwanService.AUTOWITHDRAWAL_ID)
+    if not autowithdrawal_id:
+        raise SwanApiException(
+            f"AUTOWITHDRAWAL_ID ({SwanService.AUTOWITHDRAWAL_ID}) not found in service data"
+        )
+
+    endpoint = f"/apps/v20210824/automatic-withdrawal/{autowithdrawal_id}/activate"
+    method = "POST"
+    resp = authenticated_request(
+        endpoint=endpoint,
+        method=method,
+    )
+    logger.debug(json.dumps(resp, indent=4))
+
+    """
+        {
+            "entity": "automaticWithdrawal",
+            "item": {
+                "id": "******************",
+                "minBtcThreshold": "0.01",
+                "isActive": true,
+                "isCanceled": false,
+                "createdAt": "2022-01-07T02:14:56.070Z",
+                "walletId": "******************",
+                "walletAddressId": null
+            }
+        }
+    """
+    if "item" in resp and "id" in resp["item"]:
+        if resp["item"]["id"] != service_data[SwanService.AUTOWITHDRAWAL_ID]:
+            autowithdrawal_id = resp["item"]["id"]
+    else:
+        raise SwanApiException(
+            f"No 'id' returned for the new/updated autowithdrawal: {json.dumps(resp, indent=4)}"
+        )
+
+    return resp
+
+
+def get_wallet_details(swan_wallet_id: str) -> dict:
+    """
+    {
+        "entity": "wallet",
+        "item": {
+            "id": "********",
+            "isConfirmed": false,
+            "displayName": "Specter autowithdrawal to SeedSigner demo",
+            "metadata": {
+                "oidc": {
+                    "clientId": "specter-dev"
+                },
+                "specter_wallet_alias": "seedsigner_demo"
+            },
+            "btcAddresses": []
+        }
+    }
+    """
+    resp = authenticated_request(
+        endpoint=f"/apps/v20210824/wallets/{swan_wallet_id}",
+        method="GET",
+    )
+
+    logger.debug(json.dumps(resp, indent=4))
+    return resp
+
+
+def get_wallets() -> dict:
+    """
+    Return all Swan wallet entries. Should only be one per Specter-Swan user combo (but can be more due
+    to testing/debugging, calling `/wallets` POST more than once, etc.)
+    """
+    resp = authenticated_request(
+        endpoint=f"/apps/v20210824/wallets",
+        method="GET",
+    )
+    """
+        {
+            "entity": "wallet",
+            "list": [
+                {
+                    "id": "**********",
+                    "walletAddressId": "**********",
+                    "btcAddress": "bc1q**********",
+                    "isConfirmed": false,
+                    "displayName": "Specter Desktop \"SeedSigner demo\"",
+                    "metadata": {
+                        "oidc": {
+                            "clientId": "specter-dev"
+                        },
+                        "specter_wallet_alias": "seedsigner_demo"
+                    }
+                },
+                {
+                    "id": "**********",
+                    "walletAddressId": "**********",
+                    "btcAddress": "bc1q**********",
+                    "isConfirmed": false,
+                    "displayName": "Specter Desktop \"DCA Corn\"",
+                    "metadata": {
+                        "oidc": {
+                            "clientId": "specter-dev"
+                        },
+                        "specter_wallet_alias": "dca_corn_2"
+                    }
+                },
+                ...,
+            ]
+        }
+    """
+    logger.debug(json.dumps(resp, indent=4))
+
+    return resp

src/cryptoadvance/specter/services/swan/config.py

@@ -0,0 +1,6 @@
+class BaseConfig:
+    SWAN_API_URL = "https://dev-api.swanbitcoin.com"
+
+
+class ProductionConfig(BaseConfig):
+    SWAN_API_URL = "https://api.swanbitcoin.com"

src/cryptoadvance/specter/services/swan/controller.py

@@ -0,0 +1,223 @@
+import json
+import logging
+
+from decimal import Decimal
+from flask import redirect, render_template, request, url_for, flash
+from flask import current_app as app
+from flask_babel import lazy_gettext as _
+from flask_login import current_user, login_required
+from functools import wraps
+from cryptoadvance.specter.user import User
+from cryptoadvance.specter.wallet import Wallet
+
+from cryptoadvance.specter.services.service_encrypted_storage import (
+    ServiceEncryptedStorageError,
+)
+from . import client as swan_client
+from .client import SwanApiException
+from .service import SwanService
+from ..controller import user_secret_decrypted_required
+
+
+logger = logging.getLogger(__name__)
+
+swan_endpoint = SwanService.blueprint
+
+
+def refreshtoken_required(func):
+    """Refresh token needed for any endpoint that interacts with Swan API"""
+
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        try:
+            if not SwanService.has_refresh_token():
+                logger.debug(f"No refresh token, redirecting to relink Swan account")
+                return redirect(
+                    url_for(f"{SwanService.get_blueprint_name()}.oauth2_start")
+                )
+        except ServiceEncryptedStorageError as e:
+            logger.debug(repr(e))
+            flash("Re-login required to access your protected services data")
+
+            # Use Flask's built-in re-login w/automatic redirect back to calling page
+            return app.login_manager.unauthorized()
+        return func(*args, **kwargs)
+
+    return wrapper
+
+
+@swan_endpoint.route("/")
+@login_required
+@user_secret_decrypted_required
+def index():
+    if SwanService.has_refresh_token():
+        # User has already completed Swan integration; skip ahead
+        return redirect(url_for(f"{SwanService.get_blueprint_name()}.withdrawals"))
+    return render_template(
+        "swan/index.jinja",
+    )
+
+
+@swan_endpoint.route("/withdrawals")
+@login_required
+@refreshtoken_required
+def withdrawals():
+    # The wallet currently configured for ongoing autowithdrawals
+    wallet: Wallet = SwanService.get_associated_wallet()
+
+    return render_template(
+        "swan/withdrawals.jinja",
+        # txlist=swan_txs,
+        wallet=wallet,
+        services=app.specter.service_manager.services,
+        swan_id=SwanService.id,
+        autowithdrawal_threshold=SwanService.get_current_user_service_data().get(
+            SwanService.AUTOWITHDRAWAL_THRESHOLD
+        ),
+    )
+
+
+@swan_endpoint.route("/settings", methods=["GET"])
+@login_required
+@refreshtoken_required
+def settings():
+    associated_wallet: Wallet = SwanService.get_associated_wallet()
+
+    # Get the user's Wallet objs, sorted by Wallet.name
+    wallet_names = sorted(current_user.wallet_manager.wallets.keys())
+    wallets = [current_user.wallet_manager.wallets[name] for name in wallet_names]
+
+    return render_template(
+        "swan/settings.jinja",
+        associated_wallet=associated_wallet,
+        wallets=wallets,
+        cookies=request.cookies,
+        num_reserved_addrs=SwanService.MIN_PENDING_AUTOWITHDRAWAL_ADDRS,
+    )
+
+
+@swan_endpoint.route("/settings/autowithdrawal", methods=["POST"])
+@login_required
+@refreshtoken_required
+def update_autowithdrawal():
+    threshold = request.form["threshold"]
+    destination_wallet_alias = request.form["destination_wallet"]
+    wallet = current_user.wallet_manager.get_by_alias(destination_wallet_alias)
+
+    try:
+        SwanService.set_autowithdrawal_settings(wallet=wallet, btc_threshold=threshold)
+        return redirect(url_for(f"{SwanService.get_blueprint_name()}.withdrawals"))
+    except SwanApiException as e:
+        logger.exception(e)
+        flash(_("Error communicating with Swan API"))
+        return redirect(url_for(f"{SwanService.get_blueprint_name()}.settings"))
+
+
+""" ***************************************************************************
+                                OAuth2 endpoints
+*************************************************************************** """
+
+
+@swan_endpoint.route("/oauth2/start")
+@login_required
+@user_secret_decrypted_required
+def oauth2_start():
+    """
+    Set up the Swan API integration by requesting our initial access_token and
+    refresh_token.
+    """
+    # Do we have a token already?
+    if SwanService.has_refresh_token():
+        return redirect(url_for(f"{SwanService.get_blueprint_name()}.settings"))
+
+    # Let's start the PKCE-flow
+    flow_url = swan_client.get_oauth2_start_url()
+
+    return render_template(
+        "swan/oauth2_start.jinja",
+        flow_url=flow_url,
+    )
+
+
+"""
+    Note: the callback from Swan will be treated by Flask as an AnonymousUserMixin request
+    but we need the user logged in and their user_secret decrypted. So we must require
+    @login_required here which will interrupt the return flow with the login prompt.
+"""
+
+
+@swan_endpoint.route("/oauth2/callback")
+@login_required
+def oauth2_auth():
+    if request.args.get("error"):
+        logger.error(
+            f"OAuth2 flow error: {request.args.get('error')}, {request.args.get('error_description')}"
+        )
+        return render_template(
+            "500.jinja",
+            error=request.args.get("error"),
+            traceback=request.args.get(
+                "error_description"
+            ),  # Slightly misusing the traceback field
+        )
+
+    user: User = app.specter.user_manager.get_user()
+    error = None
+
+    try:
+        swan_client.handle_oauth2_auth_callback(request)
+    except swan_client.SwanApiException as e:
+        logger.exception(e)
+        error = e
+
+    if not error:
+        try:
+            # Add the Service to the User's profile (will now appear in sidebar)
+            user = app.specter.user_manager.get_user()
+            user.add_service(SwanService.id)
+
+            # Sync this Specter instance with any previous Swan-Specter integrations
+            SwanService.sync_swan_data()
+
+            service_data = SwanService.get_current_user_service_data()
+            if service_data.get(SwanService.SPECTER_WALLET_ALIAS):
+                # We've re-synced with an existing auto-withdrawal setup. Redirect
+                # straight to the auto-withdrawals page.
+                return redirect(
+                    url_for(f"{SwanService.get_blueprint_name()}.withdrawals")
+                )
+
+            return redirect(url_for(".oauth2_success"))
+        except Exception as e:
+            logger.exception(e)
+            error = e
+
+    return render_template(
+        "error.html",
+        response=None,
+        error=str(error),
+        error_description=None,
+        cookies=request.cookies,
+    )
+
+
+@swan_endpoint.route("/oauth2/success")
+def oauth2_success():
+    """
+    The redirect from the oauth2 callback has to land on an endpoint that does not
+    have the @login_required filter set. Once we're back we can proceed to login-
+    protected pages as usual.
+    """
+    return render_template(
+        "swan/oauth2_success.jinja",
+    )
+
+
+@swan_endpoint.route("/oauth2/delete-token", methods=["POST"])
+@login_required
+@refreshtoken_required
+def oauth2_delete_token():
+    SwanService.remove_swan_integration(current_user)
+
+    url = url_for(f"{SwanService.get_blueprint_name()}.index")
+    return redirect(url_for(f"{SwanService.get_blueprint_name()}.index"))

src/cryptoadvance/specter/services/swan/service.py

@@ -0,0 +1,397 @@
+import datetime
+import json
+import logging
+import pytz
+
+from flask import current_app as app
+from flask_babel import lazy_gettext as _
+from typing import List
+
+from cryptoadvance.specter.user import User
+
+from ..service import Service, devstatus_beta
+from cryptoadvance.specter.addresslist import Address
+from cryptoadvance.specter.wallet import Wallet
+
+logger = logging.getLogger(__name__)
+
+
+class SwanService(Service):
+    id = "swan"
+    name = "Swan"
+    icon = "img/swan_icon.svg"
+    logo = "img/swan_logo.svg"
+    desc = "Auto-withdraw to your Specter wallet"
+    has_blueprint = True
+    devstatus = devstatus_beta
+
+    # TODO: As more Services are integrated, we'll want more robust categorization and sorting logic
+    sort_priority = 1
+
+    # Service-specific constants
+    MIN_PENDING_AUTOWITHDRAWAL_ADDRS = 10
+
+    # ServiceEncryptedStorage field names for Swan
+    SPECTER_WALLET_ALIAS = "wallet"
+    SWAN_WALLET_ID = "swan_wallet_id"
+    ACCESS_TOKEN = "access_token"
+    ACCESS_TOKEN_EXPIRES = "expires"
+    REFRESH_TOKEN = "refresh_token"
+    AUTOWITHDRAWAL_ID = "autowithdrawal_id"
+    AUTOWITHDRAWAL_THRESHOLD = "withdrawal_threshold"
+
+    @classmethod
+    def is_access_token_valid(cls):
+        service_data = cls.get_current_user_service_data()
+        if not service_data or not service_data.get(cls.ACCESS_TOKEN_EXPIRES):
+            return False
+        return (
+            service_data[cls.ACCESS_TOKEN_EXPIRES]
+            > datetime.datetime.now(tz=pytz.utc).timestamp()
+        )
+
+    @classmethod
+    def has_refresh_token(cls):
+        return cls.REFRESH_TOKEN in cls.get_current_user_service_data()
+
+    @classmethod
+    def get_associated_wallet(cls) -> Wallet:
+        """Get the Specter `Wallet` that is currently associated with Swan auto-withdrawals"""
+        service_data = cls.get_current_user_service_data()
+        if not service_data or cls.SPECTER_WALLET_ALIAS not in service_data:
+            # Service is not initialized; nothing to do
+            return
+        return app.specter.wallet_manager.get_by_alias(
+            service_data[cls.SPECTER_WALLET_ALIAS]
+        )
+
+    @classmethod
+    def set_associated_wallet(cls, wallet: Wallet):
+        """Set the Specter `Wallet` that is currently associated with Swan auto-withdrawals"""
+        cls.update_current_user_service_data({cls.SPECTER_WALLET_ALIAS: wallet.alias})
+
+    @classmethod
+    def reserve_addresses(
+        cls, wallet: Wallet, label: str = None, num_addresses: int = 10
+    ) -> List[str]:
+        """
+        * Reserves addresses for Swan auto-withdrawals
+        * Sets the associated Specter `Wallet` that will receive auto-withdrawals
+        * Removes any existing unused reserved addresses in the previously associated `Wallet`
+        * Performs matching cleanup and update on the Swan side
+
+        Overrides base classmethod to add Swan-specific functionality & data management.
+        """
+        from . import client as swan_client
+
+        # Update Addresses as reserved (aka "associated") with Swan in our Wallet
+        addresses = super().reserve_addresses(
+            wallet=wallet, label=label, num_addresses=num_addresses
+        )
+
+        # Clear out any prior unused reserved addresses if this is a different Wallet
+        cur_wallet = cls.get_associated_wallet()
+        if cur_wallet and cur_wallet != wallet:
+            super().unreserve_addresses(cur_wallet)
+
+        # Store our `Wallet` as the current one for Swan auto-withdrawals
+        cls.set_associated_wallet(wallet)
+
+        # Send the new list to Swan (DELETES any unused ones; creates a new SWAN_WALLET_ID if needed)
+        swan_client.update_autowithdrawal_addresses(
+            specter_wallet_name=wallet.name,
+            specter_wallet_alias=wallet.alias,
+            addresses=addresses,
+        )
+
+        return addresses
+
+    @classmethod
+    def set_autowithdrawal_settings(cls, wallet: Wallet, btc_threshold: str):
+        """
+        btc_threshold: "0", "0.01", "0.025", or "0.05"
+
+        Performs a lot of maintenance behind the scenes in order to keep Specter's
+        internal data in sync (e.g. resetting previously reserved addresses) and the same
+        in the api to keep Swan's notion of a wallet and list of addrs in sync.
+        """
+        from . import client as swan_client
+
+        # Reserve auto-withdrawal addresses for this Wallet; clear out an unused ones in a prior wallet
+        cls.reserve_addresses(
+            wallet=wallet, num_addresses=cls.MIN_PENDING_AUTOWITHDRAWAL_ADDRS
+        )
+
+        # Send the autowithdrawal threshold
+        swan_client.set_autowithdrawal(btc_threshold=btc_threshold)
+
+        # Store the threshold setting in the User's service data
+        cls.update_current_user_service_data(
+            {SwanService.AUTOWITHDRAWAL_THRESHOLD: btc_threshold}
+        )
+
+    @classmethod
+    def sync_swan_data(cls):
+        """
+        Called when the user completes the OAuth2 link with Swan.
+
+        User could be:
+        * A first-time Specter-Swan integration (so no local or Swan API wallet data).
+        * Re-linking on a previously linked Specter instance (some/all existing data).
+        * Linking a new Specter instance but had previously linked on a different Specter
+            instance; the previously linked Specter wallet may or may not be present
+            (need to resync data).
+        """
+        from . import client as swan_client
+
+        def sync_autowithdrawal_settings(service_data):
+            """
+            Retrieve the autowithdrawal objs from Swan. We only care if we find one
+            that matches our SWAN_WALLET_ID and has `isActive: true`.
+
+            Otherwise clear any local autowithdrawal data.
+            """
+            autowithdrawal_info = swan_client.get_autowithdrawal_info()
+            """
+                {
+                    "entity": "automaticWithdrawal",
+                    "list": [
+                        {
+                            "id": "2026d75b-baf0-45c3-a1c6-9e17d4f7e90f",
+                            "minBtcThreshold": "0.01",
+                            "isActive": false,
+                            "isCanceled": false,
+                            "createdAt": "2022-01-07T02:14:56.070Z",
+                            "walletId": "e26132e7-2e03-49b7-807b-9067aa3a8507",
+                            "walletAddressId": null
+                        },
+                        ...,
+                        { ... }
+                    ]
+                }
+            """
+            if autowithdrawal_info and "list" in autowithdrawal_info:
+                swan_wallet_id = service_data.get(SwanService.SWAN_WALLET_ID)
+                for entry in autowithdrawal_info["list"]:
+                    if entry.get("wallet_id") == swan_wallet_id and entry.get(
+                        "isActive"
+                    ):
+                        # Found our swan_wallet_id's active autowithdrawal!
+                        logger.debug(
+                            "Found swan_wallet_id's active autowithdrawal entry"
+                        )
+                        SwanService.update_current_user_service_data(
+                            {
+                                SwanService.AUTOWITHDRAWAL_ID: entry["id"],
+                                SwanService.AUTOWITHDRAWAL_THRESHOLD: entry[
+                                    "minBtcThreshold"
+                                ],
+                            }
+                        )
+                        return
+
+            # Did not find a matching and/or active autowithdrawal
+            logger.debug("No active autowithdrawal with swan_wallet_id found")
+
+            # Clear the autowithdrawal fields from local storage and move on
+            service_data.pop(SwanService.AUTOWITHDRAWAL_ID, None)
+            service_data.pop(SwanService.AUTOWITHDRAWAL_THRESHOLD, None)
+            SwanService.set_current_user_service_data(service_data)
+            return
+
+        service_data = cls.get_current_user_service_data()
+        if SwanService.SWAN_WALLET_ID in service_data:
+            # This user has previously/currently linked to Swan on this instance
+            swan_wallet_id = service_data[SwanService.SWAN_WALLET_ID]
+            logger.debug(f"swan_wallet_id: {swan_wallet_id}")
+
+            # Confirm that the Swan walletId exists on the Swan side
+            details = swan_client.get_wallet_details(swan_wallet_id)
+            """
+            {
+                "entity": "wallet",
+                "item": {
+                    "id": "*******",
+                    "isConfirmed": false,
+                    "displayName": "Specter autowithdrawal to SeedSigner demo",
+                    "metadata": {
+                        "oidc": {
+                            "clientId": "specter-dev"
+                        },
+                        "specter_wallet_alias": "seedsigner_demo"
+                    },
+                    "btcAddresses": []
+                }
+            }
+            """
+            if not details:
+                # Specter's SWAN_WALLET_ID is out of sync; doesn't exist on Swan's side.
+                # Clear the local SWAN_WALLET_ID and continue below to try to find one.
+                logger.debug(f"swan_wallet_id {swan_wallet_id} not found on Swan")
+                del service_data[SwanService.SWAN_WALLET_ID]
+                cls.set_current_user_service_data(service_data)
+
+            elif (
+                "item" in details
+                and "metadata" in details["item"]
+                and "specter_wallet_alias" in details["item"]["metadata"]
+            ):
+                wallet_alias = details["item"]["metadata"]["specter_wallet_alias"]
+                logger.debug(f"swan_wallet_id exists on Swan side")
+                if wallet_alias in [
+                    w.alias for w in app.specter.wallet_manager.wallets.values()
+                ]:
+                    # All is good; we've matched Swan's wallet data with a Specter `Wallet` that we recognize.
+                    logger.debug(f"Found wallet_alias {wallet_alias} in Specter")
+                    if wallet_alias != service_data.get(
+                        SwanService.SPECTER_WALLET_ALIAS
+                    ):
+                        # Our local `service_data` is out of sync; update with the
+                        # current Specter wallet_alias Swan is expecting.
+                        logger.debug(
+                            f"Updating service_data to use wallet_alias {wallet_alias}"
+                        )
+                        cls.update_current_user_service_data(
+                            {SwanService.SPECTER_WALLET_ALIAS: wallet_alias}
+                        )
+
+                    sync_autowithdrawal_settings(service_data)
+
+                    return
+                else:
+                    # Swan is out of sync with Specter; the SPECTER_WALLET_ALIAS we had
+                    # been using doesn't exist on this Specter instance.
+                    logger.warn(
+                        f"Swan referenced an unknown wallet_alias {wallet_alias}"
+                    )
+                    if SwanService.SPECTER_WALLET_ALIAS in service_data:
+                        # Clear the local reference to that unknown SPECTER_WALLET_ALIAS
+                        del service_data[SwanService.SPECTER_WALLET_ALIAS]
+                        cls.set_current_user_service_data(service_data)
+
+        # This Specter instance has no idea if there might already be wallet data on the Swan side.
+        # Fetch all Swan wallets, if any exist.
+        wallet_entries = swan_client.get_wallets().get("list")
+        if not wallet_entries:
+            # No Swan data at all yet. Nothing to do.
+            logger.debug("No wallets on the Swan side yet")
+            return
+
+        swan_wallet_id = None
+        for wallet_entry in wallet_entries:
+            swan_wallet_id = wallet_entry["id"]
+            specter_wallet_alias = wallet_entry["metadata"].get("specter_wallet_alias")
+            if specter_wallet_alias in [
+                w.alias for w in app.specter.wallet_manager.wallets.values()
+            ]:
+                # All is good; we've matched Swan's wallet data with a Specter `Wallet` that we recognize.
+                # Use this Swan walletId going forward.
+                cls.update_current_user_service_data(
+                    {
+                        SwanService.SWAN_WALLET_ID: swan_wallet_id,
+                        SwanService.SPECTER_WALLET_ALIAS: specter_wallet_alias,
+                    }
+                )
+                logger.debug(
+                    f"Found a Specter wallet that we recognize: {specter_wallet_alias}"
+                )
+
+                sync_autowithdrawal_settings(service_data)
+
+                return
+
+        # We didn't find a matching Specter `Wallet`. Clear out any nonsense settings in our local data.
+        logger.debug(
+            "Did not find any matching Specter wallets in the Swan wallet metadata"
+        )
+        if SwanService.SPECTER_WALLET_ALIAS in service_data:
+            del service_data[SwanService.SPECTER_WALLET_ALIAS]
+            SwanService.set_current_user_service_data(service_data)
+
+        # Did we at least get a Swan walletId that we can update later?
+        if swan_wallet_id:
+            cls.update_current_user_service_data(
+                {
+                    SwanService.SWAN_WALLET_ID: swan_wallet_id,
+                }
+            )
+            logger.debug(f"Setting swan_wallet_id to {swan_wallet_id}")
+
+    @classmethod
+    def remove_swan_integration(cls, user: User):
+        # Unreserve unused addresses in all wallets
+        for wallet_name, wallet in user.wallet_manager.wallets.items():
+            SwanService.unreserve_addresses(wallet=wallet)
+
+        # If an autowithdrawal setup is active, remove pending addrs from Swan
+        try:
+            service_data = SwanService.get_current_user_service_data()
+            if service_data.get(cls.SPECTER_WALLET_ALIAS) and service_data.get(
+                cls.SWAN_WALLET_ID
+            ):
+                # Import here to prevent circular dependency
+                from . import client as swan_client
+
+                swan_client.delete_autowithdrawal_addresses(
+                    service_data[cls.SWAN_WALLET_ID]
+                )
+        except Exception as e:
+            # Note the exception but proceed with clearing local data
+            logger.exception(e)
+
+        # Wipe the on-disk encrypted service data (refresh_token, etc)
+        SwanService.set_current_user_service_data({})
+
+        # Remove Swan from User's list of active Services
+        user.remove_service(SwanService.id)
+
+    """ ***********************************************************************
+                                Update hooks overrides
+    *********************************************************************** """
+
+    @classmethod
+    def update(cls):
+        """
+        Periodic or at-login call to check our Swan address status and send more when
+        needed.
+        * Check for autowithdrawals paid to addrs reserved for Swan.
+        * Add more pending autowithdrawal addrs if we're under the threshold.
+        """
+        # Which Specter `Wallet` has been configured to receive Swan autowithdrawals?
+        wallet = cls.get_associated_wallet()
+        if not wallet:
+            # Swan autowithdrawals to Specter aren't set up yet; nothing to do.
+            logger.debug(
+                "No associated wallet. Swan autowithdrawals are not set up yet."
+            )
+            return
+
+        # Scan the Wallet for any new Swan autowithdrawals
+        reserved_addresses: List[Address] = wallet.get_associated_addresses(
+            service_id=cls.id, unused_only=False
+        )
+        for addr_obj in reserved_addresses:
+            if addr_obj["used"] and addr_obj["label"] == cls.default_address_label:
+                # This addr has received an autowithdrawal since we last checked
+                logger.debug(
+                    f"Updating address label for {json.dumps(addr_obj, indent=4)}"
+                )
+                addr_obj.set_label(str(_("Swan autowithdrawal")))
+
+        num_pending_autowithdrawal_addrs = len(
+            [addr_obj for addr_obj in reserved_addresses if not addr_obj["used"]]
+        )
+        if num_pending_autowithdrawal_addrs < cls.MIN_PENDING_AUTOWITHDRAWAL_ADDRS:
+            from . import (
+                client as swan_client,
+            )  # Import here to avoid circular dependency
+
+            logger.debug("Need to send more addrs to Swan")
+
+            cls.reserve_addresses(
+                wallet=wallet, num_addresses=cls.MIN_PENDING_AUTOWITHDRAWAL_ADDRS
+            )
+
+    @classmethod
+    def on_user_login(cls):
+        cls.update()

src/cryptoadvance/specter/services/swan/static/img/swan_icon.svg

@@ -0,0 +1 @@
+<svg height="204" viewBox="0 0 204 204" width="204" xmlns="http://www.w3.org/2000/svg"><path d="m102 0c56.333044 0 102 45.6669555 102 102 0 56.333044-45.666956 102-102 102-56.3330445 0-102-45.666956-102-102 0-56.3330445 45.6669555-102 102-102zm-65.0121954 83.9261449c-.2324671.5584376-.532609 1.0989318-.6872941 1.6783765-2.2205225 8.3319247-2.7918027 16.7745736-1.8940138 25.3546446 3.3503391 32.015196 30.2343495 56.727374 59.0185259 59.505558 6.2445329.603077 12.4429244.417953 18.3016224-2.239003 11.21643-5.086771 13.284464-19.061717 3.890419-26.987505-2.491222-2.102019-5.229851-4.110819-8.18381-5.437327-9.2560583-4.157647-18.6544978-8.00369-28.0256915-11.904001-9.259135-3.853482-18.4321385-7.860578-26.4410485-14.03184-7.7975363-6.009331-13.2879793-13.485656-14.8040692-23.4421266-.1243633-.8148988-.3023391-1.6214824-.4557059-2.4320047zm98.0560854-38.7494894c-15.051039-9.212033-31.338679-12.0536753-48.6743985-8.996273-10.2311903 1.8044189-19.4692318 5.7554965-26.9385017 13.1793036-12.3893115 12.3132-13.0177197 31.7587341.5906159 43.4307812 4.5974878 3.9432 9.8897405 7.3708517 15.3832595 9.9284607 10.6451488 4.956354 21.675692 9.087742 32.5133178 13.640584 7.841042 3.294169 15.392049 7.09776 21.771053 12.879077 9.799654 8.881172 11.592594 22.456984 5.969 32.246711-.168309.293223-.233786.645529-.411762 1.155389.535686.096281.764198.207444.94437.154489.343208-.101534.673232-.272216.983921-.455153 7.015322-4.137078 13.123626-9.290372 17.7009-16.066461 9.519726-14.094424 7.513654-30.032217-5.557239-40.901181-4.559696-3.79265-9.663426-7.2176757-15.035658-9.7013228-10.857401-5.0202494-22.07339-9.2706776-33.132498-13.8602823-6.1421418-2.5492942-12.3233944-5.0276895-17.5985086-9.2299765-8.1363495-6.4819906-4.0806989-15.2760706 1.9392768-18.2945224 3.6386159-1.8245506 7.5039862-2.3208423 11.5148131-2.0578165 7.2139517.4730965 13.9005677 2.6980942 20.1824537 6.1546307 12.5251 6.8920658 22.452193 16.6857317 31.136972 27.8501082.497453.6394023 1.086312 1.2074682 1.633422 1.8087953.184567-.0949694.369135-.1903765.554142-.2853459-.129197-.6507812-.197751-1.3203812-.397699-1.9488424-1.50027-4.7182729-3.069094-9.4155388-4.503007-14.1535059-.240816-.7952047-.213131-1.9024517.165232-2.6153788 2.523301-4.7515341 2.299623-9.3757129-.406048-13.9311812-2.522423-4.246927-6.178617-7.3918588-10.327429-9.931087z" fill="#fbfbfb" fill-rule="evenodd"/></svg>

src/cryptoadvance/specter/services/swan/templates/swan/components/swan_menu.jinja

@@ -0,0 +1,16 @@
+{% from 'components/menu_item.jinja' import menu_item %}
+
+{# 
+    vaultoro_menu - Tabs menu to navigate between the vaultoro screens.
+    Parameters:
+    - active_menuitem: Current active tab. Options: 'general', 'bitcoin_core', 'auth'.
+ #}
+{% macro swan_menu(active_menuitem) -%}
+	<nav class="row collapse-on-mobile">
+		{{ menu_item(service.id, 'withdrawals', 'Auto-Withdrawals', active_menuitem, isLeft=true) }}
+		{{ menu_item(service.id, 'settings', 'Settings', active_menuitem, isRight=true) }}
+		<a href="javascript:void(0);" class="mobile-nav-icon" onclick="toggleMobileNav(this, `{{ url_for('static', filename='img/expand-more.svg') }}`, `{{ url_for('static', filename='img/expand-less.svg') }}`)">
+			<img style="width: 36px;" src="{{ url_for('static', filename='img/expand-more.svg') }}"/>
+		</a>
+	</nav>
+{%- endmacro %}

src/cryptoadvance/specter/services/swan/templates/swan/components/swan_tab.jinja

@@ -0,0 +1,8 @@
+{% extends "base.jinja" %}
+{% block main %}
+    <img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.logo) }}" width="300"/>    
+    {% from 'swan/components/swan_menu.jinja' import swan_menu with context %}
+    {{ swan_menu(tab) }}
+    {% block content %}
+    {% endblock %}
+{% endblock %}

src/cryptoadvance/specter/services/swan/templates/swan/index.jinja

@@ -0,0 +1,112 @@
+{% extends "base.jinja" %}
+
+{% block main %}
+    <style>
+        .tagline {
+            font-style: italic;
+            font-size: 1.1em;
+            text-align: center;
+            color: #aaa;
+            padding-left: 4em;
+            padding-right: 4em;
+        }
+
+        .button_wide {
+            text-decoration: none;
+            color: white;
+        }
+        .button_wide:hover {
+            color: white;
+        }
+        .big_option {
+            display: inline-block;
+            width: 12em;
+            padding: 1em;
+            vertical-align: middle;
+            height: 3.5em;
+            border: 4px solid var(--cmap-border);
+            border-radius: 0.5em;
+            background-color: var(--cmap-bg-lighter);
+            font-size: 1.5em;
+            margin: 1em;
+            position: relative;
+        }
+        .big_option:hover {
+            background-color: var(--cmap-bg-lightest);
+            border: 4px solid var(--main-color);
+            cursor: pointer;
+        }
+
+        .big_option_text {
+            margin: 0;
+            position: absolute;
+            top: 50%;
+            left: 50%;
+            -ms-transform: translate(-50%, -50%);
+            transform: translate(-50%, -50%);
+        }
+
+        .bonus {
+            color: #999;
+            font-size: 0.7em;
+            font-style: italic;
+        }
+
+        .highlights {
+            margin-top: 2em;
+            margin-bottom: 1.5em;
+        }
+
+        .show_list_bullet {
+            margin-top: 0.5em;
+            display: inline-block;
+            width: auto;
+        }
+        .show_list_bullet li {
+            text-align: left;
+            list-style-type: disc;
+            line-height: 1.7em;
+        }
+
+        .plus_sign {
+            font-size: 3em;
+            color: #ccc;
+            vertical-align: text-bottom;
+        }
+    </style>
+    <img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.logo) }}" width="300"/>
+    <div class="tagline">&ldquo;Swan is the best way to accumulate Bitcoin with automatic recurring buys and instant buys.&rdquo;</div>
+    <div class="card center" style="width: auto; min-width: 90%; margin: 40px;">
+        <a class="button_wide" href="https://www.swanbitcoin.com/Specter/" target="_swan">
+            <span class="big_option">
+                <div class="big_option_text">
+                    Join Swan!
+                    <br/>
+                    <div class="bonus">$10&nbsp;free&nbsp;Bitcoin&nbsp;bonus</div>
+                </div>
+            </span>
+        </a>
+        <span class="big_option" onclick="location.href='{{ url_for(service.id +'_endpoint.oauth2_start') }}';">
+            <div class="big_option_text">
+                Existing&nbsp;Swan&nbsp;users
+            </div>
+        </span>
+
+        <div class="highlights">
+            <img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.icon) }}" width="50"/>
+            <span class="plus_sign">+</span>
+            <img src="{{ url_for('static', filename='img/ghost_3d.png') }}" width="50"/><br/>
+            <div class="show_list_bullet">
+                <li>Auto-withdraw from Swan straight into your Specter wallet
+                <li>Swan UTXOs marked in your Specter History
+            </div>
+        </div>
+    </div>
+{% endblock %}
+
+
+
+{% block scripts %}
+    <script>
+    </script>
+{% endblock %}

src/cryptoadvance/specter/services/swan/templates/swan/oauth2_start.jinja

@@ -0,0 +1,62 @@
+{% extends "base.jinja" %}
+
+{% block main %}
+    <style>
+        .big_option {
+            display: inline-block;
+            width: 12em;
+            padding: 1em;
+            vertical-align: middle;
+            height: 3.5em;
+            border: 4px solid var(--cmap-border);
+            border-radius: 0.5em;
+            background-color: var(--cmap-bg-lighter);
+            font-size: 1.5em;
+            margin: 1em;
+            position: relative;
+        }
+        .big_option:hover {
+            background-color: var(--cmap-bg-lightest);
+            border: 4px solid var(--main-color);
+            cursor: pointer;
+        }
+
+        .big_option_text {
+            margin: 0;
+            position: absolute;
+            top: 50%;
+            left: 50%;
+            -ms-transform: translate(-50%, -50%);
+            transform: translate(-50%, -50%);
+        }
+
+        .instructions {
+            color: #999;
+            font-style: italic;
+        }
+    </style>
+
+    <img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.logo) }}" width="300"/>
+    <div class="card center" style="width: auto; min-width: 90%; margin: 40px;">
+        <h1>{{ _("Enable Swan Integration") }}</h1>
+        <br/>
+
+        <div class="big_option" onclick="location.href='{{ flow_url }}';">
+            <div class="big_option_text">
+                Link&nbsp;your&nbsp;account
+            </div>
+        </div>
+
+        <div class="instructions">
+            {{ _("Click to sign into your Swan account and authorize Specter integration") }}
+        </div>
+        <br/>
+    </div>
+{% endblock %}
+
+
+
+{% block scripts %}
+    <script>
+    </script>
+{% endblock %}

src/cryptoadvance/specter/services/swan/templates/swan/oauth2_success.jinja

@@ -0,0 +1,52 @@
+{% extends "base.jinja" %}
+
+{% block main %}
+    <style>
+        .big_option {
+            display: inline-block;
+            width: 12em;
+            padding: 1em;
+            vertical-align: middle;
+            height: 3.5em;
+            border: 4px solid var(--cmap-border);
+            border-radius: 0.5em;
+            background-color: var(--cmap-bg-lighter);
+            font-size: 1.5em;
+            margin: 1em;
+            position: relative;
+        }
+        .big_option:hover {
+            background-color: var(--cmap-bg-lightest);
+            border: 4px solid var(--main-color);
+            cursor: pointer;
+        }
+
+        .big_option_text {
+            margin: 0;
+            position: absolute;
+            top: 50%;
+            left: 50%;
+            -ms-transform: translate(-50%, -50%);
+            transform: translate(-50%, -50%);
+        }
+
+        .instructions {
+            color: #999;
+            font-style: italic;
+        }
+    </style>
+    <img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.logo) }}" width="300"/>
+    <div class="card center" style="width: auto; min-width: 90%; margin: 40px;">
+        <h1>Swan Integration Success!</h1>
+        <br/>
+
+        <div class="big_option" onclick="location.href='{{ url_for('swan_endpoint.settings') }}';"><div class="big_option_text">Continue</div></div>
+    </div>
+{% endblock %}
+
+
+
+{% block scripts %}
+    <script>
+    </script>
+{% endblock %}

src/cryptoadvance/specter/services/swan/templates/swan/settings.jinja

@@ -0,0 +1,133 @@
+{% extends "swan/components/swan_tab.jinja" %}
+{% block title %}Settings{% endblock %}
+{% set tab = 'settings' %}
+{% block content %}
+    <br/>
+    <style>
+        {# TODO: Remove these classes after Swan exits Beta #}
+        .big_option {
+            display: inline-block;
+            width: 14em;
+            padding: 1em;
+            vertical-align: middle;
+            height: 3.5em;
+            border: 4px solid var(--cmap-border);
+            border-radius: 0.5em;
+            background-color: var(--cmap-bg-lighter);
+            font-size: 1.5em;
+            margin: 1em;
+            position: relative;
+            
+        }
+        .big_option:hover {
+            background-color: var(--cmap-bg-lightest);
+            border: 4px solid var(--main-color);
+            cursor: pointer;
+        }
+
+        .big_option_text {
+            margin: 0;
+            position: absolute;
+            top: 50%;
+            left: 50%;
+            -ms-transform: translate(-50%, -50%);
+            transform: translate(-50%, -50%);
+        }
+
+        .instructions {
+            color: #999;
+            font-style: italic;
+        }
+        {# TODO: End Remove #}
+
+        .css-1rbd7t8 {
+            box-sizing: border-box;
+            margin: 60px 0px 24px;
+            min-width: 0px;
+        }
+        .css-yn7azs {
+            box-sizing: border-box;
+            margin: 0px;
+            min-width: 0px;
+            -moz-box-pack: justify;
+            justify-content: space-between;
+            position: relative;
+            font-family: "Source Serif Pro", serif;
+            font-weight: 600;
+            line-height: 1.5;
+            letter-spacing: -0.02em;
+            font-variant: common-ligatures;
+            text-rendering: optimizelegibility;
+            font-size: 16px;
+            display: flex;
+        }
+    </style>
+
+    <div class="card">
+        <h1>{{ _("Configure Auto-Withdrawal") }}</h1>
+        <br>
+
+        <div class="note">
+            {{ _("- Specter will send {} unused addresses to Swan").format(num_reserved_addrs) }}<br/>
+            {{ _("- Your wallet will show them pre-labeled as \"reserved\" for Swan") }}<br/>
+            {{ _("- Specter will check these addresses each time you log in to Specter") }}<br/>
+            {{ _("- Specter will send more unused addresses to Swan as needed") }}<br/>
+        </div>
+        <br/>
+
+        <form action="{{ url_for(service.get_blueprint_name() + '.update_autowithdrawal') }}" method="POST" role="form">
+            <input type="hidden" class="csrf-token" name="csrf_token" value="{{ csrf_token() }}"/>
+            
+            {{ _("Withdraw to wallet") }}:<br>
+            <select name="destination_wallet">
+                {% for wallet in wallets %}
+                    <option value="{{ wallet.alias }}" {% if associated_wallet == wallet %}selected{% endif %}>{{ wallet.name }}</option>
+                {% endfor %}
+            </select>
+            <br/>
+            <br/>
+            <br/>
+
+            <div>Auto-withdrawal threshold:</div>
+            <select name="threshold">
+                <option value="0"     {% if threshold == '0' %}selected{% endif %}>Weekly</option>
+                <option value="0.01"  {% if threshold == '0.01' %}selected{% endif %}>0.010 BTC</option>
+                <option value="0.025" {% if threshold == '0.025' %}selected{% endif %}>0.025 BTC</option>
+                <option value="0.05"  {% if threshold == '0.05' %}selected{% endif %}>0.050 BTC</option>
+            </select>
+            <br/>
+            <br/>
+            <br/>
+            
+            <div class="row">
+                <button type="submit" class="btn">{{ _("Update Auto-withdrawal") }}</button>
+            </div>
+
+        </form>
+    </div>
+    <br/>
+    <br/>
+    <br/>
+
+
+    <div class="card">
+        <h1>{{ _("Remove Swan Integration") }}</h1>
+        <br/>
+        <div class="note">
+            {{ _("- Halts Swan auto-withdrawals to your Specter wallet") }}<br/>
+            {{ _("- Clears Swan data from Specter") }}<br/>
+        </div>
+        <br/>
+
+        <form action="{{ url_for(service.get_blueprint_name() + '.oauth2_delete_token') }}" method="POST" role="form">
+            <input type="hidden" class="csrf-token" name="csrf_token" value="{{ csrf_token() }}"/>
+            <div class="row">
+                <button type="submit" class="btn danger">{{ _("Remove Swan Integration") }}</button>
+            </div>
+        </form>
+    </div>
+    <br/>
+    <br/>
+    <br/>
+
+{% endblock %}

src/cryptoadvance/specter/services/swan/templates/swan/withdrawals.jinja

@@ -0,0 +1,78 @@
+{% extends "swan/components/swan_tab.jinja" %}
+{% block title %}Balances{% endblock %}
+{% set tab = 'withdrawals' %}
+{% block content %}
+
+    <style>
+        h1 {
+            margin-top: 1em;
+        }
+        .linked_wallet {
+            margin-bottom: 3em;
+        }
+        .no_linked_wallet {
+            background-color: var(--cmap-bg-lighter);
+            border: 2px solid yellow;
+            border-radius: 0.5em;
+            padding: 2em 3em 2em 3em;
+            margin-bottom: 3em;
+        }
+        .no_linked_wallet .headline {
+            text-align: center;
+            font-size: 1.1em;
+            margin-bottom: 1em;
+        }
+        .footnote {
+            margin-top: 2em;
+            font-style: italic;
+            font-size: 0.85em;
+            color: #999;
+        }
+    </style>
+
+    <h1>Swan Auto-Withdrawals to Specter</h1>
+
+    {% if wallet %}
+        <div class="linked_wallet">Linked wallet:
+            <a class="explorer-link" href="{{ url_for('wallets_endpoint.addresses', wallet_alias=wallet.alias) }}">{{ wallet.name }}</a><br/>
+        </div>
+        {% if autowithdrawal_threshold %}
+            Auto-withdrawal threshold: {% if autowithdrawal_threshold == "0" %}{{ _("Weekly") }}{% else %}{{ autowithdrawal_threshold }} btc{% endif %}
+        {% endif %}
+
+    {% else %}
+        <div class="no_linked_wallet">
+            <div class="headline">{{ _("Auto-Withdrawal Not Configured") }}</div>
+            <div class="note">
+                {{ _("Go to Settings to set up auto-withdrawal to one of your Specter wallets") }}
+            </div>
+        </div>
+    {% endif %}
+
+    {# TODO: List total withdrawal value? Or just current value of withdrawn utxos? #}
+
+    <div class="table-holder">
+        {% include "includes/services-data.html" %}
+        {% include "includes/address-label.html" %}
+        {% include "includes/tx-row.html" %}
+        {% include "includes/tx-data.html" %}
+        {% include "includes/explorer-link.html" %}
+        {% include "includes/tx-table.html" %}
+        <tx-table
+            {% if specter.price_check and (specter.alt_rate and specter.alt_symbol) %}
+                price="{{ specter.alt_rate }}"
+                symbol="{{ specter.alt_symbol }}"
+            {% endif %}
+            btc-unit="{{ specter.unit }}"
+            hide-sensitive-info="{{ specter.hide_sensitive_info | lower }}"
+            service-id="{{ swan_id }}"
+            type="txlist"
+        />
+    </div>
+    <div id="tx-popup" class="hidden"></div>
+
+    <div class="footnote">
+        {{ _("<sup>*</sup>Only shows the withdrawals that this Specter instance is aware of.") }}
+    </div>
+
+{% endblock %}

src/cryptoadvance/specter/services/templates/components/menu_item.jinja

@@ -0,0 +1,17 @@
+{# 
+    menu_item - navigation bar menu item.
+    Parameters:
+    - tab: The tab name.
+    - title: The tab title.
+    - active_menuitem: Current active tab name.
+    - optional: is the tab supported for mobile.
+    - isLeft: Is the item at the left edge (first item) of the navigation bar?
+    - isRight: Is the item at the right edge (last item) of the navigation bar?
+ #}
+ {% macro menu_item(service_id, tab, title, active_menuitem, isLeft=false, isRight=false) -%}
+ <a 
+     href="{{ url_for(service_id+'_endpoint.' ~ tab)}}"
+     class="btn radio {% if isLeft %}left{% endif %} {% if isRight %}right{% endif %} {% if active_menuitem == tab %}checked{% endif %}">
+     {{ title }}
+ </a>
+{%- endmacro %}

src/cryptoadvance/specter/services/templates/includes/services-data.html

@@ -0,0 +1,15 @@
+{# Assumes that the `services` object has been injected into the Flask context. This
+script makes the Services data globally available to all JS. #}
+{# TODO: Better way to manage this JS global? #}
+<script>
+    const services = {
+        {% for service_id, service in services.items() %}
+            {{ service_id }}: {
+                name: "{{ service.name }}",
+                icon: "{{ service.icon }}",
+                logo: "{{ service.logo }}",
+                desc: "{{ service.desc }}",
+            },
+        {% endfor %}
+    }
+</script>

src/cryptoadvance/specter/services/templates/services/associate_addr.jinja

@@ -0,0 +1,83 @@
+{% extends "base.jinja" %}
+
+{% block main %}
+    <style>
+        .info {
+        }
+        .bullets {
+            margin-top: 1em;
+            text-align: left;
+        }
+        .bullets li {
+            list-style-type: disc;
+            list-style-position: inside;
+        }
+        .form_container {
+            margin-top: 2em;
+            border-top: 1px solid var(--cmap-border);
+            padding-top: 1em;
+            text-align: left;
+        }
+        .form_container .row {
+            display: block;
+            margin-top: 1em;
+            margin-bottom: 1em;
+        }
+        .form_container .label {
+            font-size: 0.85em;
+            color: #999;
+            display: block;
+        }
+        .form_container select {
+            display: block;
+        }
+        .form_container button {
+            margin: auto;
+            margin-top: 3em;
+        }
+    </style>
+    <h1 style="font-size: 2em;">{{ _("Associate Address") }}</h1>
+    <div class="card center" style="width: auto; margin: 10px;">
+        <div class="info">Associating an address with a service will:
+            <div class="bullets">
+                <ul>
+                    <li>Display the service's icon with the address label</li>
+                    <li>Include the address' UTXOs in the service's transaction totals</li>
+                </ul>
+            </div>
+        </div>
+
+        <div class="form_container">
+            <form method="post">
+        		<input type="hidden" class="csrf-token" name="csrf_token" value="{{ csrf_token() }}"/>
+                <div class="row">
+                    <div class="label">Address:</div>
+                    {{addr_obj.address}}
+                </div>
+
+                <div class="row">
+                    <div class="label">Wallet:</div>
+                    {{ wallet.name }}
+                </div>
+
+                <div class="row">
+                    <div class="label">Service:</div>
+                    <select name="service_id">
+                        {% for service in services %}
+                            <option value="{{ service.id }}">{{ service.name }}</option>
+                        {% endfor %}
+                    </select>
+                </div>
+
+                <button class="btn">Submit</button>
+            </form>
+        </div>
+    </div>
+{% endblock %}
+
+
+
+{% block scripts %}
+    <script>
+    </script>
+{% endblock %}

src/cryptoadvance/specter/services/templates/services/choose.jinja

@@ -0,0 +1,98 @@
+{% extends "base.jinja" %}
+
+{% block main %}
+    <style>
+        .service_container {
+            border: 4px solid var(--cmap-border);
+            border-radius: 0.5em;
+            display: inline-block;
+            vertical-align: middle;
+            background-color: var(--cmap-bg-lighter);
+            width: 16em;
+            height: 13em;
+            cursor: pointer;
+            padding: 1em;
+            margin: 0.75em;
+        }
+        .service_container:hover {
+            background-color: var(--cmap-bg-lightest);
+            border: 4px solid var(--main-color);
+        }
+
+        .service_name {
+            font-size: 1.5em;
+        }
+        .service_desc {
+            color: #ccc;
+        }
+
+
+        .endnote {
+            margin-top: 4em;
+        }
+
+        .login_disabled {
+            max-width: 35em;
+            border: 4px solid yellow;
+            border-radius: 0.5em;
+            background-color: var(--cmap-bg-lighter);
+            padding: 1em;
+            text-align: center;
+        }
+        .login_disabled .btn {
+            margin: auto;
+            max-width: 20em;
+        }
+    </style>
+    <h1 style="font-size: 2em;">{{ _("Services") }}</h1>
+    
+    {% if is_login_disabled %}
+        <div class="login_disabled">
+            <p>{{ _("Current Authentication method: \"None\"")}}</p>
+            <p>{{ _("Service integrations require secure per-user storage. This is only possible with a user authentication method that is password protected (any method other than \"None\").") }}</p>
+            <br/>
+            <p>{{ _("Select a new method in:") }}<br/>
+            <a class="btn" href="{{ url_for('settings_endpoint.auth', next=url_for('services_endpoint.choose')) }}">{{ _("Settings") }}</a></p>
+        </div>
+    {% endif %}
+
+    <div class="card center" style="width: auto; min-width: 90%; margin: 40px;">
+        <br>
+        {% for service in services %}
+            <div class="service_container" {% if not is_login_disabled %}onclick="location.href=`{{ url_for(service.id + '_endpoint' + '.index') }}`"{% endif %}>
+                {% if service.icon %}
+                    <img src="{{ url_for(service.id + '_endpoint' + '.static', filename=service.icon) }}" height="80px"><br/>
+                {% endif %}
+                <div class="service_name">{{ service.name }}</div>
+                <div class="service_desc">{{ service.desc }}</div>
+            </div>
+        {% endfor %}
+        <div class="service_container">
+            <img src="{{ url_for('static', filename='img/ghost_3d.png') }}" height="80px"><br/>
+            <div class="service_name">???</div>
+            <div class="service_desc">More integrations coming soon!</div>
+        </div>
+        <div class="service_container">
+            <img src="{{ url_for('static', filename='img/ghost.png') }}" height="80px"><br/>
+            <div class="service_name">???</div>
+            <div class="service_desc">More integrations coming soon!</div>
+        </div>
+        <div class="service_container">
+            <img src="{{ url_for('static', filename='img/icon.png') }}" height="80px"><br/>
+            <div class="service_name">???</div>
+            <div class="service_desc">More integrations coming soon!</div>
+        </div>
+
+
+        <div class="note endnote">
+            {{ _("Some services might not respect the Tor-only policy.") }}
+        </div>
+    </div>
+{% endblock %}
+
+
+
+{% block scripts %}
+    <script>
+    </script>
+{% endblock %}

src/cryptoadvance/specter/services/templates/services/sidebar_services_list.jinja

@@ -0,0 +1,13 @@
+<div class="separator">
+    <span id="toggle_services_list" style="cursor: pointer;">Services &nbsp; &#9660;</span>
+</div>
+<div id="services_list">
+    {% for _,service in specter.service_manager.services.items() %}
+        {% if service.id in current_user.services %}
+        <a href="{{ url_for(service.id +'_endpoint.index') }}" class="item service">
+            <img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.icon) }}" height="30px">&nbsp;{{ service.name }}
+        </a>
+        <br>
+        {% endif %}
+    {% endfor %}
+</div>

src/cryptoadvance/specter/specter.py

@@ -37,6 +37,8 @@ from .rpc import (
     RpcError,
     get_default_datadir,
 )
+from .services.service_manager import ServiceManager
+from .services.service import devstatus_alpha, devstatus_beta, devstatus_prod
 from .specter_error import ExtProcTimeoutException, SpecterError
 from .tor_daemon import TorDaemonController
 from .user import User
@@ -56,7 +58,13 @@ class Specter:
     lock = threading.Lock()
     _default_asset = None
 
-    def __init__(self, data_folder="./data", config={}, internal_bitcoind_version=""):
+    def __init__(
+        self,
+        data_folder="./data",
+        config={},
+        internal_bitcoind_version="",
+        service_devstatus_threshold=devstatus_prod,
+    ):
         if data_folder.startswith("~"):
             data_folder = os.path.expanduser(data_folder)
         data_folder = os.path.abspath(data_folder)
@@ -71,6 +79,10 @@ class Specter:
             self
         )  # has to come before calling VersionChecker()
 
+        self.service_manager = ServiceManager(
+            specter=self, devstatus_threshold=service_devstatus_threshold
+        )
+
         # version checker
         # checks for new versions once per hour
         self.version = VersionChecker(specter=self)
@@ -426,7 +438,11 @@ class Specter:
     def update_alt_symbol(self, alt_symbol, user):
         self.config_manager.update_alt_symbol(alt_symbol, user)
 
-    # mark logic!
+    def update_services(self, services):
+        """takes a list of service_names which should be activated"""
+        self.config["services"] = services
+        self._save()
+
     def update_merkleproof_settings(self, validate_bool):
         if validate_bool is True and self.info.get("pruned") is True:
             validate_bool = False

src/cryptoadvance/specter/static/styles.css

@@ -11,7 +11,7 @@ html, body{
     --cmap-red:           #A12737;
     --cmap-red-darker:    #951E2D;
     --cmap-bg:            #192432;
-    --cmap-bg-ligher:     #263044;
+    --cmap-bg-lighter:    #263044;
     --cmap-bg-lightest:   #313E50;
     --cmap-border:        #506072;
     --cmap-border-darker: #405062;
@@ -95,7 +95,7 @@ ul, li{
     align-items: center;
 }
 .settings-bar-btn:hover {
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
 }
 .logout{
     position: relative;
@@ -110,7 +110,7 @@ ul, li{
     border-radius: 0 0 0 5px;
 }
 .logout:hover{
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
 }
 .logout img{
     margin-right: 10px;
@@ -197,7 +197,7 @@ nav.side{
     width: 250px;
 }
 #side-content{
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
     border-right: 1px solid var(--cmap-bg-lightest);
     padding-top: 10px;
     display: flex;
@@ -684,7 +684,7 @@ td, th{
     border-bottom: 1px solid var(--cmap-bg-lightest);
 }
 tr, thead{
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
 }
 th{
     text-align: left;
@@ -992,7 +992,7 @@ nav.collapse-on-mobile a.mobile-nav-icon {
     bottom: 0;
     right: 0;
     width: 20px;
-    background: var(--cmap-bg-ligher);
+    background: var(--cmap-bg-lighter);
   }
   td.xpub{
     max-width: 230px;

src/cryptoadvance/specter/templates/base.jinja

@@ -71,7 +71,7 @@
 						<img src="{{ url_for('static', filename='img/gear.svg') }}" style="width: 44px;" class="svg-white"/>
 					</a>
 					<a class="settings-bar-btn" href="{{ url_for('about') }}" title="About and help">
-						<img src="{{ url_for('static', filename='img/icon.png') }}" style="width: 22px;"/>
+						<img src="{{ url_for('static', filename='img/ghost_3d.png') }}" style="width: 22px;"/>
 					</a>
 					{% if specter.config.auth.method != "none" %}
 						<a class="settings-bar-btn" href="{{ url_for('auth_endpoint.logout') }}"  style="padding: 0px 7px;">
@@ -102,9 +102,9 @@
 					<br>
 					<div style="max-width: 60%; text-align: center;">
 					<h1 style="font-size: 1.8em;">
-					<img src="{{ url_for('static', filename='img/icon.png') }}" width="40px" style="vertical-align: top; margin-right: 20px;"/>
+					<img src="{{ url_for('static', filename='img/ghost_3d.png') }}" width="40px" style="vertical-align: top; margin-right: 20px;"/>
 					{{ _('Welcome to Specter Desktop') }} 
-					<img src="{{ url_for('static', filename='img/ghost.png') }}" width="40px" style="vertical-align: top; margin-left: 20px;"/>
+					<img src="{{ url_for('static', filename='img/ghost_3d.png') }}" width="40px" style="vertical-align: top; margin-left: 20px;"/>
 					</h1>
 					<p style="font-size: 1.2em; font-weight: bolder;">{{ _("Specter Desktop is a convenient wallet interface to better use your Bitcoin Core node. It’s good for singlesig wallets, but comes with a special focus on multisignature setups for different hardware wallets and air-gapped signing devices.") }}</p>
 
@@ -223,7 +223,7 @@
 								{% if specter.chain == 'main' %}
 								<tr>
 									<td>
-										<img src="{{ url_for('static', filename='img/ghost.png')}}" width="30px">
+										<img src="{{ url_for('static', filename='img/ghost_3d.png')}}" width="30px">
 									</td>
 									<td style="text-align: left;">
 										<a href="{{ url_for('get_whitepaper') }}">

src/cryptoadvance/specter/templates/includes/address-data.html

@@ -13,6 +13,12 @@
             word-wrap: break-word;
             margin: auto;
         }
+        .service-icon {
+            margin-top: -5px;
+            margin-right: 0.5em;
+            height:24px;
+            vertical-align: middle;
+        }
     </style>
     <div class="address-data">
         <h1>{{ _("Address details") }}</h1><br>
@@ -33,6 +39,8 @@
             this.el = clone.querySelector(".address-data");
             this.note = clone.querySelector(".note");
             this.info = clone.querySelector(".address-data-info");
+
+            // Read input "data-*" attributes
             this.isVerifyQR = this.getAttribute('data-verify-qr') == 'True';
             this.isVerifyHwi = this.getAttribute('data-verify-hwi') == 'True';
             this.used = (this.getAttribute('data-used') == 'true');
@@ -40,7 +48,10 @@
             this.amount = this.getAttribute('data-amount');
             this.amountPrice = this.getAttribute('data-amount-price');
             this.address = this.getAttribute('data-address');
+            this.label = this.getAttribute('data-label');
+            this.serviceId = this.getAttribute('data-service-id');
             this.wallet = this.getAttribute('data-address-wallet');
+
             this.note.innerText = `{{ _("Loading address") }}: ${this.address} {{ _("details") }}...`;
             this.fetchAddressData();
         
@@ -67,7 +78,6 @@
                 }
                 const jsonResponse = await response.json();
                 if (jsonResponse.success) {
-                    console.log(jsonResponse);
                     let descriptor = jsonResponse.descriptor;
                     let addressIndex = jsonResponse.index;
                     let isChange = jsonResponse.change;
@@ -75,20 +85,31 @@
                     let address = jsonResponse.address;
                     let walletLink = `{{ url_for('wallets_endpoint.wallet', wallet_alias='WALLET_ALIAS') }}`.replace("WALLET_ALIAS", this.wallet);
                     let addressInfoHTML = `
-                    <qr-code style="margin: auto;" value="bitcoin:${address}" width="256"></qr-code><br>
-                    <table class="address-data-table">
-                    <tbody>
-                    <tr><td>{{ _("Address") }}:</td><td style="word-break: break-all;"><explorer-link data-type="address" data-value="${address}"></explorer-link></td></tr>
-                    <tr><td>{{ _("Label") }}:</td><td><address-label data-copy-hidden="true" data-address="${address}" data-wallet="${this.wallet}"></address-label><br></td></tr>
-                    <tr><td>{{ _("From wallet") }}:</td><td><a href=${walletLink}>${walletName}<a></td></tr>
-                    <tr><td>{{ _("Address index") }}:</td><td>${addressIndex}</td></tr>
-                    <tr><td>{{ _("Is change address") }}:</td><td>${isChange ? '{{ _("Yes") }}' : '{{ _("No") }}'}</td></tr>
-                    <tr><td>{{ _("Used") }}:</td><td>${this.used ? 'Yes' : 'No'}</td></tr>
-                    <tr><td>{{ _("UTXO count") }}:</td><td>${this.utxo}</td></tr>
-                    <tr><td>{{ _("Amount") }}:</td><td>${this.amount} <span class="amount-price note ${this.amountPrice ? '' : 'hidden'}">${this.amountPrice}</span></td></tr>
-                    </tbody>
-                    </table>
-                    `
+                        <qr-code style="margin: auto;" value="bitcoin:${address}" width="256"></qr-code><br>
+                        <table class="address-data-table">
+                        <tbody>
+                        <tr><td>{{ _("Address") }}:</td><td style="word-break: break-all;"><explorer-link data-type="address" data-value="${address}"></explorer-link></td></tr>
+                        <tr><td>{{ _("Label") }}:</td><td><address-label data-copy-hidden="true" data-address="${address}" data-wallet="${this.wallet}" data-label="${this.label}" data-service-id="${this.serviceId}"></address-label><br></td></tr>
+                        <tr><td>{{ _("From wallet") }}:</td><td><a href=${walletLink}>${walletName}<a></td></tr>
+                        <tr><td>{{ _("Address index") }}:</td><td>${addressIndex}</td></tr>
+                        <tr><td>{{ _("Is change address") }}:</td><td>${isChange ? '{{ _("Yes") }}' : '{{ _("No") }}'}</td></tr>
+                        <tr><td>{{ _("Used") }}:</td><td>${this.used ? 'Yes' : 'No'}</td></tr>
+                        <tr><td>{{ _("UTXO count") }}:</td><td>${this.utxo}</td></tr>
+                        <tr><td>{{ _("Amount") }}:</td><td>${this.amount} <span class="amount-price note ${this.amountPrice ? '' : 'hidden'}">${this.amountPrice}</span></td></tr>
+                        <tr><td>{{ _("Service") }}:</td><td>
+                    `;
+                    if (this.serviceId != "null") {
+                        // `services` obj made globally available in services-data.html
+                        addressInfoHTML += `<img class="service-icon" src='/svc/${this.serviceId}/static/${services[this.serviceId].icon}'>${services[this.serviceId].name}`;
+                    } else {
+                        let associateServiceUrl = `{{ url_for('services_endpoint.associate_addr', wallet_alias='WALLET_ALIAS', address='ADDRESS') }}`.replace('WALLET_ALIAS', this.wallet).replace('ADDRESS', jsonResponse.address);
+                        addressInfoHTML += `<button type="button" class="btn" onclick="location.href='${associateServiceUrl}';">{{ _("Associate with a service") }}</button>`
+                    }
+                    addressInfoHTML += `
+                        </td></tr>
+                        </tbody>
+                        </table>
+                    `;
 
                     if (this.isVerifyHwi) { 
                         addressInfoHTML += `

src/cryptoadvance/specter/templates/includes/address-label.html

@@ -3,6 +3,12 @@
         .address-label-form {
             display: inline;
         }
+        .service-icon {
+            margin-top: -5px;
+            margin-right: 0.25em;
+            height:24px;
+            vertical-align: middle;
+        }
         .label {
             word-break: break-all;
             background: none;
@@ -60,6 +66,7 @@
     </style>
     <form class="address-label-form">
         <input type="hidden" class="csrf-token" name="csrf_token" value="{{ csrf_token() }}"/>
+        <img class="service-icon"/>
         <a target="_blank" class="explorer-link"><span class="label" autocomplete="off" spellcheck="false">{{ _("Fetching address label...") }}</span></a>
         <button type="button" class="btn edit" title="Edit label"><img src="{{ url_for('static', filename='img/edit.svg') }}" style="width: 22px; margin-bottom:-5px;" class="svg-white"/></button>
         <button type="button" class="btn update hidden">{{ _("Update") }}</button>
@@ -76,6 +83,7 @@
             var style = document.getElementById('address-label').content;
             var clone = style.cloneNode(true);
             this.el = clone.querySelector(".address-label-form");
+            this.serviceIconImg = clone.querySelector(".service-icon");
             this.label = clone.querySelector(".label");
             this.update = clone.querySelector(".update");
             this.cancel = clone.querySelector(".cancel");
@@ -90,10 +98,16 @@
             this.address = this.getAttribute('data-address');
             this.wallet = this.getAttribute('data-wallet');
             this.labelValue = this.getAttribute('data-label');
+            this.serviceId = this.getAttribute('data-service-id');
             this.explorer = '{{ specter.explorer }}';
             this.label.title = this.address;
             this.isEditing = false;
 
+            if (this.serviceId && this.serviceId != "null") {
+                // `services` globally injected via includes/services-data.html
+                this.serviceIconImg.src = "/svc/" + this.serviceId + "/static/" + services[this.serviceId].icon;
+            }
+
             // Set the label - fetch if not specified
             if (this.labelValue) {
                 this.label.innerText = this.labelValue;

src/cryptoadvance/specter/templates/includes/address-row.html

@@ -9,6 +9,9 @@
                 max-width: 160px;
             }
         }
+        .address-row:hover {
+            background-color: var(--cmap-bg-lightest);
+        }
     </style>
     <tr class="address-row">
         <td class="index"></td>
@@ -69,15 +72,22 @@
             this.hideSensitiveInfo = this.getAttribute('data-hide-sensitive-info') == 'true';
 
             this.index.innerText = `#${this.addressData.index}`;
-            this.address.innerText = this.hideSensitiveInfo ? '###########################' : this.addressData.address;
+            this.address.innerHTML = this.hideSensitiveInfo ? '###########################' : this.addressData.address.substring(0, 7) + "&hellip;" + this.addressData.address.substring(this.addressData.address.length - 7);
             if (!this.hideSensitiveInfo) {
                 this.address.onclick = () => {
                     showAddressData(this.amountText.innerText, this.amountPrice.innerText, this.addressData, this.wallet);
                 }
             }
 
-            this.label.innerHTML = this.hideSensitiveInfo ? '############' : `<address-label data-address="${this.addressData.address}" ${this.addressData.label ? `data-label="${this.addressData.label}"` : ''} data-wallet="${this.wallet}"></address-label>`;
-
+            if  (this.hideSensitiveInfo) {
+                this.label.innerHTML = '############'
+            } else {
+                this.label.innerHTML = `<address-label \
+                    data-address="${this.addressData.address}" \
+                    ${this.addressData.label ? `data-label="${this.addressData.label}"` : ''} \
+                    ${this.addressData.service_id ? `data-service-id="${this.addressData.service_id}"` : ''} \
+                    data-wallet="${this.wallet}"></address-label>`;
+            }
             this.used.innerText = this.hideSensitiveInfo ? '###' : `${this.addressData.used ? 'Yes' : 'No'}`;
 
             this.utxo.innerText = this.hideSensitiveInfo ? '###' : this.addressData.utxo;

src/cryptoadvance/specter/templates/includes/addresses-table.html

@@ -454,7 +454,7 @@
                         let addressesList = JSON.parse(jsonResponse.addressesList);
 
                         for (let addr of addressesList) {
-
+                            // Address data is embedded into each row as stringified JSON in the "data-address" attr.
                             let addrRow = document.createRange().createContextualFragment(`
                             <address-row 
                                 data-verify-qr="{{ supports_qr_code_verify != [] }}"
@@ -464,7 +464,7 @@
                                 data-symbol="${this.symbol ? this.symbol : ''}"
                                 data-address='${JSON.stringify(addr).replace(/[\(]/g, "&lpar;").replace(/[\)]/g, "&rpar;").replace(/[\/]/g, "&sol;").replace(/[\']/g, "&apos;")}'
                                 data-hide-sensitive-info="${this.hideSensitiveInfo}"
-                                data-wallet="${this.wallet}"> 
+                                data-wallet="${this.wallet}">
                             </<address-row>`);
 
                             this.tbody.append(addrRow);
@@ -535,15 +535,17 @@
             let addressDataPopup = document.getElementById('address-popup');
             addressDataPopup.innerHTML = `
             <address-data
-            data-verify-qr="{{ supports_qr_code_verify != [] }}"
-            data-verify-hwi="{{ supports_hwi != [] and (supports_hwi_multisig_display_address != [] or not wallet.is_multisig) }}"
-            data-used="${addressData.used}"
-            data-utxo="${addressData.utxo}"
-            data-amount="${amount}"
-            data-amount-price="${amountPrice}"
-            data-address="${addressData.address}"
-            data-address-wallet="${wallet}">
-            </address-data>`;
+                data-verify-qr="{{ supports_qr_code_verify != [] }}"
+                data-verify-hwi="{{ supports_hwi != [] and (supports_hwi_multisig_display_address != [] or not wallet.is_multisig) }}"
+                data-used="${addressData.used}"
+                data-utxo="${addressData.utxo}"
+                data-amount="${amount}"
+                data-amount-price="${amountPrice}"
+                data-address="${addressData.address}"
+                data-service-id="${addressData.service_id}"
+                data-label="${addressData.label}"
+                data-address-wallet="${wallet}"
+            />`;
             showPageOverlay('address-popup');
         } else {
             copyText(address, `{{ _("Copied address") }}: ${ address }`);

src/cryptoadvance/specter/templates/includes/message-box.html

@@ -91,6 +91,9 @@ class MessageBoxElement extends HTMLElement {
       this.main.style.display = "none";
     }, 200);
   }
+  setType(mytype) {
+    this.main.classList.add("error");
+  }
   connectedCallback(){
     this.main.style.opacity = 0;
     this.main.style["animation-name"]="appear";
@@ -101,7 +104,7 @@ class MessageBoxElement extends HTMLElement {
     var timeout = parseInt(this.getAttribute('timeout'));
     this.main.classList.remove("error");
     if(type=="error"){
-      this.main.classList.add("error");
+      this.setType("error");
     }
     if(timeout > 0){
       setTimeout(()=>{

src/cryptoadvance/specter/templates/includes/sidebar/sidebar.jinja

@@ -221,12 +221,14 @@
             {% endfor %}
             {{ sidebar_btn(url_for('devices_endpoint.new_device_type'), _('Add new device'),'btn_new_device') }}
         </div>
+        {% include "services/sidebar_services_list.jinja" %}
+        {{ sidebar_btn(url_for('services_endpoint.choose'), 'Choose services', 'btn_new_service') }}
         <br>
         <div class="footer">
 	        <span>{{ _("Specter Version:") }} <strong>{{ specter.version.current }}</strong></span>
             <a style="color: #fff; font-size: 1.1em; margin: 10px; text-align: center;" href="{{ url_for('about') }}">
             {{ _("About Specter") }}<br>
-            <img src="{{ url_for('static', filename='img/icon.png') }}" style="width: 35px;"/>
+            <img src="{{ url_for('static', filename='img/ghost_3d.png') }}" style="width: 35px;"/>
             </a>
         </div>
     </nav>
@@ -300,6 +302,13 @@
             });
         }
 
+        if ('{{ specter.service_manager.service_names | length }}' == '0') {
+            document.getElementById('toggle_services_list').innerHTML = 'Services';
+        } else {
+            document.getElementById('toggle_services_list').addEventListener('click', (event) => {
+                toggleList('services');
+            });
+        }
         {% if specter.wallet_manager.is_loading %}
             updateWalletsLoadingData();
         {% endif %}

src/cryptoadvance/specter/templates/includes/tx-data.html

@@ -138,6 +138,7 @@
                     let jsonResponse = await this.fetchRawTx(rawtx.vin[i].txid);
                     if (jsonResponse !== null && jsonResponse.success) {
                         let spentOutput = jsonResponse.rawtx.vout[rawtx.vin[i].vout];
+
                         let address = (spentOutput.addresses && spentOutput.addresses.length == 1) ? spentOutput.addresses[0] : 'Unknown';
                         if (address == 'Unknown') {
                             address = (spentOutput.scriptPubKey.addresses && spentOutput.scriptPubKey.addresses.length == 1) ? spentOutput.scriptPubKey.addresses[0] : 'Unknown';
@@ -175,8 +176,23 @@
                             value = `${numberWithCommas(value.toString())}`;
                         }
 
+                        let labelAttr = "";
+                        if ("label" in spentOutput) {
+                            labelAttr = `data-label="${spentOutput.label}"`;
+                        }
+                        let serviceIdAttr = "";
+                        if ("service_id" in spentOutput) {
+                            serviceIdAttr = `data-service-id="${spentOutput.service_id}"`;
+                        }
+
                         addressAndValue = `<br>
-                            {{ _("Address:") }} <address-label data-copy-hidden="true" data-address="${address}" data-wallet="${this.wallet}"></address-label><br>
+                            {{ _("Address:") }} <address-label 
+                                data-copy-hidden="true"
+                                data-address="${address}"
+                                data-wallet="${this.wallet}"
+                                ${labelAttr}
+                                ${serviceIdAttr}
+                            /><br>
                             {{ _("Value:") }} ${value} ${assetlabel} ${price}`;
                     }
 
@@ -235,15 +251,32 @@
                     }
 
                     let bgColor = '#131a24';
-                    let isMine = await this.fetchAddressIsMine(address);
-                    if (isMine) {
+                    if ("index" in rawtx.vout[i]) {
+                        // Only addresses from this wallet will have an index; isMine = True
                         bgColor = '#154984';
                     }
 
+                    let labelAttr = "";
+                    if ("label" in rawtx.vout[i]) {
+                        labelAttr = `data-label="${rawtx.vout[i].label}"`;
+                    } else if (address != "Unknown") {
+                        labelAttr = `data-label="${address}"`;
+                    }
+                    let serviceIdAttr = "";
+                    if ("service_id" in rawtx.vout[i]) {
+                        serviceIdAttr = `data-service-id="${rawtx.vout[i].service_id}"`;
+                    }
+
                     rawtxHTML += `
                     <p class="tx_info" style="text-align: left; background-color: ${bgColor};">
                         <b>{{ _("Output #${i}") }}</b><br><br>
-                        {{ _("Address:") }} <address-label data-copy-hidden="true" data-address="${address}" data-wallet="${this.wallet}"></address-label><br>
+                        {{ _("Address:") }} <address-label
+                            data-copy-hidden="true"
+                            data-address="${address}"
+                            data-wallet="${this.wallet}"
+                            ${labelAttr}
+                            ${serviceIdAttr}
+                        /><br/>
                         {{ _("Value:") }} ${value} ${assetlabel} ${price}
                     </p>
                     `;

src/cryptoadvance/specter/templates/includes/tx-row.html

@@ -15,6 +15,20 @@
         .svg-cancelled {
             filter: invert(20%) sepia(32%) saturate(4838%) hue-rotate(332deg) brightness(81%) contrast(97%);
         }
+
+        .tx-row:hover {
+            background-color: var(--cmap-bg-lightest);
+        }
+        .address {
+            overflow: hidden;
+            text-overflow: ellipsis;
+        }
+        .service-icon {
+            margin-top: -5px;
+            margin-right: 0.5em;
+            height:24px;
+            vertical-align: middle;
+        }
     </style>
     <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='styles.css') }}">
     <tr class="tx-row">
@@ -58,6 +72,7 @@
             this.category = clone.querySelector(".category");
             this.txid = clone.querySelector(".txid .explorer-link");
             this.address = clone.querySelector(".address");
+            this.serviceIcon = clone.querySelector(".service-icon");
             this.amountText = clone.querySelector(".amount");
             this.amountPrice = clone.querySelector(".amount-price");
             this.time = clone.querySelector(".time");
@@ -109,7 +124,7 @@
 
 
             // Set txid
-            this.txid.innerText = this.hideSensitiveInfo ? '############################################' : this.tx.txid;
+            this.txid.innerHTML = this.hideSensitiveInfo ? '############################################' : this.tx.txid.slice(0,11) + '&hellip;';
             if (!this.hideSensitiveInfo) {
                 this.txid.onclick = () => {
                     showTxData(
@@ -138,7 +153,18 @@
             if (Array.isArray(this.tx.address)) {
                 this.address.innerText = this.hideSensitiveInfo ? '#########' : `${this.tx.address.length} {{ _("Recipients") }}`;
             } else {
-                this.address.innerHTML = this.hideSensitiveInfo ? '#########' : `<address-label data-address="${this.tx.address}" ${this.tx.label ? `data-label="${this.tx.label}"` : ''} data-wallet="${this.wallet ? this.wallet : this.tx.wallet_alias}"></address-label>`
+                if (this.hideSensitiveInfo) {
+                    this.address.innerHTML = '#########';
+                } else {
+                    if ("service_id" in this.tx) {
+                        this.address.innerHTML = `<img class="service-icon" src="/svc/${this.tx.service_id}/static/${services[this.tx.service_id].icon}">`;
+                    }
+                    if (this.tx.label !== undefined) {
+                        this.address.innerHTML += this.tx.label;
+                    } else {
+                        this.address.innerHTML += this.tx.address;
+                    }
+                }
             }
             
             // Set amount
@@ -190,7 +216,7 @@
 
             // Set confirmations
             if (this.tx.confirmations > 0) {
-                this.confirmations.innerHTML = this.hideSensitiveInfo ? '########' : `${this.tx.confirmations}<span class="optional"> {{ _("Confirmations") }}</span>`;
+                this.confirmations.innerHTML = this.hideSensitiveInfo ? '########' : `${this.tx.confirmations}`;
             } else if (this.tx.confirmations == 0) {
                 this.el.classList.add('unconfirmed');
                 this.confirmations.innerHTML = this.hideSensitiveInfo ? '########' : `{{ _("Unconfirmed") }}`;

src/cryptoadvance/specter/templates/includes/tx-table.html

@@ -322,12 +322,14 @@
             // Next page action
             this.paginationNext.onclick = () => {
                 this.idx++;
+                console.log("paginationNext fetch");
                 this.fetchTxItems();
             }
 
             // Previous page action
             this.paginationBack.onclick = () => {
                 this.idx--;
+                console.log("paginationBack fetch");
                 this.fetchTxItems();
             }
 
@@ -339,6 +341,7 @@
                     return;
                 }
                 this.idx = newIdx - 1;
+                console.log("paginationIdxInput fetch");
                 this.fetchTxItems();
             }
 
@@ -346,12 +349,14 @@
             this.searchInput.onchange = () => {
                 this.search = this.searchInput.value;
                 this.export.href = this.export.href.split('?')[0] + `?exportPrices=${this.exportPricesSwitch.checked}&search=${this.search}&sortby=${this.sortby}&sortdir=${this.sortdir}`;
+                console.log("searchInput fetch");
                 this.fetchTxItems();
             }
 
             // Listen to changes in page limit
             this.pageLimitSelect.onchange = () => {
                 this.limit = this.pageLimitSelect.value;
+                console.log("pageLimitSelect fetch");
                 this.fetchTxItems();
             }
 
@@ -385,6 +390,7 @@
                         this.el.querySelector(`.${this.sortby}-arrow`).classList.add(this.sortdir == 'asc' ? 'down-arrow' : 'up-arrow');
                         this.export.href = this.export.href.split('?')[0] + `?exportPrices=${this.exportPricesSwitch.checked}&search=${this.search}&sortby=${this.sortby}&sortdir=${this.sortdir}`;
                     }
+                    console.log("header fetch")
                     this.fetchTxItems();
                 }
             }
@@ -486,6 +492,9 @@
         }
 
         /**
+         * TODO: This is being called 7 times (once for each attribute) when the page is loaded, 
+         * causing up to 7 redundant fetchTxItems() hits to the server.
+         * 
          * Listens to changes on the following attributes:
          * - blockhash: Should show blockhash column. Either "true" or "false"
          * - btc-unit: Bitcoin unit to display amounts with. Either "btc" or "sat"
@@ -621,6 +630,9 @@
             formData.append('search', this.search);
             formData.append('sortby', this.sortby);
             formData.append('sortdir', this.sortdir);
+            if (this.serviceId) {
+                formData.append('service_id', this.serviceId);
+            }
             formData.append('csrf_token', '{{ csrf_token() }}');
             try {
                 let callId = this.callId;
@@ -690,6 +702,7 @@
                         // If index is greater than the pages count fetch again for the last page
                         if (this.idx >= this.pageCount) {
                             this.idx = this.pageCount - 1;
+                            console.log("index greater than pages count fetch");
                             this.fetchTxItems()
                         }
 

src/cryptoadvance/specter/templates/settings/auth_settings.jinja

@@ -1,7 +1,18 @@
 {% extends "base.jinja" %}
 {% block main %}
+	<style>
+		.hasencryptedservicedata {
+            background-color: var(--cmap-bg-lighter);
+			border: 2px solid yellow;
+			border-radius: 0.5em;
+			padding: 1em;
+			text-align: center;
+			margin-bottom: 4em !important;
+		}
+	</style>
 	<form action="?" method="POST">
 		<input type="hidden" class="csrf-token" name="csrf_token" value="{{ csrf_token() }}"/>
+		<input type="hidden" name="next" value="{{ next }}">
 		<h1 id="title" class="settings-title">Settings</h1>
 		{% from 'settings/components/settings_menu.jinja' import settings_menu %}
 		{{ settings_menu('auth', current_user) }}
@@ -16,6 +27,9 @@
 				</select>
 				<br><br>
 			{% endif %}
+			<div id="hasencryptedservicedata" style="display: none" class="hasencryptedservicedata note">
+				<p>{{ _("Note: If you set Authentication to \"None\", Specter will unlink your Service integrations as a security precaution.") }}</p>
+			</div>
 			<div id="ratelimit" class="{% if method == 'none' or not current_user.is_admin %}hidden{% endif %}">
 				{{ _("Rate Limiting (seconds between login/register attempts)") }}:<br><input id="rate_limit" type="number" name="rate_limit" min="0" step="1" value="{{ rate_limit }}"><br><br>
 			</div>
@@ -67,10 +81,15 @@
 		function toggleUsernamePassword(select) {
 			{% if current_user.is_admin %}
 				var ratelimitDiv = document.getElementById("ratelimit");
+				var hasEncryptedServiceData = document.getElementById("hasencryptedservicedata");
 				if (select.options[select.selectedIndex].value !== 'none'){
 					ratelimitDiv.style.display = 'block';
+					hasEncryptedServiceData.style.display = 'none';
 				} else {
 					ratelimitDiv.style.display = 'none';
+					{% if method != "none" and current_user.is_admin and has_service_encrypted_storage %}
+						hasEncryptedServiceData.style.display = 'block';
+					{% endif %}
 				}
 			{% endif %}
 			var usernamepasswordDiv = document.getElementById("usernamepassword");

src/cryptoadvance/specter/templates/settings/general_settings.jinja

@@ -6,7 +6,6 @@
 		{% from 'settings/components/settings_menu.jinja' import settings_menu %}
 		{{ settings_menu('general', current_user) }}
 		<div class="card" style="margin: 20px auto;">
-
 			<h1>{{ _("Language") }}</h1>
 			{% include "includes/language/language_select.jinja" %}<br/>
 			<br><br>
@@ -59,6 +58,7 @@
 				<a href="{{ url_for('settings_endpoint.backup_file') }}" class="btn" style="width: 100%; margin-top: -5px;">{{ _("Download Specter backup files") }}</a>
 			</div>
 			<br>
+			<span style="margin: auto 5px auto 0px;">Load Specter backup:</span>
 			<div class="tool-tip" style="float: right; margin-bottom: 5px;">
 				<i class="tool-tip__icon">i</i>
 				<p class="tool-tip__info">

src/cryptoadvance/specter/templates/settings/hwi_settings.jinja

@@ -2,7 +2,7 @@
 {% block main %}
 	<style>
 		.radio-btn:hover {
-			background: var(--cmap-bg-ligher);
+			background: var(--cmap-bg-lighter);
 			cursor: pointer;
 		}
 	</style>

src/cryptoadvance/specter/templates/settings/tor_settings.jinja

@@ -2,7 +2,7 @@
 {% block main %}
 	<style>
 		.radio-btn:hover {
-			background: var(--cmap-bg-ligher);
+			background: var(--cmap-bg-lighter);
 			cursor: pointer;
 		}
 		.svg-check-green {

src/cryptoadvance/specter/templates/setup/bitcoind_datadir.jinja

@@ -1,7 +1,7 @@
 {% extends "setup/setup_page.jinja" %}
 
 {% block setup %}
-    <h1><img src="{{ url_for('static', filename='img/icon.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow-right.svg') }}" style="width: 60px;" class="svg-white"/>&nbsp;{{ bitcoin_svg(network, 60) }}<br>
+    <h1><img src="{{ url_for('static', filename='img/ghost_3d.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow-right.svg') }}" style="width: 60px;" class="svg-white"/>&nbsp;{{ bitcoin_svg(network, 60) }}<br>
     <br>{{ _("Configure your node") }}</h1>
     <p id="quicksync-option" {% if network != 'main' %}class="hidden"{% endif %}>{{ _("QuickSync?") }}&nbsp;&nbsp;
         <label class="switch" id="quicksync-switch">

src/cryptoadvance/specter/templates/setup/node_type.jinja

@@ -1,7 +1,7 @@
 {% extends "setup/setup_page.jinja" %}
 
 {% block setup %}
-    <h1><img src="{{ url_for('static', filename='img/icon.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow-right.svg') }}" style="width: 60px;" class="svg-white"/>&nbsp;{{ bitcoin_svg('main', 60) }}<br>
+    <h1><img src="{{ url_for('static', filename='img/ghost_3d.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow-right.svg') }}" style="width: 60px;" class="svg-white"/>&nbsp;{{ bitcoin_svg('main', 60) }}<br>
     <br>{{ _("Would you like to setup a new Bitcoin node or connect to an existing one?") }}</h1>
     <p class="hidden warning" id="helper-text">{{ _("If you have an existing Bitcoin Core node you would like to use, either running locally on this computer or somewhere else, like on Umbrel, myNode, or RaspiBlitz, you can just connect and use it.") }}<br><br>
     {{ _("Don't have a node? Specter can help you setup a new one! This guide will help you get it up and running!") }}</p>

src/cryptoadvance/specter/templates/setup/tor.jinja

@@ -1,7 +1,7 @@
 {% extends "setup/setup_page.jinja" %}
 
 {% block setup %}
-    <h1><img src="{{ url_for('static', filename='img/icon.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow-right.svg') }}" style="width: 60px;" class="svg-white"/>&nbsp;<img style="width: 60px;" src="{{ url_for('static', filename='img/tor.svg') }}"/><br>
+    <h1><img src="{{ url_for('static', filename='img/ghost_3d.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow-right.svg') }}" style="width: 60px;" class="svg-white"/>&nbsp;<img style="width: 60px;" src="{{ url_for('static', filename='img/tor.svg') }}"/><br>
     <br>{{ _("Setup Tor daemon") }}</h1>
     <br><br>
     <div class="row">

src/cryptoadvance/specter/templates/wallet/addresses/wallet_addresses.jinja

@@ -4,6 +4,7 @@
 
 	<div class="table-holder">
 
+		{% include "includes/services-data.html" %}
 		{% include "includes/hwi/hwi.jinja" %}
 		{% include "includes/explorer-link.html" %}
 		{% include "includes/address-data.html" %}
@@ -12,8 +13,8 @@
         
 		<addresses-table
 			{% if specter.price_check and (specter.alt_rate and specter.alt_symbol) %}
-			price="{{ specter.alt_rate }}"
-			symbol="{{ specter.alt_symbol }}"
+				price="{{ specter.alt_rate }}"
+				symbol="{{ specter.alt_symbol }}"
 			{% endif %}
 			btc-unit="{{ specter.unit }}"
 			hide-sensitive-info="{{ specter.hide_sensitive_info | lower }}"

src/cryptoadvance/specter/templates/wallet/history/wallet_history.jinja

@@ -4,14 +4,15 @@
 	{% from 'wallet/history/components/total_wallet_balances.jinja' import total_wallet_balances %}
 	{{ total_wallet_balances(wallet, specter) }}
 	<div class="table-holder">
+		{% include "includes/services-data.html" %}
 		{% include "includes/tx-row.html" %}
 		{% include "includes/tx-data.html" %}
 		{% include "includes/explorer-link.html" %}
 		{% include "includes/tx-table.html" %}
 		<tx-table 
 		{% if specter.price_check and (specter.alt_rate and specter.alt_symbol) %}
-		price="{{ specter.alt_rate }}"
-		symbol="{{ specter.alt_symbol }}"
+			price="{{ specter.alt_rate }}"
+			symbol="{{ specter.alt_symbol }}"
 		{% endif %}
 		btc-unit="{{ specter.unit }}"
 		blockhash="{{ specter.config.validate_merkle_proofs | lower }}"

src/cryptoadvance/specter/templates/wallet/receive/wallet_receive.jinja

@@ -82,8 +82,8 @@
 			{% endif %}
 		</form>
 	</div>
-	<br>
-	<br>
+	<br class="bottom-space">
+	<br class="bottom-space">
 	<div class="hidden" id="new_wallet_devices_popup">
 		{% set supports_export_to_device = [] %}
 		{% for device in wallet.devices if device.exportable_to_wallet %}

src/cryptoadvance/specter/templates/wallet/send/new/wallet_send.jinja

@@ -604,7 +604,7 @@
 		}
 
 		document.addEventListener("DOMContentLoaded", function(){
-		{% if recipients %}
+		{% if fillform %}
 			{% for addr, amount, amount_unit, label in recipients %}
 			addRecipient("{{ addr }}", {{ amount }}, "{{ amount_unit }}", "{{ label }}");
 			{% endfor %}

src/cryptoadvance/specter/templates/wallet/wallets_overview.jinja

@@ -1,35 +1,36 @@
 {% extends "base.jinja" %}
 {% block main %}
     <style>
-		.arrow {
-			text-decoration: none;
-			display: inline-block;
-			margin: 0px 16px 16px;
-			border-radius: 50%;
-			color: #fff;
-			font-size: 1.5em;
-		}
+        .arrow {
+            text-decoration: none;
+            display: inline-block;
+            margin: 0px 16px 16px;
+            border-radius: 50%;
+            color: #fff;
+            font-size: 1.5em;
+        }
 
-		.disabled {
-			visibility: hidden;
-		}
-	</style>
-	{% include "includes/address-label.html" %}
+        .disabled {
+            visibility: hidden;
+        }
+    </style>
+    {% include "includes/services-data.html" %}
+    {% include "includes/address-label.html" %}
     <h1 id="title">{{ _("Wallets Overview") }}</h1>
-	<p style="width: 80%; text-align: center; margin-left: auto; margin-right: auto;">{{ _("Here you can see the combined balance and transactions history of all your Specter wallets.") }}</p>
+    <p style="width: 80%; text-align: center; margin-left: auto; margin-right: auto;">{{ _("Here you can see the combined balance and transactions history of all your Specter wallets.") }}</p>
     <h1>
         <small style="line-height:30px">Total balance:</small><br>
         <span style="color: #fff">
-			{% set fullbalance = specter.wallet_manager.wallets.values() | sum(attribute='fullbalance') %}
-			{% set balance = specter.wallet_manager.joined_balance() %}
+            {% set fullbalance = specter.wallet_manager.wallets.values() | sum(attribute='fullbalance') %}
+            {% set balance = specter.wallet_manager.joined_balance() %}
             <span id="fullbalance_amount">{{ fullbalance | btcunitamount }}</span>
             {% if specter.unit == 'sat' %}
-				sats
-			{% else %}
-				{% if specter.is_testnet %}t{%endif%}{% if specter.is_liquid %}L{%endif%}BTC
-			{% endif %}
+                sats
+            {% else %}
+                {% if specter.is_testnet %}t{%endif%}{% if specter.is_liquid %}L{%endif%}BTC
+            {% endif %}
         </span><br>
-		<span class="note">{{ fullbalance | altunit }}</span>
+        <span class="note">{{ fullbalance | altunit }}</span>
         {% if balance.get("untrusted_pending", 0) or balance.get("immature", 0) %}<br>
             <small>( {{ balance.get("trusted", 0) | btcunitamount }} {{ _("confirmed") }},
                      {% if balance.get("untrusted_pending", 0) %}
@@ -61,18 +62,18 @@
     </h1>
     <div class="table-holder">
         {% include "includes/tx-row.html" %}
-		{% include "includes/tx-data.html" %}
-		{% include "includes/explorer-link.html" %}
-		{% include "includes/tx-table.html" %}
-		<tx-table
-		{% if specter.price_check and (specter.alt_rate and specter.alt_symbol) %}
-		price="{{ specter.alt_rate }}"
-		symbol="{{ specter.alt_symbol }}"
-		{% endif %}
-		btc-unit="{{ specter.unit }}"
-		hide-sensitive-info="{{ specter.hide_sensitive_info | lower }}"
-		type="txlist">
-		</tx-table>
+        {% include "includes/tx-data.html" %}
+        {% include "includes/explorer-link.html" %}
+        {% include "includes/tx-table.html" %}
+        <tx-table
+            {% if specter.price_check and (specter.alt_rate and specter.alt_symbol) %}
+                price="{{ specter.alt_rate }}"
+                symbol="{{ specter.alt_symbol }}"
+            {% endif %}
+            btc-unit="{{ specter.unit }}"
+            hide-sensitive-info="{{ specter.hide_sensitive_info | lower }}"
+            type="txlist">
+        </tx-table>
     </div>
     <div id="tx-popup" class="hidden"></div>
 {% endblock %}

src/cryptoadvance/specter/user.py

@@ -1,7 +1,9 @@
 import base64
 import binascii
+import cryptography
 import hashlib
 import json
+import logging
 import os
 import shutil
 
@@ -18,6 +20,9 @@ from .managers.device_manager import DeviceManager
 from .helpers import deep_update
 
 
+logger = logging.getLogger(__name__)
+
+
 def hash_password(plaintext_password):
     """Hash a password for storing."""
     salt = binascii.b2a_base64(hashlib.sha256(os.urandom(60)).digest()).strip()
@@ -44,6 +49,10 @@ def verify_password(stored_password, provided_password):
     return pwdhash == binascii.a2b_base64(stored_password["pwdhash"])
 
 
+class UserSecretException(Exception):
+    pass
+
+
 class User(UserMixin):
     """
     The user_secret is used to encrypt/decrypt other user-specific data
@@ -63,7 +72,7 @@ class User(UserMixin):
         specter,
         encrypted_user_secret=None,
         is_admin=False,
-        services=None,
+        services=[],
     ):
         self.id = id
         self.username = username
@@ -95,7 +104,7 @@ class User(UserMixin):
                 "config": {},
                 "specter": specter,
                 "encrypted_user_secret": user_dict.get("encrypted_user_secret", None),
-                "services": user_dict.get("services", None),
+                "services": user_dict.get("services", []),
             }
             if not user_dict["is_admin"]:
                 user_args["config"] = user_dict["config"]
@@ -114,6 +123,10 @@ class User(UserMixin):
             return ""
         return f"_{self.id}"
 
+    @property
+    def is_user_secret_decrypted(self):
+        return self.plaintext_user_secret is not None
+
     def _encrypt_user_secret(self, plaintext_password):
         """
         Implementation taken from the pyca/cryptography docs:
@@ -163,26 +176,58 @@ class User(UserMixin):
             self._encrypt_user_secret(plaintext_password)
 
     def _generate_user_secret(self, plaintext_password):
-        # Encryption using the user_secret uses a Fernet key. But the Fernet
-        #   key itself will be encrypted with the user's password.
+        """
+        Generates and stores the user_secret in memory. Also stores it encrypted
+        to disk.
+
+        Encryption using the user_secret uses a Fernet key. But the Fernet
+        key itself will be encrypted with the user's password.
+        """
         self.plaintext_user_secret = Fernet.generate_key()
         self._encrypt_user_secret(plaintext_password)
         self.save_info()
+        logger.debug("Generated user_secret")
+
+    def delete_user_secret(self, autosave: bool = True):
+        self.encrypted_user_secret = None
+        self.plaintext_user_secret = None
+        if autosave:
+            self.save_info()
 
     def set_password(self, plaintext_password):
-        # Hash the incoming plaintext password and update the encrypted
-        #   user_secret as needed.
-        self.password_hash = hash_password(plaintext_password)
+        """Hash the incoming plaintext password and update the encrypted user_secret as
+        needed.
 
-        # Must keep encrypted_user_secret in sync with password changes
+        Remember that the underlying user_secret doesn't change if the user changes
+        their password; it's the same user_secret but it just needs to be
+        re-encrypted using the new password.
+        """
+        # Check the encrypted_user_secret before saving password change!
         if self.encrypted_user_secret is None:
+            # First time this user is initializing their user_secret
             self._generate_user_secret(plaintext_password)
         else:
             if self.plaintext_user_secret is None:
-                raise Exception(
-                    "encrypted_user_secret wasn't decrypted during user login"
-                )
-            self._encrypt_user_secret(plaintext_password)
+                # encrypted_user_secret hasn't been decrypted in memory; try to decrypt
+                # it now (will only work if we're re-enabling the same password for the
+                # admin account).
+                try:
+                    self.decrypt_user_secret(plaintext_password)
+                except cryptography.fernet.InvalidToken as e:
+                    # Existing encrypted_user_secret cannot be decrypted with this new
+                    # password! Alert the calling code to handle (either provide the
+                    # previous password to decrypt/re-encrypt or delete all existing
+                    # encrypted data for this user (admin) because it's no longer
+                    # decryptable).
+                    logger.warn(e)
+                    raise UserSecretException(
+                        "Cannot decrypt existing encrypted_user_secret with the provided password"
+                    )
+            else:
+                # Must keep re-encrypt encrypted_user_secret with the new password.
+                self._encrypt_user_secret(plaintext_password)
+
+        self.password_hash = hash_password(plaintext_password)
 
     @property
     def json(self):
@@ -255,6 +300,20 @@ class User(UserMixin):
             self.specter.delete_user(self)
         self.manager.save()
 
+    def add_service(self, service_id: str, autosave: bool = True):
+        """Add a Service to the User. Only updates what is listed in the sidebar."""
+        if service_id not in self.services:
+            self.services.append(service_id)
+        if autosave:
+            self.save_info()
+
+    def remove_service(self, service_id: str, autosave: bool = True):
+        """Remove a Service from the User. Only updates what is listed in the sidebar."""
+        if service_id in self.services:
+            self.services.remove(service_id)
+        if autosave:
+            self.save_info()
+
     # TODO: Refactor calling code to explicitly call User.save() rather than embedding
     #   self.save_info() on every update and setter. It ends up saving to disk multiple
     #   times for a single Settings submit.

src/cryptoadvance/specter/util/reflection.py

@@ -0,0 +1,71 @@
+import logging
+from importlib import import_module
+from inspect import isclass
+import os
+from pathlib import Path
+from pkgutil import iter_modules
+
+logger = logging.getLogger(__name__)
+
+
+def _get_module_from_class(clazz):
+    return import_module(clazz.__module__)
+
+
+def get_package_dir_for_subclasses_of(clazz):
+    """There are two occasions where this is used: migrations and service-classes. Depending on the clazz
+    this function is returning package_directories where subclasses from clazz are supposed to be located
+    * subclasses of SpecterMigration are located in a subpackage called migrations
+    * subclasses of Servuce are located in subpackages of package cryptoadvance.specter.services
+    I have to admit that this is not a pure util class as it's containing business-logic
+    """
+    if clazz.__name__ == "SpecterMigration":
+        return str(
+            Path(
+                Path(_get_module_from_class(clazz).__file__).resolve().parent,
+                "migrations",
+            ).resolve()
+        )
+    elif clazz.__name__ == "Service":
+        return str(
+            Path(
+                import_module("cryptoadvance.specter.services").__file__
+            ).parent.resolve()
+        )
+
+
+def get_subclasses_for_class(clazz):
+    """Returns all subclasses of class clazz located in the specific package for that class"""
+    class_list = []
+    loopdir = Path(__file__).resolve()
+    package_dir = get_package_dir_for_subclasses_of(clazz)
+    logger.info(f"Collecting subclasses of {clazz.__name__} ...")
+    for (_, module_name, _) in iter_modules(
+        [package_dir]
+    ):  # import the module and iterate through its attributes
+        # logger.debug(f"Iterating on {module_name} ")
+        if clazz.__name__ == "Service":
+            try:
+
+                module = import_module(
+                    f"cryptoadvance.specter.services.{module_name}.service"
+                )
+            except ModuleNotFoundError:
+                logger.debug(
+                    f"No Service Impl found in cryptoadvance.specter.services.{module_name}. Skipping!"
+                )
+                continue
+        elif clazz.__name__ == "SpecterMigration":
+            module = import_module(
+                f"cryptoadvance.specter.util.migrations.{module_name}"
+            )
+        for attribute_name in dir(module):
+            attribute = getattr(module, attribute_name)
+            if isclass(attribute):
+                if (
+                    issubclass(attribute, clazz)
+                    and not attribute.__name__ == clazz.__name__
+                ):
+                    class_list.append(attribute)
+                    logger.info(f"  Found class {attribute.__name__}")
+    return class_list

src/cryptoadvance/specter/wallet.py

@@ -1,31 +1,31 @@
-import copy, hashlib, json, logging, os, re, csv
-from csv import Error
-from io import StringIO
+import json, logging, os, re, csv
+import requests
+import threading
 import time
+
 from collections import OrderedDict
+from csv import Error
+from embit import bip32
+from embit.descriptor import Descriptor
+from embit.descriptor.checksum import add_checksum
+from embit.ec import PublicKey
+from embit.liquid.networks import get_network
+from embit.psbt import DerivationPath
+from embit.transaction import Transaction
+from io import StringIO
+from typing import List
+
+from .addresslist import Address, AddressList
 from .device import Device
 from .key import Key
 from .util.merkleblock import is_valid_merkle_proof
-from .helpers import der_to_bytes, get_address_from_dict
-from embit import base58, bip32
-from embit.descriptor import Descriptor
-from embit.descriptor.checksum import add_checksum
-from embit.liquid.networks import get_network
-from embit.psbt import PSBT, DerivationPath
-from embit.transaction import Transaction
-from embit.ec import PublicKey
-
-from .util.xpub import get_xpub_fingerprint
-from .util.tx import decoderawtransaction
+from .helpers import get_address_from_dict
 from .persistence import write_json_file, delete_file, delete_folder
-from io import BytesIO
 from .specter_error import SpecterError
-import threading
-import requests
-from math import ceil
-from .addresslist import AddressList
 from .txlist import TxList
 from .util.psbt import SpecterPSBT
+from .util.tx import decoderawtransaction
+from .util.xpub import get_xpub_fingerprint
 
 logger = logging.getLogger(__name__)
 LISTTRANSACTIONS_BATCH_SIZE = 1000
@@ -1002,6 +1002,7 @@ class Wallet:
         fetch_transactions=True,
         validate_merkle_proofs=False,
         current_blockheight=None,
+        service_id: str = None,
     ):
         """Returns a list of all transactions in the wallet's CSV cache - processed with information to display in the UI in the transactions list
         #Parameters:
@@ -1070,11 +1071,21 @@ class Wallet:
 
                 if isinstance(tx["address"], str):
                     tx["label"] = self.getlabel(tx["address"])
+                    addr_obj = self.get_address_obj(tx["address"])
+                    if addr_obj and addr_obj.get("service_id"):
+                        tx["service_id"] = addr_obj["service_id"]
                 elif isinstance(tx["address"], list):
+                    # TODO: Handle services integration w/batch txs
                     tx["label"] = [self.getlabel(address) for address in tx["address"]]
                 else:
                     tx["label"] = None
 
+                if service_id and (
+                    "service_id" not in tx or tx["service_id"] != service_id
+                ):
+                    # We only want `service_id`-related txs returned
+                    continue
+
                 # TODO: validate for unique txids only
                 tx["validated_blockhash"] = ""  # default is assume unvalidated
                 if validate_merkle_proofs is True and tx["confirmations"] > 0:
@@ -1347,12 +1358,16 @@ class Wallet:
         if change:
             self.change_address = address
         else:
+            addr_obj = self.get_address_obj(address)
+            if addr_obj["service_id"]:
+                # Skip addresses reserved for a Service
+                return self.getnewaddress(change, save)
             self.address = address
         if save:
             self.save_to_file()
         return address
 
-    def get_address(self, index, change=False, check_keypool=True):
+    def get_address(self, index, change=False, check_keypool=True) -> str:
         if check_keypool:
             pool = self.change_keypool if change else self.keypool
             logger.debug(
@@ -1364,6 +1379,9 @@ class Wallet:
             self.network
         )
 
+    def get_address_obj(self, address: str) -> Address:
+        return self._addresses.get(address)
+
     def derive_descriptor(self, index: int, change: bool, keep_xpubs=False):
         """
         Derives descriptor for receiving or change address with `index`.
@@ -1423,7 +1441,8 @@ class Wallet:
             ),
         }
 
-    def get_address_info(self, address):
+    def get_address_info(self, address) -> Address:
+        # TODO: This is a misleading name. This is really fetching an Address obj
         return self._addresses.get(address)
 
     def is_address_mine(self, address):
@@ -1586,10 +1605,41 @@ class Wallet:
         self.save_to_file()
         return end
 
+    def associate_address_with_service(
+        self, address: str, service_id: str, label: str, autosave: bool = True
+    ):
+        """
+        Links the Address to the specified Service.id
+        """
+        self._addresses.associate_with_service(
+            address=address, service_id=service_id, label=label, autosave=autosave
+        )
+
+    def deassociate_address(self, address: str, autosave: bool = True):
+        """
+        Clears any Service associations on the Address.
+        """
+        self._addresses.deassociate(address=address, autosave=autosave)
+
+    def get_associated_addresses(
+        self, service_id: str, unused_only: bool = False
+    ) -> List[Address]:
+        """
+        Return the Wallet's Address objs that are associated with the specified Service.id
+        """
+        addrs = []
+        for addr, addr_obj in self._addresses.items():
+            if addr_obj["service_id"] == service_id:
+                if not unused_only or not addr_obj["used"]:
+                    addrs.append(addr_obj)
+        return addrs
+
     def setlabel(self, address, label):
         self._addresses.set_label(address, label)
 
     def getlabel(self, address):
+        # TODO: This is confusing. The Address["label"] attr may be blank but the
+        # Address.label property will auto-populate a value (e.g. "Address #4").
         if address in self._addresses:
             return self._addresses[address].label
         else:
@@ -1631,8 +1681,8 @@ class Wallet:
 
     def createpsbt(
         self,
-        addresses: [str],
-        amounts: [float],
+        addresses: List[str],
+        amounts: List[float],
         subtract: bool = False,
         subtract_from: int = 0,
         fee_rate: float = 0,  # fee rate to use, if less than MIN_FEE_RATE will use MIN_FEE_RATE, 0 for automatic
@@ -1960,10 +2010,19 @@ class Wallet:
         self.save_pending_psbt(psbt)
         return psbt.to_dict()
 
-    def addresses_info(self, is_change):
+    def addresses_info(
+        self,
+        is_change: bool = False,
+        service_id: str = None,
+        include_wallet_alias: bool = False,
+    ):
         """Create a list of (receive or change) addresses from cache and retrieve the
         related UTXO and amount.
-        Parameters: is_change: if true, return the change addresses else the receive ones.
+        Parameters:
+            * is_change: if true, return the change addresses else the receive ones.
+            * service_id: just return addresses associated for the given Service
+            * include_wallet_alias: adds `wallet_alias` to each output (useful when this
+                is called by WalletManager.full_addresses_info() across all Wallets)
         """
 
         addresses_info = []
@@ -1973,7 +2032,6 @@ class Wallet:
         ]
 
         for addr in addresses_cache:
-
             addr_utxo = 0
             addr_amount = 0
 
@@ -1983,17 +2041,26 @@ class Wallet:
                 addr_amount = addr_amount + utxo["amount"]
                 addr_utxo = addr_utxo + 1
 
-            addresses_info.append(
-                {
-                    "index": addr.index,
-                    "address": addr.address,
-                    "label": addr.label,
-                    "amount": addr_amount,
-                    "used": bool(addr.used),
-                    "utxo": addr_utxo,
-                    "type": "change" if is_change else "receive",
-                }
-            )
+            if service_id and (
+                "service_id" not in addr or addr["service_id"] != service_id
+            ):
+                # Filter this address out
+                continue
+
+            addr_info = {
+                "index": addr.index,
+                "address": addr.address,
+                "label": addr.label,
+                "amount": addr_amount,
+                "used": bool(addr.used),
+                "utxo": addr_utxo,
+                "type": "change" if is_change else "receive",
+                "service_id": addr.service_id,
+            }
+            if include_wallet_alias:
+                addr_info["wallet_alias"] = self.alias
+
+            addresses_info.append(addr_info)
 
         return addresses_info
 

tests/conftest.py

@@ -10,6 +10,7 @@ import time
 import docker
 import pytest
 from cryptoadvance.specter.managers.device_manager import DeviceManager
+from cryptoadvance.specter.managers.user_manager import UserManager
 from cryptoadvance.specter.process_controller.bitcoind_controller import (
     BitcoindPlainController,
 )
@@ -20,7 +21,7 @@ from cryptoadvance.specter.process_controller.elementsd_controller import (
     ElementsPlainController,
 )
 from cryptoadvance.specter.rpc import BitcoinRPC
-from cryptoadvance.specter.server import create_app, init_app
+from cryptoadvance.specter.server import SpecterFlask, create_app, init_app
 from cryptoadvance.specter.specter import Specter
 from cryptoadvance.specter.specter_error import SpecterError
 from cryptoadvance.specter.user import User, hash_password
@@ -368,6 +369,34 @@ def device_manager(devices_filled_data_folder):
     return DeviceManager(os.path.join(devices_filled_data_folder, "devices"))
 
 
+# @pytest.fixture
+# def user_manager(empty_data_folder) -> UserManager:
+#     """A UserManager having users alice, bob and eve"""
+#     specter = Specter(data_folder=empty_data_folder)
+#     user_manager = UserManager(specter=specter)
+#     config = {}
+#     user_manager.get_user("admin").decrypt_user_secret("admin")
+#     user_manager.create_user(
+#         user_id="alice",
+#         username="alice",
+#         plaintext_password="plain_pass_alice",
+#         config=config,
+#     )
+#     user_manager.create_user(
+#         user_id="bob",
+#         username="bob",
+#         plaintext_password="plain_pass_bob",
+#         config=config,
+#     )
+#     user_manager.create_user(
+#         user_id="eve",
+#         username="eve",
+#         plaintext_password="plain_pass_eve",
+#         config=config,
+#     )
+#     return user_manager
+
+
 @pytest.fixture
 def specter_regtest_configured(bitcoin_regtest, devices_filled_data_folder):
     assert bitcoin_regtest.get_rpc().test_connection()
@@ -447,7 +476,7 @@ def specter_regtest_configured(bitcoin_regtest, devices_filled_data_folder):
 
 
 @pytest.fixture
-def app(specter_regtest_configured):
+def app(specter_regtest_configured) -> SpecterFlask:
     """the Flask-App, but uninitialized"""
     app = create_app(config="cryptoadvance.specter.config.TestConfig")
     app.app_context().push()
@@ -459,6 +488,19 @@ def app(specter_regtest_configured):
     return app
 
 
+@pytest.fixture
+def app_no_node(empty_data_folder) -> SpecterFlask:
+    specter = Specter(data_folder=empty_data_folder)
+    app = create_app(config="cryptoadvance.specter.config.TestConfig")
+    app.app_context().push()
+    app.config["TESTING"] = True
+    app.testing = True
+    app.tor_service_id = None
+    app.tor_enabled = False
+    init_app(app, specter=specter)
+    return app
+
+
 @pytest.fixture
 def client(app):
     """a test_client from an initialized Flask-App"""

tests/install_noded.sh

@@ -153,9 +153,10 @@ function build_node_impl {
 function sub_help {
     echo "This script will result in having bitcoind or elementsd binaries, either by binary download or via compilation"
     echo "Do one of these:"
-    echo "$ ./install_noded.sh --bitcoin compile"
-    echo "$ ./install_noded.sh --elements compile"
-    echo "$ ./install_noded.sh --bitcoin binary # only works for bitcoind currently, no binaries for elements"
+    echo "$ ./install_node.sh --bitcoin binary"
+    echo "$ ./install_node.sh --bitcoin compile"
+    echo "$ ./install_node.sh --elements binary"
+    echo "$ ./install_node.sh --elements compile"
     echo "For more context, see https://github.com/cryptoadvance/specter-desktop/blob/master/docs/development.md#how-to-run-the-tests"
 }
 
@@ -266,7 +267,7 @@ function sub_binary {
 
 function parse_and_execute() {
   if [[ $# = 0 ]]; then
-    sub_default
+    sub_help
     exit 0
   fi 
 

tests/test_services.py

@@ -0,0 +1,218 @@
+import json
+import pytest
+
+from flask_login import current_user
+from flask_login.utils import login_user, logout_user
+from mock import patch, Mock
+from tempfile import TemporaryDirectory
+from unittest.mock import MagicMock
+from unittest import TestCase
+
+from cryptoadvance.specter.managers.user_manager import UserManager
+from cryptoadvance.specter.server import SpecterFlask
+from cryptoadvance.specter.services.service import Service
+from cryptoadvance.specter.services.service_encrypted_storage import (
+    ServiceEncryptedStorage,
+    ServiceEncryptedStorageError,
+    ServiceEncryptedStorageManager,
+)
+from cryptoadvance.specter.services.service_manager import ServiceManager
+from cryptoadvance.specter.user import User, hash_password
+
+
+class FakeService(Service):
+    # A dummy Service just used by the test suite
+    id = "test_service"
+    name = "Test Service"
+    has_blueprint = False
+
+
+# @patch("cryptoadvance.specter.services.service_manager.app")
+# def test_ServiceManager_loads_services(empty_data_folder, app):
+#     # app.config = MagicMock()
+#     # app.config.get.return_value = "prod"
+#     specter_mock = MagicMock()
+#     specter_mock.data_folder.return_value = empty_data_folder
+
+#     service_manager = ServiceManager(specter=specter_mock, devstatus_threshold="alpha")
+#     services = service_manager.services
+#     assert "swan" in services
+
+
+def test_ServiceEncryptedStorage(empty_data_folder):
+    specter_mock = Mock()
+    specter_mock.config = {"uid": ""}
+    specter_mock.user_manager = Mock()
+    specter_mock.user_manager.users = [""]
+
+    user1 = User.from_json(
+        user_dict={
+            "id": "user1",
+            "username": "user1",
+            "password": hash_password("somepassword"),
+            "config": {},
+            "is_admin": False,
+            "services": None,
+        },
+        specter=specter_mock,
+    )
+    user2 = User.from_json(
+        user_dict={
+            "id": "user2",
+            "username": "user2",
+            "password": hash_password("somepassword"),
+            "config": {},
+            "is_admin": False,
+            "services": None,
+        },
+        specter=specter_mock,
+    )
+    user1._generate_user_secret("muh")
+
+    # Can set and get service storage fields
+    service_storage = ServiceEncryptedStorage(empty_data_folder, user1)
+    service_storage.set_service_data("a_service_id", {"somekey": "green"})
+    assert service_storage.get_service_data("a_service_id") == {"somekey": "green"}
+    assert service_storage.get_service_data("another_service_id") == {}
+
+    # We expect a call for a user that isn't logged in to fail
+    with pytest.raises(ServiceEncryptedStorageError) as execinfo:
+        ServiceEncryptedStorage(empty_data_folder, user2)
+    assert "must be authenticated with password" in str(execinfo.value)
+
+
+def test_access_encrypted_storage_after_login(app_no_node: SpecterFlask):
+    """ServiceEncryptedStorage should be accessible (decryptable) after user login"""
+    # Create test users; automatically generates their `user_secret` and kept decrypted
+    # in memory.
+    user_manager: UserManager = app_no_node.specter.user_manager
+    user_manager.create_user(
+        user_id="bob",
+        username="bob",
+        plaintext_password="plain_pass_bob",
+        config={},
+    )
+    user_manager.create_user(
+        user_id="alice",
+        username="alice",
+        plaintext_password="plain_pass_alice",
+        config={},
+    )
+
+    # The ServiceEncryptedStorageManager singleton creates a mess in the test suite. Have
+    # to override its (potentially) already-set values with references to this test's
+    # new instances and temp storage dir.
+    storage_manager = ServiceEncryptedStorageManager.get_instance()
+    storage_manager.data_folder = user_manager.data_folder
+    storage_manager.user_manager = user_manager
+    storage_manager.storage_by_user = {}
+
+    # Need a simulated request context to enable `current_user` lookup
+    with app_no_node.test_request_context():
+        login_user(user_manager.get_user("bob"))
+
+        # Test user's service_data should be decryptable but initially empty
+        service_data = storage_manager.get_current_user_service_data(
+            service_id=FakeService.id
+        )
+        assert service_data == {}
+        storage_manager.update_current_user_service_data(
+            service_id=FakeService.id, service_data={"somekey": "green"}
+        )
+
+        logout_user()
+
+        # Meanwhile Alice's account storage will still be blank...
+        login_user(user_manager.get_user("alice"))
+        service_data = storage_manager.get_current_user_service_data(
+            service_id=FakeService.id
+        )
+        assert (
+            storage_manager.get_current_user_service_data(service_id=FakeService.id)
+            == {}
+        )
+
+        logout_user()
+
+        # ...while Bob's can be retrieved when he's logged in.
+        login_user(user_manager.get_user("bob"))
+        service_data = storage_manager.get_current_user_service_data(
+            service_id=FakeService.id
+        )
+        assert storage_manager.get_current_user_service_data(
+            service_id=FakeService.id
+        ) == {"somekey": "green"}
+
+
+def test_remove_all_services_from_user(app_no_node: SpecterFlask, empty_data_folder):
+    """ServiceEncryptedStorage should be accessible (decryptable) after user login"""
+    # Create test users; automatically generates their `user_secret` and kept decrypted
+    # in memory.
+    user_manager: UserManager = app_no_node.specter.user_manager
+    user_manager.create_user(
+        user_id="bob",
+        username="bob",
+        plaintext_password="plain_pass_bob",
+        config={},
+    )
+
+    # The ServiceEncryptedStorageManager singleton creates a mess in the test suite. Have
+    # to override its (potentially) already-set values with references to this test's
+    # new instances and temp storage dir.
+    storage_manager = ServiceEncryptedStorageManager.get_instance()
+    storage_manager.data_folder = user_manager.data_folder
+    storage_manager.user_manager = user_manager
+    storage_manager.storage_by_user = {}
+
+    # Need a simulated request context to enable `current_user` lookup
+    with app_no_node.test_request_context():
+        user = user_manager.get_user("bob")
+        login_user(user)
+
+        # Test user's service_data should be decryptable but initially empty
+        service_data = storage_manager.get_current_user_service_data(
+            service_id=FakeService.id
+        )
+        assert service_data == {}
+        storage_manager.update_current_user_service_data(
+            service_id=FakeService.id, service_data={"somekey": "green"}
+        )
+
+        # Check on disk. The <user>_services.json should now have our data.
+        service_encrypted_storage_file = (
+            storage_manager._get_current_user_service_storage().data_file
+        )
+        with open(service_encrypted_storage_file, "r") as storage_json_file:
+            data_on_disk = json.load(storage_json_file)
+
+        # Can't test the actual values because they're encrypted, but the Service.id key is plaintext
+        assert FakeService.id in data_on_disk
+
+        # Now remove all
+        app_no_node.specter.service_manager.remove_all_services_from_user(user)
+
+        # Verify data on disk; Bob's user should have his user_secret cleared.
+        users_file = app_no_node.specter.user_manager.users_file
+        with open(users_file, "r") as storage_json_file:
+            data_on_disk = json.load(storage_json_file)
+        found_bob = False
+        for user_entry in data_on_disk:
+            if user_entry["id"] == "bob":
+                found_bob = True
+                assert user_entry.get("encrypted_user_secret") is None
+        assert found_bob
+
+        # With no `user_secret` the decryption attempt must fail.
+        with pytest.raises(ServiceEncryptedStorageError) as execinfo:
+            storage_manager._get_current_user_service_storage()
+        assert "must be authenticated with password" in str(execinfo.value)
+
+        # Should now be cleared in memory (have to request raw data since we can't
+        # decrypt)...
+        service_encrypted_storage = storage_manager.get_raw_encrypted_data(user)
+        assert service_encrypted_storage == {}
+
+        # ...and on disk
+        with open(service_encrypted_storage_file, "r") as storage_json_file:
+            data_on_disk = json.load(storage_json_file)
+        assert data_on_disk == {}

tests/test_user.py

@@ -1,7 +1,6 @@
 import json, logging, pytest
 from cryptoadvance.specter.specter import Specter
 from cryptoadvance.specter.user import User, hash_password, verify_password
-from cryptoadvance.specter.managers.user_manager import UserManager
 
 
 def test_password_hash():

tests/test_util_reflection.py

@@ -0,0 +1,37 @@
+import logging
+from pathlib import Path
+from cryptoadvance.specter.util.reflection import (
+    get_subclasses_for_class,
+    _get_module_from_class,
+    get_package_dir_for_subclasses_of,
+)
+from cryptoadvance.specter.util.specter_migrator import SpecterMigration
+from cryptoadvance.specter.util.migrations.migration_0000 import SpecterMigration_0000
+from cryptoadvance.specter.services.service_manager import Service
+from cryptoadvance.specter.services.swan.service import SwanService
+from cryptoadvance.specter.services.bitcoinreserve.service import BitcoinReserveService
+
+
+def test_get_module_from_class():
+    assert (
+        _get_module_from_class(SpecterMigration).__name__
+        == "cryptoadvance.specter.util.specter_migrator"
+    )
+
+
+def test_get_package_dir_for_subclasses_of():
+    assert get_package_dir_for_subclasses_of(SpecterMigration).endswith(
+        "cryptoadvance/specter/util/migrations"
+    )
+    assert get_package_dir_for_subclasses_of(Service).endswith(
+        "cryptoadvance/specter/services"
+    )
+
+
+def test_get_subclasses_in_packagedir(caplog):
+    caplog.set_level(logging.INFO)
+    classlist = get_subclasses_for_class(SpecterMigration)
+    assert SpecterMigration_0000 in classlist
+    classlist = get_subclasses_for_class(Service)
+    assert SwanService in classlist
+    assert BitcoinReserveService in classlist