GlcoinCore/labelbase
1
0
Fork 0
mirror of https://github.com/Labelbase/Labelbase.git synced 2026-08-14 12:43:22 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Compare commits

base: GlcoinCore:master
Branches Tags
GlcoinCore:master
GlcoinCore:legacy
GlcoinCore:2.3.0
GlcoinCore:2.2.3
GlcoinCore:2.2.2
GlcoinCore:2.2.1
GlcoinCore:2.2.0
GlcoinCore:2.1.1
GlcoinCore:2.1.0
GlcoinCore:2.0.0
..
compare: GlcoinCore:2.0.0
Branches Tags
GlcoinCore:master
GlcoinCore:legacy
GlcoinCore:2.3.0
GlcoinCore:2.2.3
GlcoinCore:2.2.2
GlcoinCore:2.2.1
GlcoinCore:2.2.0
GlcoinCore:2.1.1
GlcoinCore:2.1.0
GlcoinCore:2.0.0

No commits in common. "master" and "2.0.0" have entirely different histories.
master ... 2.0.0

192 changed files with 948 additions and 21247 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.dockerignore
View file

@ -7,7 +7,6 @@ db/
*.pyc
__pycache__
django/importer/uploadeddata/*
django/attachments/attachment/*
_scratches
exports.sh
.last_git_commit

2
.gitignore vendored
View file

@ -5,11 +5,9 @@ labelbase.log
labelbase.log.*
bgt.log
db/
backup_*
*.pyc
__pycache__
django/importer/uploadeddata/*
django/attachments/attachment/*
#_scratches
exports.sh
.last_git_commit

896
BACKUP_AND_MIGRATION_GUIDE.md
View file

@ -1,896 +0,0 @@
# Labelbase Backup and Migration Guide
A comprehensive guide for safely backing up your Labelbase database and running Django migrations.
## Table of Contents
- [Why Backup?](#why-backup)
- [Quick Backup](#quick-backup)
- [Automated Backup Script](#automated-backup-script)
- [Running Migrations Safely](#running-migrations-safely)
- [Upgrading Labelbase](#upgrading-labelbase)
- [Restoring from Backup](#restoring-from-backup)
- [Scheduled Backups](#scheduled-backups)
- [Best Practices](#best-practices)
---
## Why Backup?
**Always backup before running migrations!** Migrations can:
- Alter database table structures in irreversible ways
- Delete data if there are bugs in the migration code
- Fail mid-execution, leaving your database inconsistent
- Introduce conflicts with existing data
A backup takes 30 seconds. Recovery without one could take hours or days.
---
## Quick Backup
From your Labelbase directory:
```bash
# Navigate to Labelbase directory
cd Labelbase
# Source environment variables
source exports.sh
# Create backup with timestamp
docker-compose exec -T labelbase_mysql mysqldump -u root -p"${MYSQL_ROOT_PASSWORD}" labelbase > backup_$(date +%Y%m%d_%H%M%S).sql
```
This creates a backup file like `backup_20250119_143022.sql`.
### Backup config.ini (Important!)
The `config.ini` file contains encryption keys and other critical settings. Always back it up too:
```bash
# Navigate to Labelbase directory
cd Labelbase
# Source environment variables
source exports.sh
# Create backup with timestamp
docker-compose exec -T labelbase_django cat /app/config.ini > backup_$(date +%Y%m%d_%H%M%S)_config.ini
```
This creates a backup file like `backup_20250119_143022_config.ini`.
---
## Automated Backup Script
Create a reusable backup script that handles everything automatically.
### Create the Script
Save this as `backup-labelbase.sh` in your Labelbase directory:
```bash
#!/bin/bash
# Labelbase Database Backup Script
# Usage: ./backup-labelbase.sh
# Configuration
LABELBASE_DIR="/path/to/Labelbase" # CHANGE THIS to your actual path
BACKUP_DIR="$LABELBASE_DIR/backups"
KEEP_BACKUPS=10 # Number of backups to keep
# Colors for output
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Change to Labelbase directory
cd "$LABELBASE_DIR" || exit 1
# Source environment variables for MySQL passwords
if [ ! -f "exports.sh" ]; then
echo -e "${RED}✗ Error: exports.sh not found!${NC}"
echo "Make sure you're in the Labelbase directory and exports.sh exists."
exit 1
fi
source exports.sh
# Check if MySQL password is set
if [ -z "$MYSQL_ROOT_PASSWORD" ]; then
echo -e "${RED}✗ Error: MYSQL_ROOT_PASSWORD not set!${NC}"
echo "Make sure exports.sh contains the MySQL password."
exit 1
fi
# Create backup directory if it doesn't exist
mkdir -p "$BACKUP_DIR"
# Generate backup filename with timestamp
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="$BACKUP_DIR/labelbase_backup_$TIMESTAMP.sql"
CONFIG_BACKUP_FILE="$BACKUP_DIR/config_backup_$TIMESTAMP.ini"
echo -e "${YELLOW}Starting backup...${NC}"
echo "Database backup: $BACKUP_FILE"
echo "Config backup: $CONFIG_BACKUP_FILE"
# Backup config.ini first (contains encryption keys!)
echo "Backing up config.ini..."
docker-compose exec -T labelbase_django cat /app/config.ini > "$CONFIG_BACKUP_FILE"
if [ $? -eq 0 ] && [ -s "$CONFIG_BACKUP_FILE" ]; then
echo -e "${GREEN}✓ Config backup successful${NC}"
else
echo -e "${YELLOW}⚠ Config backup failed or file is empty${NC}"
fi
# Create database backup
docker-compose exec -T labelbase_mysql mysqldump \
-u root \
-p"${MYSQL_ROOT_PASSWORD}" \
--single-transaction \
--quick \
--lock-tables=false \
labelbase > "$BACKUP_FILE"
# Check if backup was successful
if [ $? -eq 0 ] && [ -s "$BACKUP_FILE" ]; then
echo -e "${GREEN}✓ Backup successful!${NC}"
# Get file size
SIZE=$(du -h "$BACKUP_FILE" | cut -f1)
echo "Backup size: $SIZE"
# Compress backup to save space
echo "Compressing backup..."
gzip "$BACKUP_FILE"
if [ $? -eq 0 ]; then
COMPRESSED_SIZE=$(du -h "${BACKUP_FILE}.gz" | cut -f1)
echo -e "${GREEN}✓ Compressed to: $COMPRESSED_SIZE${NC}"
echo "Backup location: ${BACKUP_FILE}.gz"
else
echo -e "${YELLOW}⚠ Compression failed, keeping uncompressed backup${NC}"
fi
# Clean up old backups (keep only last N backups)
echo "Cleaning up old backups (keeping last $KEEP_BACKUPS)..."
BACKUP_COUNT=$(ls -1 "$BACKUP_DIR"/labelbase_backup_*.sql.gz 2>/dev/null | wc -l)
if [ "$BACKUP_COUNT" -gt "$KEEP_BACKUPS" ]; then
ls -t "$BACKUP_DIR"/labelbase_backup_*.sql.gz | tail -n +$((KEEP_BACKUPS + 1)) | xargs -r rm
echo -e "${GREEN}✓ Cleaned up old backups${NC}"
else
echo "No cleanup needed ($BACKUP_COUNT backups exist)"
fi
echo ""
echo -e "${GREEN}=== Backup Complete ===${NC}"
echo "Database: ${BACKUP_FILE}.gz"
echo "Config: ${CONFIG_BACKUP_FILE}"
else
echo -e "${RED}✗ Backup failed!${NC}"
# Remove empty or failed backup file
[ -f "$BACKUP_FILE" ] && rm "$BACKUP_FILE"
echo "Troubleshooting:"
echo "1. Check if MySQL container is running: docker-compose ps"
echo "2. Check MySQL logs: docker-compose logs labelbase_mysql"
echo "3. Verify password in exports.sh"
exit 1
fi
```
### Make Script Executable
```bash
chmod +x backup-labelbase.sh
```
### Edit Configuration
Open `backup-labelbase.sh` and change this line to your actual Labelbase path:
```bash
LABELBASE_DIR="/path/to/Labelbase" # CHANGE THIS!
```
For example:
```bash
LABELBASE_DIR="/root/Labelbase"
# or
LABELBASE_DIR="/home/username/Labelbase"
```
### Run the Backup
```bash
./backup-labelbase.sh
```
You'll see output like:
```
Starting backup...
Backup file: /path/to/Labelbase/backups/labelbase_backup_20250119_143022.sql
✓ Backup successful!
Backup size: 15M
Compressing backup...
✓ Compressed to: 3.2M
Backup location: /path/to/Labelbase/backups/labelbase_backup_20250119_143022.sql.gz
Cleaning up old backups (keeping last 10)...
No cleanup needed (3 backups exist)
=== Backup Complete ===
Location: /path/to/Labelbase/backups/labelbase_backup_20250119_143022.sql.gz
```
---
## Running Migrations Safely
**Always follow this order:**
### Step 1: Create a Backup
```bash
./backup-labelbase.sh
```
### Step 2: Check Migration Status
```bash
source exports.sh
docker-compose exec labelbase_django python manage.py showmigrations
```
This shows which migrations are applied (marked with `[X]`) and pending (marked with `[ ]`).
### Step 3: Review Pending Migrations
Look for any unapplied migrations. If you see pending migrations for critical apps, review them carefully.
### Step 4: Apply Migrations
```bash
# If you've modified models, create new migrations first
docker-compose exec labelbase_django python manage.py makemigrations
# Apply all pending migrations
docker-compose exec labelbase_django python manage.py migrate
```
### Step 5: Verify Application
After migrations complete:
1. Check for any error messages
2. Visit your Labelbase site
3. Test critical functionality
4. Check Django logs: `docker-compose logs labelbase_django`
### Step 6: If Something Goes Wrong
If migrations fail or break functionality, restore from backup (see below).
---
## Upgrading Labelbase
When new versions of Labelbase are released, follow this workflow to safely upgrade.
### Complete Upgrade Workflow
**Step 1: Backup First (Critical!)**
```bash
cd Labelbase
./backup-labelbase.sh
```
**Step 2: Pull Latest Code**
```bash
git pull origin master
```
**Step 3: Rebuild Containers (if needed)**
If dependencies or Docker configuration changed:
```bash
source exports.sh && docker-compose up --build -d
```
Or use the main script:
```bash
source exports.sh && ./build-and-run-labelbase.sh
```
**⚠️ IMPORTANT**: These commands are SAFE - they rebuild containers but preserve your data in Docker volumes. Your database and uploaded files are NOT deleted.
**❌ DANGER ZONE - Commands that DELETE data:**
```bash
# NEVER run these unless you want to lose ALL data:
docker-compose down -v # The -v flag deletes volumes = data loss!
docker volume prune # Deletes unused volumes
docker system prune -a # Nuclear option - deletes everything
```
**Step 4: Apply Migrations and collect static files **
```bash
source exports.sh
docker-compose exec labelbase_django python manage.py showmigrations
docker-compose exec labelbase_django python manage.py migrate
docker-compose exec labelbase_django python manage.py collectstatic --noinput
```
**Step 5: Restart Services**
```bash
docker-compose restart labelbase_django
```
**Step 6: Verify Everything Works**
1. Visit your Labelbase site
2. Test critical functionality
3. Check logs: `docker-compose logs -f labelbase_django`
### Quick Upgrade Script
Create `update-and-migrate.sh` for a streamlined upgrade process:
```bash
#!/bin/bash
# Labelbase Quick Update & Migration Script
# Usage: source exports.sh && ./update-and-migrate.sh
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m'
echo -e "${YELLOW}=== Labelbase Update & Migrate ===${NC}"
# Check if we're in the right directory
if [ ! -f "docker-compose.yml" ]; then
echo -e "${RED}Error: Not in Labelbase directory (docker-compose.yml not found)${NC}"
exit 1
fi
# Check env vars
if [[ -z "${MYSQL_ROOT_PASSWORD}" ]]; then
echo -e "${RED}Error: Run 'source exports.sh' first!${NC}"
exit 1
fi
# Step 1: Backup
echo -e "${YELLOW}Step 1: Creating backup...${NC}"
if [ -f "backup-labelbase.sh" ]; then
./backup-labelbase.sh
if [ $? -ne 0 ]; then
echo -e "${RED}Backup failed! Aborting upgrade.${NC}"
exit 1
fi
else
echo -e "${YELLOW}⚠ Warning: backup-labelbase.sh not found, skipping backup${NC}"
read -p "Continue without backup? (yes/no): " -r
if [[ ! $REPLY =~ ^[Yy][Ee][Ss]$ ]]; then
echo "Upgrade cancelled."
exit 0
fi
fi
# Step 2: Pull latest code
echo ""
echo -e "${YELLOW}Step 2: Pulling latest changes...${NC}"
git pull origin master
if [ $? -ne 0 ]; then
echo -e "${RED}Git pull failed!${NC}"
exit 1
fi
# Step 3: Check for pending migrations
echo ""
echo -e "${YELLOW}Step 3: Checking for migrations...${NC}"
PENDING=$(docker-compose exec -T labelbase_django python manage.py showmigrations --plan 2>/dev/null | grep "\[ \]" | wc -l)
if [ $PENDING -gt 0 ]; then
echo -e "${YELLOW}Found $PENDING pending migration(s)${NC}"
# Show what will be migrated
echo "Pending migrations:"
docker-compose exec -T labelbase_django python manage.py showmigrations | grep "\[ \]"
echo ""
read -p "Apply migrations now? (y/n): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
echo "Applying migrations..."
docker-compose exec -T labelbase_django python manage.py migrate --noinput
if [ $? -eq 0 ]; then
echo -e "${GREEN}✓ Migrations applied successfully${NC}"
else
echo -e "${RED}✗ Migrations failed!${NC}"
echo "Check logs: docker-compose logs labelbase_django"
exit 1
fi
else
echo -e "${YELLOW}⚠ Skipping migrations${NC}"
echo "Run manually later: docker-compose exec labelbase_django python manage.py migrate"
fi
else
echo -e "${GREEN}✓ No pending migrations${NC}"
fi
# Step 4: Restart Django
echo ""
echo -e "${YELLOW}Step 4: Restarting Django...${NC}"
docker-compose restart labelbase_django
echo ""
echo -e "${GREEN}=== Update Complete! ===${NC}"
echo "Check logs: docker-compose logs -f labelbase_django"
echo "Visit your site to verify everything works"
```
Make it executable:
```bash
chmod +x update-and-migrate.sh
```
### Using the Quick Upgrade Script
```bash
# Navigate to Labelbase
cd Labelbase
# Source environment and run update
source exports.sh && ./update-and-migrate.sh
```
The script will:
1. ✓ Create automatic backup
2. ✓ Pull latest code from git
3. ✓ Detect pending migrations
4. ✓ Ask for confirmation before applying
5. ✓ Restart services
6. ✓ Provide verification steps
### When to Rebuild vs. Restart
**Just restart** (`docker-compose restart`) when:
- Only Django code changed (Python files)
- No dependency updates
- No Dockerfile changes
- Fastest option
**Full rebuild** (`docker-compose up --build -d`) when:
- requirements.txt changed
- Dockerfile modified
- New system packages needed
- Docker configuration changed
**Data Safety Note**: Both `restart` and `--build` are SAFE - they preserve your data. Docker stores your database and files in **volumes** that persist across rebuilds.
If unsure, rebuild - it's safer and only takes a minute longer.
### What Actually Deletes Data
Only these commands delete data (requires `-v` flag):
```bash
# DANGER: This deletes ALL data including database!
docker-compose down -v
# To safely stop without deleting data, use:
docker-compose down # Safe - keeps volumes
docker-compose stop # Safe - just stops containers
```
**Rule of thumb**: If you see `-v` flag, your data is at risk!
### Rollback After Failed Upgrade
If something goes wrong:
```bash
# 1. Stop services
docker-compose down
# 2. Restore previous code
git reset --hard HEAD~1
# 3. Restore database
./restore-labelbase.sh backups/labelbase_backup_TIMESTAMP.sql.gz
# 4. Restart
source exports.sh && docker-compose up -d
```
---
## Restoring from Backup
If something goes wrong, you can restore your database from a backup.
### Quick Restore
```bash
# Navigate to Labelbase directory
cd Labelbase
# Source environment variables
source exports.sh
# Stop Django to prevent conflicts
docker-compose stop labelbase_django
# Decompress and restore backup
gunzip -c backups/labelbase_backup_20250119_143022.sql.gz | \
docker-compose exec -T labelbase_mysql mysql -u root -p"${MYSQL_ROOT_PASSWORD}" labelbase
# Restart all services
docker-compose up -d
# Check logs
docker-compose logs -f labelbase_django
```
### Restore Script (Optional)
Create `restore-labelbase.sh`:
```bash
#!/bin/bash
# Labelbase Database Restore Script
# Usage: ./restore-labelbase.sh <backup-file>
LABELBASE_DIR="/path/to/Labelbase" # CHANGE THIS
BACKUP_FILE="$1"
# Colors
GREEN='\033[0;32m'
RED='\033[0;31m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Check if backup file provided
if [ -z "$BACKUP_FILE" ]; then
echo -e "${RED}✗ Error: No backup file specified${NC}"
echo "Usage: ./restore-labelbase.sh <backup-file>"
echo ""
echo "Available backups:"
ls -lh "$LABELBASE_DIR/backups/"*.sql.gz 2>/dev/null || echo "No backups found"
exit 1
fi
# Check if backup file exists
if [ ! -f "$BACKUP_FILE" ]; then
echo -e "${RED}✗ Error: Backup file not found: $BACKUP_FILE${NC}"
exit 1
fi
# Change to Labelbase directory
cd "$LABELBASE_DIR" || exit 1
# Source environment variables
source exports.sh
echo -e "${YELLOW}⚠ WARNING: This will overwrite your current database!${NC}"
echo "Backup file: $BACKUP_FILE"
read -p "Are you sure you want to continue? (yes/no): " -r
echo
if [[ ! $REPLY =~ ^[Yy][Ee][Ss]$ ]]; then
echo "Restore cancelled."
exit 0
fi
echo "Stopping Django container..."
docker-compose stop labelbase_django
echo "Restoring database..."
# Check if file is compressed
if [[ "$BACKUP_FILE" == *.gz ]]; then
gunzip -c "$BACKUP_FILE" | \
docker-compose exec -T labelbase_mysql mysql -u root -p"${MYSQL_ROOT_PASSWORD}" labelbase
else
cat "$BACKUP_FILE" | \
docker-compose exec -T labelbase_mysql mysql -u root -p"${MYSQL_ROOT_PASSWORD}" labelbase
fi
if [ $? -eq 0 ]; then
echo -e "${GREEN}✓ Database restored successfully${NC}"
else
echo -e "${RED}✗ Restore failed${NC}"
exit 1
fi
echo "Restarting services..."
docker-compose up -d
# Optional: Restore config.ini if you have a backup from the same time
# CONFIG_FILE="${BACKUP_FILE%_backup_*}_config_backup_${BACKUP_FILE##*_backup_}"
# CONFIG_FILE="${CONFIG_FILE%.sql.gz}.ini"
# if [ -f "$CONFIG_FILE" ]; then
# echo "Found config backup: $CONFIG_FILE"
# read -p "Restore config.ini too? (y/n) " -n 1 -r
# echo
# if [[ $REPLY =~ ^[Yy]$ ]]; then
# cat "$CONFIG_FILE" | docker-compose exec -T labelbase_django bash -c "cat > /app/config.ini"
# echo -e "${GREEN}✓ Config restored${NC}"
# docker-compose restart labelbase_django
# fi
# fi
echo ""
echo -e "${GREEN}=== Restore Complete ===${NC}"
echo "Check logs with: docker-compose logs -f labelbase_django"
```
Make executable and configure:
```bash
chmod +x restore-labelbase.sh
# Edit LABELBASE_DIR in the script
```
Usage:
```bash
./restore-labelbase.sh backups/labelbase_backup_20250119_143022.sql.gz
```
---
## Scheduled Backups
### Using Cron (Linux/Unix)
Automate daily backups at 2 AM:
```bash
# Edit crontab
crontab -e
# Add this line (adjust path to your Labelbase directory)
0 2 * * * cd /path/to/Labelbase && ./backup-labelbase.sh >> /var/log/labelbase-backup.log 2>&1
```
This runs the backup script daily at 2:00 AM and logs output to `/var/log/labelbase-backup.log`.
### Verify Cron Job
```bash
# List current cron jobs
crontab -l
# Check backup log
tail -f /var/log/labelbase-backup.log
```
### Alternative: Weekly Backups
```bash
# Every Sunday at 3 AM
0 3 * * 0 cd /path/to/Labelbase && ./backup-labelbase.sh >> /var/log/labelbase-backup.log 2>&1
```
---
## Best Practices
### Before Migrations
1. ✓ **Always create a backup first**
2. ✓ Review what migrations will be applied
3. ✓ Have a rollback plan ready
4. ✓ Test migrations on a development copy if possible
5. ✓ Schedule migrations during low-traffic periods
### Backup Storage
1. ✓ Keep backups in multiple locations
2. ✓ Regularly test your restore process
3. ✓ Keep at least 7-10 recent backups
4. ✓ Store critical backups off-server (external drive, cloud storage)
5. ✓ Monitor backup script success/failure
### Security
1. ✓ Protect `exports.sh` - it contains database passwords
2. ✓ Secure backup files - they contain all your data
3. ✓ Protect `config.ini` backups - they contain encryption keys
4. ✓ Use appropriate file permissions:
```bash
chmod 600 exports.sh
chmod 700 backups/
chmod 600 backups/*.sql.gz
chmod 600 backups/*_config.ini
```
### Regular Maintenance
1. ✓ Run backups before any system updates
2. ✓ Test restore process quarterly
3. ✓ Monitor backup file sizes (unexpected changes may indicate issues)
4. ✓ Keep backup logs for troubleshooting
### Docker Data Safety
1. ✓ **SAFE commands** (preserve data):
- `docker-compose up --build -d` - Rebuild containers
- `docker-compose restart` - Restart services
- `docker-compose down` - Stop without deleting volumes
- `docker-compose stop` - Pause containers
2. ✓ **DANGEROUS commands** (delete data):
- `docker-compose down -v` - ⚠️ Deletes ALL volumes/data
- `docker volume prune` - ⚠️ Removes unused volumes
- `docker system prune -a` - ⚠️ Nuclear option
3. ✓ **Remember**: The `-v` flag means "delete volumes" = data loss!
---
## Troubleshooting
### "MYSQL_ROOT_PASSWORD not set"
**Cause**: `exports.sh` not sourced or doesn't contain password
**Solution**:
```bash
# Check if exports.sh exists
ls -la exports.sh
# Source it
source exports.sh
# Verify password is set
echo $MYSQL_ROOT_PASSWORD
```
### "Access denied for user 'root'"
**Cause**: Wrong password in `exports.sh`
**Solution**:
1. Check MySQL container logs: `docker-compose logs labelbase_mysql`
2. Verify password in `exports.sh` matches what MySQL expects
3. If lost, you may need to reset MySQL root password
### Backup File is Empty or Very Small
**Cause**: MySQL container not running or database empty
**Solution**:
```bash
# Check container status
docker-compose ps
# Check MySQL logs
docker-compose logs labelbase_mysql
# Verify database exists
docker-compose exec labelbase_mysql mysql -u root -p -e "SHOW DATABASES;"
```
### Restore Fails with "Unknown Database"
**Cause**: Database doesn't exist in MySQL
**Solution**:
```bash
# Create database first
docker-compose exec labelbase_mysql mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS labelbase;"
# Then restore
./restore-labelbase.sh backups/your_backup.sql.gz
```
---
## Complete Safe Migration Workflow
Here's the complete workflow combining backup and migration:
```bash
# 1. Navigate to Labelbase
cd Labelbase
# 2. Create backup
./backup-labelbase.sh
# 3. Source environment
source exports.sh
# 4. Check what migrations will run
docker-compose exec labelbase_django python manage.py showmigrations
# 5. Apply migrations
docker-compose exec labelbase_django python manage.py makemigrations
docker-compose exec labelbase_django python manage.py migrate
# 6. Check for errors
docker-compose logs labelbase_django | tail -50
# 7. Test your application
# Visit site and verify functionality
# 8. If problems occur, restore:
# ./restore-labelbase.sh backups/labelbase_backup_TIMESTAMP.sql.gz
```
---
## Quick Command Reference
```bash
# Quick upgrade (recommended)
source exports.sh && ./update-and-migrate.sh
# Manual upgrade workflow
./backup-labelbase.sh
git pull origin master
source exports.sh && docker-compose up --build -d
docker-compose exec labelbase_django python manage.py migrate
docker-compose restart labelbase_django
# Create backup (database + config.ini)
./backup-labelbase.sh
# Manual database backup
docker-compose exec -T labelbase_mysql mysqldump -u root -p"${MYSQL_ROOT_PASSWORD}" labelbase > backup_$(date +%Y%m%d_%H%M%S).sql
# Manual config.ini backup
docker-compose exec -T labelbase_django cat /app/config.ini > backup_$(date +%Y%m%d_%H%M%S)_config.ini
# List backups
ls -lh backups/
# Check migration status
docker-compose exec labelbase_django python manage.py showmigrations
# Apply migrations
docker-compose exec labelbase_django python manage.py migrate
# Restore backup
./restore-labelbase.sh backups/labelbase_backup_20250119_143022.sql.gz
# View recent Django logs
docker-compose logs labelbase_django | tail -100
# Access MySQL directly
docker-compose exec labelbase_mysql mysql -u root -p labelbase
```
---
## Additional Resources
- [Labelbase Development Guide](DEVELOPMENT_GUIDE.md)
- [Django Migrations Documentation](https://docs.djangoproject.com/en/stable/topics/migrations/)
- [mysqldump Documentation](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html)
---
## Support
If you encounter issues:
1. Check the troubleshooting section above
2. Review Docker logs: `docker-compose logs -f`
3. Verify all services are running: `docker-compose ps`
4. Check Labelbase GitHub issues: https://github.com/Labelbase/Labelbase/issues
---
**Remember: A backup today saves recovery tomorrow. Always backup before migrations!**

357
BARE_METALL_INSTALL.md
View file

@ -1,357 +0,0 @@
# Labelbase Bare Metal Installation Guide
This guide will help you install and run Labelbase directly on macOS and Linux systems, based on the RaspiBlitz installation script.
## Prerequisites
### Common Requirements
- Git
- Python 3.8 or higher
- pip (Python package manager)
- virtualenv
- MySQL/MariaDB server
### System-Specific Requirements
#### macOS
```bash
# Install Homebrew if not already installed
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install required packages
brew install python3 mysql git
brew install pkg-config mysql-client
```
#### Linux (Ubuntu/Debian)
```bash
# Update package list
sudo apt update
# Install required packages
sudo apt install -y \
mariadb-server \
mariadb-client \
default-libmysqlclient-dev \
build-essential \
python3 \
python3-pip \
python3-venv \
git \
libpcre3-dev
```
#### Linux (CentOS/RHEL/Fedora)
```bash
# For CentOS/RHEL 8+
sudo dnf install -y \
mariadb-server \
mariadb-devel \
python3 \
python3-pip \
python3-virtualenv \
git \
gcc \
gcc-c++ \
make
# For older versions, use yum instead of dnf
```
## Installation Steps
### 1. Create Application User (Optional but Recommended)
#### macOS
```bash
# Create a new user (optional on macOS for development)
sudo dscl . -create /Users/labelbase
sudo dscl . -create /Users/labelbase UserShell /bin/bash
sudo dscl . -create /Users/labelbase RealName "Labelbase User"
sudo dscl . -create /Users/labelbase UniqueID 1001
sudo dscl . -create /Users/labelbase PrimaryGroupID 20
sudo dscl . -create /Users/labelbase NFSHomeDirectory /Users/labelbase
sudo createhomedir -c -u labelbase
```
#### Linux
```bash
# Create system user
sudo adduser --system --group --shell /bin/bash --home /home/labelbase labelbase
sudo -u labelbase cp -r /etc/skel/. /home/labelbase/
```
### 2. Start Database Service
#### macOS
```bash
# Start MySQL service
brew services start mysql
# Secure the installation
mysql_secure_installation
```
#### Linux
```bash
# Start MariaDB service
sudo systemctl enable mariadb
sudo systemctl start mariadb
# Secure the installation
sudo mysql_secure_installation
```
### 3. Download and Setup Labelbase
```bash
# Switch to labelbase user (if created) or use your regular user
# sudo su - labelbase # (if using dedicated user)
# Set variables
LABELBASE_HOME="$HOME" # or /home/labelbase if using dedicated user
LABELBASE_REPO="https://github.com/Labelbase/Labelbase/"
LABELBASE_VERSION="2.2.1"
# Clone the repository
git clone $LABELBASE_REPO $LABELBASE_HOME/labelbase
cd $LABELBASE_HOME/labelbase
# Checkout specific version
git checkout $LABELBASE_VERSION
# Create virtual environment
python3 -m venv $LABELBASE_HOME/ENV
# Activate virtual environment
source $LABELBASE_HOME/ENV/bin/activate
# Install Python dependencies
pip install --upgrade pip
pip install --no-cache-dir -r $LABELBASE_HOME/labelbase/django/requirements.txt
```
### 4. Database Configuration
```bash
# Generate a secure password
MYSQL_PASSWORD=$(openssl rand -base64 32 | tr -d '+/' | fold -w 32 | head -n 1)
# Create exports file
cat > $LABELBASE_HOME/exports.sh << EOF
export MYSQL_PASSWORD=$MYSQL_PASSWORD
export DATABASE_URL=mysql://ulabelbase:$MYSQL_PASSWORD@localhost:3306/labelbase
EOF
chmod 755 $LABELBASE_HOME/exports.sh
```
#### Create Database and User
##### macOS
```bash
# Connect to MySQL
mysql -u root -p
# In MySQL prompt:
CREATE DATABASE labelbase;
CREATE USER 'ulabelbase'@'localhost' IDENTIFIED BY 'YOUR_GENERATED_PASSWORD';
GRANT ALL PRIVILEGES ON labelbase.* TO 'ulabelbase'@'localhost';
FLUSH PRIVILEGES;
EXIT;
```
##### Linux
```bash
# Connect to MariaDB
sudo mysql
# In MariaDB prompt:
CREATE DATABASE labelbase;
CREATE USER 'ulabelbase'@'localhost' IDENTIFIED BY 'YOUR_GENERATED_PASSWORD';
GRANT ALL PRIVILEGES ON labelbase.* TO 'ulabelbase'@'localhost';
FLUSH PRIVILEGES;
EXIT;
```
**Note:** Replace `YOUR_GENERATED_PASSWORD` with the password generated in the previous step.
### 5. Django Setup
```bash
# Navigate to Django directory
cd $LABELBASE_HOME/labelbase/django
# Activate virtual environment and load environment variables
source $LABELBASE_HOME/ENV/bin/activate
source $LABELBASE_HOME/exports.sh
# Run Django migrations
python manage.py makemigrations --noinput
python manage.py migrate --noinput
python manage.py collectstatic --noinput
# Create a superuser (optional)
python manage.py createsuperuser
```
### 6. Running Labelbase
#### Development Mode
```bash
# Activate environment
source $LABELBASE_HOME/ENV/bin/activate
source $LABELBASE_HOME/exports.sh
# Navigate to Django directory
cd $LABELBASE_HOME/labelbase/django
# Run development server
python manage.py runserver 0.0.0.0:8089
# Access at: http://localhost:8089
```
#### Production Mode (using Gunicorn)
```bash
# Install Gunicorn if not already installed
pip install gunicorn
# Run with Gunicorn
source $LABELBASE_HOME/ENV/bin/activate
source $LABELBASE_HOME/exports.sh
cd $LABELBASE_HOME/labelbase/django
gunicorn labellabor.wsgi:application -b 0.0.0.0:8089 --reload
```
## Creating System Services (Optional)
### macOS (using LaunchAgent)
Create a plist file at `~/Library/LaunchAgents/com.labelbase.app.plist`:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.labelbase.app</string>
<key>ProgramArguments</key>
<array>
<string>/bin/bash</string>
<string>-c</string>
<string>cd /Users/labelbase/labelbase/django && source /Users/labelbase/ENV/bin/activate && source /Users/labelbase/exports.sh && gunicorn labellabor.wsgi:application -b 0.0.0.0:8089</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>StandardOutPath</key>
<string>/Users/labelbase/labelbase.out</string>
<key>StandardErrorPath</key>
<string>/Users/labelbase/labelbase.err</string>
</dict>
</plist>
```
Load the service:
```bash
launchctl load ~/Library/LaunchAgents/com.labelbase.app.plist
```
### Linux (using systemd)
Create `/etc/systemd/system/labelbase.service`:
```ini
[Unit]
Description=Labelbase Application
After=mariadb.service
[Service]
Type=simple
User=labelbase
WorkingDirectory=/home/labelbase/labelbase/django
Environment="HOME_PATH=/home/labelbase"
ExecStart=/bin/bash -c 'source /home/labelbase/ENV/bin/activate && source /home/labelbase/exports.sh && gunicorn labellabor.wsgi:application -b 0.0.0.0:8089'
Restart=always
TimeoutSec=120
RestartSec=30
[Install]
WantedBy=multi-user.target
```
Enable and start the service:
```bash
sudo systemctl enable labelbase
sudo systemctl start labelbase
sudo systemctl status labelbase
```
## Troubleshooting
### Common Issues
1. **Database Connection Errors**
- Ensure MySQL/MariaDB is running
- Verify database credentials in `exports.sh`
- Check if the database user has proper permissions
2. **Python Package Issues**
- Make sure virtual environment is activated
- Try upgrading pip: `pip install --upgrade pip`
- Install packages one by one if requirements.txt fails
3. **Port Already in Use**
- Change the port in the run command: `gunicorn ... -b 0.0.0.0:8090`
- Find and kill processes using the port: `lsof -ti:8089 | xargs kill -9`
4. **Permission Errors**
- Ensure proper ownership of files: `sudo chown -R labelbase:labelbase /home/labelbase`
- Check file permissions: `chmod 755 /home/labelbase/exports.sh`
### Logs and Debugging
- **Development server logs**: Displayed in terminal
- **Gunicorn logs**: Use `--log-file` flag for logging
- **System service logs** (Linux): `sudo journalctl -u labelbase -f`
- **Database logs**: Check MySQL/MariaDB error logs
### Stopping the Application
```bash
# If running in development mode
Ctrl+C
# If running as system service
# macOS:
launchctl unload ~/Library/LaunchAgents/com.labelbase.app.plist
# Linux:
sudo systemctl stop labelbase
```
## Security Considerations
1. **Firewall Configuration**
- Only expose necessary ports
- Consider using a reverse proxy (nginx/apache) for production
2. **Database Security**
- Use strong passwords
- Limit database user permissions
- Consider encrypting database connections
3. **Application Security**
- Keep dependencies updated
- Use HTTPS in production
- Regular security updates
4. **File Permissions**
- Ensure proper file ownership and permissions
- Limit access to configuration files
This guide provides a foundation for running Labelbase on bare metal systems. Adjust paths, ports, and configurations according to your specific needs and security requirements.

312
DEVELOPMENT_GUIDE.md
View file

@ -1,312 +0,0 @@
# Labelbase Development Guide
Quick reference for working with and developing Labelbase in Docker.
## Initial Setup
### 1. Clone and Setup
```bash
git clone https://github.com/Labelbase/Labelbase/
cd Labelbase
```
### 2. Generate MySQL Passwords
```bash
./make-exports.sh
```
This creates `exports.sh` with random passwords. **Backup this file!**
### 3. Build and Run
```bash
source exports.sh && ./build-and-run-labelbase.sh
```
The `source exports.sh` loads the passwords into your shell, then the script uses them.
Access at: http://127.0.0.1:8080
---
## Daily Development Workflow
### Start/Stop Services
```bash
# Start (always source exports.sh first!)
source exports.sh && docker-compose up -d
# Stop
docker-compose down
# Rebuild and restart (after code changes)
source exports.sh && docker-compose up --build -d
# Or use the main script
source exports.sh && ./build-and-run-labelbase.sh
```
### View Logs
```bash
# All services
docker-compose logs -f
# Specific service
docker-compose logs -f labelbase_django
docker-compose logs -f labelbase_mysql
docker-compose logs -f labelbase_nginx
# Search logs
docker-compose logs labelbase_django | grep -i error
```
### Access Container Shell
```bash
# Django container (most common)
docker-compose exec labelbase_django bash
# MySQL container
docker-compose exec labelbase_mysql bash
# Nginx container
docker-compose exec labelbase_nginx sh
```
### Django Management Commands
```bash
# From host
docker-compose exec labelbase_django python manage.py <command>
# Or from inside container
docker-compose exec labelbase_django bash
python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser
python manage.py shell
```
### Database Operations
```bash
# Access MySQL CLI
docker-compose exec labelbase_mysql mysql -u ulabelbase -p labelbase
# Backup database
docker-compose exec labelbase_mysql mysqldump -u root -p labelbase > backup.sql
# Restore database
docker-compose exec -T labelbase_mysql mysql -u root -p labelbase < backup.sql
```
---
## Common Tasks
### Reset config.ini (if passwords change)
```bash
docker-compose exec labelbase_django bash
rm /app/config.ini
python manage.py make_config
exit
docker-compose restart labelbase_django
```
### Clean Rebuild (fresh start, deletes volumes!!)
❌ DANGER ZONE
```bash
docker-compose down -v
source exports.sh && docker-compose up --build -d
```
⚠️ **Warning**: `-v` deletes volumes including database data!
### Check What Django Sees
```bash
# Check environment variables
docker-compose exec labelbase_django env | grep MYSQL
# Check config.ini
docker-compose exec labelbase_django cat /app/config.ini
# Check database connection
docker-compose exec labelbase_django python manage.py dbshell
```
### Update from Git
```bash
git pull origin master
source exports.sh && ./build-and-run-labelbase.sh
```
The script automatically detects updates and rebuilds if needed.
---
## Configuration Files
### exports.sh (IMPORTANT - backup this!)
```bash
#!/bin/bash
export MYSQL_ROOT_PASSWORD="your_password_here"
export MYSQL_PASSWORD="your_password_here"
```
- Generated by `./make-exports.sh`
- Contains MySQL passwords
- Source before running docker-compose: `source exports.sh`
- **Add to `.gitignore`** - contains secrets!
### config.ini (auto-generated, persisted)
- Located at `/app/config.ini` inside Django container
- Generated from environment variables on first run by `python manage.py make_config`
- **Not overwritten** on subsequent runs (preserves user settings)
- Force regenerate: `rm /app/config.ini` then restart
### docker-compose.yml
Uses environment variables like `${MYSQL_PASSWORD}` from your shell (after sourcing exports.sh).
---
## Troubleshooting
### "Access denied for user 'ulabelbase'@'localhost'"
**Cause**: Old `config.ini` with wrong password from previous build
**Solution**:
```bash
docker-compose exec labelbase_django bash
rm /app/config.ini
python manage.py make_config
exit
docker-compose restart labelbase_django
```
### "MYSQL_PASSWORD variable is not set" warnings
**Cause**: Forgot to source exports.sh
**Solution**: Always use `source exports.sh && docker-compose up`
These warnings appear at parse time but are harmless if you source exports.sh before running the command.
### "Can't connect to MySQL server"
**Cause**: Database not ready yet
**Solution**: Wait 15 seconds (run.sh has a built-in delay) or check logs:
```bash
docker-compose logs labelbase_mysql
```
### Django can't see code changes
- Volume mounting issue
- Solution: `docker-compose restart labelbase_django`
- Or use `--reload` in gunicorn (already enabled)
### Port 8080 already in use
```bash
# Find what's using it
lsof -i :8080
# Change port in docker-compose.yml
ports:
- "127.0.0.1:8081:8080" # Use 8081 instead
```
---
## Development Tips
### Live Code Reloading
Django container mounts `./django:/app`, so changes are live. Gunicorn runs with `--reload` flag.
### Static Files
After changing CSS/JS:
```bash
docker-compose exec labelbase_django python manage.py collectstatic --noinput
```
### Running Tests
```bash
docker-compose exec labelbase_django python manage.py test
```
### Python Dependencies
Add to `requirements.txt`, then:
```bash
docker-compose up --build -d
```
### Database Migrations
```bash
# Create migrations
docker-compose exec labelbase_django python manage.py makemigrations
# Apply migrations
docker-compose exec labelbase_django python manage.py migrate
# Show migration status
docker-compose exec labelbase_django python manage.py showmigrations
```
---
## Production Deployment
See main README for:
- Setting up nginx reverse proxy
- SSL certificates with certbot
- Domain configuration
- Firewall rules
---
## Quick Command Reference
```bash
# Build and run
source exports.sh && ./build-and-run-labelbase.sh
# Stop
docker-compose down
# Logs
docker-compose logs -f
# Shell
docker-compose exec labelbase_django bash
or
source exports.sh && docker-compose exec labelbase_django bash
# Reset everything (DANGER: deletes data!)
docker-compose down -v && source exports.sh && docker-compose up --build -d
# Reset config.ini (if passwords changed)
docker-compose exec labelbase_django rm /app/config.ini
docker-compose restart labelbase_django
```
---
## Environment Variables Reference
| Variable | Purpose | Default |
|----------|---------|---------|
| `MYSQL_ROOT_PASSWORD` | MySQL root password | (required) |
| `MYSQL_PASSWORD` | MySQL user password | (required) |
| `MYSQL_DATABASE` | Database name | `labelbase` |
| `MYSQL_USER` | Database user | `ulabelbase` |
| `MYSQL_HOST` | Database hostname | `labelbase_mysql` |
| `MYSQL_PORT` | Database port | `3306` |
Set in `exports.sh` file and load with `source exports.sh` before running docker-compose.
**Note I**: You may see warnings like "variable is not set" when running docker-compose commands. These appear at parse time but are harmless - the variables are properly set when you source exports.sh before the command.
**NOTE II**: Some of the variables are hard coded or may need to be changed in the docker-compose.yml file manually.
---
## Need Help?
- Check logs: `docker-compose logs -f`
- View settings: `docker-compose exec labelbase_django cat /app/config.ini`
- Check environment: `docker-compose exec labelbase_django env | grep MYSQL`
- Access Django shell: `docker-compose exec labelbase_django python manage.py shell`

10
build-and-run-labelbase.sh
View file

@ -7,19 +7,17 @@ docker-compose down # make sure Labelbase is terminated.
if [[ -z "${MYSQL_ROOT_PASSWORD}" ]]; then
echo "Error: MYSQL_ROOT_PASSWORD environment variable is not set"
exit 1
else
export $MYSQL_ROOT_PASSWORD
fi
if [[ -z "${MYSQL_PASSWORD}" ]]; then
echo "Error: MYSQL_PASSWORD environment variable is not set"
exit 1
else
export $MYSQL_PASSWORD
fi
export MYSQL_ROOT_PASSWORD
export MYSQL_PASSWORD
# Check git next
LAST_GIT_COMMIT_FILE=".last_git_commit"

4
django/Dockerfile
View file

@ -1,4 +1,4 @@
FROM python:3.11
FROM python:3.9
ENV PYTHONUNBUFFERED 1
@ -11,7 +11,7 @@ RUN apt-get update && \
default-libmysqlclient-dev \
build-essential \
cron vim logrotate \
libpcre2-dev \
libpcre3-dev \
default-mysql-client \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --upgrade pip \

3
django/attachments/README.md
View file

@ -1,3 +0,0 @@
Friendly-forked at commit https://github.com/bartTC/django-attachments/commit/d0c3e7b366691f2a57329c804b0dd002064396a6 (version 1.11) and extended for Labelbase.
BSD 3-Clause "New" or "Revised" License https://github.com/bartTC/django-attachments/blob/master/LICENSE

1
django/attachments/__init__.py
View file

@ -1 +0,0 @@
default_app_config = "attachments.apps.AttachmentsConfig"

21
django/attachments/admin.py
View file

@ -1,21 +0,0 @@
from __future__ import unicode_literals
from django.contrib.contenttypes.admin import GenericStackedInline
from django.contrib import admin
from .models import LabelAttachment
from .models import Attachment
class AttachmentInlines(GenericStackedInline):
model = Attachment
exclude = ()
extra = 1
admin.site.register(LabelAttachment)

9
django/attachments/apps.py
View file

@ -1,9 +0,0 @@
from __future__ import unicode_literals
from django.apps import AppConfig
from django.utils.translation import gettext_lazy as _
class AttachmentsConfig(AppConfig):
default_auto_field = 'django.db.models.AutoField'
name = "attachments"
verbose_name = _("Attachments")

46
django/attachments/forms.py
View file

@ -1,46 +0,0 @@
from __future__ import unicode_literals
from django import forms
from django.apps import apps
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.template.defaultfilters import filesizeformat
from django.utils.translation import gettext_lazy as _
from .models import Attachment
config = apps.get_app_config("attachments")
def validate_max_size(data):
if (
hasattr(settings, "FILE_UPLOAD_MAX_SIZE")
and data.size > settings.FILE_UPLOAD_MAX_SIZE
):
raise forms.ValidationError(
_("File exceeds maximum size of {size}").format(
size=filesizeformat(settings.FILE_UPLOAD_MAX_SIZE)
)
)
def custom_attachment_validators(uploaded_file):
for validator in getattr(config, "attachment_validators", ()):
validator(uploaded_file)
class AttachmentForm(forms.ModelForm):
attachment_file = forms.FileField(
label=_("Upload attachment"),
validators=[validate_max_size, custom_attachment_validators]
)
class Meta:
model = Attachment
fields = ("attachment_file",)
def save(self, request, obj, *args, **kwargs):
self.instance.creator = request.user
self.instance.content_type = ContentType.objects.get_for_model(obj)
self.instance.object_id = obj.pk
super(AttachmentForm, self).save(*args, **kwargs)

11
django/attachments/locale/README.transifex
View file

@ -1,11 +0,0 @@
Transifex.net Token Verification
=================================
The list of tokens bellow guarantee the respective users to be able to enable
submission on components using the following repository url:
git://github.com/bartTC/django-attachments.git
Tokens:
24u9WhbEZQbCGDn3ruWJ6c7YpDpSxCCf / bartTC

BIN
django/attachments/locale/da/LC_MESSAGES/django.mo
View file

Binary file not shown.

54
django/attachments/locale/da/LC_MESSAGES/django.po
View file

@ -1,54 +0,0 @@
# django-attachments in Danish
# django-attachments på Dansk
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# Michael Lind Mortensen <illio@cs.au.dk>, 2009.
#
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2009-08-27 23:52+0200\n"
"PO-Revision-Date: 2010-04-04 00:24+0100\n"
"Last-Translator: Martin Mahner <martin@mahner.org>\n"
"Language-Team: da <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Upload vedhæftning"
#: models.py:29
msgid "creator"
msgstr "skaber"
#: models.py:30
msgid "attachment"
msgstr "vedhæftning"
#: models.py:31
msgid "created"
msgstr "skabt"
#: models.py:32
msgid "modified"
msgstr "ændret"
#: views.py:33
msgid "Your attachment was uploaded."
msgstr "Din vedhæftning blev uploadet."
#: views.py:51
msgid "Your attachment was deleted."
msgstr "Din vedhæftning blev slettet."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Tilføj vedhæftning"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Slet vedhæftning"

BIN
django/attachments/locale/de/LC_MESSAGES/django.mo
View file

Binary file not shown.

53
django/attachments/locale/de/LC_MESSAGES/django.po
View file

@ -1,53 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2010-04-04 00:19+0200\n"
"PO-Revision-Date: 2010-04-04 00:29+0100\n"
"Last-Translator: Martin Mahner <martin@mahner.org>\n"
"Language-Team: de <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Anhang hochladen"
#: models.py:29
msgid "creator"
msgstr "Autor"
#: models.py:30
msgid "attachment"
msgstr "Anhang"
#: models.py:31
msgid "created"
msgstr "Erstellt"
#: models.py:32
msgid "modified"
msgstr "Geändert"
#: views.py:33
msgid "Your attachment was uploaded."
msgstr "Dein Anhang wurde hochgeladen."
#: views.py:51
msgid "Your attachment was deleted."
msgstr "Dein Anhang wurde gelöscht."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Anhang hinzufügen"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Anhang löschen"

BIN
django/attachments/locale/el/LC_MESSAGES/django.mo
View file

Binary file not shown.

53
django/attachments/locale/el/LC_MESSAGES/django.po
View file

@ -1,53 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# Panos Laganakos <panos.laganakos@gmail.com>, 2010.
#
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2010-04-04 02:52+0300\n"
"PO-Revision-Date: 2010-04-04 15:07+0100\n"
"Last-Translator: Martin Mahner <martin@mahner.org>\n"
"Language-Team: gr <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Φώρτωση συνημμένου"
#: models.py:29
msgid "creator"
msgstr "δημιουργός"
#: models.py:30
msgid "attachment"
msgstr "συνημμένο"
#: models.py:31
msgid "created"
msgstr "δημιουργήθηκε"
#: models.py:32
msgid "modified"
msgstr "τροποποιήθηκε"
#: views.py:33
msgid "Your attachment was uploaded."
msgstr "Η επισύναψη σας ολοκληρώθηκε."
#: views.py:51
msgid "Your attachment was deleted."
msgstr "Το συνήμμενο σας διεγράφει."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Προσθέστε ένα συνημμένο"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Διαγράψτε το συνημμένο"

BIN
django/attachments/locale/en/LC_MESSAGES/django.mo
View file

Binary file not shown.

53
django/attachments/locale/en/LC_MESSAGES/django.po
View file

@ -1,53 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2010-04-04 00:18+0200\n"
"PO-Revision-Date: 2010-04-04 00:28+0100\n"
"Last-Translator: Martin Mahner <martin@mahner.org>\n"
"Language-Team: en <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Upload attachment"
#: models.py:29
msgid "creator"
msgstr "creator"
#: models.py:30
msgid "attachment"
msgstr "attachment"
#: models.py:31
msgid "created"
msgstr "created"
#: models.py:32
msgid "modified"
msgstr "modified"
#: views.py:33
msgid "Your attachment was uploaded."
msgstr "Your attachment was uploaded."
#: views.py:51
msgid "Your attachment was deleted."
msgstr "Your attachment was deleted."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Add attachment"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Delete attachment"

BIN
django/attachments/locale/es_AR/LC_MESSAGES/django.mo
View file

Binary file not shown.

54
django/attachments/locale/es_AR/LC_MESSAGES/django.po
View file

@ -1,54 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# Gonzalo Bustos, 2015.
#
msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2014-05-31 17:20-0300\n"
"PO-Revision-Date: 2015-10-11 22:04-0300\n"
"Last-Translator: Gonzalo Bustos\n"
"Language-Team: Spanish (Argentina)\n"
"Language: es_AR\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: Poedit 1.6.10\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Subir adjunto"
#: models.py:32
msgid "creator"
msgstr "creador"
#: models.py:33 models.py:38
msgid "attachment"
msgstr "adjunto"
#: models.py:34
msgid "created"
msgstr "creado"
#: models.py:35
msgid "modified"
msgstr "modificado"
#: views.py:34
msgid "Your attachment was uploaded."
msgstr "Su adjunto fue subido."
#: views.py:52
msgid "Your attachment was deleted."
msgstr "Su adjunto fue eliminado."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Agregar adjunto"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Eliminar adjunto"

BIN
django/attachments/locale/fi/LC_MESSAGES/django.mo
View file

Binary file not shown.

54
django/attachments/locale/fi/LC_MESSAGES/django.po
View file

@ -1,54 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2016-06-11 15:55+0300\n"
"PO-Revision-Date: 2016-06-11 16:03+0300\n"
"Last-Translator: Aleksi Häkli <aleksi.hakli@iki.fi>\n"
"Language-Team: fi <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Language: fi\n"
"X-Generator: Poedit 1.8.8\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Lähetä liite"
#: models.py:29
msgid "creator"
msgstr "luoja"
#: models.py:30
msgid "attachment"
msgstr "liite"
#: models.py:31
msgid "created"
msgstr "luotu"
#: models.py:32
msgid "modified"
msgstr "muokattu"
#: views.py:33
msgid "Your attachment was uploaded."
msgstr "Liitteesi ladattiin palvelimelle."
#: views.py:51
msgid "Your attachment was deleted."
msgstr "Liitteesi poistettiin palvelimelta."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Lisää liite"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Poista liite"

BIN
django/attachments/locale/fr/LC_MESSAGES/django.mo
View file

Binary file not shown.

53
django/attachments/locale/fr/LC_MESSAGES/django.po
View file

@ -1,53 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2010-04-04 00:18+0200\n"
"PO-Revision-Date: 2010-04-04 00:28+0100\n"
"Last-Translator: AERT <dev.aert@gmail.com>\n"
"Language-Team: en <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Ajouter une pièce jointe"
#: models.py:29
msgid "creator"
msgstr "créateur"
#: models.py:30
msgid "attachment"
msgstr "pièce jointe"
#: models.py:31
msgid "created"
msgstr "créé"
#: models.py:32
msgid "modified"
msgstr "modifié"
#: views.py:33
msgid "Your attachment was uploaded."
msgstr "Votre pièce jointe a été ajoutée."
#: views.py:51
msgid "Your attachment was deleted."
msgstr "Votre pièce jointe a été supprimée."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Ajouter une pièce jointe"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Supprimer la pièce jointe"

BIN
django/attachments/locale/it/LC_MESSAGES/django.mo
View file

Binary file not shown.

77
django/attachments/locale/it/LC_MESSAGES/django.po
View file

@ -1,77 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2018-05-29 16:35+0200\n"
"PO-Revision-Date: 2018-05-29 16:39+0200\n"
"Language: it\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"Last-Translator: Mario Orlandi <morlandi@brainstorm.it>\n"
"Language-Team: \n"
"X-Generator: Poedit 2.0.7\n"
#: apps.py:7
msgid "Attachments"
msgstr "Allegati"
#: forms.py:13
#, python-brace-format
msgid "File exceeds maximum size of {size}"
msgstr "Il file supera le dimensioni massime di {size}"
#: forms.py:18
msgid "Upload attachment"
msgstr "Invia allegato"
#: models.py:35
msgid "creator"
msgstr "autore"
#: models.py:36 models.py:41
msgid "attachment"
msgstr "allegato"
#: models.py:37
msgid "created"
msgstr "creato il"
#: models.py:38
msgid "modified"
msgstr "modificato il"
#: models.py:42
msgid "attachments"
msgstr "allegati"
#: models.py:45
msgid "Can delete foreign attachments"
msgstr "Può eliminare gli allegati esterni"
#: models.py:49
#, python-brace-format
msgid "{username} attached {filename}"
msgstr "{username} ha allegato {filename}"
#: templates/attachments/add_form.html:8
msgid "Add attachment"
msgstr "Aggiungi allegato"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Elimina allegato"
#: views.py:51
msgid "Your attachment was uploaded."
msgstr "L'allegato è stato inviato."
#: views.py:75
msgid "Your attachment was deleted."
msgstr "L'allegato è stato eliminato."

BIN
django/attachments/locale/pt_BR/LC_MESSAGES/django.mo
View file

Binary file not shown.

54
django/attachments/locale/pt_BR/LC_MESSAGES/django.po
View file

@ -1,54 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2014-05-31 17:20-0300\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: forms.py:7
msgid "Upload attachment"
msgstr "Subir anexo"
#: models.py:32
msgid "creator"
msgstr "criador"
#: models.py:33 models.py:38
msgid "attachment"
msgstr "anexo"
#: models.py:34
msgid "created"
msgstr "criado"
#: models.py:35
msgid "modified"
msgstr "modificado"
#: views.py:34
msgid "Your attachment was uploaded."
msgstr "Seu anexo foi enviado."
#: views.py:52
msgid "Your attachment was deleted."
msgstr "Seu anexo foi excluido."
#: templates/attachments/add_form.html:7
msgid "Add attachment"
msgstr "Adicionar anexo"
#: templates/attachments/delete_link.html:2
msgid "Delete attachment"
msgstr "Excluir anexo"

61
django/attachments/locale/ru/LC_MESSAGES/django.po
View file

@ -1,61 +0,0 @@
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: django-attachments\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2017-04-07 23:35+0300\n"
"PO-Revision-Date: 2017-04-07 23:26+0300:MI+ZONE\n"
"Last-Translator: Maxim Barabanov <maxim.barabanov@gmail.com>\n"
"Language-Team: RU <LL@li.org>\n"
"Language: Russian\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=4; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n"
"%10<=4 && (n%100<12 || n%100>14) ? 1 : n%10==0 || (n%10>=5 && n%10<=9) || (n"
"%100>=11 && n%100<=14)? 2 : 3);\n"
#: .\attachments\forms.py:9
msgid "Upload attachment"
msgstr "Загрузить вложение"
#: .\attachments\models.py:35
msgid "creator"
msgstr "создатель"
#: .\attachments\models.py:36
msgid "attachment"
msgstr "вложение"
#: .\attachments\models.py:37
msgid "created"
msgstr "создано"
#: .\attachments\models.py:38
msgid "modified"
msgstr "изменено"
#: .\attachments\models.py:43
msgid "Can delete foreign attachments"
msgstr "Может удалять чужие вложения"
#: .\attachments\templates\attachments\add_form.html:8
msgid "Add attachment"
msgstr "Добавить вложение"
#: .\attachments\templates\attachments\delete_link.html:2
msgid "Delete attachment"
msgstr "Удалить вложение"
#: .\attachments\views.py:40
msgid "Your attachment was uploaded."
msgstr "Ваше вложение было загружено."
#: .\attachments\views.py:62
msgid "Your attachment was deleted."
msgstr "Ваше вложение было удалено"

0
django/attachments/management/__init__.py
View file

0
django/attachments/management/commands/__init__.py
View file

44
django/attachments/management/commands/delete_stale_attachments.py
View file

@ -1,44 +0,0 @@
from django.core.management.base import BaseCommand
from attachments.models import Attachment
from attachments.views import remove_file_from_disk
class Command(BaseCommand):
help = ("Remove attachments for which the related objects "
"don't exist anymore!")
def add_arguments(self, parser):
parser.add_argument(
'-y', '--yes', default='x', action='store_const', const='y',
dest='answer', help='Automatically confirm deletion',
)
def handle(self, *args, **kwargs):
verbose = kwargs['verbosity'] >= 1
answer = kwargs['answer']
# -v0 sets --yes
if not verbose:
answer = 'y'
for att in Attachment.objects.all():
if att.content_object is None:
if verbose:
self.stdout.write(
"Attachment `%s' to non-existing `%s' with PK `%s'" %
(att, att.content_type.model, att.object_id))
while answer not in 'yn':
answer = input("Do you wish to delete? [yN] ")
if not answer:
answer = 'x'
continue
answer = answer[0].lower()
if answer == 'y' :
remove_file_from_disk(att.attachment_file)
att.delete()
if verbose:
self.stdout.write("Deleted attachment `%s'" % att)

34
django/attachments/migrations/0001_initial.py
View file

@ -1,34 +0,0 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
from django.conf import settings
import attachments.models
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('contenttypes', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='Attachment',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('object_id', models.PositiveIntegerField()),
('attachment_file', models.FileField(upload_to=attachments.models.attachment_upload, verbose_name='attachment')),
('created', models.DateTimeField(auto_now_add=True, verbose_name='created')),
('modified', models.DateTimeField(auto_now=True, verbose_name='modified')),
('content_type', models.ForeignKey(to='contenttypes.ContentType', on_delete=models.CASCADE)),
('creator', models.ForeignKey(related_name='created_attachments', verbose_name='creator', to=settings.AUTH_USER_MODEL, on_delete=models.CASCADE)),
],
options={
'ordering': ['-created'],
'permissions': (('delete_foreign_attachments', 'Can delete foreign attachments'),),
},
bases=(models.Model,),
),
]

19
django/attachments/migrations/0002_auto_20180104_1247.py
View file

@ -1,19 +0,0 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.11 on 2018-01-04 12:47
from __future__ import unicode_literals
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('attachments', '0001_initial'),
]
operations = [
migrations.AlterModelOptions(
name='attachment',
options={'ordering': ['-created'], 'permissions': (('delete_foreign_attachments', 'Can delete foreign attachments'),), 'verbose_name': 'attachment', 'verbose_name_plural': 'attachments'},
),
]

18
django/attachments/migrations/0003_auto_20190722_1216.py
View file

@ -1,18 +0,0 @@
# Generated by Django 2.2.3 on 2019-07-22 12:16
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('attachments', '0002_auto_20180104_1247'),
]
operations = [
migrations.AlterField(
model_name='attachment',
name='object_id',
field=models.TextField(),
),
]

25
django/attachments/migrations/0004_db_index.py
View file

@ -1,25 +0,0 @@
# Generated by Django 3.0.9 on 2020-08-17 13:29
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('attachments', '0003_auto_20190722_1216'),
]
operations = [
migrations.AlterField(
model_name='attachment',
name='created',
field=models.DateTimeField(auto_now_add=True, db_index=True,
verbose_name='created'),
),
migrations.AlterField(
model_name='attachment',
name='modified',
field=models.DateTimeField(auto_now=True, db_index=True,
verbose_name='modified'),
),
]

16
django/attachments/migrations/0005_object_id_charfield.py
View file

@ -1,16 +0,0 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('attachments', '0004_db_index'),
]
operations = [
migrations.AlterField(
model_name='attachment',
name='object_id',
field=models.CharField(db_index=True, max_length=64),
),
]

19
django/attachments/migrations/0006_alter_attachment_attachment_file.py
View file

@ -1,19 +0,0 @@
# Generated by Django 3.2.25 on 2024-04-11 14:35
from django.db import migrations, models
import uuid_upload_path.storage
class Migration(migrations.Migration):
dependencies = [
('attachments', '0005_object_id_charfield'),
]
operations = [
migrations.AlterField(
model_name='attachment',
name='attachment_file',
field=models.FileField(upload_to=uuid_upload_path.storage.upload_to, verbose_name='attachment'),
),
]

28
django/attachments/migrations/0007_labelattachment.py
View file

@ -1,28 +0,0 @@
# Generated by Django 3.2.25 on 2024-04-11 15:06
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('attachments', '0006_alter_attachment_attachment_file'),
]
operations = [
migrations.CreateModel(
name='LabelAttachment',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('type_ref_hash', models.CharField(blank=True, help_text='Reflects type + ref, where type could be any bip-329 defined type', max_length=64)),
('network', models.CharField(choices=[('mainnet', 'Mainnet'), ('testnet', 'Testnet')], default='mainnet', help_text="Choose the network for this labelbase's label attachement.", max_length=10)),
('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
],
options={
'unique_together': {('user', 'network', 'type_ref_hash')},
},
),
]

0
django/attachments/migrations/__init__.py
View file

111
django/attachments/models.py
View file

@ -1,111 +0,0 @@
from __future__ import unicode_literals
import os
from django.conf import settings
from django.contrib.contenttypes.fields import GenericForeignKey
from django.contrib.contenttypes.models import ContentType
from django.db import models
from django.utils.translation import gettext_lazy as _
from six import python_2_unicode_compatible
from uuid_upload_path import upload_to
from django.contrib.auth.models import User
import logging
logger = logging.getLogger('labelbase')
def attachment_upload(instance, filename):
pass # used for compatibility / migrations only.
class AttachmentManager(models.Manager):
def attachments_for_object(self, obj):
object_type = ContentType.objects.get_for_model(obj)
return self.filter(content_type__pk=object_type.id, object_id=obj.pk)
@python_2_unicode_compatible
class Attachment(models.Model):
objects = AttachmentManager()
content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
object_id = models.CharField(db_index=True, max_length=64)
content_object = GenericForeignKey("content_type", "object_id")
creator = models.ForeignKey(
settings.AUTH_USER_MODEL,
related_name="created_attachments",
verbose_name=_("creator"),
on_delete=models.CASCADE,
)
attachment_file = models.FileField(
_("attachment"), upload_to=upload_to
)
created = models.DateTimeField(_("created"), auto_now_add=True, db_index=True)
modified = models.DateTimeField(_("modified"), auto_now=True, db_index=True)
class Meta:
verbose_name = _("attachment")
verbose_name_plural = _("attachments")
ordering = ["-created"]
permissions = (
("delete_foreign_attachments", _("Can delete foreign attachments")),
)
def __str__(self):
return _("{username} attached {filename}").format(
username=self.creator.get_username(),
filename=self.attachment_file.name,
)
@property
def filename(self):
return os.path.split(self.attachment_file.name)[1]
def attach_to(self, new_object, update_path=False):
"""
Attach to a new object and possibly move the actual file on disk!
.. important::
As long as path names are valid you can continue serving
the files from their original path and not change it!
"""
self.object_id = new_object.pk
self.content_type = ContentType.objects.get_for_model(new_object)
self.save()
if update_path:
old_path = self.attachment_file.path
self.attachment_file.name = upload_to
self.attachment_file.save()
os.makedirs(
os.path.dirname(self.attachment_file.path), exist_ok=True)
os.rename(old_path, self.attachment_file.path)
class LabelAttachment(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE, null=True, blank=True)
type_ref_hash = models.CharField(max_length=64,
blank=True,
help_text="Reflects type + ref, where type could be any bip-329 defined type")
MAINNET = 'mainnet'
TESTNET = 'testnet'
NETWORK_CHOICES = [
(MAINNET, 'Mainnet'),
(TESTNET, 'Testnet'),
]
network = models.CharField(
max_length=10,
choices=NETWORK_CHOICES,
default='mainnet',
help_text="Choose the network for this labelbase's label attachement."
)
class Meta:
unique_together = (("user", "network", "type_ref_hash"),)

0
django/attachments/templatetags/__init__.py
View file

69
django/attachments/templatetags/attachments_tags.py
View file

@ -1,69 +0,0 @@
from django.template import Library
from django.urls import reverse
from ..forms import AttachmentForm
from ..models import Attachment
from ..views import add_url_for_obj
register = Library()
@register.filter(name='endswith')
def endswith(value, arg):
"""Checks if the value ends with a certain string."""
if isinstance(value, str):
return value.endswith(arg)
return False
@register.inclusion_tag("attachments/add_form.html", takes_context=True)
def attachment_form(context, obj, **kwargs):
"""
Renders a "upload attachment" form.
The user must own ``attachments.add_attachment permission`` to add
attachments.
"""
return {
"form": AttachmentForm(),
"form_url": add_url_for_obj(obj),
"next": context.request.path,
}
@register.inclusion_tag("attachments/delete_link.html", takes_context=True)
def attachment_delete_link(context, attachment, **kwargs):
if context["user"] == attachment.creator:
return {
"next": context.request.path,
"delete_url": reverse(
"attachments:delete", kwargs={"attachment_pk": attachment.pk}
),
}
@register.simple_tag
def attachments_count(obj):
"""
Counts attachments that are attached to a given object::
{% attachments_count obj %}
"""
attachments_count = Attachment.objects.attachments_for_object(obj).count()
print("obj: {}, attachments_count: {}".format(obj.id, attachments_count))
return attachments_count
@register.simple_tag
def get_attachments_for(obj, *args, **kwargs):
"""
Resolves attachments that are attached to a given object. You can specify
the variable name in the context the attachments are stored using the `as`
argument.
Syntax::
{% get_attachments_for obj as "my_attachments" %}
"""
return Attachment.objects.attachments_for_object(obj)

0
django/attachments/tests/__init__.py
View file

68
django/attachments/tests/base.py
View file

@ -1,68 +0,0 @@
# -*- encoding: utf-8 -*-
from __future__ import unicode_literals
from django.contrib.auth.models import Permission, User
from django.contrib.contenttypes.models import ContentType
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase
from django.urls import reverse
from ..models import Attachment
from .testapp.models import TestModel
class BaseTestCase(TestCase):
target_model_class = TestModel
def setUp(self):
"""
Create two users with `attachments.add_attachment` permission
and one object to attach files to.
"""
content_type = ContentType.objects.get_for_model(Attachment)
self.add_permission = Permission.objects.get(
content_type=content_type, codename="add_attachment"
)
self.del_permission = Permission.objects.get(
content_type=content_type, codename="delete_attachment"
)
self.del_foreign_permission = Permission.objects.get(
content_type=content_type, codename="delete_foreign_attachments"
)
self.cred_jon = {"username": "jon", "password": "foobar"}
self.cred_jane = {"username": "jane", "password": "foobar"}
self.jon = User.objects.create_user(**self.cred_jon)
self.jon.user_permissions.add(self.add_permission)
self.jon.user_permissions.add(self.del_permission)
self.jane = User.objects.create_user(**self.cred_jane)
self.jane.user_permissions.add(self.add_permission)
self.jane.user_permissions.add(self.del_permission)
self.obj = self.target_model_class.objects.create(title="My first test item")
def _upload_testfile(self, file_obj=None, file_content=b"file content", **extra):
"""
Uploads a sample file for the given user.
"""
add_url = reverse(
"attachments:add",
kwargs={
"app_label": "testapp",
"model_name": self.target_model_class.__name__.lower(),
"pk": self.obj.pk,
},
)
if not file_obj:
file_obj = SimpleUploadedFile(
"Ünicode Filename 🙂.jpg",
file_content,
content_type="image/jpeg",
)
return self.client.post(
add_url, {"attachment_file": file_obj}, follow=True,
**extra
)

19
django/attachments/tests/test_integrity.py
View file

@ -1,19 +0,0 @@
from django.core.management import call_command
from django.test import TestCase
from six import StringIO
class IntegrityTestCase(TestCase):
"""
Very basic tests around the app itself, not the code.
"""
def test_no_pending_migrations(self):
"""
Make sure all model changes are reflected with Django migrations.
"""
output = StringIO()
call_command(
"makemigrations", "--dry-run", interactive=False, stdout=output
)
self.assertTrue("No changes detected" in output.getvalue())

69
django/attachments/tests/test_template.py
View file

@ -1,69 +0,0 @@
from django.urls import reverse
from ..models import Attachment
from .base import BaseTestCase
class ViewTestCase(BaseTestCase):
def setUp(self):
super(ViewTestCase, self).setUp()
self.item_url = reverse("testapp-detail", kwargs={"pk": self.obj.pk})
def test_uploaded_attachment_urls_are_listed(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
response = self.client.get(self.item_url)
attachment = Attachment.objects.attachments_for_object(self.obj)[0]
self.assertTrue(attachment.attachment_file.url in str(response.content))
def test_attachment_count_is_listed(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
self._upload_testfile()
response = self.client.get(self.item_url)
attachment_count = Attachment.objects.attachments_for_object(
self.obj
).count()
self.assertTrue(
"Object has %d attachments" % attachment_count
in str(response.content)
)
def test_upload_form_is_listed_with_add_permission(self):
self.client.login(**self.cred_jon)
response = self.client.get(self.item_url)
self.assertTrue("<form" in str(response.content))
def test_upload_form_is_not_listed_without_add_permission(self):
self.jon.user_permissions.remove(self.add_permission)
self.client.login(**self.cred_jon)
response = self.client.get(self.item_url)
self.assertFalse("<form" in str(response.content))
def test_delete_link_is_listed_with_delete_permission(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
response = self.client.get(self.item_url)
self.assertTrue("delete-attachment" in str(response.content))
def test_delete_link_is_not_listed_without_delete_permission(self):
self.jon.user_permissions.remove(self.del_permission)
self.client.login(**self.cred_jon)
self._upload_testfile()
response = self.client.get(self.item_url)
self.assertFalse("delete-attachment" in str(response.content))
def test_delete_link_is_listed_with_foreign_delete_permission(self):
self.jon.user_permissions.add(self.del_foreign_permission)
self.client.login(**self.cred_jane)
self._upload_testfile()
self.client.login(**self.cred_jon)
response = self.client.get(self.item_url)
self.assertTrue("delete-attachment" in str(response.content))
def test_delete_link_is_not_listed_for_others_attachments(self):
self.client.login(**self.cred_jane)
self._upload_testfile()
self.client.login(**self.cred_jon)
response = self.client.get(self.item_url)
self.assertFalse("delete-attachment" in str(response.content))

273
django/attachments/tests/test_views.py
View file

@ -1,273 +0,0 @@
import os
import json
import mock
from http import HTTPStatus
from django.urls import reverse
from ..models import Attachment
from .base import BaseTestCase
from .testapp.models import ModelWithUuidPk
class ViewTestCase(BaseTestCase):
def test_empty_post_to_form_wont_create_attachment(self):
add_url = reverse(
"attachments:add",
kwargs={
"app_label": "testapp",
"model_name": "testmodel",
"pk": self.obj.pk,
},
)
response = self.client.post(add_url)
self.assertEqual(302, response.status_code)
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)
def test_invalid_model_wont_fail(self):
add_url = reverse(
"attachments:add",
kwargs={
"app_label": "thisdoes",
"model_name": "notexist",
"pk": self.obj.pk,
},
)
response = self.client.post(add_url)
self.assertEqual(302, response.status_code)
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)
def test_invalid_attachment_wont_fail(self):
self.client.login(**self.cred_jon)
self._upload_testfile(file_obj="Not a UploadedFile object")
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)
def test_upload_size_less_than_limit(self):
# NOTE: in all of the other tests there's no limit specified
# so they will cover the branch where the setting is missing
with self.settings(FILE_UPLOAD_MAX_SIZE=1024):
self.client.login(**self.cred_jon)
self._upload_testfile()
self.assertEqual(Attachment.objects.count(), 1)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 1
)
def test_upload_size_more_than_limit(self):
# we set a limit of 1 byte b/c the file used for testing
# is very small
with self.settings(FILE_UPLOAD_MAX_SIZE=1):
self.client.login(**self.cred_jon)
response = self._upload_testfile()
self.assertContains(response, "File exceeds maximum size of 1")
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)
def test_upload_without_permission(self):
"""
Remove the 'add permission' and try to upload a file.
"""
self.jon.user_permissions.remove(self.add_permission)
self.client.login(**self.cred_jon)
self._upload_testfile()
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)
def test_upload_with_permission(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
self.assertEqual(Attachment.objects.count(), 1)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 1
)
def test_unauthed_user_cant_delete_attachment(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
att = Attachment.objects.first()
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": att.pk}
)
self.client.logout()
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 1)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 1
)
def test_author_can_delete_attachment(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
att = Attachment.objects.first()
file_path = att.attachment_file.path
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": att.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 0)
# file on disk is still present b/c setting not specified
self.assertTrue(os.path.exists(file_path))
def test_author_cant_delete_attachment_if_no_delete_permission(self):
self.jon.user_permissions.remove(self.del_permission)
self.client.login(**self.cred_jon)
self._upload_testfile()
att = Attachment.objects.first()
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": att.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 1)
def test_author_cant_delete_others_attachment(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
obj1 = Attachment.objects.order_by("-created")[0]
self.client.login(**self.cred_jane)
self._upload_testfile()
obj2 = Attachment.objects.order_by("-created")[0]
self.assertNotEqual(obj1, obj2)
# Jon can't delete Janes attachment
self.client.login(**self.cred_jon)
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": obj2.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 2)
def test_author_can_delete_others_attachment_with_permission(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
obj1 = Attachment.objects.order_by("-created")[0]
path1 = obj1.attachment_file.path
self.client.login(**self.cred_jane)
self._upload_testfile()
obj2 = Attachment.objects.order_by("-created")[0]
path2 = obj2.attachment_file.path
self.assertNotEqual(obj1, obj2)
# Jon has the `delete_foreign_attachments` permission so he can
# delete Janes attachment
self.jon.user_permissions.add(self.del_foreign_permission)
self.client.login(**self.cred_jon)
# explicitly set the delete setting to False to
# cover that branch as well
with self.settings(DELETE_ATTACHMENT_FILE=False):
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": obj2.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 1)
self.assertTrue(os.path.exists(path1))
self.assertTrue(os.path.exists(path2))
def test_delete_removes_files_from_disk_if_settings(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
att = Attachment.objects.first()
file_path = att.attachment_file.path
with self.settings(DELETE_ATTACHMENTS_FROM_DISK=True):
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": att.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 0)
self.assertFalse(os.path.exists(file_path))
def test_delete_does_not_raise_if_settings_and_file_missing(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
att = Attachment.objects.first()
file_path = att.attachment_file.path
# remove the file before hand
os.remove(file_path)
with self.settings(DELETE_ATTACHMENTS_FROM_DISK=True):
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": att.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 0)
self.assertFalse(os.path.exists(file_path))
def test_delete_does_not_raise_if_os_remove_raises(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
att = Attachment.objects.first()
with mock.patch("attachments.views.os.remove") as _mock:
_mock.side_effect = OSError("Test file does not exist")
with self.settings(DELETE_ATTACHMENTS_FROM_DISK=True):
del_url = reverse(
"attachments:delete", kwargs={"attachment_pk": att.pk}
)
self.client.get(del_url, follow=True)
self.assertEqual(Attachment.objects.count(), 0)
# NOTE: we don't assert the file path here because
# the mock which raises will not actually delete it
class UUIDTestCase(BaseTestCase):
target_model_class = ModelWithUuidPk
def test_upload_with_permission(self):
self.client.login(**self.cred_jon)
self._upload_testfile()
self.assertEqual(Attachment.objects.count(), 1)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 1
)
class CustomValidatorsTestCase(BaseTestCase):
def test_deny_specific_content(self):
self.client.login(**self.cred_jon)
response = self._upload_testfile(file_content=b"<xml>this is not allowed</xml>")
self.assertContains(response, "XML is forbidden")
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)
def test_form_errors_are_returned_as_json(self):
self.client.login(**self.cred_jon)
response = self._upload_testfile(
file_content=b"<xml>this is not allowed</xml>",
HTTP_X_RETURN_FORM_ERRORS=True,
)
self.assertEqual(response.status_code, HTTPStatus.BAD_REQUEST)
self.assertEqual(response.headers.get("Content-Type"), "application/json")
# this should be a dict
errors = json.loads(response.content)
# note: field errors are a list of string messages
self.assertEqual(errors["attachment_file"], ["XML is forbidden"])
self.assertEqual(Attachment.objects.count(), 0)
self.assertEqual(
Attachment.objects.attachments_for_object(self.obj).count(), 0
)

0
django/attachments/tests/testapp/__init__.py
View file

11
django/attachments/tests/testapp/admin.py
View file

@ -1,11 +0,0 @@
from django.contrib import admin
from ...admin import AttachmentInlines
from .models import TestModel
class TestModelAdmin(admin.ModelAdmin):
inlines = [AttachmentInlines]
admin.site.register(TestModel, TestModelAdmin)

19
django/attachments/tests/testapp/apps.py
View file

@ -1,19 +0,0 @@
"""
Custom app config to demonstrate and test the 'custom attachment validators'
functionality.
"""
from attachments.apps import AttachmentsConfig
from django.forms import ValidationError
def deny_xml_uploads(uploaded_file):
if uploaded_file.read().find(b"<xml>") > -1:
raise ValidationError("XML is forbidden")
class CustomizedAttachmentsApp(AttachmentsConfig):
"""
Adds a custom form validator function.
"""
attachment_validators = (deny_xml_uploads,)

16
django/attachments/tests/testapp/migrations/0001_initial.py
View file

@ -1,16 +0,0 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
operations = [
migrations.CreateModel(
name='TestModel',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('title', models.CharField(max_length=100)),
],
bases=(models.Model,),
),
]

19
django/attachments/tests/testapp/migrations/0002_auto_20180104_1247.py
View file

@ -1,19 +0,0 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.11 on 2018-01-04 12:47
from __future__ import unicode_literals
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('testapp', '0001_initial'),
]
operations = [
migrations.AlterModelTable(
name='testmodel',
table='testapp_testmodel',
),
]

24
django/attachments/tests/testapp/migrations/0003_model_with_uuid_pk.py
View file

@ -1,24 +0,0 @@
# Generated by Django 3.2.7 on 2023-03-11 11:15
from django.db import migrations, models
import uuid
class Migration(migrations.Migration):
dependencies = [
('testapp', '0002_auto_20180104_1247'),
]
operations = [
migrations.CreateModel(
name='ModelWithUuidPk',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)),
('title', models.CharField(max_length=100)),
],
options={
'db_table': 'testapp_uuid4_model',
},
),
]

0
django/attachments/tests/testapp/migrations/__init__.py
View file

23
django/attachments/tests/testapp/models.py
View file

@ -1,23 +0,0 @@
import uuid
from django.db import models
class TestModel(models.Model):
title = models.CharField(max_length=100)
class Meta:
db_table = "testapp_testmodel"
def get_absolute_url(self):
return "/"
class ModelWithUuidPk(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
title = models.CharField(max_length=100)
class Meta:
db_table = "testapp_uuid4_model"
def get_absolute_url(self):
return "/"

80
django/attachments/tests/testapp/settings.py
View file

@ -1,80 +0,0 @@
import os
DEBUG = True
TESTAPP_DIR = os.path.abspath(os.path.dirname(__file__))
SECRET_KEY = "testsecretkey"
if os.environ.get("DJANGO_DATABASE_ENGINE") == "postgresql":
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"USER": "postgres",
"NAME": "attachments",
"HOST": "localhost",
"PORT": 5432,
}
}
elif os.environ.get("DJANGO_DATABASE_ENGINE") == "mysql":
DATABASES = {
"default": {
"ENGINE": "django.db.backends.mysql",
"USER": "root",
"NAME": "attachments",
"HOST": "127.0.0.1",
"PORT": 3306,
}
}
else:
DATABASES = {
"default": {"ENGINE": "django.db.backends.sqlite3", "NAME": "tests.db"}
}
DATABASES["default"].update(
{
"PASSWORD": os.environ.get("DATABASE_PASSWORD", "testing"),
}
)
MEDIA_ROOT = os.path.join(TESTAPP_DIR, "uploads")
ROOT_URLCONF = "attachments.tests.testapp.urls"
INSTALLED_APPS = [
"attachments.tests.testapp",
"attachments.tests.testapp.apps.CustomizedAttachmentsApp",
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
]
MIDDLEWARE = (
"django.contrib.sessions.middleware.SessionMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
)
MIDDLEWARE_CLASSES = (
"django.contrib.sessions.middleware.SessionMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
)
TEMPLATES = [
{
"BACKEND": "django.template.backends.django.DjangoTemplates",
"DIRS": [os.path.join(TESTAPP_DIR, "templates")],
"APP_DIRS": True,
"OPTIONS": {
"context_processors": [
"django.template.context_processors.debug",
"django.template.context_processors.request",
"django.template.context_processors.i18n",
"django.contrib.auth.context_processors.auth",
"django.contrib.messages.context_processors.messages",
]
},
}
]

16
django/attachments/tests/testapp/templates/testmodel_detail.html
View file

@ -1,16 +0,0 @@
{% load attachments_tags %}
<h1>{{ object.title }}</h1>
<h2>Object has {% attachments_count object %} attachments</h2>
{% get_attachments_for object as attachments_list %}
{% for att in attachments_list %}
<p>
{{ att }}
{{ att.attachment_file.url }}
{{ att.filename }}
{% attachment_delete_link att %}
</p>
{% endfor %}
{% attachment_form object %}

25
django/attachments/tests/testapp/urls.py
View file

@ -1,25 +0,0 @@
try:
from django.urls import re_path as url
except ImportError:
from django.conf.urls import url
from django.conf.urls import include
from django.contrib import admin
from django.views.generic import DetailView
from .models import TestModel
admin.autodiscover()
urlpatterns = [
url(r"^attachments/", include("attachments.urls", namespace="attachments")),
url(r"^admin/", admin.site.urls),
url(
r"^testapp/(?P<pk>\d+)/$",
DetailView.as_view(
template_name="testmodel_detail.html",
queryset=TestModel.objects.all(),
),
name="testapp-detail",
),
]

24
django/attachments/urls.py
View file

@ -1,24 +0,0 @@
from __future__ import unicode_literals
try:
from django.urls import re_path as url
except ImportError:
from django.conf.urls import url
from .views import add_attachment, delete_attachment
app_name = "attachments"
urlpatterns = [
url(
r"^add-for/(?P<app_label>[\w\-]+)/(?P<model_name>[\w\-]+)/(?P<pk>\d+)/$",
add_attachment,
name="add",
),
url(
r"^add-for/(?P<app_label>[\w\-]+)/(?P<model_name>[\w\-]+)/(?P<pk>[a-f0-9]{8}-?[a-f0-9]{4}-?4[a-f0-9]{3}-?[89ab][a-f0-9]{3}-?[a-f0-9]{12})/$",
add_attachment,
name="add",
),
url(r"^delete/(?P<attachment_pk>\d+)/$", delete_attachment, name="delete"),
]

81
django/attachments/views.py
View file

@ -1,81 +0,0 @@
from __future__ import unicode_literals
import os
from http import HTTPStatus
from django.apps import apps
from django.conf import settings
from django.contrib import messages
from django.contrib.auth.decorators import login_required
from django.http import HttpResponseRedirect, JsonResponse
from django.shortcuts import get_object_or_404, render
from django.urls import reverse
from django.utils.translation import gettext
from django.views.decorators.http import require_POST
from .forms import AttachmentForm
from .models import Attachment
def add_url_for_obj(obj):
return reverse(
"attachments:add",
kwargs={
"app_label": obj._meta.app_label,
"model_name": obj._meta.model_name,
"pk": obj.pk,
},
)
def remove_file_from_disk(f):
if getattr(
settings, "DELETE_ATTACHMENTS_FROM_DISK", False
) and os.path.exists(f.path):
try:
os.remove(f.path)
except OSError:
pass
@require_POST
@login_required
def add_attachment(
request,
app_label,
model_name,
pk,
template_name="attachments/add.html",
extra_context=None,
):
next_ = request.POST.get("next", "/")
model = apps.get_model(app_label, model_name)
obj = get_object_or_404(model, pk=pk)
obj = obj.get_label_attachment() # our label to attachment proxy
form = AttachmentForm(request.POST, request.FILES)
if form.is_valid():
form.save(request, obj)
messages.success(request, gettext("Your attachment was uploaded."))
return HttpResponseRedirect(next_)
if request.headers.get("X-Return-Form-Errors", None):
return JsonResponse(form.errors, status=HTTPStatus.BAD_REQUEST)
template_context = {
"form": form,
"form_url": add_url_for_obj(obj),
"next": next_,
}
template_context.update(extra_context or {})
return render(request, template_name, template_context)
@login_required
def delete_attachment(request, attachment_pk):
g = get_object_or_404(Attachment, pk=attachment_pk)
if request.user == g.creator:
remove_file_from_disk(g.attachment_file)
g.delete()
messages.success(request, gettext("Your attachment was deleted."))
next_ = request.GET.get("next") or "/"
return HttpResponseRedirect(next_)

3
django/background_task/settings.py
View file

@ -10,7 +10,8 @@ except Exception:
class AppSettings(object):
""" """
"""
"""
@property
def MAX_ATTEMPTS(self):
"""Control how many times a task will be attempted."""

3
django/background_task/tasks.py
View file

@ -38,6 +38,7 @@ def bg_runner(proxy_task, task=None, loop=None, *args, **kwargs):
task = task_qs[0]
if func is None:
raise BackgroundTaskError("Function is None, can't execute!")
print("bg_runner, loop {}".format(loop))
kwargs['loop'] = loop
func(*args, **kwargs)
@ -95,6 +96,8 @@ class Tasks(object):
return _decorator
def run_task(self, task_name, loop, args=None, kwargs=None):
print("run_task loop {}".format(loop))
# task_name can be either the name of a task or a Task instance.
if isinstance(task_name, Task):
task = task_name
task_name = task.task_name

0
django/background_task/templatetags/__init__.py
View file

12
django/background_task/templatetags/backgroundtask_tags.py
View file

@ -1,12 +0,0 @@
from django import template
from background_task.models import Task
register = template.Library()
@register.simple_tag
def is_label_id_in_queue(label_id):
try:
return Task.objects.filter(task_name="finances.tasks.check_spent",
task_params__contains=label_id).exists()
except:
return False

2
django/connectrum/README.md
View file

@ -1,3 +1,3 @@
Friendly-forked at commit https://github.com/coinkite/connectrum/commit/c893bc2100de6acebbdf0bf67b62e6cb9ce1c7be and extended for Labelbase.
Copy-and-pasted at commit https://github.com/coinkite/connectrum/commit/c893bc2100de6acebbdf0bf67b62e6cb9ce1c7be and extended for Labelbase.
MIT Licence https://github.com/coinkite/connectrum/blob/master/LICENSE

11
django/finances/admin.py
View file

@ -2,16 +2,7 @@ from django.contrib import admin
from .models import OutputStat, HistoricalPrice
class OutputStatAdmin(admin.ModelAdmin):
list_display = ('type_ref_hash',
'value',
'confirmed_at_block_height',
'confirmed_at_block_time',
'get_spent_status',
'spent',
'network',
'user',
'next_enc_input_attrs',
'last_error')
list_display = ('type_ref_hash', 'value', 'confirmed_at_block_height', 'confirmed_at_block_time', 'get_spent_status', 'spent', 'network', 'user')
list_filter = ('network', 'spent')
search_fields = ('type_ref_hash',)
ordering = ('-confirmed_at_block_time',)

254
django/finances/electrum.py
View file

@ -1,32 +1,34 @@
from connectrum.client import StratumClient
from connectrum.svr_info import ServerInfo
from connectrum import ElectrumErrorResponse
from labelbase.models import Label
from finances.models import OutputStat, HistoricalPrice
import logging
import logging
logger = logging.getLogger('labelbase')
async def interact(conn, server_info, method, utxo):
try:
await conn.connect(server_info, "s", use_tor=server_info.is_onion,
disable_cert_verify=True, short_term=True)
disable_cert_verify=True, short_term=True)
txid, index = utxo.split(":")
try:
txn = await conn.RPC(method, txid, True)
if txn:
try:
blocktime = int(txn.get('blocktime', 0))
logger.debug(f"blocktime: {blocktime}")
logger.debug("blocktime: {}".format(blocktime))
except Exception as ex:
blocktime = 0
logger.error(f"Can't get blocktime: {ex}")
utxo = txn.get('vout')[int(index)]
logger.error("Can't get blocktime: {}".format(ex))
address = txn.get('vout')[int(index)].get('scriptPubKey', {}).get('address')
value = txn.get('vout')[int(index)].get('value') * 100000000
return txid, index, address, value, blocktime, utxo
value = txn.get('vout')[int(index)].get('value')*100000000
return (txid, index, address, value, blocktime)
except ElectrumErrorResponse as ex:
logger.error(f"ERROR: {ex} {conn.last_error}")
logger.error("ERROR: {} {}".format(ex, conn.last_error))
finally:
conn.close()
@ -34,117 +36,185 @@ async def interact(conn, server_info, method, utxo):
async def interact_addr(conn, server_info, method, addr):
try:
await conn.connect(server_info, "s", use_tor=server_info.is_onion,
disable_cert_verify=True, short_term=True)
disable_cert_verify=True, short_term=True)
try:
hextx = await conn.RPC(method, addr)
if hextx is not None:
print(hextx)
return hextx
else:
print("Failed to fetch transaction.")
except ElectrumErrorResponse as ex:
logger.error(ex)
print(ex)
finally:
conn.close()
def is_valid_output_ref(ref):
return ":" in ref if ref else False
if not ref:
return False
if ":" in ref:
return True
return False
def checkup_label(label_id, loop):
if not label_id or not loop:
logger.error(f"Invalid input: label_id={label_id}, loop={loop}")
return
if label_id and loop:
try:
elem = Label.objects.get(id=label_id)
output = OutputStat.objects.filter(user=elem.labelbase.user,
type_ref_hash=elem.type_ref_hash,
network=elem.labelbase.network).last()
if not output:
output = OutputStat(user=elem.labelbase.user,
type_ref_hash=elem.type_ref_hash,
network=elem.labelbase.network, value=0)
try:
elem = Label.objects.get(id=label_id)
output = OutputStat.objects.filter(
user=elem.labelbase.user,
type_ref_hash=elem.type_ref_hash,
network=elem.labelbase.network
).last()
if not output:
output = OutputStat(
user=elem.labelbase.user,
type_ref_hash=elem.type_ref_hash,
network=elem.labelbase.network,
value=0,
spent=None,
confirmed_at_block_height=0,
confirmed_at_block_time=0
)
output.save()
logger.debug(f"Output before processing: {output.output_metrics_dict()}")
if elem.type == "output" and is_valid_output_ref(elem.ref) and (
output.spent is not True or output.confirmed_at_block_time is None
):
# Determine server info based on network
if elem.labelbase.is_mainnet:
electrum_hostname = elem.labelbase.user.profile.electrum_hostname or "fulcrum.sethforprivacy.com"
if elem.type == "output" and is_valid_output_ref(elem.ref) and \
(output.spent is not True or output.confirmed_at_block_time == 0):
electrum_hostname = elem.labelbase.user.profile.electrum_hostname or "electrum.emzy.de"
electrum_ports = elem.labelbase.user.profile.electrum_ports or "s50002"
elif elem.labelbase.is_testnet:
electrum_hostname = elem.labelbase.user.profile.electrum_hostname_test or "testnet.qtornado.com"
electrum_ports = elem.labelbase.user.profile.electrum_ports_test or "s51002"
else:
raise ValueError("Unknown network type.")
server_info = ServerInfo(electrum_hostname, electrum_hostname, ports=(electrum_ports))
server_info = ServerInfo(electrum_hostname, electrum_hostname, ports=(electrum_ports))
conn = StratumClient()
utxo = elem.ref
utxo_resp = loop.run_until_complete(interact(conn, server_info, "blockchain.transaction.get", utxo))
if utxo_resp:
txid, index, address, value, blocktime = utxo_resp
if blocktime:
HistoricalPrice.get_or_create_from_api(timestamp=blocktime)
unspents = loop.run_until_complete(interact_addr(conn, server_info, "blockchain.address.listunspent", address))
utxo_value = 0
utxo_height = 0
if unspents:
for unspent in unspents:
if unspent.get('tx_hash') == txid and \
unspent.get('tx_pos') == int(index) and \
unspent.get('height') > 0 and \
unspent.get('value') > 0: # Output is confirmed, but not spent yet
output.spent = False
utxo_value = unspent.get('value')
utxo_height = unspent.get('height')
output.network = elem.labelbase.network
if utxo_height:
output.confirmed_at_block_height = utxo_height
if blocktime:
output.confirmed_at_block_time = blocktime
if utxo_value:
output.value = utxo_value
elif value:
output.value = value
break
#
elif conn.last_error:
output.last_error = conn.last_error
else:
output.last_error = {}
output.save()
try:
conn.close()
except:
pass
except Exception as e:
logger.error("Error processing label {}: {}".format(label_id, e))
else:
if not label_id:
logger.error("Can't get label_id! {}".format(label_id))
if not loop:
logger.error("Can't get loop!")
def checkup_label_buggy(label_id, loop):
if label_id and loop:
elem = Label.objects.get(id=label_id)
output = OutputStat.objects.filter(user=elem.labelbase.user,
type_ref_hash=elem.type_ref_hash,
network=elem.labelbase.network).last()
if not output:
print("Creating OutputStat")
output = OutputStat(user=elem.labelbase.user,
type_ref_hash=elem.type_ref_hash,
network=elem.labelbase.network, value=0)
print("Using OutputStat id {}".format(output))
print("elem.type {} {} {} {}".format(elem.type, is_valid_output_ref(elem.ref), elem.ref, output.spent))
if elem.type == "output" and is_valid_output_ref(elem.ref) and \
(output.spent is not True or output.confirmed_at_block_time == 0):
electrum_hostname = elem.labelbase.user.profile.electrum_hostname
if not electrum_hostname:
electrum_hostname = "electrum.emzy.de"
electrum_ports = elem.labelbase.user.profile.electrum_ports
if not electrum_ports:
electrum_ports = "s50002"
print("going for server_info")
server_info = ServerInfo(electrum_hostname, electrum_hostname, ports=((electrum_ports)))
print("server_info: {}".format(server_info))
conn = StratumClient()
assert elem.type_ref_hash
utxo = elem.ref
# Fetch transaction details
tx_hash, tx_pos = elem.ref.split(":")
utxo_resp = loop.run_until_complete(interact(conn, server_info, "blockchain.transaction.get", utxo))
blocktime = 0
if utxo_resp:
txid, index, address, value, blocktime, utxo_data = utxo_resp
logger.debug(f"Transaction {txid} fetched with blocktime {blocktime}")
print("utxo_resp {}".format(utxo_resp))
txid, index, address, value, blocktime = utxo_resp
if blocktime:
output.confirmed_at_block_time = blocktime
HistoricalPrice.get_or_create_from_api(None, timestamp=blocktime)
# Fetch all unspents for the address
print("Found blocktime {} for label id {}.".format(blocktime, label_id))
HistoricalPrice.get_or_create_from_api(timestamp=blocktime)
try:
unspents = loop.run_until_complete(interact_addr(conn, server_info, "blockchain.address.listunspent", address))
except:
conn.last_error = None
conn.last_error = None # reset error if needed
unspents = loop.run_until_complete(interact_addr(conn, server_info, "blockchain.scripthash.listunspent", address))
logger.debug(f"Unspents for address {address}: {unspents}")
unspent_utxo = False
utxo_value = 0
utxo_height = 0
print("unspents: {}".format(unspents))
utxo_found = False
for unspent in unspents:
if unspent.get('tx_hash') == txid and unspent.get('tx_pos') == int(index):
if unspents:
for unspent in unspents:
if unspent.get('tx_hash') == tx_hash and \
unspent.get('tx_pos') == int(tx_pos) and \
unspent.get('height') > 0 and \
unspent.get('value') > 0:
unspent_utxo = True
utxo_value = unspent.get('value')
utxo_height = unspent.get('height')
print("found unspent: {}".format(unspent))
break
if output:
output.network = elem.labelbase.network
if utxo_height:
output.confirmed_at_block_height = utxo_height
if blocktime:
output.confirmed_at_block_time = blocktime
if utxo_value:
output.value = utxo_value
elif value: # take value from TX
output.value = value
if unspent_utxo:
output.spent = False
output.value = unspent.get('value', None)
output.confirmed_at_block_height = unspent.get('height', None)
utxo_found = True
# Ensure all key details are stored
output.network = elem.labelbase.network
if unspent.get('height'):
output.confirmed_at_block_height = unspent.get('height')
if blocktime:
output.confirmed_at_block_time = blocktime
if unspent.get('value') is not None:
output.value = unspent.get('value')
elif value:
output.value = value
break
if not utxo_found:
output.spent = True
logger.warning(f"UTXO {txid}:{index} not found in unspent outputs.")
else:
output.spent = True
output.last_error = {}
elif conn.last_error:
# Damn...
output.last_error = conn.last_error
else:
logger.warning(f"Unknown error occurred for UTXO {utxo}")
output.last_error = {"error": "Unknown issue"}
logger.debug(f"Output after processing (before save): {output.output_metrics_dict()}")
output.last_error = {}
output.save()
output.refresh_from_db()
logger.debug(f"Output after saving: {output.output_metrics_dict()}")
except Exception as e:
logger.error(f"Error processing label {label_id}: {e}")
print("output id {} saved".format(output.id))
try:
conn.close()
except:
pass
else:
if not label_id:
logger.error("Can't get label_id! {}".format(label_id))
if not loop:
logger.error("Can't get loop!")

19
django/finances/migrations/0010_outputstat_next_input_attributes.py
View file

@ -1,19 +0,0 @@
# Generated by Django 3.2.24 on 2024-03-29 17:38
from django.db import migrations
import jsonfield.fields
class Migration(migrations.Migration):
dependencies = [
('finances', '0009_alter_outputstat_last_error'),
]
operations = [
migrations.AddField(
model_name='outputstat',
name='next_input_attributes',
field=jsonfield.fields.JSONField(default={}),
),
]

18
django/finances/migrations/0011_alter_outputstat_network.py
View file

@ -1,18 +0,0 @@
# Generated by Django 3.2.25 on 2024-04-11 15:06
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('finances', '0010_outputstat_next_input_attributes'),
]
operations = [
migrations.AlterField(
model_name='outputstat',
name='network',
field=models.CharField(choices=[('mainnet', 'Mainnet'), ('testnet', 'Testnet')], default='mainnet', help_text="Choose the network for this labelbase's label output.", max_length=10),
),
]

22
django/finances/migrations/0012_auto_20240701_0932.py
View file

@ -1,22 +0,0 @@
# Generated by Django 3.2.25 on 2024-07-01 09:32
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('finances', '0011_alter_outputstat_network'),
]
operations = [
migrations.RemoveField(
model_name='outputstat',
name='next_input_attributes',
),
migrations.AddField(
model_name='outputstat',
name='next_enc_input_attrs',
field=models.TextField(default=None, null=True),
),
]

66
django/finances/models.py
View file

@ -1,6 +1,5 @@
import requests
from django.db import models
from django.contrib import messages
from djmoney.models.fields import MoneyField
from decimal import Decimal
import datetime
@ -8,13 +7,14 @@ from pymempool import MempoolAPI
from labelbase.receivers import compute_type_ref_hash
from django.conf import settings
from jsonfield import JSONField
import json
from django.contrib.auth.models import User
from shared.encryption import get_fernet_key, cipher_suite
import logging
logger = logging.getLogger('labelbase')
class OutputStat(models.Model):
"""
These fields are unencrypted. Why?
@ -43,19 +43,6 @@ class OutputStat(models.Model):
confirmed_at_block_time = models.IntegerField(default=0)
last_error = JSONField(default={})
next_enc_input_attrs = models.TextField(default=None, null=True) # will be used for fee estimation
def set_next_input_attributes(self, data):
json_data = json.dumps(data).encode('utf-8')
encrypted_data = cipher_suite.encrypt(json_data)
self.next_enc_input_attrs = encrypted_data.decode('utf-8')
def next_input_attributes(self):
if self.next_enc_input_attrs:
encrypted_data = self.next_enc_input_attrs.encode('utf-8')
decrypted_data = cipher_suite.decrypt(encrypted_data)
return json.loads(decrypted_data.decode('utf-8'))
return json.loads("{}")
MAINNET = 'mainnet'
TESTNET = 'testnet'
@ -69,7 +56,7 @@ class OutputStat(models.Model):
max_length=10,
choices=NETWORK_CHOICES,
default='mainnet',
help_text="Choose the network for this labelbase's label output."
help_text="Choose the network for this labelbase."
)
class Meta:
@ -77,12 +64,12 @@ class OutputStat(models.Model):
@property
def get_spent_status(self):
if self.confirmed_at_block_time == 0:
return "unconfirmed"
if self.spent:
return "spent"
elif not self.spent:
if not self.spent:
return "unspent"
elif self.confirmed_at_block_time == 0:
return "unconfirmed"
def output_metrics_dict(self, tracked_fiat_value=0, fiat_currency="USD"):
"""
@ -102,8 +89,8 @@ class OutputStat(models.Model):
# Check if the block time is confirmed
if self.confirmed_at_block_time:
# Get or create HistoricalPrice instance for the confirmed block time
obj, created = HistoricalPrice.get_or_create_from_api(self.user,
timestamp=self.confirmed_at_block_time
obj, created = HistoricalPrice.get_or_create_from_api(
timestamp=self.confirmed_at_block_time
)
if obj is None:
logger.error("No price info found for {}".format(self.confirmed_at_block_time))
@ -126,8 +113,8 @@ class OutputStat(models.Model):
timestamp = int(current_datetime.timestamp())
# Get or create HistoricalPrice instance for the current time in UTC
obj_now, created = HistoricalPrice.get_or_create_from_api(self.user,
timestamp=timestamp
obj_now, created = HistoricalPrice.get_or_create_from_api(
timestamp=timestamp
)
# Calculate the current price
@ -158,7 +145,7 @@ class OutputStat(models.Model):
Parses 'tracked_fiat_value' and 'fiat_currency' information from the given label.
"""
if self.confirmed_at_block_time:
obj, created = HistoricalPrice.get_or_create_from_api(self.user,
obj, created = HistoricalPrice.get_or_create_from_api(
timestamp=self.confirmed_at_block_time)
performance = 0
@ -206,11 +193,13 @@ class OutputStat(models.Model):
network=network).last()
if cached_data:
print("found cached data {} for type_ref_hash {}".format(cached_data, type_ref_hash))
return cached_data, False
def get_value_and_spent(txid, vout):
mempool_api = MempoolAPI()
res0 = mempool_api.get_transaction(txid)
print("res0 {}".format(res0))
vouts = res0.get("vout", [])
if vouts:
value = vouts[int(vout)].get("value", 0)
@ -222,9 +211,10 @@ class OutputStat(models.Model):
if txid and vout:
res = get_value_and_spent(txid, vout)
print (res)
if res:
value, spent, confirmed_at_block_height, confirmed_at_block_time = res
print("called data {} {} for type_ref_hash {}".format(value, spent, type_ref_hash))
obj, created = cls.objects.get_or_create(user=user,
type_ref_hash=type_ref_hash, network=network,
defaults={
@ -276,31 +266,17 @@ class HistoricalPrice(models.Model):
ordering = ['-timestamp']
@classmethod
def get_or_create_from_api(cls, user=None, timestamp=-1):
def get_or_create_from_api(cls, timestamp=-1):
print("running get_or_create_from_api @ timestamp {}".format(timestamp))
if timestamp == -1:
current_datetime = datetime.datetime.now()
timestamp = int(current_datetime.timestamp())
cached_data = cls.objects.filter(timestamp=timestamp).first()
if cached_data:
return cached_data, False
try:
if user:
mempool_endpoint = user.profile.mempool_endpoint
else:
mempool_endpoint = "https://mempool.space"
url = f"{mempool_endpoint}/api/v1/historical-price?timestamp={timestamp}"
response = requests.get(url)
api_response = response.json()
except Exception as ex:
logger.error(ex, exc_info=True)
try:
from threadlocals.threadlocals import get_current_request
request = get_current_request()
if request:
messages.error(request, "<strong>Connection Error:</strong> Could not connect to Mempool to retrieve historical price.")
except Exception as ex2:
logger.error(ex2, exc_info=True)
return None, None
url = f"https://mempool.space/api/v1/historical-price?timestamp={timestamp}"
response = requests.get(url)
api_response = response.json()
try:
obj, created = cls.objects.get_or_create(timestamp=timestamp, defaults={
'usd_price': Decimal(str(api_response['prices'][0]['USD'])),

31
django/finances/signals.py
View file

@ -1,25 +1,20 @@
from django.contrib.auth.signals import user_logged_in
from django.contrib import messages
from django.dispatch import receiver
from finances.models import HistoricalPrice
from django.contrib import messages
@receiver(user_logged_in)
def perform_tasks_on_login(sender, user, request, **kwargs):
try:
if user.profile.update_utxo_on_login:
from finances.tasks import check_all_outputs
from labelbase.models import Label
check_all_outputs(user.id)
if Label.objects.filter(labelbase__user_id=user.id).exists():
messages.info(request, (
"<strong>Sync in progress:</strong> "
"We are checking your unspent transaction outputs now."
))
except Exception as ex:
messages.info(request, (
"<strong>Oups:</strong> "
f"{ex}"
))
""" """
if user.profile.update_utxo_on_login:
from finances.tasks import check_all_outputs
from labelbase.models import Label
check_all_outputs(user.id)
if Label.objects.filter(labelbase__user_id=user.id).exists():
messages.info(request, "<strong>Sync in progress:</strong> We are checking your unspent transaction outputs now.")
# Store nearest price information.
HistoricalPrice.get_or_create_from_api(user, -1)
HistoricalPrice.get_or_create_from_api(-1)

2
django/finances/tasks.py
View file

@ -1,6 +1,8 @@
import logging
from background_task import background
from background_task.management.commands.remove_completed import _remove_completed_task
from labelbase.models import Label
from finances.electrum import checkup_label

223
django/finances/tx_math.py
View file

@ -1,223 +0,0 @@
# Constants
P2PKH_IN_SIZE = 148
P2PKH_OUT_SIZE = 34
P2SH_OUT_SIZE = 32
P2SH_P2WPKH_OUT_SIZE = 32
P2SH_P2WSH_OUT_SIZE = 32
P2WPKH_OUT_SIZE = 31
P2WSH_OUT_SIZE = P2TR_OUT_SIZE = 43
PUBKEY_SIZE = 33
SIGNATURE_SIZE = 72
def get_size_of_var_int(length):
if length <= 252:
return 1
elif length <= 0xffff:
return 3
elif length <= 0xffffffff:
return 5
else:
return 9
def get_size_of_script_length_element(length):
if length < 75:
return 1
elif length <= 255:
return 2
elif length <= 65535:
return 3
elif length <= 4294967295:
return 5
else:
raise ValueError("Script too large")
def get_input_size(input_attr):
"""Return (base_size, witness_size) for a single input."""
script_type = input_attr['input_script']
m = input_attr.get('input_m', 0)
n = input_attr.get('input_n', 0)
if script_type == "P2PKH":
return P2PKH_IN_SIZE, 0
elif script_type == "P2SH":
# Redeem script: OP_m + n_pubkeys + OP_n + OP_CHECKMULTISIG
redeem_script_size = 1 + n * (1 + PUBKEY_SIZE) + 1 + 1
# scriptSig: OP_0 + m_sigs + push_opcode + redeem_script
script_sig_size = 1 + m * (1 + SIGNATURE_SIZE) + get_size_of_script_length_element(redeem_script_size) + redeem_script_size
input_base_size = 32 + 4 + get_size_of_var_int(script_sig_size) + script_sig_size + 4
return input_base_size, 0
elif script_type == "P2SH-P2WPKH":
input_base_size = 32 + 4 + 1 + 23 + 4
input_witness_size = 107
return input_base_size, input_witness_size
elif script_type == "P2WPKH":
input_base_size = 32 + 4 + 1 + 4
input_witness_size = 107
return input_base_size, input_witness_size
elif script_type == "P2WSH":
input_base_size = 32 + 4 + 1 + 4
witness_script_size = 1 + n * (1 + PUBKEY_SIZE) + 1 + 1
num_stack_items = 1 + m + 1 # OP_0 + m sigs + witness script
input_witness_size = (
get_size_of_var_int(num_stack_items) +
1 + # OP_0 length
m * (1 + SIGNATURE_SIZE) +
get_size_of_var_int(witness_script_size) +
witness_script_size
)
return input_base_size, input_witness_size
elif script_type == "P2TR":
input_base_size = 32 + 4 + 1 + 4
input_witness_size = 65
return input_base_size, input_witness_size
else:
raise ValueError(f"Unsupported input script type: {script_type}")
def calculate_transaction_size(inputs, output_counts):
"""
inputs: list of dicts with keys: input_script, input_m, input_n
output_counts: dict with counts per output type
"""
total_base = 0
total_witness = 0
# Total inputs / outputs
input_count = len(inputs)
output_count = sum(output_counts.values())
# Transaction overhead: version(4) + varints + locktime(4)
tx_base_size = 4 + get_size_of_var_int(input_count) + get_size_of_var_int(output_count) + 4
# Segwit marker + flag if any input is segwit
has_witness = any(inp['input_script'] in ["P2SH-P2WPKH", "P2WPKH", "P2WSH", "P2TR"] for inp in inputs)
if has_witness:
total_witness += 2 # marker + flag
for inp in inputs:
base, witness = get_input_size(inp)
total_base += base
total_witness += witness
# Sum output sizes
output_size = (P2PKH_OUT_SIZE * output_counts.get('p2pkh', 0) +
P2SH_OUT_SIZE * output_counts.get('p2sh', 0) +
P2SH_P2WPKH_OUT_SIZE * output_counts.get('p2sh_p2wpkh', 0) +
P2SH_P2WSH_OUT_SIZE * output_counts.get('p2sh_p2wsh', 0) +
P2WPKH_OUT_SIZE * output_counts.get('p2wpkh', 0) +
P2WSH_OUT_SIZE * output_counts.get('p2wsh', 0) +
P2TR_OUT_SIZE * output_counts.get('p2tr', 0))
# Total base size
tx_total_base_size = tx_base_size + total_base + output_size
# Transaction weight and vbytes
tx_weight = tx_total_base_size * 4 + total_witness
tx_vbytes = tx_weight / 4
# Raw bytes (base + witness discounted by 1/4)
tx_bytes = tx_total_base_size + total_witness / 4
return {
'txBytes': round(tx_bytes),
'txVBytes': round(tx_vbytes),
'txWeight': tx_weight
}
def calculate_fee(tx_vbytes, fee_rate_sats_per_vbyte):
"""
tx_vbytes: virtual size from calculate_transaction_size()
fee_rate_sats_per_vbyte: fee rate in sats per vbyte
"""
return round(tx_vbytes * fee_rate_sats_per_vbyte)
def run_tests():
fee_rate = 20 # sats per vbyte
test_cases = [
# 1. Single P2PKH input -> single P2PKH output
{
'inputs': [{'input_script': 'P2PKH'}],
'outputs': {'p2pkh': 1},
'description': 'Single P2PKH -> P2PKH'
},
# 2. Two P2WPKH inputs -> two P2WPKH outputs
{
'inputs': [{'input_script': 'P2WPKH'}, {'input_script': 'P2WPKH'}],
'outputs': {'p2wpkh': 2},
'description': 'Two P2WPKH -> Two P2WPKH'
},
# 3. Single P2SH 2-of-3 multisig input -> two P2PKH outputs
{
'inputs': [{'input_script': 'P2SH', 'input_m': 2, 'input_n': 3}],
'outputs': {'p2pkh': 2},
'description': 'P2SH 2-of-3 multisig -> 2x P2PKH'
},
# 4. Mixed inputs: P2PKH + P2WPKH + P2TR -> P2PKH + P2WPKH
{
'inputs': [
{'input_script': 'P2PKH'},
{'input_script': 'P2WPKH'},
{'input_script': 'P2TR'}
],
'outputs': {'p2pkh': 1, 'p2wpkh': 1},
'description': 'Mixed inputs -> mixed outputs'
},
# 5. Two P2WSH multisig inputs -> P2WSH outputs
{
'inputs': [
{'input_script': 'P2WSH', 'input_m': 2, 'input_n': 3},
{'input_script': 'P2WSH', 'input_m': 1, 'input_n': 2}
],
'outputs': {'p2wsh': 2},
'description': 'Two P2WSH -> Two P2WSH'
}
]
for idx, test in enumerate(test_cases, 1):
# Step 1: calculate size
tx_size = calculate_transaction_size(test['inputs'], test['outputs'])
# Step 2: calculate fee
fee_sats = calculate_fee(tx_size['txVBytes'], fee_rate)
print(f"Test {idx}: {test['description']}")
print(f" txBytes: {tx_size['txBytes']}, txVBytes: {tx_size['txVBytes']}, txWeight: {tx_size['txWeight']}")
print(f" Fee (@ {fee_rate} sats/vbyte): {fee_sats} sats\n")
# Example usage
if __name__ == "__main__":
inputs = [
{'input_script': 'P2PKH'},
{'input_script': 'P2WPKH'},
{'input_script': 'P2SH', 'input_m': 2, 'input_n': 3},
]
output_counts = {
'p2pkh': 1,
'p2sh': 0,
'p2sh_p2wpkh': 1,
'p2sh_p2wsh': 0,
'p2wpkh': 0,
'p2wsh': 0,
'p2tr': 0
}
tx_size = calculate_transaction_size(inputs, output_counts)
print(tx_size)
print("*"*80)
run_tests()

48
django/finances/tx_math_tests.py
View file

@ -1,48 +0,0 @@
def run_tests():
test_cases = [
# 1. Single P2PKH input -> single P2PKH output
{
'inputs': [{'input_script': 'P2PKH'}],
'outputs': {'p2pkh': 1},
'description': 'Single P2PKH -> P2PKH'
},
# 2. Two P2WPKH inputs -> two P2WPKH outputs
{
'inputs': [{'input_script': 'P2WPKH'}, {'input_script': 'P2WPKH'}],
'outputs': {'p2wpkh': 2},
'description': 'Two P2WPKH -> Two P2WPKH'
},
# 3. Single P2SH 2-of-3 multisig input -> two P2PKH outputs
{
'inputs': [{'input_script': 'P2SH', 'input_m': 2, 'input_n': 3}],
'outputs': {'p2pkh': 2},
'description': 'P2SH 2-of-3 multisig -> 2x P2PKH'
},
# 4. Mixed inputs: P2PKH + P2WPKH + P2TR -> P2PKH + P2WPKH
{
'inputs': [
{'input_script': 'P2PKH'},
{'input_script': 'P2WPKH'},
{'input_script': 'P2TR'}
],
'outputs': {'p2pkh': 1, 'p2wpkh': 1},
'description': 'Mixed inputs -> mixed outputs'
},
# 5. Two P2WSH multisig inputs -> P2WSH outputs
{
'inputs': [
{'input_script': 'P2WSH', 'input_m': 2, 'input_n': 3},
{'input_script': 'P2WSH', 'input_m': 1, 'input_n': 2}
],
'outputs': {'p2wsh': 2},
'description': 'Two P2WSH -> Two P2WSH'
}
]
for idx, test in enumerate(test_cases, 1):
result = calculate_transaction_size(test['inputs'], test['outputs'])
print(f"Test {idx}: {test['description']}")
print(f" txBytes: {result['txBytes']}, txVBytes: {result['txVBytes']}, txWeight: {result['txWeight']}\n")
if __name__ == "__main__":
run_tests()

5
django/importer/forms.py
View file

@ -8,8 +8,3 @@ class UploadFileForm(forms.Form):
choices=IMPORTER_CHOICES
)
file = forms.FileField()
passphrase = forms.CharField(
widget=forms.PasswordInput(),
required=False,
max_length=100
)

9
django/importer/models.py
View file

@ -2,18 +2,15 @@ from django.db import models
from django.contrib.auth.models import User
from labelbase.models import Labelbase
from uuid_upload_path import upload_to
from django.conf import settings
IMPORTER_CHOICES = [
IMPORTER_CHOICES = (
("BIP-0329", "BIP-329 .jsonl"),
# TODO: ("BIP-0329-7z-enc" , "BIP-329 (encrypted) .7z"),
("csv-bluewallet", "BlueWallet .csv"),
("csv-bitbox", "BitBox .csv"),
("pocket-accointing", "Pocket Accointing .csv"),
]
("pocket-accointing", "Pocket Accointing .csv")
)
if settings.SELF_HOSTED:
IMPORTER_CHOICES.append(("samourai", "Samourai .txt, (v2)"))
class UploadedData(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)

4
django/importer/pocket.py
View file

@ -1,5 +1,7 @@
import csv
import json
import json
from labelbase.models import Label
from labelbase.models import Labelbase
def validate_csv_format(csv_file_path):

151
django/importer/samourai.py
View file

@ -1,151 +0,0 @@
import json
from Crypto.Cipher import AES
from Crypto.Hash import SHA256
from Crypto.Protocol.KDF import PBKDF2
import hashlib
import re
import base64
from labelbase.serializers import LabelSerializer
from labelbase.models import Label
import logging
logger = logging.getLogger('labelbase')
DefaultPBKDF2Iterations = 5000
DefaultPBKDF2HMACSHA256Iterations = 15000
DefaultSamouraiImportLabel = "Imported form samourai.txt"
def decrypt_v1(payload, password, iterations=DefaultPBKDF2Iterations):
# V1 uses PBKDF2 for key derivation and AES for decryption
AESBlockSize = 16
cipherdata = base64.b64decode(payload)
iv = cipherdata[:AESBlockSize]
input_data = cipherdata[AESBlockSize:]
key = PBKDF2(password, iv, dkLen=32, count=iterations)
cipher = AES.new(key, AES.MODE_CBC, iv)
decrypted = cipher.decrypt(input_data)
return decrypted.rstrip(b"\x00").decode('utf-8')
def decrypt_v2(payload, password, iterations=DefaultPBKDF2HMACSHA256Iterations):
# V2 uses SHA256 for key derivation and AES for decryption
encrypted_bytes = base64.b64decode(payload.replace("\n", ""))
salt = encrypted_bytes[8:16]
cipher_text = encrypted_bytes[16:]
key_iv = PBKDF2(password, salt, dkLen=48, count=iterations, hmac_hash_module=SHA256)
key = key_iv[:32]
iv = key_iv[32:]
cipher = AES.new(key, AES.MODE_CBC, iv)
decrypted = cipher.decrypt(cipher_text)
pad_len = decrypted[-1]
decrypted = decrypted[:-pad_len]
return decrypted.decode('utf-8')
def import_samourai_labels(labelbase, content, passphrase):
content = content.decode('utf-8')
pattern = re.compile(r'\{.*?\}')
match = pattern.search(content)
imported_lables = 0
payload = None
if match:
json_content = match.group(0)
try:
logger.info(f"json_content {json_content}")
data = json.loads(json_content)
logger.info(f"data: {data}")
version = data.get("version", 1)
payload = data.get("payload", "")
if payload:
if version in [1, "1"]:
decrypted_data = decrypt_v1(payload, passphrase)
elif version in [2, "2"]:
decrypted_data = decrypt_v2(payload, passphrase)
else:
logger.error(f"Unsupported backup version: {version}")
raise ValueError(f"Unsupported backup version: {version}")
logger.info(decrypted_data)
samourai_data = json.loads(decrypted_data)
logger.info(samourai_data)
"""
DOC/KB: If the labelbase where you import your samourai.txt into, labelbase will set the fingerprint,
"""
labels = Label.objects.filter(labelbase__id=labelbase.id)
if labels.count() == 0:
if not labelbase.fingerprint:
labelbase.fingerprint = samourai_data.get('wallet').get('fingerprint')
if samourai_data.get('wallet').get('testnet'):
labelbase.network == labelbase.TESTNET
else:
labelbase.network == labelbase.MAINNET
labelbase.save()
xpub = samourai_data.get('wallet', {}).get('accounts')[0].get('xpub')
ypub = samourai_data.get('wallet', {}).get('bip49_accounts')[0].get('ypub')
zpub = samourai_data.get('wallet', {}).get('bip84_accounts')[0].get('zpub')
for pub in [xpub, ypub, zpub]:
if pub:
_data = {
"type": Label.TYPE_XPUB,
"ref": pub,
"label": DefaultSamouraiImportLabel,
}
_data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=_data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
utxo_notes = samourai_data.get('meta', {}).get('utxo_notes')
logger.info(utxo_notes)
for note in utxo_notes:
_data = {
"type": Label.TYPE_TX,
"ref": note[0],
"label": note[1],
}
_data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=_data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
blocked_utxos = samourai_data.get('meta', {}).get('blocked_utxos',{}).get('blocked')
logger.info(blocked_utxos)
for utxo in blocked_utxos:
_data = {
"type": Label.TYPE_OUTPUT,
"ref": utxo.get('id','').replace("-", ":"),
"label": DefaultSamouraiImportLabel,
"spendable": False
}
_data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=_data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
return imported_lables
else:
print("No payload found in the JSON content.")
logger.error("No payload found in the JSON content.")
return imported_lables
except json.JSONDecodeError as e:
print(f"JSONDecodeError: {e}")
logger.error(f"JSONDecodeError: {e}")
return imported_lables
except Exception as ex:
print(f"An error occurred: {ex}")
logger.error(f"An error occurred: {ex}")
logger.error(ex, exc_info=True)
return imported_lables
else:
print("No JSON found in file.")
logger.error("No JSON found in file.")
return imported_lables

261
django/importer/tasks.py
View file

@ -2,10 +2,9 @@ from background_task import background
import json
import decimal
from labelbase.models import Labelbase
from labelbase.serializers import LabelSerializer
import logging
logger = logging.getLogger('labelbase')
from .models import UploadedData
@ -13,135 +12,93 @@ EOLSTOP = [b"", "", None, "\n"]
@background(schedule=1)
def process_uploaded_data(uploaded_data_id, passphrase=None, loop=None):
try:
uploaded_data = UploadedData.objects.get(pk=uploaded_data_id)
imported_lables = 0
labelbase = uploaded_data.labelbase
fp = uploaded_data.file.open()
def process_uploaded_data(uploaded_data_id, loop=None):
imported_lables = 0
uploaded_data = UploadedData.objects.get(pk=uploaded_data_id)
labelbase = uploaded_data.labelbase
fp = uploaded_data.file.open()
# BIP-0329
if uploaded_data.import_type == "BIP-0329":
while True:
buf = fp.readline()
if buf in EOLSTOP:
break
data = json.loads(buf)
logger.info(f"Parsed data: {data}")
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
elif uploaded_data.import_type == "BIP-0329-7z-enc":
# TODO: Implementation needed.
pass
elif uploaded_data.import_type == "samourai":
buf = fp.read()
logger.info(buf)
print(buf)
from .samourai import import_samourai_labels
import_samourai_labels(labelbase, buf, passphrase)
# Bitbox App
elif uploaded_data.import_type == "csv-bitbox":
while True:
buf = fp.readline()
if buf in EOLSTOP:
break
try:
buf = str(buf.decode("utf-8"))
sbuf = buf.split(",")
# Time,Type,Amount,Unit,Fee,Address,Transaction ID,Note
for elem in [("tx", 6), ("addr", 5)]:
data = {
"type": elem[0],
"ref": sbuf[elem[1]],
"label": " ".join(sbuf[7:]),
}
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
else:
messages.add_message(
request,
messages.ERROR,
'Could not process line "{}".'.format(buf),
)
except Exception as ex:
messages.add_message(
request,
messages.ERROR,
'Could not process line "{}", {}.'.format(buf, ex),
)
# Pocket Accointing
elif uploaded_data.import_type == "pocket-accointing":
fp.close()
csv_file_path = fp.name
mempool_api = labelbase.get_mempool_api()
from .pocket import validate_csv_format, parse_csv_to_json
if validate_csv_format(csv_file_path):
for item in parse_csv_to_json(csv_file_path):
label = "Got {} {} for {:.2f} {} with reference: {} #Pocket".format(item[0].get('outSellAmount'), item[1].get(
'inBuyAsset'), decimal.Decimal(item[2].get('inBuyAmount')), item[2].get('inBuyAsset'), item[1].get('operationId'))
txid = item[0].get('operationId')
tx = mempool_api.get_transaction(txid)
potential_utxos = []
vouts = tx.get("vout", [])
for i in range(len(vouts)):
if vouts[i].get('value', 0) == decimal.Decimal(item[0].get('outSellAmount'))*100000000:
potential_utxos.append("{}:{}".format(txid, i ))
data = {}
if len(potential_utxos) == 1:
# label UTXO/output of tx
data = {
"type": "output",
"ref": potential_utxos[0],
"label": label,
}
if len(potential_utxos) > 1:
# mark tx, add warning tag
data = {
"type": "tx",
"ref": txid,
"label": "{} {}".format(label, "#W001_UTXO_NOT_FOUND"),
}
if data:
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
else:
# messages.add_message(
# request,
# messages.ERROR,
# 'Could not process record "{}".'.format(item),
# )
pass
else:
print("ERROR") # TODO
# BlueWallet
elif uploaded_data.import_type == "csv-bluewallet":
header_row = True
while True:
buf = fp.readline()
if buf in EOLSTOP:
break
if header_row:
header_row = False
continue
try:
buf = str(buf.decode("utf-8"))
sbuf = buf.split(",")
# BIP-0329
if uploaded_data.import_type == "BIP-0329":
while True:
buf = fp.readline()
if buf in EOLSTOP:
break
data = json.loads(buf)
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
elif uploaded_data.import_type == "BIP-0329-7z-enc":
# TODO: Implementation needed.
pass
# Bitbox App
elif uploaded_data.import_type == "csv-bitbox":
while True:
buf = fp.readline()
if buf in EOLSTOP:
break
try:
buf = str(buf.decode("utf-8"))
sbuf = buf.split(",")
# Time,Type,Amount,Unit,Fee,Address,Transaction ID,Note
for elem in [("tx", 6), ("addr", 5)]:
data = {
"type": elem[0],
"ref": sbuf[elem[1]],
"label": " ".join(sbuf[7:]),
}
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
else:
messages.add_message(
request,
messages.ERROR,
'Could not process line "{}".'.format(buf),
)
except Exception as ex:
messages.add_message(
request,
messages.ERROR,
'Could not process line "{}", {}.'.format(buf, ex),
)
# Pocket Accointing
elif uploaded_data.import_type == "pocket-accointing":
fp.close()
csv_file_path = fp.name
mempool_api = labelbase.get_mempool_api()
from .pocket import validate_csv_format, parse_csv_to_json
if validate_csv_format(csv_file_path):
for item in parse_csv_to_json(csv_file_path):
label = "Got {} {} for {:.2f} {} with reference: {} #Pocket".format(item[0].get('outSellAmount'), item[1].get(
'inBuyAsset'), decimal.Decimal(item[2].get('inBuyAmount')), item[2].get('inBuyAsset'), item[1].get('operationId'))
txid = item[0].get('operationId')
tx = mempool_api.get_transaction(txid)
potential_utxos = []
vouts = tx.get("vout", [])
for i in range(len(vouts)):
if vouts[i].get('value', 0) == decimal.Decimal(item[0].get('outSellAmount'))*100000000:
potential_utxos.append("{}:{}".format(txid, i ))
data = {}
if len(potential_utxos) == 1:
# label UTXO/output of tx
data = {
"type": "output",
"ref": potential_utxos[0],
"label": label,
}
if len(potential_utxos) > 1:
# mark tx, add warning tag
data = {
"type": "tx",
"ref": sbuf[1],
"label": " ".join(sbuf[3:]),
"ref": txid,
"label": "{} {}".format(label, "#W001_UTXO_NOT_FOUND"),
}
if data:
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
@ -151,20 +108,50 @@ def process_uploaded_data(uploaded_data_id, passphrase=None, loop=None):
# messages.add_message(
# request,
# messages.ERROR,
# 'Could not process line "{}".'.format(buf),
# 'Could not process record "{}".'.format(item),
# )
pass
except Exception as ex:
else:
print("ERROR") # TODO
# BlueWallet
elif uploaded_data.import_type == "csv-bluewallet":
header_row = True
while True:
buf = fp.readline()
if buf in EOLSTOP:
break
if header_row:
header_row = False
continue
try:
buf = str(buf.decode("utf-8"))
sbuf = buf.split(",")
data = {
"type": "tx",
"ref": sbuf[1],
"label": " ".join(sbuf[3:]),
}
data["labelbase"] = labelbase.id
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()
imported_lables += 1
else:
# messages.add_message(
# request,
# messages.ERROR,
# 'Could not process line "{}", {}.'.format(buf, ex),
# 'Could not process line "{}".'.format(buf),
# )
pass
# Clean up – Note: Currently we delete the upload from the file system,
# later we can store the messages.add_message messages, the state and the
# amount of importet labels in it to propagate the messages to the
# frontend/user interface.
uploaded_data.delete()
except Exception as ex:
logger.error(ex, exc_info=True)
except Exception as ex:
# messages.add_message(
# request,
# messages.ERROR,
# 'Could not process line "{}", {}.'.format(buf, ex),
# )
pass
# Clean up – Note: Currently we delete the upload from the file system,
# later we can store the messages.add_message messages, the state and the
# amount of importet labels in it to propagate the messages to the
# frontend/user interface.
uploaded_data.delete()

25
django/importer/templatetags/importer_tags.py
View file

@ -10,4 +10,27 @@ def genericlabeluploadform(labelbase_id):
form.fields["labelbase_id"].initial = labelbase_id
form.fields["import_type"].initial = "BIP-0329"
return form
@register.simple_tag
def bip0329labeluploadform(labelbase_id):
form = UploadFileForm()
form.fields["labelbase_id"].initial = labelbase_id
form.fields["import_type"].initial = "BIP-0329"
return form
@register.simple_tag
def csvBlueWalletlabeluploadform(labelbase_id):
form = UploadFileForm()
form.fields["labelbase_id"].initial = labelbase_id
form.fields["import_type"].initial = "csv-bluewallet"
return form
@register.simple_tag
def csvBitBoxLabeluploadform(labelbase_id):
form = UploadFileForm()
form.fields["labelbase_id"].initial = labelbase_id
form.fields["import_type"].initial = "csv-bitbox"
return form

2
django/importer/views.py
View file

@ -30,7 +30,7 @@ def upload_labels(request):
file=request.FILES["file"],
)
# Schedule the background task to process the uploaded data
process_uploaded_data(uploaded_data.id, passphrase=form.cleaned_data.get("passphrase", None))
process_uploaded_data(uploaded_data.id)
messages.add_message(
request,
messages.INFO,

34
django/labelbase/admin.py
View file

@ -1,36 +1,6 @@
from django.contrib import admin
from .models import Labelbase
from .models import Label
from django.conf import settings
class LabelbaseAdmin(admin.ModelAdmin):
list_display = ['id', 'user', 'name', 'network', 'operation_mode']
list_filter = ['network', 'operation_mode']
search_fields = ['name', 'fingerprint']
class LabelAdmin(admin.ModelAdmin):
list_display = ['id', 'type', 'labelbase', 'label']
list_filter = ['type', 'labelbase__network']
search_fields = ['ref', 'label']
# Organize fields into logical sections
fieldsets = (
('Core BIP-329 Fields', {
'fields': ('labelbase', 'type', 'ref', 'label', 'origin', 'spendable')
}),
('Additional BIP-329 Fields', {
'fields': ('height', 'time', 'fee', 'value', 'rate', 'keypath', 'fmv', 'heights'),
'classes': ('collapse',), # Make this section collapsible
}),
('Internal', {
'fields': ('type_ref_hash',),
'classes': ('collapse',),
}),
)
readonly_fields = ['type_ref_hash']
if settings.DEBUG:
admin.site.register(Labelbase, LabelbaseAdmin)
admin.site.register(Label, LabelAdmin)
admin.site.register(Labelbase)
admin.site.register(Label)

5
django/labelbase/api.py
View file

@ -10,9 +10,6 @@ from labelbase.models import Labelbase, Label
from labelbase.serializers import LabelbaseSerializer, LabelSerializer
import logging
logger = logging.getLogger('labelbase')
class LabelbaseAPIView(APIView):
"""
Labelbase
@ -93,7 +90,7 @@ class LabelAPIView(APIView):
"spendable": request.data.get("spendable", "null"),
}
#logger.debug(f"data: {data}")
serializer = LabelSerializer(data=data)
if serializer.is_valid():
serializer.save()

2
django/labelbase/forms.py
View file

@ -36,7 +36,7 @@ class LabelForm(forms.ModelForm):
for field_name in self.fields:
self.fields[field_name].label = mark_safe(
f'<label class="bip329-attr">{self.fields[field_name].label}</label>')
class ExportLabelsForm(forms.Form):
""" """

54
django/labelbase/migrations/0012_auto_20251122_0800.py
View file

@ -1,54 +0,0 @@
# Generated by Django 3.2.25 on 2025-11-22 08:00
from django.db import migrations, models
import django_cryptography.fields
class Migration(migrations.Migration):
dependencies = [
('labelbase', '0011_alter_labelbase_operation_mode'),
]
operations = [
migrations.AddField(
model_name='label',
name='fee',
field=django_cryptography.fields.encrypt(models.CharField(blank=True, help_text='Transaction fee in satoshis (stored as string)', max_length=32, null=True)),
),
migrations.AddField(
model_name='label',
name='fmv',
field=django_cryptography.fields.encrypt(models.TextField(blank=True, help_text='Fair market value (JSON string)', null=True)),
),
migrations.AddField(
model_name='label',
name='height',
field=django_cryptography.fields.encrypt(models.CharField(blank=True, help_text='Block height where transaction was confirmed', max_length=16, null=True)),
),
migrations.AddField(
model_name='label',
name='heights',
field=django_cryptography.fields.encrypt(models.TextField(blank=True, help_text='Block heights for address activity (JSON array as string)', null=True)),
),
migrations.AddField(
model_name='label',
name='keypath',
field=django_cryptography.fields.encrypt(models.CharField(blank=True, help_text='Key derivation path (e.g., /1/123)', max_length=256, null=True)),
),
migrations.AddField(
model_name='label',
name='rate',
field=django_cryptography.fields.encrypt(models.TextField(blank=True, help_text='Exchange rate at transaction time (JSON string)', null=True)),
),
migrations.AddField(
model_name='label',
name='time',
field=django_cryptography.fields.encrypt(models.CharField(blank=True, help_text='ISO-8601 timestamp of the block', max_length=64, null=True)),
),
migrations.AddField(
model_name='label',
name='value',
field=django_cryptography.fields.encrypt(models.CharField(blank=True, help_text='Transaction value in satoshis, signed (stored as string)', max_length=32, null=True)),
),
]

194
django/labelbase/models.py
View file

@ -2,14 +2,12 @@ from django.db import models
from django.contrib.auth.models import User
from django.urls import reverse
from django_cryptography.fields import encrypt
from django.utils.safestring import mark_safe
from pymempool import MempoolAPI
from labellabor.utils import extract_fiat_value
from labelbase.utils import compute_type_ref_hash
from finances.models import OutputStat
from attachments.models import LabelAttachment
class Labelbase(models.Model):
"""
@ -113,13 +111,6 @@ class Labelbase(models.Model):
def get_hashtags_url(self):
return reverse('labelbase_hashtags', kwargs={'labelbase_id': self.id})
def get_xpub_url(self):
for label in self.label_set.all():
if label.type == "xpub": # and is_valid_xpub() ...
# returns the first xpub, works for single signature only at the moment.
return reverse('edit_label', kwargs={'pk': label.id})
return None
class Label(models.Model):
"""
@ -131,7 +122,7 @@ class Label(models.Model):
TYPE_PUBKEY = "pubkey"
TYPE_INPUT = "input"
TYPE_OUTPUT = "output"
TYPE_XPUB = "xpub"
TYPE_XPUT = "xpub"
TYPE_CHOICES = [
(TYPE_TX, "tx"),
@ -139,7 +130,7 @@ class Label(models.Model):
(TYPE_PUBKEY, "pubkey"),
(TYPE_INPUT, "input"),
(TYPE_OUTPUT, "output"),
(TYPE_XPUB, "xpub"),
(TYPE_XPUT, "xpub"),
]
type = models.CharField(
@ -181,77 +172,9 @@ class Label(models.Model):
)
)
labelbase = models.ForeignKey(
Labelbase,
on_delete=models.CASCADE
)
labelbase = models.ForeignKey(Labelbase, on_delete=models.CASCADE)
type_ref_hash = models.CharField(
max_length=64,
blank=True)
# All additional fields (from the BIP-329 upgrade) encrypted for maximum privacy
height = encrypt(
models.CharField(
max_length=16,
null=True,
blank=True,
help_text="Block height where transaction was confirmed"
)
)
time = encrypt(
models.CharField(
max_length=64,
null=True,
blank=True,
help_text="ISO-8601 timestamp of the block"
)
)
fee = encrypt(
models.CharField(
max_length=32,
null=True,
blank=True,
help_text="Transaction fee in satoshis (stored as string)"
)
)
value = encrypt(
models.CharField(
max_length=32,
null=True,
blank=True,
help_text="Transaction value in satoshis, signed (stored as string)"
)
)
rate = encrypt(
models.TextField(
null=True,
blank=True,
help_text="Exchange rate at transaction time (JSON string)"
)
)
keypath = encrypt(
models.CharField(
max_length=256,
null=True,
blank=True,
help_text="Key derivation path (e.g., /1/123)"
)
)
fmv = encrypt(
models.TextField(
null=True,
blank=True,
help_text="Fair market value (JSON string)"
)
)
heights = encrypt(
models.TextField(
null=True,
blank=True,
help_text="Block heights for address activity (JSON array as string)"
)
)
type_ref_hash = models.CharField(max_length=64, blank=True)
def get_extracted_fiat_value(self):
return extract_fiat_value(self.label)
@ -266,18 +189,10 @@ class Label(models.Model):
val, cur = extract_fiat_value(self.label)
return output.output_metrics_dict(tracked_fiat_value=val, fiat_currency=cur)
def get_label_attachment(self):
type_ref_hash = compute_type_ref_hash(self.type, self.ref)
label_attachment, _ = LabelAttachment.objects.get_or_create(
user=self.labelbase.user,
type_ref_hash=type_ref_hash,
network=self.labelbase.network)
return label_attachment
def get_absolute_url(self):
"""
Is used by "edit label" functionality.
This brings us back to the labelbase once the label was saved.
This brings us back to the labelbase once the lable was saved.
"""
return self.labelbase.get_absolute_url()
@ -299,102 +214,3 @@ class Label(models.Model):
except:
pass
return ""
def get_fee_health_status(self):
"""
Calculate fee health status for this label if it's a spendable unspent output.
"""
# Only calculate for spendable outputs
if self.type != self.TYPE_OUTPUT or not self.spendable:
return {
'status': None,
'fee_sats': None,
'value_sats': None,
'fee_percentage': None,
'threshold_healthy': None,
'threshold_warning': None,
'threshold_high': None
}
try:
value_sats = int(self.value) if self.value else None
except (ValueError, TypeError):
value_sats = None
if not value_sats or value_sats <= 0:
return {
'status': None,
'fee_sats': None,
'value_sats': value_sats,
'fee_percentage': None,
'threshold_healthy': None,
'threshold_warning': None,
'threshold_high': None
}
# Get user's fee rate from profile
user_fee_rate = self.labelbase.user.profile.my_fee # sats per vbyte
threshold_adjustment = self.labelbase.user.profile.my_fee_rate_threshold # percentage points
# Use P2WPKH as default - most common modern type
# Simple 1-in, 2-out transaction
from finances.tx_math import calculate_transaction_size, calculate_fee
inputs = [{'input_script': 'P2WPKH'}]
output_counts = {'p2wpkh': 2}
tx_size = calculate_transaction_size(inputs, output_counts)
fee_sats = calculate_fee(tx_size['txVBytes'], user_fee_rate)
# Calculate fee as percentage of output value
fee_percentage = (fee_sats / value_sats) * 100
# Define thresholds (base + user adjustment)
threshold_healthy = 1.0 + threshold_adjustment
threshold_warning = 3.0 + threshold_adjustment
# Determine status
if fee_percentage < threshold_healthy:
status = 'green'
elif fee_percentage < threshold_warning:
status = 'yellow'
else:
status = 'red'
return {
'status': status,
'fee_sats': fee_sats,
'value_sats': value_sats,
'fee_percentage': round(fee_percentage, 3),
'threshold_healthy': threshold_healthy,
'threshold_warning': threshold_warning,
'threshold_high': threshold_warning
}
@property
def get_fee_health_status_display(self):
"""
Returns text representation of fee health status for DataTables display.
"""
health = self.get_fee_health_status()
if not health['status']:
return ''
status_map = {
'green': '🟢',
'yellow': '🟡',
'red': '🔴'
}
# FIXME: escaping in data tables
#status_map = {
# 'green': '<span data-feather="circle-check" class="align-text-bottom"></span>',
# 'yellow': '<span data-feather="alert-circle" class="align-text-bottom"></span>',
# 'red': '<span color:red; data-feather="alert-triangle" class="align-text-bottom"></span>'
#}
emoji = status_map.get(health['status'], '')
return f"{emoji} {health['fee_percentage']}%"

166
django/labelbase/serializers.py
View file

@ -1,9 +1,8 @@
import json
from rest_framework import serializers
from labelbase.models import Labelbase, Label
class LabelSerializer_v1(serializers.ModelSerializer):
class LabelSerializer(serializers.ModelSerializer):
class Meta:
model = Label
fields = [
@ -20,169 +19,6 @@ class LabelSerializer_v1(serializers.ModelSerializer):
]
class LabelSerializer(serializers.ModelSerializer):
# Additional BIP-329 fields
height = serializers.IntegerField(required=False, allow_null=True)
time = serializers.CharField(required=False, allow_null=True, allow_blank=True)
fee = serializers.IntegerField(required=False, allow_null=True)
value = serializers.IntegerField(required=False, allow_null=True)
rate = serializers.JSONField(required=False, allow_null=True)
keypath = serializers.CharField(required=False, allow_null=True, allow_blank=True)
fmv = serializers.JSONField(required=False, allow_null=True)
heights = serializers.ListField(
child=serializers.IntegerField(),
required=False,
allow_null=True
)
class Meta:
model = Label
fields = [
"id",
"labelbase",
"type",
"ref",
"label",
"origin",
"spendable",
# Additional BIP-329 fields
"height",
"time",
"fee",
"value",
"rate",
"keypath",
"fmv",
"heights",
]
read_only_fields = [
"id",
]
def validate(self, data):
"""Validate BIP-329 field combinations based on type"""
label_type = data.get('type')
# Define valid fields per type (from BIP-329 spec)
valid_fields = {
'tx': {'height', 'time', 'fee', 'value', 'rate'},
'addr': {'keypath', 'heights'},
'pubkey': {'keypath'},
'input': {'keypath', 'value', 'fmv', 'height', 'time'},
'output': {'spendable', 'keypath', 'value', 'fmv', 'height', 'time'},
'xpub': set()
}
# Get allowed additional fields for this type
allowed = valid_fields.get(label_type, set())
# Check for invalid field combinations
additional_fields = {'height', 'time', 'fee', 'value', 'rate', 'keypath', 'fmv', 'heights', 'spendable'}
for field in additional_fields:
if field in data and data[field] is not None:
# Allow origin for all types
if field == 'origin':
continue
# Check if field is valid for this type
if field not in allowed and field in additional_fields - {'origin'}:
# Remove invalid field instead of raising error (for compatibility)
data.pop(field, None)
return data
def create(self, validated_data):
"""Override create to convert data types for storage"""
# Convert integers to strings for storage
if 'height' in validated_data and validated_data['height'] is not None:
validated_data['height'] = str(validated_data['height'])
if 'fee' in validated_data and validated_data['fee'] is not None:
validated_data['fee'] = str(validated_data['fee'])
if 'value' in validated_data and validated_data['value'] is not None:
validated_data['value'] = str(validated_data['value'])
# Convert JSON objects to strings
if 'rate' in validated_data and validated_data['rate'] is not None:
validated_data['rate'] = json.dumps(validated_data['rate'])
if 'fmv' in validated_data and validated_data['fmv'] is not None:
validated_data['fmv'] = json.dumps(validated_data['fmv'])
if 'heights' in validated_data and validated_data['heights'] is not None:
validated_data['heights'] = json.dumps(validated_data['heights'])
instance = super().create(validated_data)
return instance
def update(self, instance, validated_data):
"""Override update to convert data types for storage"""
# Convert integers to strings for storage
if 'height' in validated_data and validated_data['height'] is not None:
validated_data['height'] = str(validated_data['height'])
if 'fee' in validated_data and validated_data['fee'] is not None:
validated_data['fee'] = str(validated_data['fee'])
if 'value' in validated_data and validated_data['value'] is not None:
validated_data['value'] = str(validated_data['value'])
# Convert JSON objects to strings
if 'rate' in validated_data and validated_data['rate'] is not None:
validated_data['rate'] = json.dumps(validated_data['rate'])
if 'fmv' in validated_data and validated_data['fmv'] is not None:
validated_data['fmv'] = json.dumps(validated_data['fmv'])
if 'heights' in validated_data and validated_data['heights'] is not None:
validated_data['heights'] = json.dumps(validated_data['heights'])
return super().update(instance, validated_data)
def to_representation(self, instance):
"""Convert stored data back to API format"""
data = super().to_representation(instance)
# Convert string integers back to integers
if data.get('height'):
try:
data['height'] = int(data['height'])
except (ValueError, TypeError):
data['height'] = None
if data.get('fee'):
try:
data['fee'] = int(data['fee'])
except (ValueError, TypeError):
data['fee'] = None
if data.get('value'):
try:
data['value'] = int(data['value'])
except (ValueError, TypeError):
data['value'] = None
# Convert JSON strings back to objects
if data.get('rate'):
try:
data['rate'] = json.loads(data['rate'])
except (json.JSONDecodeError, TypeError):
data['rate'] = None
if data.get('fmv'):
try:
data['fmv'] = json.loads(data['fmv'])
except (json.JSONDecodeError, TypeError):
data['fmv'] = None
if data.get('heights'):
try:
data['heights'] = json.loads(data['heights'])
except (json.JSONDecodeError, TypeError):
data['heights'] = None
return data
class LabelbaseSerializer(serializers.ModelSerializer):
def __init__(self, *args, **kwargs):
super(LabelbaseSerializer, self).__init__(*args, **kwargs)

BIN
django/labelbase/static/Cloud.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 89 KiB

BIN
django/labelbase/static/bitcoin.pdf
View file

Binary file not shown.

7
django/labelbase/static/js/bootstrap.bundle.min.js vendored
View file

File diff suppressed because one or more lines are too long

13
django/labelbase/static/js/feather.min.js vendored
View file

File diff suppressed because one or more lines are too long

2
django/labelbase/static/js/jquery-3.5.1.min.js vendored
View file

File diff suppressed because one or more lines are too long

4
django/labelbase/static/js/jquery.dataTables.min.js vendored
View file

File diff suppressed because one or more lines are too long

9
django/labelbase/static/js/pdfobject.min.js vendored
View file

File diff suppressed because one or more lines are too long

1
django/labelbase/static/js/qrcode.min.js vendored
View file

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show more