nhyatt/bitwarden
1
0
Fork 0
mirror of https://github.com/bitwarden/server.git synced 2025-04-09 23:28:12 -05:00
Code

updated readme(s)

Browse Source
This commit is contained in:
Robert Y 2025-03-13 16:36:04 -06:00
parent 4737d9d95c
commit ea1e9fd1f2
2 changed files with 235 additions and 62 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

141
util/DbSeederUtility/README.md Normal file
View File

@ -0,0 +1,141 @@
# Bitwarden Database Seeder Utility
A command-line utility for generating and managing test data for Bitwarden databases.
## Overview
DbSeederUtility is an executable wrapper around the Seeder class library that provides a convenient command-line interface for:
1. **Generating** test data as JSON files
2. **Loading** test data into a database
3. **Extracting** database data into seed files
4. **Generating and loading** data in a single operation
## Commands
The utility provides the following commands:
### generate
Generates seed data as JSON files.
```
DbSeeder.exe generate --users <count> --ciphers-per-user <count> --seed-name <name>
```
Options:
- `-u, --users`: Number of users to generate
- `-c, --ciphers-per-user`: Number of ciphers per user to generate
- `-n, --seed-name`: Name for the seed data files
Example:
```
DbSeeder.exe generate --users 10 --ciphers-per-user 5 --seed-name test_data
```
### load
Loads seed data from JSON files into the database.
```
DbSeeder.exe load --seed-name <name> [--timestamp <timestamp>] [--dry-run]
```
Options:
- `-n, --seed-name`: Name of the seed data to load
- `-t, --timestamp`: Specific timestamp of the seed data to load (defaults to most recent)
- `-d, --dry-run`: Validate the seed data without actually loading it
Example:
```
DbSeeder.exe load --seed-name test_data
```
### generate-direct-load
Generates seed data and loads it directly into the database without creating JSON files.
```
DbSeeder.exe generate-direct-load --users <count> --ciphers-per-user <count> --seed-name <name>
```
Options:
- `-u, --users`: Number of users to generate
- `-c, --ciphers-per-user`: Number of ciphers per user to generate
- `-n, --seed-name`: Name identifier for this seed operation
Example:
```
DbSeeder.exe generate-direct-load --users 3 --ciphers-per-user 5 --seed-name direct_test_data
```
### extract
Extracts data from the database into seed files.
```
DbSeeder.exe extract --seed-name <name>
```
Options:
- `-n, --seed-name`: Name for the extracted seed
Example:
```
DbSeeder.exe extract --seed-name extracted_data
```
## Configuration
DbSeederUtility uses the same configuration as the Seeder library. See the Seeder README for details on configuration options and file structure.
## Installation
The utility can be built and run as a .NET 8 application:
```
dotnet build
dotnet run -- <command> [options]
```
Or directly using the compiled executable:
```
DbSeeder.exe <command> [options]
```
## Examples
### Generate and load test data
```bash
# Generate 10 users, each with 5 ciphers
DbSeeder.exe generate --users 10 --ciphers-per-user 5 --seed-name demo_data
# Load the generated data
DbSeeder.exe load --seed-name demo_data
```
### Extract and reload data
```bash
# Extract existing data
DbSeeder.exe extract --seed-name production_backup
# Load the extracted data
DbSeeder.exe load --seed-name production_backup
```
### One-step generation and loading
```bash
# Generate and load in one step
DbSeeder.exe generate-direct-load --users 5 --ciphers-per-user 10 --seed-name quick_test
```
## Dependencies
This utility depends on:
- The Seeder class library
- CommandDotNet for command-line parsing
- .NET 8.0 runtime

156
util/seeder/README.md
View File

@ -1,98 +1,130 @@
# Bitwarden Database Seeder # Bitwarden Database Seeder
The Bitwarden Database Seeder utility is a tool for generating and loading test data for Bitwarden databases. A class library for generating, loading, and extracting test data for Bitwarden databases.
## Features ## Overview
- Generate random user accounts with associated ciphers (passwords, secure notes, etc.) The Seeder library provides functionality to:
- Load generated seeds into the database
- Configure database connections using the same configuration approach as the Bitwarden Migrator 1. **Generate** realistic test data for Bitwarden, including users and ciphers
2. **Load** previously generated test data into the database
3. **Extract** existing data from a database into seed files
4. **Generate and load** data in a single operation
## Project Structure
The project is organized into these main components:
### Commands
- **ExtractCommand** - Extracts existing data from the database into seed files
- **GenerateCommand** - Generates new test data as seed files or directly loads it
- **LoadCommand** - Loads previously generated seed files into the database
### Services
- **DatabaseContext** - EF Core DbContext connecting to the configured database
- **DatabaseService** - Provides database operations (save, retrieve, clear data)
- **EncryptionService** - Handles security operations (password hashing, encryption)
- **SeederService** - Core service that generates realistic test data using Bogus
### Settings
- **GlobalSettings** - Configuration model for database connections
- **GlobalSettingsFactory** - Loads and caches settings from config sources
## Usage ## Usage
The seeder utility can be run in two ways: The Seeder library is designed to be used by the DbSeederUtility executable. For direct usage in code:
1. Using `dotnet run` from the source: ```csharp
``` // Get seeder service from dependency injection
cd util/DbSeederUtility var seederService = serviceProvider.GetRequiredService<ISeederService>();
dotnet run -- generate-seeds --users 10 --ciphers 5 --output ./seeds
dotnet run -- load-seeds --path ./seeds
```
2. As a standalone executable: // Generate seed data
``` await seederService.GenerateSeedsAsync(
DbSeeder.exe generate-seeds --users 10 --ciphers 5 --output ./seeds userCount: 10,
DbSeeder.exe load-seeds --path ./seeds ciphersPerUser: 5,
``` seedName: "test_data"
);
## Commands // Load seed data
await seederService.LoadSeedsAsync(
seedName: "test_data",
timestamp: null // Use most recent if null
);
### Generate Seeds // Extract data
await seederService.ExtractSeedsAsync(
seedName: "extracted_data"
);
Generates random seed data for users and ciphers. // Generate and load in one step
await seederService.GenerateAndLoadSeedsAsync(
``` userCount: 10,
DbSeeder.exe generate-seeds [options] ciphersPerUser: 5,
seedName: "direct_load"
Options: );
-u, --users <NUMBER> Number of users to generate (default: 10)
-c, --ciphers <NUMBER> Number of ciphers per user (default: 5)
-o, --output <DIRECTORY> Output directory for seed files (default: seeds)
```
### Load Seeds
Loads generated seed data into the database.
```
DbSeeder.exe load-seeds [options]
Options:
-p, --path <DIRECTORY> Path to the directory containing seed data
--dry-run Preview the operation without making changes
``` ```
## Configuration ## Configuration
The utility uses the same configuration approach as other Bitwarden utilities: The library uses the following configuration sources (in order of precedence):
1. **User Secrets** - For local development 1. Environment variables
2. **appsettings.json** - For general settings 2. User secrets (with ID "Bit.Seeder")
3. **Environment variables** - For deployment environments 3. appsettings.{Environment}.json
4. appsettings.json
### Database Configuration The expected configuration structure is:
Configure the database connection in user secrets or appsettings.json:
```json ```json
{ {
"globalSettings": { "globalSettings": {
"databaseProvider": "postgresql", // or "sqlserver", "mysql", "sqlite" "selfHosted": true,
"postgreSql": { "databaseProvider": "postgres",
"connectionString": "Host=localhost;Port=5432;Database=vault_dev;Username=postgres;Password=YOURPASSWORD;Include Error Detail=true"
},
"sqlServer": { "sqlServer": {
"connectionString": "Data Source=localhost;Initial Catalog=vault_dev;Integrated Security=SSPI;MultipleActiveResultSets=true" "connectionString": "..."
},
"postgreSql": {
"connectionString": "..."
},
"mySql": {
"connectionString": "..."
},
"sqlite": {
"connectionString": "..."
} }
} }
} }
``` ```
## Building the Executable ## Seed File Structure
To build the standalone executable: Seed files are organized as follows:
``` ```
cd util seeds/
.\publish-seeder.ps1 ├── {seed_name}/
│ └── {timestamp}/
│ ├── users/
│ │ └── users.json
│ └── ciphers/
│ ├── {user_id1}.json
│ ├── {user_id2}.json
│ └── ...
``` ```
This will create the executable in the `util/Seeder/publish/win-x64` directory. You can also specify a different runtime: ## Dependencies
``` - **EntityFrameworkCore** - Database access
.\publish-seeder.ps1 -runtime linux-x64 - **Bogus** - Realistic test data generation
``` - **CommandDotNet** - Used by DbSeederUtility for CLI commands
- **DataProtection** - Used for secure data handling
## Integration with Bitwarden Server ## Best Practices
The seeder utility is designed to work seamlessly with Bitwarden Server. It uses the same database models and configuration approach as the server, ensuring compatibility with the core repository. - Clear the database before loading new seed data
- Use consistent seed names for related operations
- Store sensitive connection strings in user secrets or environment variables
- Use the DbSeederUtility executable for command-line operations