Skip to content

07_CONFIGURATION.md

Latest commit

 

History

History
573 lines (329 loc) · 5.49 KB

07_CONFIGURATION.md

File metadata and controls

573 lines (329 loc) · 5.49 KB
Raw

Configuration Specification

Project: Collect Files

Version: 1.0

Status: Draft

1. Purpose

This document defines every configurable aspect of the Collect Files engine.

Configuration determines how the engine behaves.

Configuration never determines what the engine is.

Business logic must remain independent from configuration.

2. Design Principles

Configuration must satisfy the following principles.

  • Explicit
  • Immutable during execution
  • Strongly typed
  • Validated before execution
  • Platform-independent
  • Deterministic
  • Human-readable
  • Easy to serialize

3. Configuration Sources

Configuration may originate from:

Priority 1

Command-line arguments

Priority 2

Configuration file (future)

Priority 3

Environment variables (future)

Priority 4

Built-in defaults

The highest-priority source always overrides lower priorities.

4. The Job Object

The Job dataclass is the single source of truth for runtime configuration.

Every pipeline stage receives the same immutable Job instance.

No module may modify it.

5. Runtime Configuration

Source Directory

Field

source: Path

Requirements

  • must exist
  • must be readable
  • must be a directory
  • must not be empty string

Destination Directory

Field

destination: Path

Requirements

  • may not exist
  • parent directory must be writable
  • may equal source only if explicitly supported

Destination creation belongs to the execution stage.

Extensions

Field

extensions: frozenset[str]

Normalization Rules

  • lowercase
  • unique
  • leading dot required

Examples

Valid

.png
.jpg
.pdf

Invalid

PNG
Pdf
jpeg

(after normalization these become valid)

Operation

Field

operation: Operation

Supported values

MOVE

COPY

Exactly one operation must be selected.

Sort Mode

Field

sort_mode: SortMode

Supported values

EXIF

CREATED

MODIFIED

NAME

SIZE

Sorting must always be deterministic.

Files Per Folder

Field

files_per_folder: int

Requirements

> 0

Recommended

100

250

500

1000

Dry Run

Field

dry_run: bool

Behavior

All planning stages execute.

Filesystem mutations are skipped.

Follow Symlinks

Field

follow_symlinks: bool

Behavior

False

Ignore symbolic links.

True

Traverse symbolic links.

6. Validation Rules

Validation occurs before pipeline execution.

Validation failures are fatal.

Rules

Source exists.

Source is directory.

Extensions not empty.

Files per folder > 0.

Operation specified.

Sort mode specified.

Job internally consistent.

7. Invalid Configurations

Examples

files_per_folder = 0

Rejected.

extensions = {}

Rejected.

source = missing_directory

Rejected.

operation = None

Rejected.

Execution must never begin with an invalid Job.

8. Default Values

Recommended defaults

Operation

MOVE

Sort Mode

EXIF

Files per Folder

100

Dry Run

False

Follow Symlinks

False

Defaults may evolve between versions.

9. Internal Constants

Configuration differs from internal constants.

Configuration is user-controlled.

Constants are developer-controlled.

Examples of constants

DEFAULT_BATCH_SIZE

DEFAULT_LOG_LEVEL

DEFAULT_FOLDER_PATTERN

SUPPORTED_TIMESTAMP_FIELDS

Constants belong in:

core/constants.py

not inside config.py.

10. Environment Variables (Future)

Reserved names

COLLECT_FILES_LOG_LEVEL

COLLECT_FILES_CACHE

COLLECT_FILES_CONFIG

COLLECT_FILES_TEMP

Environment variables must never override explicit CLI arguments.

11. Configuration File (Future)

Preferred format

TOML

Example

[source]
path="/data"

[destination]
path="/archive"

[execution]
operation="move"

sort="exif"

files_per_folder=100

dry_run=false

YAML may be supported later.

JSON is optional.

12. CLI Mapping

Example

collect-files \
  --source ~/Pictures \
  --destination ~/Archive \
  --extensions png jpg \
  --operation move \
  --sort exif \
  --batch-size 100

maps directly to

Job(...)

CLI parsing must never leak into business modules.

13. Immutability

After validation,

Job becomes immutable.

Pipeline stages may read configuration.

They may never modify it.

This guarantees deterministic execution.

14. Configuration Lifecycle

CLI

↓

Parse

↓

Normalize

↓

Validate

↓

Create Job

↓

Freeze

↓

Execute Engine

Job creation occurs exactly once.

15. Configuration Errors

Fatal examples

Invalid directory.

Invalid extension.

Invalid enum.

Negative batch size.

Unknown sort mode.

Unknown operation.

Errors should clearly explain:

  • what failed
  • expected value
  • received value
  • suggested correction

16. Future Configuration Options

Reserved options

Maximum recursion depth.

Ignore hidden files.

Include hidden files.

Minimum file size.

Maximum file size.

Filename filters.

Regex filters.

Timestamp priority list.

Hash duplicate detection.

Plugin configuration.

These additions must preserve backward compatibility.

17. Configuration Invariants

Configuration must always satisfy:

Exactly one Job.

Immutable during execution.

Validated before pipeline start.

Platform-independent.

Serializable.

Strongly typed.

Independent of CLI implementation.

Free from business logic.