nixos/docs/agents.md

# NixOS Configuration Repository - AI Agent Guide

This guide provides everything you need to know to work efficiently with this NixOS configuration repository. Keep this file handy when making changes.

## Repository Overview

This repository manages NixOS configurations for multiple systems using flakes and Home Manager. All systems are defined declaratively in Nix expressions.

### Managed Systems

| Hostname | Type | Hardware | Special Notes |
|----------|------|----------|---------------|
| `framework` | Desktop/Laptop | Framework AMD AI 300 series | User: jsutter, Uses nixos-hardware module |
| `aurora` | Desktop | Custom | Users: jsutter, isutter, aksutter |
| `labrizor` | Desktop | Custom | User: jsutter |
| `skip01` | Server | Custom | Headless, Docker support |
| `warp` | Server | Custom | Minimal headless configuration |

### Core Technologies

- **NixOS**: 25.11 (stable channel)
- **Nixpkgs Unstable**: For packages requiring latest versions
- **Home Manager**: Release 25.11
- **Plasma Manager**: KDE Plasma configuration management
- **Flakes**: For reproducible, declarative system definitions
- **App Flakes**: Standalone flakes for server applications (Immich, Octofriend, etc.)

## Project Structure

```
nixos/
├── appflakes/           # Standalone flake inputs for server applications
│   ├── immich/          # Self-hosted photo/video backup (Docker Compose)
│   └── octofriend/      # CLI coding assistant
├── desktop/             # Desktop environment configurations
│   ├── plasma.nix       # KDE Plasma + Plasma Manager
│   ├── dev.nix          # Development tools and languages
│   ├── office.nix       # Office applications
│   ├── gaming.nix       # Games and gaming platforms
│   ├── media.nix        # Audio/video software
│   ├── virtualization.nix  # VMs and containers
│   ├── tailscale.nix    # VPN configuration
│   ├── 2dprinting.nix   # Printer setup
│   ├── 3dprinting.nix   # 3D printer tools
│   ├── dnm.nix          # Framework-specific config
│   └── stp-elc-udmshare.nix  # Specific mount/share config
├── docs/                # Documentation
│   └── agents.md        # This file
├── servers/             # Server service modules
│   ├── common.nix       # Common server settings (Docker enablement)
│   ├── forgejo.nix      # Forgejo Git server module
│   └── hugo.nix         # Hugo web server module (Docker compose)
├── systems/             # Hardware-specific configurations
│   ├── common.nix       # Shared system settings (bootloader, nix config, etc.)
│   ├── framework.nix    # Framework laptop hardware config
│   ├── aurora.nix       # Aurora desktop hardware config
│   ├── labrizor.nix     # Labrizor desktop hardware config
│   ├── skip01.nix       # Skip01 server config
│   └── warp.nix         # Warp server config
├── users/               # User configurations with Home Manager
│   ├── common-home.nix  # Shared user packages and settings
│   ├── jsutter.nix      # Primary user configuration
│   ├── isutter.nix      # Secondary user configuration
│   └── aksutter.nix     # Secondary user configuration
└── flake.nix            # Top-level flake definition
```

### App Flakes Architecture


The `appflakes/` directory contains standalone flakes for server applications. These are self-contained, reusable flake definitions that can be used independently or integrated into NixOS configurations.

**Current App Flakes:**
- `immich/`: Self-hosted photo and video backup solution (Docker Compose deployment)
- `octofriend/`: CLI coding assistant packaged for Nix/NixOS

**App Flake Structure:**
```nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    # Additional inputs as needed
  };

  outputs = { self, nixpkgs, flake-utils, ... }:
    {
      # NixOS module (if applicable)
      nixosModules.app-name = nixosModule;

      # Package definition
      packages = flake-utils.lib.eachDefaultSystem (system: { ... });

      # App entry (if CLI tool)
      apps.default = ...;
    };
}
```

**Using App Flakes:**

1. **As a local input in main flake.nix:**
   ```nix
   inputs = {
     immich = { url = "path:./appflakes/immich"; };
   };

   outputs = { self, nixpkgs, immich, ... }: {
     nixosConfigurations.server = nixpkgs.lib.nixosSystem {
       modules = [
         immich.nixosModules.immich-docker
         ./systems/server.nix
       ];
     };
   };
   ```

2. **Directly on a system:**
   ```bash
   # Use app flake package on any system
   nix run .#appflakes/octofriend
   ```

### Server Service Modules

The `servers/` directory contains NixOS modules for server-specific services. These differ from appflakes in that they are modules designed to be included in NixOS configurations rather than standalone flakes.

**Server Module Pattern:**

1. **Define options module-wide:**
   ```nix
   { config, pkgs, lib, ... }:
   let
     cfg = config.services.service-name;
   in {
     options.services.service-name = {
       enable = mkEnableOption "Service description";

       domain = mkOption {
         type = types.str;
         example = "service.example.com";
         description = "Domain name for the service";
       };

       # Additional options...
     };
   }
   ```

2. **Configure service only if enabled:**
   ```nix
   config = mkIf cfg.enable {
     # Service dependencies (PostgreSQL, Nginx, etc.)

     # Main service configuration
     services.service-name = {
       enable = true;
       settings = { ... };
     };

     # Reverse proxy configuration (Nginx)
     services.nginx.virtualHosts.${cfg.domain} = { ... };

     # Firewall rules
     networking.firewall.allowedTCPPorts = [ ... ];

     # Backup systemd services
     systemd.services.service-name-backup = { ... };
   };
   ```

**Common Server Module Elements:**

- **Database backing:** PostgreSQL (`services.postgresql`)
- **Reverse proxy:** Nginx with SSL/TLS (`services.nginx`)
- **Firewall:** Open required ports (`networking.firewall`)
- **Backups:** Systemd timers for automated backups
- **Logging:** Centralized logging configuration
- **Monitoring:** Optional Prometheus/metrics endpoints
- **Security:** Hardening options, secret management

**Creating a New Server Module:**

1. Create `servers/<service-name>.nix`
2. Follow the standard module pattern with options
3. Use `mkIf cfg.enable` for conditional configuration
4. Include backup systemd service with timer
5. Add firewall rules if service is network-accessible
6. Document required secrets/password files
7. Test on a headless system first (`skip01` or `warp`)

**Server Module Organization:**

- `servers/common.nix`: Shared server infrastructure (Docker, etc.)
- `servers/forgejo.nix`: Forgejo Git server complete configuration
- `servers/hugo.nix`: Hugo static site server with Docker compose
- Additional server modules can be added as needed

**Headless System Configuration:**

Headless systems (`skip01`, `warp`) follow simplified patterns:

```nix
{
  networking.hostName = "hostname";

  # Minimal hardware configuration
  boot.initrd.availableKernelModules = [ "nvme" "xhci_pci" "ahci" ];
  boot.kernelModules = [ "kvm-intel" ];

  # Enable container support
  virtualisation.docker.enable = true;

  # SSH for remote administration
  services.openssh = {
    enable = true;
    settings.PasswordAuthentication = false;
  };

  # Disable desktop services
  services.xserver.enable = false;
}
```

**Deploying Server Service Modules:**

1. Add server module to system configuration:
   ```nix
   in {
     nixosConfigurations.skip01 = nixpkgs.lib.nixosSystem {
       modules = [
         ./systems/common.nix
         ./servers/forgejo.nix  # Add server module
         ./systems/skip01.nix
         { services.forgejo.enable = true; }
       ];
     };
   }
   ```

2. Configure service options:
   ```nix
   {
     services.forgejo = {
       domain = "git.example.com";
       database.passwordFile = "/run/keys/forgejo-db";
     };
   }
   ```

3. Test on server system:
   ```bash
   nixos-rebuild dry-run --flake .#skip01
   sudo nixos-rebuild switch --flake .#skip01
   ```

**Secrets Management for Servers:**

Server services often require secrets (database passwords, API keys, OAuth secrets). Store these securely:

- Use `/run/keys/` for runtime secrets (systemd credentials)
- Use environment files with restricted permissions
- Never commit secrets to the repository
- Example: `passwordFile` options that point to secure paths

**Backup Strategy for Server Services:**

Server modules include automated backup services:

```nix
systemd.services.service-backup = {
  description = "Service backup service";
  serviceConfig = {
    Type = "oneshot";
    User = "service-user";
    ExecStart = "${pkgs.service}/bin/service backup";
  };
};

systemd.timers.service-backup = {
  wantedBy = [ "timers.target" ];
  timerConfig = {
    OnCalendar = "daily";
    Persistent = true;
  };
};
```

**Testing Server Modules:**

Always test server modules on headless systems before deploying to production:

```bash
# Check flake outputs
nix flake show

# Dry-run for server system
sudo nixos-rebuild dry-run --flake .#server-hostname

# Build without switching
sudo nixos-rebuild build --flake .#server-hostname

# Switch to new configuration
sudo nixos-rebuild switch --flake .#server-hostname
```

**Monitoring and Maintenance:**

- Check service status: `systemctl status service-name`
- View logs: `journalctl -u service-name`
- Monitor backup timers: `systemctl list-timers`
- Verify firewall rules: `nft list ruleset`
- Check Docker containers: `docker ps` (if using Docker-based services)

```

## Core Principles

### 1. Declarative Configuration
- All system state must be defined in Nix expressions
- Never use imperative commands for permanent changes
- Every configuration change should be reproducible

### 2. Modular Architecture
- Maintain separation between hardware, software, and user configurations
- Use existing modules before creating new ones
- Follow the established directory structure strictly

### 3. Security First
- Never expose sensitive data (passwords, private keys, API tokens)
- Use proper user privilege separation
- Validate all external inputs and configurations

## Configuration Patterns

### Function Parameters

Always include the appropriate parameters based on what the module needs:

```nix
# For modules using unstable packages
{ config, pkgs, pkgs-unstable, lib, ... }:

# For stable-only modules
{ config, pkgs, lib, ... }:

# For modules using special args from flake
{ config, pkgs, pkgs-unstable, lib, octofriend, ... }:

# For Home Manager modules
{ pkgs, ... }:
```

### System Configuration Pattern

```nix
{ config, lib, pkgs, ... }:
{
  # Always include hostname first
  networking.hostName = "system-name";
  
  # Hardware configuration
  fileSystems."/" = { 
    device = "/dev/disk/by-partlabel/primary";
    fsType = "ext4";
  };
  fileSystems."/boot" = {
    device = "/dev/disk/by-partlabel/ESP";
    fsType = "vfat";
  };

  # System-specific settings
  # Group related configurations together
}
```

### User Configuration Pattern

```nix
{ config, pkgs, pkgs-unstable, home-manager, ... }:
{
  # System user definition
  users.users.username = {
    shell = pkgs.zsh;
    isNormalUser = true;
    description = "User Description";
    extraGroups = [ "networkmanager" "wheel" ];  # Only essential groups
  };

  # Home Manager configuration
  home-manager.users.username = {
    home.username = "username";
    home.homeDirectory = "/home/username";
    home.stateVersion = "25.05";

    # User configurations
  };
}
```

### Desktop Module Pattern

```nix
{ config, pkgs, pkgs-unstable, lib, ... }:
{
  # System packages for this category
  environment.systemPackages = with pkgs; [
    package1
    package2
  ];

  # Configuration options
  services.service-name.enable = true;

  # Home Manager integrations (if applicable)
}
```

## Package Management Guidelines

### Stable vs Unstable Packages

```nix
# PREFERRED: Use stable packages by default
environment.systemPackages = with pkgs; [
  firefox
  git
  vim
];

# ACCEPTABLE: Use unstable when necessary
# Add comments explaining why unstable is needed
environment.systemPackages = with pkgs; [
  firefox
  pkgs-unstable.windsurf  # Latest features not in stable
];
```

### Package Selection Criteria

1. **Use stable unless**: Package doesn't exist OR needs newer version
2. **Document reasons**: Always comment why unstable is used
3. **Test thoroughly**: Unstable packages may have issues
4. **Review regularly**: Check if unstable packages can move to stable

### Common Package Sets by Module

- `dev.nix`: IDEs, languages, development tools
- `office.nix`: LibreOffice, productivity apps
- `gaming.nix`: Steam, Lutris, game launchers
- `media.nix`: Audio/video players, editors
- `virtualization.nix`: VirtualBox, Docker, other virt tools
- `plasma.nix`: KDE Plasma applications and themes

## System-Specific Information

### Framework (`framework.nix`)
- Uses `nixos-hardware` module: `framework-amd-ai-300-series`
- Additional modules: `dnm.nix`, `stp-elc-udmshare.nix`
- Single user: jsutter

### Aurora (`aurora.nix`)
- Multi-user system: jsutter, isutter, aksutter
- Additional modules: `3dprinting.nix`, `2dprinting.nix`
- No specialized hardware module

### Labrizor (`labrizor.nix`)
- Single user: jsutter
- Additional modules: `3dprinting.nix`
- No specialized hardware module

### Common Settings (`systems/common.nix`)
Includes:
- Hardware firmware and fwupd
- NetworkManager with VPN plugins (OpenVPN, OpenConnect)
- Systemd-boot EFI bootloader
- Latest kernel
- Nix settings with experimental features (flakes, nix-command)
- Weekly garbage collection (7-day retention)
- Auto-optimized Nix store
- Trusted users: root, jsutter
- Timezone: America/Los_Angeles
- Locale: en_US.UTF-8
- PipeWire audio stack
- SSH service enabled
- Zsh shell enabled
- Passwordless sudo for wheel group
- Flatpak enabled
- USB plugdev group rules

### Server Common Settings (`servers/common.nix`)
Includes:
- Docker containerization enabled for server services

## File Naming Conventions

- Use lowercase with hyphens: `vpn-config.nix`, `development-tools.nix`
- System configurations must match hostname: `framework.nix`, `aurora.nix`
- User configurations must match username: `jsutter.nix`, `isutter.nix`
- Common configurations: `common.nix`, `common-home.nix`

## Code Style

- Use 2-space indentation
- Align attribute lists consistently
- Add comments for non-obvious configurations
- Group related configurations together

Example:
```nix
{
  # Audio configuration
  sound.enable = true;
  hardware.pulseaudio.enable = false;
  security.rtkit.enable = true;
  services.pipewire = {
    enable = true;
    alsa.enable = true;
    alsa.support32Bit = true;
    pulse.enable = true;
  };
}
```

Server module example:
```nix
{ config, pkgs, lib, ... }:

let
  cfg = config.services.example-service;
in {
  options.services.example-service = {
    enable = mkEnableOption "Example service";
    domain = mkOption {
      type = types.str;
      example = "service.example.com";
      description = "Domain name for the service";
    };
    port = mkOption {
      type = types.port;
      default = 8080;
      description = "Service port number";
    };
  };

  config = mkIf cfg.enable {
    systemd.services.example-service = {
      description = "Example service";
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        ExecStart = "${pkgs.example}/bin/example --port ${toString cfg.port}";
        Restart = "on-failure";
      };
    };

    services.nginx.virtualHosts.${cfg.domain} = {
      forceSSL = true;
      enableACME = true;
      locations."/".proxyPass = "http://localhost:${toString cfg.port}";
    };

    networking.firewall.allowedTCPPorts = [ 80 443 ];
  };
}
```

## Common Workflows

### Adding a New Package

1. Determine appropriate module (desktop/, systems/, users/)
2. Check if package exists in stable channel: `nix search nixpkgs package-name`
3. Use unstable only if necessary with documentation
4. Add to appropriate environment.systemPackages or home.packages
5. Run dry-run before applying
6. Update relevant documentation if it affects multiple systems

Example:
```nix
# In desktop/dev.nix
environment.systemPackages = with pkgs; [
  # Added for Python development
  python312
  python312Packages.pip
  pkgs-unstable.windsurf  # Latest AI features not in stable
];
```

### Creating a New Desktop Module

1. Create file in `desktop/` with descriptive name
2. Include proper function parameters
3. Add packages, services, and configurations
4. Test module independently
5. Add to `commonDesktopModules` in flake.nix if for all desktops
6. Add to specific system in flake.nix if selective

### Adding a New System

1. Create `systems/<hostname>.nix` following existing patterns
2. Set `networking.hostName = "<hostname>"`
3. Configure hardware (fileSystems, hardware-specific settings)
4. Add system entry to `flake.nix`:
```nix
new-system = mkSystem {
  modules = commonDesktopModules ++ [
    ./systems/new-system.nix
    ./users/username.nix
  ];
};
```
5. Test with dry-run and build

### Modifying User Configuration

1. Edit appropriate file in `users/`
2. For shared changes: modify `common-home.nix`
3. For user-specific changes: modify `<username>.nix`
4. Home Manager configuration goes in `home-manager.users.<username>`

### Creating a New Server Module

1. Create `servers/<service-name>.nix` following the module pattern
2. Define options using `mkOption` for all configurable parameters
3. Use `mkIf cfg.enable` to wrap all configuration
4. Include database, reverse proxy (Nginx), and firewall configuration
5. Add automated backup systemd service with timer
6. Document required secrets (passwordFile paths, API keys)
7. Test on headless system first

### Creating a New App Flake

1. Create `appflakes/<app-name>/` directory with `flake.nix`
2. Include `nixpkgs` and `flake-utils` as inputs
3. Define outputs with `packages` (and optionally `nixosModules`, `apps`)
4. Use `flake-utils.lib.eachDefaultSystem` for multi-system support
5. Add local input reference in main `flake.nix` if needed
6. Test build with `nix build .#appflakes/<app-name>`

### Adding a Server Service to a System

1. Choose target system (typically headless: `skip01` or `warp`)
2. Add server module to system's modules list in `flake.nix`:
```nix
nixosConfigurations.server = mkSystem {
  modules = [
    ./systems/common.nix
    ./servers/common.nix
    ./servers/service-name.nix
    ./systems/server.nix
    { services.service-name.enable = true; }
  ];
};
```
3. Configure service-specific options in host config or separate file
4. Set up secret files (e.g., `/run/keys/service-db`)
5. Run dry-run and test before switching

## Testing and Validation

### Pre-commit Testing

ALWAYS run these commands before submitting changes:

```bash
# 1. Syntax check
nix flake check

# 2. Dry run for affected systems (run for ALL affected)
sudo nixos-rebuild dry-run --flake .#framework
sudo nixos-rebuild dry-run --flake .#aurora
sudo nixos-rebuild dry-run --flake .#labrizor

# 3. Check flake outputs
nix flake show

# For server configurations, also test:
sudo nixos-rebuild dry-run --flake .#skip01
sudo nixos-rebuild dry-run --flake .#warp
```

### Applying Changes

```bash
# Build and switch (active system only)
sudo nixos-rebuild switch --flake .#hostname

# Build only (test without switching)
sudo nixos-rebuild build --flake .#hostname

# Build and boot (adds to GRUB menu)
sudo nixos-rebuild boot --flake .#hostname
```

### Build Verification

- Test on at least one system before merging
- Verify all services start correctly: `systemctl status`
- Check for package conflicts
- Validate hardware-specific configurations
- For server modules: verify Docker containers run, check systemd timers, validate firewall rules

### Updating Packages

```bash
# Update flake inputs
nix flake update

# Check what changed
nix flake diff

# Rebuild after update
sudo nixos-rebuild switch --flake .#hostname
```

## Security Rules

### Sensitive Data Handling

- **NEVER** commit plaintext passwords
- **NEVER** commit private SSH keys
- **NEVER** commit API tokens or secrets
- **ALWAYS** use hashed passwords: `hashedPassword = "$6$..."`
- **CONSIDER** using sops-nix for advanced secret management

### User Privilege Management

```nix
# GOOD: Minimal necessary groups
extraGroups = [ "networkmanager" "wheel" "audio" ];

# BAD: Over-privileged users
extraGroups = [ "networkmanager" "wheel" "audio" "video" "input" "disk" "lp" "scanner" ];
```

### SSH Key Management

```nix
# GOOD: Use authorizedKeys for SSH access
openssh.authorizedKeys.keys = [
  "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBNVUh+RrcOSMRV6qysnsdPs5AyK8dSm4QhhnwgpikyI user@domain"
];

# FORBIDDEN: Never include private keys
```

## Rollback Procedures

### Configuration Rollback

```bash
# List available generations
sudo nix-env --list-generations --profile /nix/var/nix/profiles/system

# Rollback to previous working generation
sudo nixos-rebuild switch --profile-name generation-123

# Or use rollback shortcut
sudo nixos-rebuild switch --rollback

# Rollback specific system
sudo nixos-rebuild switch --rollback --flake .#hostname
```

### Emergency System Recovery

If a system fails to boot after changes:

1. Boot from installation media
2. Mount root filesystem: `mount /dev/disk/by-partlabel/primary /mnt`
3. Roll back: `sudo nixos-rebuild switch --rollback --install-bootloader /mnt`
4. Identify and fix configuration issues
5. Test before rebooting

### Booting into Previous Generations

At boot time (GRUB menu):
1. Hold Shift during boot to see GRUB menu
2. Select "Advanced options for NixOS"
3. Choose previous generation

## NixOS Installation (New System)

```bash
# Partition disk (example for NVMe)
sudo parted /dev/nvme0n1 -- mklabel gpt
sudo parted /dev/nvme0n1 -- mkpart primary ext4 512MB 100%
sudo parted /dev/nvme0n1 -- mkpart ESP fat32 1MB 512MB
sudo parted /dev/nvme0n1 -- set 2 esp on
sudo mkfs.ext4 /dev/disk/by-partlabel/primary
sudo mkfs.vfat /dev/disk/by-partlabel/ESP

# Mount filesystems
sudo mount -o rw /dev/disk/by-partlabel/primary /mnt/
sudo mkdir /mnt/boot
sudo mount -o rw /dev/disk/by-partlabel/ESP /mnt/boot/
sudo mkdir /mnt/root
sudo git clone https://git.symbiotrip.com/jsutter/nixos /mnt/root/nixos

# Install (no root password, impure for git access)
sudo -i
cd /mnt/root/nixos
nixos-install --flake .#<name> --no-root-password --impure

# Set password after install
nixos-enter --root '/mnt'
passwd jsutter
```

## Module Dependencies

### Known Dependencies

- `plasma.nix` depends on: `common.nix`, `home-manager`
- `dev.nix` is included in all desktop systems
- All desktop systems include: `common.nix`, `plasma.nix`, `dev.nix`, `office.nix`, `gaming.nix`, `media.nix`, `virtualization.nix`, `tailscale.nix`
- User files include Home Manager configuration

### Avoiding Circular Dependencies

- Don't import system modules into user modules
- Use `common.nix` for shared system settings only
- Use `common-home.nix` for shared user settings only
- Hardware-specific modules should not depend on desktop modules

## Emergency Tools

### Useful Commands

```bash
# Check what packages provide a command
nix-locate bin/commandname

# Search for packages
nix search nixpkgs package-name
nix search nixpkgs-unstable package-name

# Check Nix store size
du -sh /nix/store

# Garbage collect manually
sudo nix-collect-garbage -d

# Profile Nix operations
sudo nix-env --profile /nix/var/nix/profiles/system --list-generations

# Check hardware info
lshw
lspci
lsusb
lsblk
hwinfo
```

## Troubleshooting Common Issues

### Build Errors

1. Read the error message carefully
2. Check for syntax errors in the modified file
3. Verify all attributes are closed properly
4. Run `nix flake check` to validate syntax

### Service Failures

```bash
# Check service status
systemctl status service-name

# View service logs
journalctl -u service-name -e

# Restart service
systemctl restart service-name
```

### Package Conflicts

- If two modules try to enable the same service with different settings
- Use `lib.mkForce` to override settings if necessary
- Or consolidate the configuration in one place

## Flakes Reference

### Input Structure

```nix
inputs = {
  nixpkgs.url = "nixpkgs/nixos-25.11";              # Stable
  nixpkgs-unstable.url = "nixpkgs/nixos-unstable";  # Unstable
  nixos-hardware.url = "github:NixOS/nixos-hardware/master";
  home-manager.url = "github:nix-community/home-manager/release-25.11";
  plasma-manager.url = "github:nix-community/plasma-manager";
  octofriend.url = "path:./appflakes/octofriend";   # Local flake
};
```

### Special Args

Available in all modules:
- `pkgs`: Stable Nix packages
- `pkgs-unstable`: Unstable Nix packages
- `octofriend`: Local octofriend flake

### Output Structure

```nix
outputs = {
  self,                             # Self reference
  nixpkgs,                          # Stable packages
  nixpkgs-unstable,                 # Unstable packages
  nixos-hardware,                   # Hardware modules
  home-manager,                     # Home Manager
  plasma-manager,                   # Plasma Manager
  octofriend                        # Custom app flake
}: {
  nixosConfigurations = {
    framework = { ... };  # Define each system
    aurora = { ... };
    labrizor = { ... };
  };
};
```

## Documentation Requirements

### Code Comments

- Explain non-obvious configurations
- Document reasons for unstable package usage
- Note hardware-specific requirements
- Reference external dependencies

Example:
```nix
# Use unstable neovim for LuaLS integration not in stable
pkgs-unstable.neovim  # Latest features needed for Lua development
```

### Commit Messages

Use clear, descriptive commit messages:
- Reference affected systems/modules
- Explain reasoning for significant changes
- Include testing steps when applicable

Example:
```
feat(framework): Add 3D printing support for Prusa printers

- Added 3dprinting.nix module with PrusaSlicer and Cura
- Installed USB rules for Prusa devices
- Added to framework system configuration

Tested: Prusa MK4 recognized and can start print via USB
```

## Performance Considerations

### Nix Settings

The repository uses optimized Nix settings:
- `auto-optimise-store`: Auto-optimizes the Nix store
- `keep-outputs = true`: Keeps build outputs
- `keep-derivations = true`: Keeps derivations for evaluation
- `download-buffer-size`: 512MB for faster downloads

### Garbage Collection

Automatically runs weekly:
- Deletes builds older than 7 days
- Run manually: `sudo nix-collect-garbage -d`

### Build Caching

Consider using `nix-serve` or Cachix for:
- Faster rebuilds
- Shared cache across systems
- CI/CD integration

## Resources

### Official Documentation
- [NixOS Manual](https://nixos.org/manual/nixos/stable/)
- [Home Manager Manual](https://nix-community.github.io/home-manager/)
- [Nix Pills](https://nixos.org/guides/nix-pills/)
- [NixOS Wiki](https://nixos.wiki/)

### Community Resources
- [NixOS Discourse](https://discourse.nixos.org/)
- [NixOS subreddit](https://reddit.com/r/NixOS/)
- [NixOS Matrix](https://matrix.to/#/#nixos:nixos.org)

### Package Search
- [search.nixos.org](https://search.nixos.org/packages)
- Command line: `nix search nixpkgs package-name`

## Quick Reference

### Essential Commands

| Action | Command |
|--------|---------|
| Update system | `sudo nixos-rebuild switch --flake .#hostname` |
| Dry run | `sudo nixos-rebuild dry-run --flake .#hostname` |
| Rollback | `sudo nixos-rebuild switch --rollback` |
| Update flakes | `nix flake update` |
| Check flakes | `nix flake check` |
| Search packages | `nix search nixpkgs name` |
| Garbage collect | `sudo nix-collect-garbage -d` |

**Server-Specific Commands:**
| Action | Command |
|--------|---------|
| Check service status | `systemctl status service-name` |
| View service logs | `journalctl -u service-name` |
| List active timers | `systemctl list-timers` |
| Check Docker containers | `docker ps` |
| View Docker logs | `docker logs container-name` |
| Restart docker-compose stack | `docker-compose -f /etc/compose/service/docker-compose.yml restart` |
| Check firewall rules | `nft list ruleset` |
| Verify SSL certificates | `certbot certificates` |
| List backup directory | `ls -lh /var/backups/service-name` |
| Run manual backup | `systemctl start service-backup` |

### File Paths Reference

| Type | Location |
|------|----------|
| Common system settings | `systems/common.nix` |
| System hardware config | `systems/<hostname>.nix` |
| Common user settings | `users/common-home.nix` |
| User config | `users/<username>.nix` |
| Desktop apps | `desktop/<category>.nix` |
| Server common settings | `servers/common.nix` |
| Server service modules | `servers/<service-name>.nix` |
| App flakes | `appflakes/<app-name>/flake.nix` |
| Docker compose configs | `/etc/compose/<service>/docker-compose.yml` |
| Backup directories | `/var/backups/<service-name>/` |
| Runtime secrets | `/run/keys/<service-name>` |
| Flake definition | `flake.nix` |

## Maintenance & Changelog

This section tracks changes to the repository structure and when this document was updated. Keep this in sync!

### Current Repository State
- **Last Updated**: February 2026
- **NixOS Version**: 25.11
- **Managed Systems**: 5 (framework, aurora, labrizor, skip01, warp)
- **Users**: jsutter (primary), isutter, aksutter
- **App Flakes**: 2 (immich, octofriend)
- **Server Modules**: 3 (common, forgejo, hugo)

### Repository Change Log

| Date | Change | Updated Sections |
|------|--------|------------------|
| 2026-02 - Created | Created `agents.md` with comprehensive guide | All sections |
| 2026-02 - Removed | Removed .clinerules file, consolidated into agents.md | Maintenance section |
| 2026-02 - Cleanup | Removed unused gnome.nix module, common-headless.nix; removed allowBroken setting | System structure |
| 2026-02 - Cleanup | Removed commented packages across multiple files; fixed trailing whitespace | Code quality |
| 2026-02 - Cleanup | Removed .clinerules file; deleted gnome.nix (unused); common-headless.nix (duplicated) | File structure |
| 2026-02 - Improvement | Added result, *.swp, *~ to .gitignore | Maintenance |
| 2026-02 - Added | Added appflakes directory (immich, octofriend) and servers directory (forgejo, hugo) | Project structure, all server sections |
| 2026-02 - Documentation | Added comprehensive server module patterns, appflake architecture, server workflows, server-specific commands | Repository Overview, Workflows, Quick Reference |

### When to Update This Document

Update `agents.md` when any of the following occur:

- **Structural Changes**:
  - New system added to `flake.nix`
  - New user configuration added
  - New or renamed desktop module
  - Directory restructuring
  - New app flake added to `appflakes/`
  - New server module added to `servers/`

- **Configuration Changes**:
  - New shared settings in `systems/common.nix`
  - New shared user settings in `users/common-home.nix`
  - New shared settings in `servers/common.nix`
  - Changes to module pattern or function parameters
  - New package categories or organizational changes
  - Changes to server module standard pattern
  - Updates to headless system configuration patterns

- **Technology Updates**:
  - NixOS version changes
  - Channel updates (stable/unstable)
  - New major inputs added to flake
  - Significant Nix configuration changes
  - New app flake with different architecture
  - Changes to Docker/compose deployment patterns

- **Procedure Changes**:
  - New testing workflow
  - Changed emergency procedures
  - Updated installation steps
  - New common commands
  - New server deployment or backup procedures
  - Changes to secrets management approach for servers

- **Server Changes**:
  - New server service added to production
  - Changes to backup strategy for server services
  - Updates to monitoring or logging for servers
  - New headless system deployed or configured
  - Security hardening changes for server modules

### Update Template

When making changes, add an entry using this format:

```
| YYYY-MM - One-line description | More detail if needed | Section(s) affected |
| --- | --- | --- |
```

### Version History for agents.md

- **v1.1** (2026-02): Added comprehensive server module patterns, appflake architecture, server workflows, and server-specific commands
- **v1.0** (2025-01): Initial creation with full repository documentation

## Getting Help

When unsure about something:
1. Check this documentation first
2. Review existing similar configurations
3. Search NixOS manual and wiki
4. Test changes with dry-run before applying

---

**Remember**: This repository values declarative, reproducible configurations. When in doubt, prefer explicit configuration over implicit behavior. Test thoroughly, document changes, and never skip the dry-run step!