From 7c014f6534176bc8b9241c856c7f78149ab45974 Mon Sep 17 00:00:00 2001
From: Julian Sutter <jsutter@symbiotip.com>
Date: Mon, 16 Feb 2026 22:05:18 -0800
Subject: [PATCH] Add development standards and procedures for app deployment

- Add curl timeout requirement (5 seconds)
- Add comprehensive 8-step workflow for new application deployment
- Add troubleshooting procedures for domain availability and SSL certs
- Add infrastructure roadmap for Borg backup server and sops-nix
- Update README with session start protocol and infrastructure tasks
---
 README.md | 17 ++++++++++-
 agents.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 101 insertions(+), 4 deletions(-)

diff --git a/README.md b/README.md
index 44c5f15..2f4a973 100755
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
 # NixOS Configuration Repository
 
 ## Session Start Protocol
-Always begin by reading agents.md for workflow instructions.
+Always begin by reading agents.md for workflow instructions and development standards.
 
 ## System Installation
 
@@ -37,3 +37,18 @@ nixos-install --flake .#<name> --no-root-password --impure
 nixos-enter --root '/mnt'
 passwd jsutter
 ```
+
+## Infrastructure Roadmap
+
+### Planned Work
+
+#### Borg Backup Server
+- Set up a dedicated Borg backup server for automated backups
+- Configure backup schedules for critical systems
+- Implement retention policies and pruning rules
+
+#### Secrets Management with sops-nix
+- Implement sops-nix for secrets management
+- Move all hardcoded secrets from server configs into sops-nix
+- Set up encryption keys and key rotation policies
+- Document the secrets management workflow
diff --git a/agents.md b/agents.md
index 190b507..6c517ec 100644
--- a/agents.md
+++ b/agents.md
@@ -23,11 +23,93 @@
 4. Check success message: `"Done. The new configuration is /nix/store/..."`
 
 ## Important
-- Server configs may contain hardcoded credentials - use agenix or systemd credentials for production
-- **Always carefully inspect the NixOS wiki for instructions before adding new applications to the repo**
-- Both warp and skip build successfully
+- Server configs may contain hardcoded credentials
+- Always carefully inspect the NixOS wiki for instructions before adding new applications to the repo
+- Do not editorialize or pass judgement.  Be a robot.
 - Repository root: `/home/jsutter/src/nixos`
 
+## Development Standards
+
+### curl Usage
+When using curl commands, always set a timeout to 5 seconds:
+```bash
+curl --max-time 5 <url>
+# or
+curl -m 5 <url>
+```
+
+## Procedures
+
+### Adding a New Application to the Repository
+
+1. **Gather Requirements**
+   - Ask the user what server to deploy to
+   - Ask the user what domain name the app will be available on
+
+2. **Research and Planning**
+   - Build a brief plan to construct the app
+   - Review the NixOS wiki (https://nixos.org/nixos/manual/) to see if packages are available
+   - Check for existing NixOS modules or services that can be used
+   - Identify dependencies and configuration requirements
+
+3. **Implementation**
+   - Follow the plan constructed in step 2
+   - Add the necessary configuration to the appropriate server file in `servers/`
+   - Include nginx reverse proxy configuration if the app needs to be accessible via HTTP/HTTPS
+   - Add any required firewall rules, services, or users
+
+4. **Local Testing**
+   - Test the build locally: `nixos-rebuild build --flake .#<system>`
+   - Refine the configuration until the build succeeds
+   - Review the generated configuration for correctness
+
+5. **Remote Deployment**
+   - Push the repo to the remote machine: `git push origin master`
+   - SSH to the target server
+   - Pull the changes: `cd ~/src/nixos && git pull origin master`
+   - Build and switch to the new config: `sudo nixos-rebuild switch --flake .#`
+
+6. **Verification**
+   - Ensure the service is available on the chosen domain
+   - Ensure the certificate is issued by Let's Encrypt (check with: `openssl s_client -connect <domain>:443 | openssl x509 -noout -issuer`)
+   - Test basic functionality of the application
+
+7. **Troubleshooting**
+   - If the app isn't available on the chosen domain:
+     - Check service status: `systemctl status <service>`
+     - Check nginx logs: `journalctl -u nginx -f`
+     - Check application logs: `journalctl -u <service> -f`
+     - Verify DNS resolution
+     - Check firewall rules
+     - Verify nginx configuration syntax: `nginx -t`
+   - If the certificate isn't issued by Let's Encrypt:
+     - Check ACME challenge configuration
+     - Verify domain ownership record
+     - Check Let's Encrypt logs: `journalctl -u certbot -f`
+     - Manually trigger certificate renewal if needed
+
+8. **Process Improvement**
+   - After successful deployment, propose 3 suggestions to add to agents.md that would help with future deployments:
+     1. [Specific pattern or configuration approach discovered]
+     2. [Common pitfall to avoid]
+     3. [Useful command or tool discovered]
+
+### Infrastructure Tasks
+
+#### Planned Work
+
+1. **Borg Backup Server**
+   - Set up a dedicated Borg backup server for automated backups
+   - Configure backup schedules for critical systems
+   - Implement retention policies and pruning rules
+
+2. **Secrets Management with sops-nix**
+   - Implement sops-nix for secrets management
+   - Move all hardcoded secrets from server configs into sops-nix
+   - Set up encryption keys and key rotation policies
+   - Document the secrets management workflow
+
+
 ## Remote System Management
 
 ### Access Systems