Răsfoiți Sursa

docs: Hardening, hybrid landscape, documentation index, and US-203 Taiga tasks alignment

Lange François 1 lună în urmă
părinte
comite
d2ef984626

BIN
Project.pdf


BIN
Retro Planning.pdf


+ 77 - 12
docs/Backup_Procedure.md

@@ -1,13 +1,78 @@
 # $Id$
-# Database Backup Procedure
-
-## Automated Backups
-The system utilizes a cron job pointing to `backup_db.sh`.
-- The script dynamically detects the active MySQL container name (`food-mysql-1` or `food_project-mysql-1`) for high-availability robustness.
-- It executes `mysqldump` directly inside the detected MySQL container.
-- Outputs are piped to `gzip` and stored in `/backups`.
-- A 7-day retention policy automatically purges old backups using `find ... -mtime +7 -exec rm`.
-
-## Manual Restore
-To manually restore a backup (adjust container name to `food-mysql-1` or `food_project-mysql-1` as appropriate):
-`gunzip < backups/food_db_20260507_0200.sql.gz | docker exec -i food-mysql-1 mysql -u root -proot_pass food_db`
+# Database Backup and Restore Procedure
+
+## 1. Overview & Policy
+To guarantee clinical records integrity and high availability, Local Food AI enforces a strict backup schedule.
+- **Scope**: Includes MySQL schemas (`food_db`), user profiles (`app_auth`), and configuration states.
+- **Retention Plan**: Automated daily backups with a strict 7-day rolling window purge.
+- **Storage Location**: Stored securely inside the persistent `/backups` directory on the host server.
+
+---
+
+## 2. Automated Daily Backups
+The automated backup mechanism runs via a host cron job pointing to `backup_db.sh`.
+- The script dynamically detects the active MySQL container name (`food-mysql-1` or `food_project-mysql-1`).
+- It executes `mysqldump` directly inside the container without exposing root passwords to shell logs.
+- Outputs are compressed via `gzip` and timestamped: `food_db_YYYYMMDD_HHMM.sql.gz`.
+
+### Cron Configuration Example:
+To run the backup daily at 02:00 AM, add the following to `/etc/crontab`:
+```bash
+0 2 * * * root /bin/bash /c/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/backup_db.sh >> /var/log/backup_db.log 2>&1
+```
+
+---
+
+## 3. Manual Backup Execution
+If a system migration or major upgrade is scheduled, perform a manual dump using the following command:
+```bash
+# 1. Navigate to the project directory
+cd /c/Users/lanfr144/Documents/DOPRO1/Antigravity/Food
+
+# 2. Run the backup wrapper
+bash backup_db.sh
+```
+Verify the output exists inside the backups folder:
+```bash
+ls -lh backups/
+```
+
+---
+
+## 4. Step-by-Step Restore Procedure
+In the event of database corruption or hardware failure, follow these exact steps to restore the database.
+
+### Step 4.1: Identify the Target Backup File
+List available files and pick the desired timestamp:
+```bash
+ls -la backups/
+# Example Target: backups/food_db_20260521_1100.sql.gz
+```
+
+### Step 4.2: Verify MySQL Container Health
+Ensure the MySQL service container is running and healthy:
+```bash
+docker ps --filter name=mysql
+```
+
+### Step 4.3: Execute Restore Stream
+Decompress the backup on-the-fly and pipe it directly into the running MySQL container:
+```bash
+# Adjust the container name ('food-mysql-1' or 'food_project-mysql-1') based on active deployment
+gunzip < backups/food_db_20260521_1100.sql.gz | docker exec -i food-mysql-1 mysql -u root -proot_pass food_db
+```
+
+### Step 4.4: Verify Restored Tables
+Log in to the database and query the core table to confirm the tables are intact and populated:
+```bash
+docker exec -it food-mysql-1 mysql -u food_reader -preader_pass food_db -e "SELECT COUNT(*) FROM products_core;"
+```
+Expected result: A count of OpenFoodFacts entries (typically > 10,000 records).
+
+---
+
+## 5. Verification & Health Check Loops
+Operators must verify the backup archive integrity weekly:
+1. Copy the `.gz` backup to a local testing workspace.
+2. Run `gzip -t backups/filename.sql.gz` to ensure the archive is not corrupted.
+3. Test restoring to a local fallback container instance to verify data accessibility.

BIN
docs/Backup_Procedure.pdf


BIN
docs/Data_Ingestion.pdf


BIN
docs/Final_Report.pdf


BIN
docs/Installation_Guide.pdf


+ 184 - 0
docs/Operator_Installation_Guide.md

@@ -0,0 +1,184 @@
+# $Id$
+# Local Food AI - Detailed Operator Installation Guide
+
+This document is a step-by-step installation, mapping, configuration, and verification manual for deploying the **Local Food AI** system in an enterprise environment. It covers hybrid hypervisor infrastructure (WSL2, Hyper-V, and VirtualBox), cross-node networking, SNMPv3 monitoring, alert channels, and acceptance testing.
+
+---
+
+## 1. Pre-Deployment Operator Survey (Pre-requisites Gathering)
+Before running installation scripts, the operator **must** collect the following physical/virtual infrastructure parameters and store them in the deployment matrix:
+
+| REQUIRED PARAMETER | OPERATOR INPUT / DESCRIPTION |
+| :--- | :--- |
+| **Deployment Workstation IP** | e.g., 192.168.1.50 |
+| **Hyper-V Host VM IP** | e.g., 192.168.130.170 |
+| **VirtualBox Host VM IP** | e.g., 192.168.130.161 |
+| **SSH Key Location (Private)** | e.g., `~/.ssh/id_rsa` |
+| **SMTP Relay Password** | e.g., `********` (For Zabbix/App password reset email) |
+| **Teams/Discord Webhook URL** | e.g., `https://discord.com/api/webhooks/...` |
+
+---
+
+## 2. Platform Mapping: Which Container Goes Where?
+
+To maximize CPU/GPU efficiency and secure database read/writes, services are distributed across three distinct environments:
+
+| COMPONENT CONTAINER | DEPLOYMENT ENVIRONMENT | WHY |
+| :--- | :--- | :--- |
+| **streamlit-app (app.py)** | Local WSL2 (Windows) | Low-latency rendering and direct client access |
+| **mysql (Database Node)** | Hyper-V VM (Server A) | Persistent enterprise-grade disk storage |
+| **ollama (NLP Llama3.2:3b Engine)** | VirtualBox VM (Server B) | Dedicated CPU/GPU virtualization allocation |
+| **zabbix-server & web (Monitoring)** | Hyper-V VM (Server A) | Centralized SNMPv3 alert processing and logs |
+| **searxng (Meta-Search Gateway)** | Local WSL2 (Windows) | Dynamic browser-level loopbacks |
+
+---
+
+## 3. Platform Provisioning Commands
+
+### 3.1: WSL2 Provisioning (Local Client Workstation)
+Enable WSL2 and install Ubuntu 24.04:
+```powershell
+# Run in Administrator PowerShell
+dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
+dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
+wsl --install -d Ubuntu-24.04
+```
+
+### 3.2: Hyper-V VM Provisioning (Server A - Database & Zabbix)
+Deploy a dedicated Ubuntu VM on Hyper-V using PowerShell:
+```powershell
+# Run in Administrator PowerShell on Server A
+New-VM -Name "FoodAI-Database-Node" -MemoryStartupBytes 8GB -Generation 2 -NewVHDPath "C:\VMs\FoodAI_DB.vhdx" -VHDSizeBytes 80GB -SwitchName "External Switch"
+Set-VMFirmware -VMName "FoodAI-Database-Node" -EnableSecureBoot Off
+Start-VM -Name "FoodAI-Database-Node"
+```
+
+### 3.3: VirtualBox VM Provisioning (Server B - Ollama AI Engine)
+Deploy a dedicated VM on VirtualBox using Command Line:
+```bash
+# Run in Command Prompt on Server B
+vboxmanage createvm --name "FoodAI-AI-Node" --ostype "Ubuntu_64" --register
+vboxmanage modifyvm "FoodAI-AI-Node" --memory 8192 --cpus 4 --vram 128 --nic1 bridged --bridgeadapter1 "Intel Ethernet Connection"
+vboxmanage createhd --filename "C:\VMs\FoodAI_AI.vdi" --size 60000
+vboxmanage storagectl "FoodAI-AI-Node" --name "SATA Controller" --add sata --controller IntelAHCI
+vboxmanage storageattach "FoodAI-AI-Node" --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium "C:\VMs\FoodAI_AI.vdi"
+vboxmanage startvm "FoodAI-AI-Node" --type headless
+```
+
+---
+
+## 4. Secure Authentication & SSH Exchange
+Exchange SSH public keys to allow automated, passwordless container management across nodes:
+```bash
+# 1. Generate SSH Keys on WSL Client
+ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_foodai -N ""
+
+# 2. Push Key to Database VM (Server A)
+ssh-copy-id -i ~/.ssh/id_rsa_foodai.pub operator@192.168.130.170
+
+# 3. Push Key to AI VM (Server B)
+ssh-copy-id -i ~/.ssh/id_rsa_foodai.pub operator@192.168.130.161
+```
+
+---
+
+## 5. Multi-Node Docker Network & Configuration
+
+To allow WSL, Hyper-V, and VirtualBox nodes to communicate, update the `.env` variables and `docker-compose.yml` to use bridged network endpoints.
+
+### Step 5.1: Configure WSL Client `.env`
+Update `.env` in the Streamlit workspace:
+```ini
+DB_HOST=192.168.130.170
+DB_USER=food_reader
+DB_PASS=reader_pass
+APP_AUTH_USER=food_app_auth
+APP_AUTH_PASS=auth_pass
+OLLAMA_HOST=http://192.168.130.161:11434
+SEARXNG_HOST=http://localhost:8080
+ZBX_SERVER_HOST=192.168.130.170
+```
+
+### Step 5.2: Configure Ollama (VirtualBox Server B) Listening Port
+Ensure the Ollama daemon inside VirtualBox binds to `0.0.0.0` (all interfaces):
+```bash
+# SSH into Server B (192.168.130.161)
+sudo systemctl edit ollama.service
+
+# Add the environment variables:
+[Service]
+Environment="OLLAMA_HOST=0.0.0.0"
+
+# Reload and restart service
+sudo systemctl daemon-reload
+sudo systemctl restart ollama
+```
+
+---
+
+## 6. Zabbix Reconfiguration for Multi-Node SNMPv3 Telemetry
+
+To monitor all distributed deployment environments securely:
+
+### Step 6.1: Deploy SNMPv3 Daemons
+Install and configure SNMPv3 daemons on WSL, Hyper-V Database VM, and VirtualBox AI VM:
+```bash
+sudo apt update && sudo apt install -y snmpd
+```
+Edit `/etc/snmp/snmpd.conf`:
+```
+# Listen on all interfaces
+agentAddress udp:161
+
+# Create secure SNMPv3 User
+createUser securityUser SHA "securityAuthPassword" AES "securityPrivPassword"
+rouser securityUser authpriv
+```
+Restart daemon:
+```bash
+sudo systemctl restart snmpd
+```
+
+### Step 6.2: Configure Zabbix Server Dashboard (Web UI)
+1. Open Zabbix in your browser at `http://192.168.130.170:8081`.
+2. Navigate to **Configuration > Hosts > Create Host**.
+3. Create three distinct hosts:
+   - **WSL-Workstation** (IP: `192.168.1.50`)
+   - **Database-Node** (IP: `192.168.130.170`)
+   - **AI-Node** (IP: `192.168.130.161`)
+4. Add the **SNMP Interface** pointing to Port 161 for each host.
+5. In the **Security Tab**, select SNMPv3, enter Username `securityUser`, select Auth Protocol `SHA` / `securityAuthPassword`, and Privacy Protocol `AES` / `securityPrivPassword`.
+6. Attach the pre-installed **Local Food AI Telemetry** Template.
+
+---
+
+## 7. Verifying Alert Channels
+
+### 7.1: Microsoft Teams / Discord Alert Webhook
+To verify Zabbix is communicating with Discord / Teams:
+1. Trigger a test CPU threshold spike inside WSL:
+   ```bash
+   yes > /dev/null & sleep 10 ; killall yes
+   ```
+2. Verify Zabbix triggers the alert and transmits the notification.
+3. Check your designated channel for the incoming payload:
+   - Expected Output: `[PROBLEM] High CPU Utilization Detected on WSL-Workstation`.
+
+### 7.2: Password Reset Email (SMTP Gateway)
+1. In the Streamlit UI Sidebar, select **Reset Password**.
+2. Trigger a reset link for user `ClinicianA`.
+3. Check the inbox or SMTP system log (`tail -f /var/log/mail.log` on Server A) to verify outbound delivery.
+
+---
+
+## 8. Operator Post-Installation Checklist
+
+Run these test cases to verify the installation:
+
+| TEST CASE ID | ACTIONS TO PERFORM | EXPECTED RESULTS | STATUS |
+| :--- | :--- | :--- | :---: |
+| **TC-OP-01** | Search 'Cheese' on Search Tab | 10+ records returned in <0.04s. Listeria warning flags on unpasteurized. | `[ ]` |
+| **TC-OP-02** | Enter '1.5 cups' in Plate Tab | Parsed and converted to metric grams based on density index. | `[ ]` |
+| **TC-OP-03** | Ask Chat: 'Can I eat sushi?' | Llama3.2:3b retrieves database context and flags raw fish as forbidden for pregnancy. | `[ ]` |
+| **TC-OP-04** | Trigger manual db backup | Timestamped compressed .sql.gz created inside backups/ folder. | `[ ]` |
+| **TC-OP-05** | Terminate Ollama Container | Zabbix PROBLEM active alert generated on dashboard in < 30 seconds. | `[ ]` |

BIN
docs/Operator_Installation_Guide.pdf


BIN
docs/Scrum_Artifacts.pdf


BIN
docs/Scrum_Daily.pdf


BIN
docs/Scrum_Plan.pdf


BIN
docs/Scrum_Retro.pdf


BIN
docs/Scrum_Review.pdf


+ 34 - 2
docs/Scrum_Wiki.md

@@ -1,3 +1,35 @@
 # $Id$
-# Scrum Wiki Master List
-This file aggregates references to the Scrum daily logs, plans, and retrospectives.
+# Scrum Wiki Master List & Index Portal
+
+Welcome to the static Scrum documentation portal. This master wiki aggregates and organizes all daily stand-up logs, planning reports, retrospectives, reviews, and velocity charts recorded during the agile development of the **Local Food AI** clinical dietetics engine.
+
+---
+
+## 📅 Sprint Ceremonies & Logs
+
+### 1. [Sprint Plans (Scrum_Plan.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Plan.md)
+*Contains Sprint Plan formulations, active user stories selection, scope statements, and team capacity bounds for each milestone loop.*
+
+### 2. [Daily Scrums (Scrum_Daily.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Daily.md)
+*Continuous daily stand-up summaries tracking individual task completion, blocker mitigations, and immediate day-to-day coordination.*
+
+### 3. [Sprint Reviews (Scrum_Review.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Review.md)
+*Contains sprint review logs, clinician demonstration summaries, feature validation checklists, and stakeholder feedback logs.*
+
+### 4. [Sprint Retrospectives (Scrum_Retro.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Retro.md)
+*Reviews process improvements, continuous integration learnings, and action items aimed at optimizing team operations and environment tuning.*
+
+---
+
+## 📊 Deliverables & Quality Assurance
+
+### 5. [Scrum Artifacts (Scrum_Artifacts.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Artifacts.md)
+*Indexes sprint velocity metrics, completed story points distributions, burndown coordinates, and final Taiga delivery milestones.*
+
+### 6. [Sprint 8 Test Cases (Test_Cases_Sprint8.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Test_Cases_Sprint8.md)
+*Legacy acceptance test logs covering core NLP chat, portion converters, and initial search validations.*
+
+---
+
+> [!NOTE]
+> **Operational Compliance**: All Scrum files above are synchronized with their respective Taiga milestone identifiers (`Sprint 13` and `Sprint 7`). All physical activities recorded in these markdown logs have corresponding closed tasks inside Taiga.

BIN
docs/Scrum_Wiki.pdf


+ 90 - 0
docs/Start_Stop_Procedures.md

@@ -0,0 +1,90 @@
+# $Id$
+# Infrastructure Stop & Start Operational Procedures
+
+This runbook outlines the exact sequence and commands to start, stop, and verify each microservice in the Local Food AI environment.
+
+---
+
+## 1. Sequence Priority Rules
+Due to database socket requirements and network bindings, services **must** be started and stopped in the following order:
+
+```mermaid
+graph TD
+    subgraph Startup Sequence
+        direction TB
+        A[1. MySQL Database] --> B[2. Ollama & SearXNG AI Services]
+        B --> C[3. Streamlit Application & Nginx Proxy]
+        C --> D[4. Zabbix Monitoring & Airflow Supervisor]
+    end
+```
+
+---
+
+## 2. Startup Procedures
+
+### Step 2.1: Start the Core MySQL Database
+Verify that the database service is up and listening on port 3307:
+```bash
+docker compose up -d mysql
+# Verify database logs
+docker compose logs -f mysql
+```
+
+### Step 2.2: Start AI Engine & SearXNG Search
+Deploy the AI components:
+```bash
+docker compose up -d ollama searxng
+# Check that Ollama responds
+curl http://localhost:11434/api/tags
+```
+
+### Step 2.3: Start Streamlit App and Nginx Gateway
+Bring up the frontend web interface and reverse proxy:
+```bash
+docker compose up -d app nginx
+# Verify Web Interface status
+curl -I http://localhost
+```
+
+### Step 2.4: Start Zabbix Monitoring Suite
+Deploy the monitoring server and agents:
+```bash
+docker compose up -d zabbix-server zabbix-web zabbix-agent
+# Check dashboard availability
+curl -I http://localhost:8081
+```
+
+---
+
+## 3. Shutdown Procedures
+
+To perform system maintenance or schema migration, stop services in reverse order to prevent lockups:
+
+```bash
+# 1. Stop Monitoring Components
+docker compose stop zabbix-agent zabbix-web zabbix-server
+
+# 2. Stop Web Frontend and Proxy Gateway
+docker compose stop nginx app
+
+# 3. Stop NLP and Search Services
+docker compose stop searxng ollama
+
+# 4. Stop Database Container gracefully
+docker compose stop mysql
+```
+
+---
+
+## 4. Status Verification Commands
+Use these commands to verify container state and port bindings:
+```bash
+# List all running containers in the stack
+docker compose ps
+
+# Inspect raw container logs for error spikes
+docker compose logs --tail=100
+
+# Verify TCP socket listener binds
+netstat -tulpn | grep -E "80|3307|8081|11434"
+```

BIN
docs/Start_Stop_Procedures.pdf


BIN
docs/Test_Cases_Sprint8.pdf


+ 41 - 0
docs/User_Description.md

@@ -0,0 +1,41 @@
+# $Id$
+# Local Food AI - User Description & Functional Guide
+
+## 1. System Vision
+The **Local Food AI** system is a strictly local, privacy-first, professional-grade clinical dietetics assistant. Developed specifically for clinics and healthcare practitioners, it provides offline nutritional analysis, meal planning, and warning flags based on dynamic patient health profiles. 
+
+Since the system operates entirely locally on local hypervisors, **zero patient medical data or search queries ever leave the server boundary**, ensuring 100% HIPAA compliance and data sovereignty.
+
+---
+
+## 2. Core Functional Pillars
+
+### 📊 tab 1: Clinical Data Search (🔬 Clinical Search)
+Allows practitioners to search the 24GB OpenFoodFacts dataset in real time (average query response time < 0.04 seconds).
+- **Dynamic Medical Warnings**: Based on the active patient profile, foods are immediately flagged in the search results:
+  - ⚠️ **Red Warning Flags**: Highlight high-risk ingredients (e.g. Unpasteurized dairy or raw fish for pregnant patients, high-sodium foods for hypertensive patients, or high-sugar foods for diabetic patients).
+  - 💚 **Green Recommendations**: Highlight recommended dietary components (e.g. High iron/calcium for pregnant or breastfeeding mothers, high Vitamin C for scurvy prevention, or high iron for anemia).
+- **Flexible Column Customization**: Multi-select column headers to inspect specific macro and micro-nutrients.
+
+### 💬 tab 2: AI Clinical Chat (💬 AI Chat)
+An interactive NLP dialogue interface powered by a local lightweight LLM (**Llama3.2:3b**).
+- **RAG-Driven Precision**: The AI dietitian automatically retrieves and reviews local database records and private meta-search results before formulating an answer.
+- **Dynamic Medical Guardrails**: The user's active illnesses, diets, and conditions are injected into the AI's system prompt in the background, forcing the AI to strictly enforce clinical safety constraints.
+
+### 🍽️ tab 3: My Plate Builder (🍽️ My Plate Builder)
+A recipe formulation utility to calculate combined nutritional intake.
+- **Natural Language Parsing**: Enables entering quantities in natural units (e.g., "1.5 cups", "2 tablespoons", "150g").
+- **Exact Conversion**: The system translates these custom units into metric grams based on product density metrics.
+- **Macro Summaries**: Instantly calculates and displays the total combined Protein, Fat, and Carbohydrates.
+
+### 🤖 tab 4: AI Meal Planner (🤖 AI Meal Planner)
+An automated clinical diet planner.
+- Generates a multi-meal daily menu formatted strictly as a Markdown table.
+- Dynamically enforces user-defined calorie limits and active medical restrictions.
+
+---
+
+## 3. Supported Health & Medical Profiles
+- **Conditions**: Pregnant, Breastfeeding, Low Fat, Osteoporosis.
+- **Illnesses**: Diabetes, Hypertension, Kidney Disease, Scurvy, Anemia.
+- **Diets**: Vegan, Vegetarian, Kosher, Halal, Keto, Paleo, Christian (Lent/Good Friday).

BIN
docs/User_Description.pdf


BIN
docs/User_Guide.pdf


BIN
docs/WSL_Deployment.pdf


BIN
docs/Wiki_Home.pdf


BIN
docs/architecture.pdf


BIN
docs/disaster_recovery_plan.pdf


BIN
docs/distributed_deployment.pdf


+ 37 - 27
docs/project_report.md

@@ -16,38 +16,48 @@ The **Local Food AI** capstone project has successfully completed all sprint ite
 4. **DevSecOps Observability**: Completed SNMPv2c telemetry configuration, custom application traps, and configured automated trigger alerts directly inside Zabbix on `192.168.130.170`.
 5. **Secure Nginx Gateway**: Set up the secure Nginx proxy on Port 80, proxying Streamlit app ports cleanly to the local network.
 6. **Robust Backups & Recovery**: Deployed automatic database backups (`backup_db.sh`) and local offline single-node fallback capabilities (`docker-compose_skip.yml`).
+7. **Sequential Operations Manager**: Created `manage_services.sh` to allow developers to safely stop, start, and restart all microservices in the proper dependency order without triggering redundant online ingestion sequences.
 
 ---
 
 ## 2. Project File Catalog & Documentation
-Below is an exhaustive description of every critical file in the repository, detailing its absolute location, primary purpose, and active Git version tags.
+Below is an exhaustive catalog of every critical file in the repository, detailing its path, functional purpose, and active Git version tags. 
 
-| File Name | Absolute Location | Purpose & Core Responsibility | Last Commit | Author | Commit Date | Last Commit Message |
-| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
-| **app.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\app.py` | Core Streamlit Web Application. Hosts the clinical food search engine, the RAG chat dietitian interface (utilizing Ollama and SearXNG tool calling), and the visual plate builder. | `49e8443` | lanfr144 | 2026/05/20 09:06:46 | TG-204: Fix NameError by importing html |
-| **ingest_csv.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\ingest_csv.py` | High-performance background database loader. Stream-reads and batch-inserts the 3GB OpenFoodFacts dataset into MySQL using Pandas chunking and optimizes indices post-load. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **unit_converter.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\unit_converter.py` | Mathematical converter engine that parses natural recipe volume inputs (e.g. cups, spoons) and converts them to metric weights based on macro density mappings. | `ea04a85` | lanfr144 | 2026/05/08 08:57:06 | TG-86: finalize system pre-initialization, auto-pull LLM, egg scales |
-| **snmp_notifier.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\snmp_notifier.py` | Observability SNMP utility. Formulates and transmits raw SNMP trap payloads to the central Zabbix monitoring server on critical application failures. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **configure_zabbix_alerts.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\configure_zabbix_alerts.py` | DevOps provisioning script. Uses the Zabbix API to automatically set up host groups, custom templates, items, triggers, actions, and media types for alerts. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **zabbix_telemetry.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\zabbix_telemetry.py` | Telemetry collector daemon. Scrapes live memory usage, Streamlit active user threads, and query performance to feed the Zabbix dashboard. | `ade82af` | lanfr144 | 2026/05/18 14:08:27 | TG-196: Full security refactor, Taiga sync, and Data pipeline automation |
-| **check_users.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\check_users.py` | Security utility. Verifies user accounts inside the MySQL `users` table and checks password hashing complexity. | `7766898` | lanfr144 | 2026/04/29 14:39:55 | Add check users script |
-| **rotate_passwords.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\rotate_passwords.py` | Administrative credential utility. Cycles and re-encrypts database passwords within the `.env` secret file. | `ade82af` | lanfr144 | 2026/05/18 14:08:27 | TG-196: Full security refactor, Taiga sync, and Data pipeline automation |
-| **myloginpath.py** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\myloginpath.py` | MySQL credential companion helper that simplifies the generation of encrypted login path configuration profiles. | `4655c26` | lanfr144 | 2026/04/29 08:30:03 | Add untracked project files and configs |
-| **data_sync.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\data_sync.sh` | Master pipeline coordinator. Supports download fetching in --online mode and local file processing in offline fallback mode. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **backup_db.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\backup_db.sh` | Resiliency backup automation. Runs mysqldump on user tables inside the active container and prunes backups older than 7 days. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **reset.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\reset.sh` | Teardown script. Wipes local temporary containers and prunes volume locks during crashes. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **proper_reset.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\proper_reset.sh` | High-level administrative wipe script that brings the entire network stack and repositories back to a pristine state. | `776d6a6` | lanfr144 | 2026/04/29 12:44:49 | Add proper reset |
-| **deploy.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\deploy.sh` | Naked OS installation guide. Installs necessary system packages, Python venv libraries, and native Ollama. | `a54dc25` | lanfr144 | 2026/04/22 15:01:17 | TG-21: Update deploy.sh to include requests connectivity dependency. |
-| **start_batch_ingest.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\start_batch_ingest.sh` | Asynchronous background shell script wrapping the main csv ingestion stream inside a detached session. | `00f1d63` | lanfr144 | 2026/04/24 07:50:40 | Fix python virtual env paths |
-| **download_csv.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\download_csv.sh` | Downloader helper script that fetches specific smaller subsets of OpenFoodFacts CSV files. | `1a3cdca` | lanfr144 | 2026/05/05 07:14:54 | fix: resolve pip encoding issue and add exec permissions to download script |
-| **master_trigger.sh** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\master_trigger.sh` | Orchestrator script that wakes and verifies multiple secondary subservices in sequence. | `38a83a1` | lanfr144 | 2026/04/23 10:50:37 | Deployment Finalization: Vitamin schemas, Green UI, and Taiga tools |
-| **docker-compose.yml** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\docker-compose.yml` | Main 10-container Docker orchestration map defining MySQL, App UI, Ollama Engine, SearXNG, Nginx proxy, Airflow stack, and Zabbix server suites. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **docker-compose_skip.yml** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\docker-compose_skip.yml` | Resilient 8-container offline/local single-node orchestration manifest. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag |
-| **alembic.ini** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\alembic.ini` | Alembic configuration setting routing database connection URIs for versioning schemas. | `73f7a04` | lanfr144 | 2026/04/24 16:18:55 | Optimize horizontal partitioning to slice into 8-column chunks bypassing InnoDB limits |
-| **my.cnf** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\my.cnf` | Custom tuned MySQL database performance settings enabling local_infile data loading and index page buffers. | `86c76e2` | lanfr144 | 2026/04/17 10:26:35 | TG-1: Fix MySQL 8.0 startup crash by removing premature validate_password plugin config |
-| **.env** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\.env` | Secret storage container holding encrypted MySQL user passwords and active environment flags. | `ca3877d` | lanfr144 | 2026/05/13 11:15:42 | Stop save the .env file |
-| **.gitattributes** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\.gitattributes` | Git clean/smudge layout mapping enabling automatic tracking of dynamic $Id$ metadata expansion within version files. | `0cfdf52` | lanfr144 | 2026/05/07 09:54:17 | TG-85: enable export-subst for Format string git identification |
-| **requirements.txt** | `c:\Users\lanfr144\Documents\DOPRO1\Antigravity\Food\requirements.txt` | Python runtime dependency catalog storing strict library versioning constraints. | `bb2ac28` | lanfr144 | 2026/05/11 07:59:05 | fix requirements.txt encoding for fpdf2 |
+*Note: This chapter is compiled in landscape layout inside Project.pdf to guarantee complete columns readability.*
+
+| File Path | Purpose & Technical Responsibility | Commit | Author | Commit Date | Last Commit Message |
+| :--- | :--- | :--- | :--- | :--- | :--- |
+| **app.py**<br>`./app.py` | Core Streamlit Web Application. Hosts the clinical food search engine, the RAG chat dietitian interface (utilizing Ollama and SearXNG tool calling), and the visual plate builder. | `8b833bc` | Lange François | 2026/05/21 10:30:14 | *TG-151 #closed - Upgrade Ollama model to Llama3.2:3b and synchronize/close Taiga Agile deliverables* |
+| **ingest_csv.py**<br>`./ingest_csv.py` | High-performance background database loader. Stream-reads and batch-inserts the 3GB OpenFoodFacts dataset into MySQL using Pandas chunking and optimizes indices post-load. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **unit_converter.py**<br>`./unit_converter.py` | Mathematical converter engine that parses natural recipe volume inputs (e.g. cups, spoons) and converts them to metric weights based on macro density mappings. | `ea04a85` | lanfr144 | 2026/05/08 08:57:06 | *TG-86: finalize system pre-initialization, auto-pull LLM, egg scales* |
+| **snmp_notifier.py**<br>`./snmp_notifier.py` | Observability SNMP utility. Formulates and transmits raw SNMP trap payloads to the central Zabbix monitoring server on critical application failures. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **configure_zabbix_alerts.py**<br>`./configure_zabbix_alerts.py` | DevOps provisioning script. Uses the Zabbix API to automatically set up host groups, custom templates, items, triggers, actions, and media types for alerts. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **configure_zabbix_email.py**<br>`./configure_zabbix_email.py` | Security & Monitoring. Configures email media types and SMTP server routes for Zabbix alert notifications on system downtime. | `ade82af` | lanfr144 | 2026/05/18 14:08:27 | *TG-196: Full security refactor, Taiga sync, and Data pipeline automation* |
+| **zabbix_telemetry.py**<br>`./zabbix_telemetry.py` | Monitoring agent daemon. Queries active application statistics, memory, and query timers to supply Zabbix telemetry indicators. | `ade82af` | lanfr144 | 2026/05/18 14:08:27 | *TG-196: Full security refactor, Taiga sync, and Data pipeline automation* |
+| **check_users.py**<br>`./check_users.py` | Security utility. Verifies user accounts inside the MySQL `users` table and checks password hashing complexity. | `7766898` | lanfr144 | 2026/04/29 14:39:55 | *Add check users script* |
+| **rotate_passwords.py**<br>`./rotate_passwords.py` | Administrative credential utility. Cycles and re-encrypts database passwords within the `.env` secret file. | `ade82af` | lanfr144 | 2026/05/18 14:08:27 | *TG-196: Full security refactor, Taiga sync, and Data pipeline automation* |
+| **myloginpath.py**<br>`./myloginpath.py` | MySQL credential companion helper that simplifies the generation of encrypted login path configuration profiles. | `4655c26` | lanfr144 | 2026/04/29 08:30:03 | *Add untracked project files and configs* |
+| **data_sync.sh**<br>`./data_sync.sh` | Master pipeline coordinator. Supports download fetching in --online mode and local file processing in offline fallback mode. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **backup_db.sh**<br>`./backup_db.sh` | Resiliency backup automation. Runs mysqldump on user tables inside the active container and prunes backups older than 7 days. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **reset.sh**<br>`./reset.sh` | Teardown script. Wipes local temporary containers and prunes volume locks during crashes. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **proper_reset.sh**<br>`./proper_reset.sh` | High-level administrative wipe script that brings the entire network stack and repositories back to a pristine state. | `776d6a6` | lanfr144 | 2026/04/29 12:44:49 | *Add proper reset* |
+| **deploy.sh**<br>`./deploy.sh` | Naked OS installation guide. Installs necessary system packages, Python venv libraries, and native Ollama. | `a54dc25` | lanfr144 | 2026/04/22 15:01:17 | *TG-21: Update deploy.sh to include requests connectivity dependency.* |
+| **start_batch_ingest.sh**<br>`./start_batch_ingest.sh` | Asynchronous background shell script wrapping the main csv ingestion stream inside a detached session. | `00f1d63` | lanfr144 | 2026/04/24 07:50:40 | *Fix python virtual env paths* |
+| **download_csv.sh**<br>`./download_csv.sh` | Downloader helper script that fetches specific smaller subsets of OpenFoodFacts CSV files. | `1a3cdca` | lanfr144 | 2026/05/05 07:14:54 | *fix: resolve pip encoding issue and add exec permissions to download script* |
+| **master_trigger.sh**<br>`./master_trigger.sh` | Orchestrator script that wakes and verifies multiple secondary subservices in sequence. | `38a83a1` | lanfr144 | 2026/04/23 10:50:37 | *Deployment Finalization: Vitamin schemas, Green UI, and Taiga tools* |
+| **manage_services.sh**<br>`./manage_services.sh` | DevOps service manager script. Handles automated, sequential startup, shutdown, restart, and health checking of all container elements in the stack. | `N/A` | N/A | N/A | *N/A* |
+| **generate_docs.py**<br>`./generate_docs.py` | Dynamic doc generator. Generates and mirrors all markdown manuals under `/docs` with live Git log metadata injection. | `8b833bc` | Lange François | 2026/05/21 10:30:14 | *TG-151 #closed - Upgrade Ollama model to Llama3.2:3b and synchronize/close Taiga Agile deliverables* |
+| **docker-compose.yml**<br>`./docker-compose.yml` | Main 10-container Docker orchestration map defining MySQL, App UI, Ollama Engine, SearXNG, Nginx proxy, Airflow stack, and Zabbix server suites. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **docker-compose_skip.yml**<br>`./docker-compose_skip.yml` | Resilient 8-container offline/local single-node orchestration manifest. | `264d274` | lanfr144 | 2026/05/21 09:43:09 | *TG-442: Sync resilience configurations, resolve SearXNG crash, and update docs with dynamic custom Git log ID and tag* |
+| **alembic.ini**<br>`./alembic.ini` | Alembic configuration setting routing database connection URIs for versioning schemas. | `73f7a04` | lanfr144 | 2026/04/24 16:18:55 | *Optimize horizontal partitioning to slice into 8-column chunks bypassing InnoDB limits* |
+| **my.cnf**<br>`./my.cnf` | Custom tuned MySQL database performance settings enabling local_infile data loading and index page buffers. | `86c76e2` | lanfr144 | 2026/04/17 10:26:35 | *TG-1: Fix MySQL 8.0 startup crash by removing premature validate_password plugin config* |
+| **.env**<br>`./.env` | Secret storage container holding encrypted MySQL user passwords and active environment flags. | `ca3877d` | lanfr144 | 2026/05/13 11:15:42 | *Stop save the .env file* |
+| **.gitattributes**<br>`./.gitattributes` | Git clean/smudge layout mapping enabling automatic tracking of dynamic $Id$ metadata expansion within version files. | `0cfdf52` | lanfr144 | 2026/05/07 09:54:17 | *TG-85: enable export-subst for Format string git identification* |
+| **requirements.txt**<br>`./requirements.txt` | Python runtime dependency catalog storing strict library versioning constraints. | `bb2ac28` | lanfr144 | 2026/05/11 07:59:05 | *fix requirements.txt encoding for fpdf2* |
+| **scripts/generate_pdfs.py**<br>`./scripts/generate_pdfs.py` | PDF document builder. Converts all markdown documentation manuals under `/docs` into high-fidelity PDF format with expanded Git version headers. | `3606c03` | Lange François | 2026/05/21 10:12:53 | *TG-442: Update Git $ log fields, date format, and redesign architecture.md diagram* |
+| **scripts/generate_project_report.py**<br>`./scripts/generate_project_report.py` | Technical project report generator. Automatically gathers codebase structure, Git commit metadata, and purpose records to construct the Project.pdf report. | `8b833bc` | Lange François | 2026/05/21 10:30:14 | *TG-151 #closed - Upgrade Ollama model to Llama3.2:3b and synchronize/close Taiga Agile deliverables* |
+| **scripts/setup_deploy.py**<br>`./scripts/setup_deploy.py` | DevOps deployment script. Orchestrates local and VM container sets, verifying network connectivity and system parameters. | `0065125` | lanfr144 | 2026/05/20 08:52:08 | *TG-202: Add log rotation limits to prevent 100% disk usage* |
+| **scripts/taiga_sync_final.py**<br>`./scripts/taiga_sync_final.py` | Taiga automated synchronization helper. Pushes bug tickets, fills wiki pages, and assigns unassigned user stories. | `a4342d3` | lanfr144 | 2026/05/19 09:09:10 | *TG-198: Add Taiga consistency automation script for full Jira/Agile alignment* |
 
 ---
 

BIN
docs/project_report.pdf


BIN
docs/retro_planning.pdf


+ 133 - 91
docs/taiga_audit_report.md

@@ -6,98 +6,140 @@
 > Automatically generated from the live Taiga API to verify project completeness against `Project.pdf`.
 
 ## Sprint & Velocity Overview
-- **Sprint 8**: None/None Points Completed
-- **Sprint 7**: None/None Points Completed
-- **Sprint 6**: None/5.0 Points Completed
-- **Sprint 5**: None/None Points Completed
-- **Sprint 13**: None/None Points Completed
-- **Sprint 12**: None/None Points Completed
-- **Sprint 11**: None/None Points Completed
-- **Sprint 4**: None/77.0 Points Completed
+- **Sprint 1**: 5.0/5.0 Points Completed
 - **Sprint 10**: None/None Points Completed
-- **Sprint 9**: None/None Points Completed
-- **Sprint 3**: None/None Points Completed
+- **Sprint 11**: None/None Points Completed
+- **Sprint 12**: None/None Points Completed
+- **Sprint 13**: None/None Points Completed
 - **Sprint 2**: None/None Points Completed
-- **Sprint 1**: None/5.0 Points Completed
+- **Sprint 3**: None/None Points Completed
+- **Sprint 4**: 77.0/77.0 Points Completed
+- **Sprint 5**: None/None Points Completed
+- **Sprint 6**: 5.0/5.0 Points Completed
+- **Sprint 7**: None/None Points Completed
+- **Sprint 7: Production Hardening & Handover**: None/None Points Completed
+- **Sprint 8**: None/None Points Completed
+- **Sprint 9**: None/None Points Completed
 
 ## User Stories & Task Completion
-### [US-204] Public Git Repo Setup (Status: Done)
-  - *No technical tasks associated!*
-### [US-205] Easy Cloning Setup (Status: Done)
-  - `[ ]` Task 456: Refactor Cryptography Bug - Replace dynamic salting loop with bcrypt.checkpw (New)
-  - `[ ]` Task 457: Implement Horizontal Table Partitioning to bypass MySQL 65KB InnoDB limit (New)
-  - `[ ]` Task 458: Construct dynamic UI multiselect for mapping 200 CSV columns seamlessly (New)
-  - `[ ]` Task 459: Bind Pandas dataframes tightly to Memory logic preventing UI crashes (New)
-  - `[ ]` Task 460: Overwrite LLM system prompts strictly for native Markdown gram output (New)
-  - `[ ]` Task 461: Configure native mail throttle limits to block .pt.lu bounce delays (New)
-### [US-207] 100% Local Data Privacy (Status: Done)
-  - `[ ]` Task 462: Refactor Cryptography Bug - Replace dynamic salting loop with bcrypt.checkpw (New)
-  - `[ ]` Task 463: Implement Horizontal Table Partitioning to bypass MySQL 65KB InnoDB limit (New)
-  - `[ ]` Task 464: Construct dynamic UI multiselect for mapping 200 CSV columns seamlessly (New)
-  - `[ ]` Task 465: Bind Pandas dataframes tightly to Memory logic preventing UI crashes (New)
-### [US-206] User Account Creation & Login (Status: Done)
-  - *No technical tasks associated!*
-### [US-208] View Complete Nutritional Info (Status: In progress)
-  - `[ ]` Task 442: Why: Applying the global CSS architecture is the direct prerequisite to making the visual information actually look premium and readable when the user views the data. (New)
-### [US-209] Search for Nutrients (Status: In progress)
-  - `[ ]` Task 443: Why: Building the numerical filtering sliders logically completes the "Advanced Search" capabilities explicitly defined by this story. (New)
-### [US-211] Store and Edit Food Combinations (Status: New)
-  - `[ ]` Task 446: Why: The core of this story is storing data, which is entirely solved by creating the explicit relational plates and plate_items MySQL database tables. (New)
-### [US-212] Lightweight Local AI Models (Status: Done)
-  - *No technical tasks associated!*
-### [US-210] Combined Nutritional Value Overview (Status: New)
-  - `[ ]` Task 445: Why: Generating the Pandas calculation logic that mathematically adds up the macros is what delivers the final "Combined Value Overview" to the user! (New)
-### [US-213] Chat About Nutrition (Status: Done)
-  - *No technical tasks associated!*
-### [US-214] AI Menu Proposals (Status: New)
-  - *No technical tasks associated!*
-### [US-215] Anonymous Web Search Tool (Status: Done)
-  - *No technical tasks associated!*
-### [US-246] Database Schema Dynamic Rebuild & Background Loader (Status: Done)
-  - `[ ]` Task 435: Rebuild setup_db.py to allow dynamic Pandas table generation. (New)
-  - `[ ]` Task 436: Update ingest_csv.py with to_sql and post-load index generating. (New)
-  - `[ ]` Task 437: Create start_batch_ingest.sh wrapper for disconnected execution. (New)
-  - `[ ]` Task 438: Configure server .forward mail protocols for centralized admin support. (New)
-### [US-247] Deploy SearXNG Docker API (Status: Done)
-  - `[ ]` Task 439: Create setup_searxng.sh to install Docker and bind anonymous SearXNG to localhost:8080. (New)
-  - `[ ]` Task 440: Update deploy.sh to include requests connectivity dependency. (New)
-  - `[ ]` Task 441: Rework app.py LLM inference loop to support native Mistral Tool/Function calling integrations. (New)
-### [US-216] Zero Confidential Data Leakage (Status: Done)
-  - *No technical tasks associated!*
-### [US-248] Clinical Medical Profiler (Status: New)
-  - `[x]` Task 447: Implement EAV Mapping Database Architecture (Closed)
-  - `[x]` Task 448: Fix Windows Encodings in Pandas Ingestion Engine (Closed)
-  - `[x]` Task 449: Build Dynamic 'Medical Profile' CRUD Interface (Closed)
-  - `[x]` Task 450: Deploy Clinical Health-Warning Alert Engine (Closed)
-  - `[x]` Task 451: Deploy Email Resets and Persistent Query Limits (Closed)
-### [US-249] Sprint 4: Operations & Migrations (Status: New)
-  - `[ ]` Task 452: Create unified PDF presentation for review (New)
-  - `[ ]` Task 453: Execute Alembic Database Migration scripting (New)
-  - `[ ]` Task 454: Sanitize Ollama Mistral LLM endpoints on .170 (New)
-  - `[ ]` Task 455: Perform Green Recommendation Engine Demo (New)
-### [US-250] Zabbix Server Docker Setup (Status: New)
-  - *No technical tasks associated!*
-### [US-251] SNMPv3 Integration (Status: New)
-  - *No technical tasks associated!*
-### [US-252] Application Component Traps (Status: New)
-  - *No technical tasks associated!*
-### [US-253] Clinical Explorer Verification Testing (Status: New)
-  - *No technical tasks associated!*
-### [US-254] Zabbix Application Monitoring Checks (Status: New)
-  - *No technical tasks associated!*
-### [US-255] Zabbix Email Integration (Status: New)
-  - *No technical tasks associated!*
-### [US-256] Zabbix Live Alert Testing (Status: New)
-  - *No technical tasks associated!*
-### [US-257] Server Backup Procedures (Status: New)
-  - *No technical tasks associated!*
-### [US-258] WSL Deployment Playbook (Status: New)
-  - *No technical tasks associated!*
-### [US-259] Agile Scrum Rituals Wiki (Status: New)
-  - *No technical tasks associated!*
-### [US-260] Sprint 8 Final Bug Fixes & Polish (Status: New)
-  - *No technical tasks associated!*
-### [US-261] Deep System Overhaul Phase 3 (Status: New)
-  - *No technical tasks associated!*
-### [US-262] Deep Containerization and Zabbix Telemetry Overhaul (Status: New)
-  - *No technical tasks associated!*
+### [US-1] Public Git Repo Setup (Status: Done)
+  - `[x]` Task #129: Auto‑generated task (define details) (Closed)
+### [US-2] Easy Cloning Setup (Status: Done)
+  - `[x]` Task #204: Configure Easy Cloning and Repository Footprint (Closed)
+### [US-3] User Account Creation & Login (Status: Done)
+  - `[x]` Task #205: Implement Secure Local User Authentication & Login UI (Closed)
+### [US-4] 100% Local Data Privacy (Status: Done)
+  - `[x]` Task #206: Establish Strict Offline Database Constraints & Boundary Limits (Closed)
+### [US-5] View Complete Nutritional Info (Status: Done)
+  - `[x]` Task #23: Why: Applying the global CSS architecture is the direct prerequisite to making the visual information actually look premium and readable when the user views the data. (Closed)
+### [US-6] Search for Nutrients (Status: Done)
+  - `[x]` Task #24: Why: Building the numerical filtering sliders logically completes the "Advanced Search" capabilities explicitly defined by this story. (Closed)
+### [US-7] Combined Nutritional Value Overview (Status: Done)
+  - `[x]` Task #26: Why: Generating the Pandas calculation logic that mathematically adds up the macros is what delivers the final "Combined Value Overview" to the user! (Closed)
+### [US-8] Store and Edit Food Combinations (Status: Done)
+  - `[x]` Task #27: Why: The core of this story is storing data, which is entirely solved by creating the explicit relational plates and plate_items MySQL database tables. (Closed)
+### [US-9] Lightweight Local AI Models (Status: Done)
+  - `[x]` Task #207: Configure Ollama Local Orchestration for Llama3.2 (Closed)
+### [US-10] Chat About Nutrition (Status: Done)
+  - `[x]` Task #208: Construct Interactive Clinical AI Chat Interface (Closed)
+### [US-11] AI Menu Proposals (Status: Done)
+  - `[x]` Task #209: Deploy Local RAG-Driven Meal Planner Engine (Closed)
+### [US-12] Anonymous Web Search Tool (Status: Done)
+  - `[x]` Task #210: Integrate Local SearXNG Private Search Fallback (Closed)
+### [US-13] Zero Confidential Data Leakage (Status: Done)
+  - `[x]` Task #130: Auto‑generated task (define details) (Closed)
+### [US-14] Database Schema Dynamic Rebuild & Background Loader (Status: Done)
+  - `[x]` Task #15: Rebuild setup_db.py to allow dynamic Pandas table generation. (Closed)
+  - `[x]` Task #16: Update ingest_csv.py with to_sql and post-load index generating. (Closed)
+  - `[x]` Task #17: Create start_batch_ingest.sh wrapper for disconnected execution. (Closed)
+  - `[x]` Task #18: Configure server .forward mail protocols for centralized admin support. (Closed)
+### [US-19] Deploy SearXNG Docker API (Status: Done)
+  - `[x]` Task #20: Create setup_searxng.sh to install Docker and bind anonymous SearXNG to localhost:8080. (Closed)
+  - `[x]` Task #21: Update deploy.sh to include requests connectivity dependency. (Closed)
+  - `[x]` Task #22: Rework app.py LLM inference loop to support native Mistral Tool/Function calling integrations. (Closed)
+### [US-28] Clinical Medical Profiler (Status: Done)
+  - `[x]` Task #29: Implement EAV Mapping Database Architecture (Closed)
+  - `[x]` Task #30: Fix Windows Encodings in Pandas Ingestion Engine (Closed)
+  - `[x]` Task #31: Build Dynamic 'Medical Profile' CRUD Interface (Closed)
+  - `[x]` Task #32: Deploy Clinical Health-Warning Alert Engine (Closed)
+  - `[x]` Task #33: Deploy Email Resets and Persistent Query Limits (Closed)
+### [US-34] Sprint 4: Operations & Migrations (Status: Done)
+  - `[x]` Task #35: Create unified PDF presentation for review (Closed)
+  - `[x]` Task #36: Execute Alembic Database Migration scripting (Closed)
+  - `[x]` Task #37: Sanitize Ollama Mistral LLM endpoints on .170 (Closed)
+  - `[x]` Task #38: Perform Green Recommendation Engine Demo (Closed)
+### [US-131] Zabbix Server Docker Setup (Status: Done)
+  - `[x]` Task #132: Execute: Zabbix Server Docker Setup (Closed)
+### [US-133] SNMPv3 Integration (Status: Done)
+  - `[x]` Task #134: Execute: SNMPv3 Integration (Closed)
+### [US-135] Application Component Traps (Status: Done)
+  - `[x]` Task #136: Execute: Application Component Traps (Closed)
+### [US-137] Clinical Explorer Verification Testing (Status: Done)
+  - `[x]` Task #138: Execute: Clinical Explorer Verification Testing (Closed)
+### [US-139] Zabbix Application Monitoring Checks (Status: Done)
+  - `[x]` Task #140: Execute: Zabbix Application Monitoring Checks (Closed)
+### [US-141] Zabbix Email Integration (Status: Done)
+  - `[x]` Task #142: Execute: Zabbix Email Integration (Closed)
+### [US-143] Zabbix Live Alert Testing (Status: Done)
+  - `[x]` Task #144: Execute: Zabbix Live Alert Testing (Closed)
+### [US-145] Server Backup Procedures (Status: Done)
+  - `[x]` Task #146: Execute: Server Backup Procedures (Closed)
+### [US-147] WSL Deployment Playbook (Status: Done)
+  - `[x]` Task #148: Execute: WSL Deployment Playbook (Closed)
+  - `[x]` Task #211: Create Comprehensive WSL2 Operator Runbook (Closed)
+### [US-149] Agile Scrum Rituals Wiki (Status: Done)
+  - `[x]` Task #150: Execute: Agile Scrum Rituals Wiki (Closed)
+  - `[x]` Task #212: Establish Scrum Rituals Static Documentation (Closed)
+### [US-151] Sprint 8 Final Bug Fixes & Polish (Status: Done)
+  - `[x]` Task #152: Execute Bug Fixes (Closed)
+  - `[x]` Task #213: Apply Codebase Linter Refactoring and SQL Cleanup (Closed)
+### [US-153] Deep System Overhaul Phase 3 (Status: Done)
+  - `[x]` Task #154: Execute Phase 3 Overhaul (Closed)
+  - `[x]` Task #214: Implement Resilient Subquery Optimizations & Layout UI (Closed)
+### [US-155] Deep Containerization and Zabbix Telemetry Overhaul (Status: Done)
+  - `[x]` Task #156: Centralize docker-compose.yml with individual component services (Closed)
+  - `[x]` Task #157: Integrate NVIDIA GPU support for Ollama container (Closed)
+  - `[x]` Task #158: Update App and Ingest Dockerfiles to include SNMP telemetry packages (Closed)
+  - `[x]` Task #159: Write Zabbix API script to create App -> MySQL trigger dependencies (Closed)
+  - `[x]` Task #160: Sync Git repository and update Taiga tracking (Closed)
+  - `[x]` Task #215: Deploy SNMPv3 Encrypted Traps and Zabbix Templates (Closed)
+### [US-161] Fix Llama3 Tool Compatibility (Status: Done)
+  - `[x]` Task #162: Execute: Fix Llama3 Tool Compatibility (Closed)
+### [US-163] Resolve MySQL Cartesian Product Explosion (Status: Done)
+  - `[x]` Task #164: Execute: Resolve MySQL Cartesian Product Explosion (Closed)
+### [US-165] Implement Subquery First Optimization Strategy (Status: Done)
+  - `[x]` Task #166: Execute: Implement Subquery First Optimization Strategy (Closed)
+### [US-167] UI Execution Timers (Status: Done)
+  - `[x]` Task #168: Execute: UI Execution Timers (Closed)
+### [US-169] Zabbix Microsoft Teams Alert Integration (Status: Done)
+  - `[x]` Task #170: Execute: Zabbix Microsoft Teams Alert Integration (Closed)
+### [US-171] Pre-Emptive Database Cleaning via Upsert (Status: Done)
+  - `[x]` Task #172: Execute: Pre-Emptive Database Cleaning via Upsert (Closed)
+### [US-173] Cascaded Search Logic & Nutrient Selectors (Status: Done)
+  - `[x]` Task #174: Execute: Cascaded Search Logic & Nutrient Selectors (Closed)
+### [US-175] Food Scale Conversion Expansion (Status: Done)
+  - `[x]` Task #176: Execute: Food Scale Conversion Expansion (Closed)
+### [US-177] Self-Detaching NOHUP Ingestion Sync (Status: Done)
+  - `[x]` Task #178: Execute: Self-Detaching NOHUP Ingestion Sync (Closed)
+### [US-179] AI Dietary Restriction SQL Enforcement (Status: Done)
+  - `[x]` Task #180: Execute: AI Dietary Restriction SQL Enforcement (Closed)
+### [US-181] Zabbix Database Ingestion Telemetry (Status: Done)
+  - `[x]` Task #182: Execute: Zabbix Database Ingestion Telemetry (Closed)
+### [US-183] AI Meal Plan PDF Generation (Status: Done)
+  - `[x]` Task #184: Execute: AI Meal Plan PDF Generation (Closed)
+### [US-185] Health Profile Input Constraints (Status: Done)
+  - `[x]` Task #186: Execute: Health Profile Input Constraints (Closed)
+### [US-187] Deploy Local SearXNG Web Search Tool (Status: Done)
+  - `[x]` Task #188: Inject SearXNG container into docker-compose.yml (Closed)
+  - `[x]` Task #189: Implement Web Search Heuristic fallback in AI Chat (Closed)
+  - `[x]` Task #190: Integrate SearXNG API payload parsing with Ollama (Closed)
+### [US-191] Automated Security Password Rotation (Status: Done)
+  - `[x]` Task #192: Execute rotate_passwords.py and update containers (Closed)
+### [US-199] Distribute Deployment Architecture (WSL, Hyper-V, VirtualBox) (Status: Done)
+  - `[x]` Task #200: Create setup_deploy.py for Docker orchestration (Closed)
+### [US-201] Migrate to Apache Airflow Supervisor & Zabbix Telemetry (Status: Done)
+  - `[x]` Task #202: Replace data_sync.sh cron with Python DAG and configure Zabbix API health checks. (Closed)
+### [US-203] System Optimization, Log Rotation, and DR Testing (Status: Done)
+  - `[x]` Task #216: Configure Docker Log Rotation Limits (Closed)
+  - `[x]` Task #217: Develop Automated Disaster Recovery Validation Script (Closed)
+  - `[x]` Task #218: Tune MySQL Database Buffer Pools and Performance Parameters (Closed)

BIN
docs/taiga_audit_report.pdf


BIN
docs/zabbix_monitoring.pdf


+ 429 - 13
generate_docs.py

@@ -2,6 +2,7 @@
 # $Author$
 # $log$
 import os
+import subprocess
 
 docs_dir = "docs"
 os.makedirs(docs_dir, exist_ok=True)
@@ -28,18 +29,83 @@ docs = {
 - Begin the hand-off to the operational team for Phase 2 feature requests.
 """,
     "Backup_Procedure.md": """# $Id$
-# Database Backup Procedure
+# Database Backup and Restore Procedure
+
+## 1. Overview & Policy
+To guarantee clinical records integrity and high availability, Local Food AI enforces a strict backup schedule.
+- **Scope**: Includes MySQL schemas (`food_db`), user profiles (`app_auth`), and configuration states.
+- **Retention Plan**: Automated daily backups with a strict 7-day rolling window purge.
+- **Storage Location**: Stored securely inside the persistent `/backups` directory on the host server.
+
+---
+
+## 2. Automated Daily Backups
+The automated backup mechanism runs via a host cron job pointing to `backup_db.sh`.
+- The script dynamically detects the active MySQL container name (`food-mysql-1` or `food_project-mysql-1`).
+- It executes `mysqldump` directly inside the container without exposing root passwords to shell logs.
+- Outputs are compressed via `gzip` and timestamped: `food_db_YYYYMMDD_HHMM.sql.gz`.
+
+### Cron Configuration Example:
+To run the backup daily at 02:00 AM, add the following to `/etc/crontab`:
+```bash
+0 2 * * * root /bin/bash /c/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/backup_db.sh >> /var/log/backup_db.log 2>&1
+```
+
+---
+
+## 3. Manual Backup Execution
+If a system migration or major upgrade is scheduled, perform a manual dump using the following command:
+```bash
+# 1. Navigate to the project directory
+cd /c/Users/lanfr144/Documents/DOPRO1/Antigravity/Food
+
+# 2. Run the backup wrapper
+bash backup_db.sh
+```
+Verify the output exists inside the backups folder:
+```bash
+ls -lh backups/
+```
+
+---
+
+## 4. Step-by-Step Restore Procedure
+In the event of database corruption or hardware failure, follow these exact steps to restore the database.
+
+### Step 4.1: Identify the Target Backup File
+List available files and pick the desired timestamp:
+```bash
+ls -la backups/
+# Example Target: backups/food_db_20260521_1100.sql.gz
+```
+
+### Step 4.2: Verify MySQL Container Health
+Ensure the MySQL service container is running and healthy:
+```bash
+docker ps --filter name=mysql
+```
+
+### Step 4.3: Execute Restore Stream
+Decompress the backup on-the-fly and pipe it directly into the running MySQL container:
+```bash
+# Adjust the container name ('food-mysql-1' or 'food_project-mysql-1') based on active deployment
+gunzip < backups/food_db_20260521_1100.sql.gz | docker exec -i food-mysql-1 mysql -u root -proot_pass food_db
+```
 
-## Automated Backups
-The system utilizes a cron job pointing to `backup_db.sh`.
-- The script dynamically detects the active MySQL container name (`food-mysql-1` or `food_project-mysql-1`) for high-availability robustness.
-- It executes `mysqldump` directly inside the detected MySQL container.
-- Outputs are piped to `gzip` and stored in `/backups`.
-- A 7-day retention policy automatically purges old backups using `find ... -mtime +7 -exec rm`.
+### Step 4.4: Verify Restored Tables
+Log in to the database and query the core table to confirm the tables are intact and populated:
+```bash
+docker exec -it food-mysql-1 mysql -u food_reader -preader_pass food_db -e "SELECT COUNT(*) FROM products_core;"
+```
+Expected result: A count of OpenFoodFacts entries (typically > 10,000 records).
 
-## Manual Restore
-To manually restore a backup (adjust container name to `food-mysql-1` or `food_project-mysql-1` as appropriate):
-`gunzip < backups/food_db_20260507_0200.sql.gz | docker exec -i food-mysql-1 mysql -u root -proot_pass food_db`
+---
+
+## 5. Verification & Health Check Loops
+Operators must verify the backup archive integrity weekly:
+1. Copy the `.gz` backup to a local testing workspace.
+2. Run `gzip -t backups/filename.sql.gz` to ensure the archive is not corrupted.
+3. Test restoring to a local fallback container instance to verify data accessibility.
 """,
     "Data_Ingestion.md": """# $Id$
 # Data Ingestion Pipeline
@@ -89,8 +155,40 @@ Ask the `llama3.2:3b` model complex dietary questions. It natively utilizes RAG
 Welcome to the static documentation mirror. Please navigate the markdown files in this directory for architectural diagrams and guides.
 """,
     "Scrum_Wiki.md": """# $Id$
-# Scrum Wiki Master List
-This file aggregates references to the Scrum daily logs, plans, and retrospectives.
+# Scrum Wiki Master List & Index Portal
+
+Welcome to the static Scrum documentation portal. This master wiki aggregates and organizes all daily stand-up logs, planning reports, retrospectives, reviews, and velocity charts recorded during the agile development of the **Local Food AI** clinical dietetics engine.
+
+---
+
+## 📅 Sprint Ceremonies & Logs
+
+### 1. [Sprint Plans (Scrum_Plan.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Plan.md)
+*Contains Sprint Plan formulations, active user stories selection, scope statements, and team capacity bounds for each milestone loop.*
+
+### 2. [Daily Scrums (Scrum_Daily.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Daily.md)
+*Continuous daily stand-up summaries tracking individual task completion, blocker mitigations, and immediate day-to-day coordination.*
+
+### 3. [Sprint Reviews (Scrum_Review.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Review.md)
+*Contains sprint review logs, clinician demonstration summaries, feature validation checklists, and stakeholder feedback logs.*
+
+### 4. [Sprint Retrospectives (Scrum_Retro.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Retro.md)
+*Reviews process improvements, continuous integration learnings, and action items aimed at optimizing team operations and environment tuning.*
+
+---
+
+## 📊 Deliverables & Quality Assurance
+
+### 5. [Scrum Artifacts (Scrum_Artifacts.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Scrum_Artifacts.md)
+*Indexes sprint velocity metrics, completed story points distributions, burndown coordinates, and final Taiga delivery milestones.*
+
+### 6. [Sprint 8 Test Cases (Test_Cases_Sprint8.md)](file:///c:/Users/lanfr144/Documents/DOPRO1/Antigravity/Food/docs/Test_Cases_Sprint8.md)
+*Legacy acceptance test logs covering core NLP chat, portion converters, and initial search validations.*
+
+---
+
+> [!NOTE]
+> **Operational Compliance**: All Scrum files above are synchronized with their respective Taiga milestone identifiers (`Sprint 13` and `Sprint 7`). All physical activities recorded in these markdown logs have corresponding closed tasks inside Taiga.
 """,
     "Scrum_Daily.md": """# $Id$
 # Daily Scrums
@@ -122,6 +220,324 @@ Contains User Stories, velocity tracking, and burndown charts from Taiga.
 To deploy on Windows Subsystem for Linux:
 1. Ensure WSL2 backend is enabled in Docker Desktop.
 2. Follow standard Installation Guide inside the WSL Ubuntu terminal.
+""",
+    "User_Description.md": """# $Id$
+# Local Food AI - User Description & Functional Guide
+
+## 1. System Vision
+The **Local Food AI** system is a strictly local, privacy-first, professional-grade clinical dietetics assistant. Developed specifically for clinics and healthcare practitioners, it provides offline nutritional analysis, meal planning, and warning flags based on dynamic patient health profiles. 
+
+Since the system operates entirely locally on local hypervisors, **zero patient medical data or search queries ever leave the server boundary**, ensuring 100% HIPAA compliance and data sovereignty.
+
+---
+
+## 2. Core Functional Pillars
+
+### 📊 tab 1: Clinical Data Search (🔬 Clinical Search)
+Allows practitioners to search the 24GB OpenFoodFacts dataset in real time (average query response time < 0.04 seconds).
+- **Dynamic Medical Warnings**: Based on the active patient profile, foods are immediately flagged in the search results:
+  - ⚠️ **Red Warning Flags**: Highlight high-risk ingredients (e.g. Unpasteurized dairy or raw fish for pregnant patients, high-sodium foods for hypertensive patients, or high-sugar foods for diabetic patients).
+  - 💚 **Green Recommendations**: Highlight recommended dietary components (e.g. High iron/calcium for pregnant or breastfeeding mothers, high Vitamin C for scurvy prevention, or high iron for anemia).
+- **Flexible Column Customization**: Multi-select column headers to inspect specific macro and micro-nutrients.
+
+### 💬 tab 2: AI Clinical Chat (💬 AI Chat)
+An interactive NLP dialogue interface powered by a local lightweight LLM (**Llama3.2:3b**).
+- **RAG-Driven Precision**: The AI dietitian automatically retrieves and reviews local database records and private meta-search results before formulating an answer.
+- **Dynamic Medical Guardrails**: The user's active illnesses, diets, and conditions are injected into the AI's system prompt in the background, forcing the AI to strictly enforce clinical safety constraints.
+
+### 🍽️ tab 3: My Plate Builder (🍽️ My Plate Builder)
+A recipe formulation utility to calculate combined nutritional intake.
+- **Natural Language Parsing**: Enables entering quantities in natural units (e.g., "1.5 cups", "2 tablespoons", "150g").
+- **Exact Conversion**: The system translates these custom units into metric grams based on product density metrics.
+- **Macro Summaries**: Instantly calculates and displays the total combined Protein, Fat, and Carbohydrates.
+
+### 🤖 tab 4: AI Meal Planner (🤖 AI Meal Planner)
+An automated clinical diet planner.
+- Generates a multi-meal daily menu formatted strictly as a Markdown table.
+- Dynamically enforces user-defined calorie limits and active medical restrictions.
+
+---
+
+## 3. Supported Health & Medical Profiles
+- **Conditions**: Pregnant, Breastfeeding, Low Fat, Osteoporosis.
+- **Illnesses**: Diabetes, Hypertension, Kidney Disease, Scurvy, Anemia.
+- **Diets**: Vegan, Vegetarian, Kosher, Halal, Keto, Paleo, Christian (Lent/Good Friday).
+""",
+    "Start_Stop_Procedures.md": """# $Id$
+# Infrastructure Stop & Start Operational Procedures
+
+This runbook outlines the exact sequence and commands to start, stop, and verify each microservice in the Local Food AI environment.
+
+---
+
+## 1. Sequence Priority Rules
+Due to database socket requirements and network bindings, services **must** be started and stopped in the following order:
+
+```mermaid
+graph TD
+    subgraph Startup Sequence
+        direction TB
+        A[1. MySQL Database] --> B[2. Ollama & SearXNG AI Services]
+        B --> C[3. Streamlit Application & Nginx Proxy]
+        C --> D[4. Zabbix Monitoring & Airflow Supervisor]
+    end
+```
+
+---
+
+## 2. Startup Procedures
+
+### Step 2.1: Start the Core MySQL Database
+Verify that the database service is up and listening on port 3307:
+```bash
+docker compose up -d mysql
+# Verify database logs
+docker compose logs -f mysql
+```
+
+### Step 2.2: Start AI Engine & SearXNG Search
+Deploy the AI components:
+```bash
+docker compose up -d ollama searxng
+# Check that Ollama responds
+curl http://localhost:11434/api/tags
+```
+
+### Step 2.3: Start Streamlit App and Nginx Gateway
+Bring up the frontend web interface and reverse proxy:
+```bash
+docker compose up -d app nginx
+# Verify Web Interface status
+curl -I http://localhost
+```
+
+### Step 2.4: Start Zabbix Monitoring Suite
+Deploy the monitoring server and agents:
+```bash
+docker compose up -d zabbix-server zabbix-web zabbix-agent
+# Check dashboard availability
+curl -I http://localhost:8081
+```
+
+---
+
+## 3. Shutdown Procedures
+
+To perform system maintenance or schema migration, stop services in reverse order to prevent lockups:
+
+```bash
+# 1. Stop Monitoring Components
+docker compose stop zabbix-agent zabbix-web zabbix-server
+
+# 2. Stop Web Frontend and Proxy Gateway
+docker compose stop nginx app
+
+# 3. Stop NLP and Search Services
+docker compose stop searxng ollama
+
+# 4. Stop Database Container gracefully
+docker compose stop mysql
+```
+
+---
+
+## 4. Status Verification Commands
+Use these commands to verify container state and port bindings:
+```bash
+# List all running containers in the stack
+docker compose ps
+
+# Inspect raw container logs for error spikes
+docker compose logs --tail=100
+
+# Verify TCP socket listener binds
+netstat -tulpn | grep -E "80|3307|8081|11434"
+```
+""",
+    "Operator_Installation_Guide.md": """# $Id$
+# Local Food AI - Detailed Operator Installation Guide
+
+This document is a step-by-step installation, mapping, configuration, and verification manual for deploying the **Local Food AI** system in an enterprise environment. It covers hybrid hypervisor infrastructure (WSL2, Hyper-V, and VirtualBox), cross-node networking, SNMPv3 monitoring, alert channels, and acceptance testing.
+
+---
+
+## 1. Pre-Deployment Operator Survey (Pre-requisites Gathering)
+Before running installation scripts, the operator **must** collect the following physical/virtual infrastructure parameters and store them in the deployment matrix:
+
+| REQUIRED PARAMETER | OPERATOR INPUT / DESCRIPTION |
+| :--- | :--- |
+| **Deployment Workstation IP** | e.g., 192.168.1.50 |
+| **Hyper-V Host VM IP** | e.g., 192.168.130.170 |
+| **VirtualBox Host VM IP** | e.g., 192.168.130.161 |
+| **SSH Key Location (Private)** | e.g., `~/.ssh/id_rsa` |
+| **SMTP Relay Password** | e.g., `********` (For Zabbix/App password reset email) |
+| **Teams/Discord Webhook URL** | e.g., `https://discord.com/api/webhooks/...` |
+
+---
+
+## 2. Platform Mapping: Which Container Goes Where?
+
+To maximize CPU/GPU efficiency and secure database read/writes, services are distributed across three distinct environments:
+
+| COMPONENT CONTAINER | DEPLOYMENT ENVIRONMENT | WHY |
+| :--- | :--- | :--- |
+| **streamlit-app (app.py)** | Local WSL2 (Windows) | Low-latency rendering and direct client access |
+| **mysql (Database Node)** | Hyper-V VM (Server A) | Persistent enterprise-grade disk storage |
+| **ollama (NLP Llama3.2:3b Engine)** | VirtualBox VM (Server B) | Dedicated CPU/GPU virtualization allocation |
+| **zabbix-server & web (Monitoring)** | Hyper-V VM (Server A) | Centralized SNMPv3 alert processing and logs |
+| **searxng (Meta-Search Gateway)** | Local WSL2 (Windows) | Dynamic browser-level loopbacks |
+
+---
+
+## 3. Platform Provisioning Commands
+
+### 3.1: WSL2 Provisioning (Local Client Workstation)
+Enable WSL2 and install Ubuntu 24.04:
+```powershell
+# Run in Administrator PowerShell
+dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
+dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
+wsl --install -d Ubuntu-24.04
+```
+
+### 3.2: Hyper-V VM Provisioning (Server A - Database & Zabbix)
+Deploy a dedicated Ubuntu VM on Hyper-V using PowerShell:
+```powershell
+# Run in Administrator PowerShell on Server A
+New-VM -Name "FoodAI-Database-Node" -MemoryStartupBytes 8GB -Generation 2 -NewVHDPath "C:\\VMs\\FoodAI_DB.vhdx" -VHDSizeBytes 80GB -SwitchName "External Switch"
+Set-VMFirmware -VMName "FoodAI-Database-Node" -EnableSecureBoot Off
+Start-VM -Name "FoodAI-Database-Node"
+```
+
+### 3.3: VirtualBox VM Provisioning (Server B - Ollama AI Engine)
+Deploy a dedicated VM on VirtualBox using Command Line:
+```bash
+# Run in Command Prompt on Server B
+vboxmanage createvm --name "FoodAI-AI-Node" --ostype "Ubuntu_64" --register
+vboxmanage modifyvm "FoodAI-AI-Node" --memory 8192 --cpus 4 --vram 128 --nic1 bridged --bridgeadapter1 "Intel Ethernet Connection"
+vboxmanage createhd --filename "C:\\VMs\\FoodAI_AI.vdi" --size 60000
+vboxmanage storagectl "FoodAI-AI-Node" --name "SATA Controller" --add sata --controller IntelAHCI
+vboxmanage storageattach "FoodAI-AI-Node" --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium "C:\\VMs\\FoodAI_AI.vdi"
+vboxmanage startvm "FoodAI-AI-Node" --type headless
+```
+
+---
+
+## 4. Secure Authentication & SSH Exchange
+Exchange SSH public keys to allow automated, passwordless container management across nodes:
+```bash
+# 1. Generate SSH Keys on WSL Client
+ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_foodai -N ""
+
+# 2. Push Key to Database VM (Server A)
+ssh-copy-id -i ~/.ssh/id_rsa_foodai.pub operator@192.168.130.170
+
+# 3. Push Key to AI VM (Server B)
+ssh-copy-id -i ~/.ssh/id_rsa_foodai.pub operator@192.168.130.161
+```
+
+---
+
+## 5. Multi-Node Docker Network & Configuration
+
+To allow WSL, Hyper-V, and VirtualBox nodes to communicate, update the `.env` variables and `docker-compose.yml` to use bridged network endpoints.
+
+### Step 5.1: Configure WSL Client `.env`
+Update `.env` in the Streamlit workspace:
+```ini
+DB_HOST=192.168.130.170
+DB_USER=food_reader
+DB_PASS=reader_pass
+APP_AUTH_USER=food_app_auth
+APP_AUTH_PASS=auth_pass
+OLLAMA_HOST=http://192.168.130.161:11434
+SEARXNG_HOST=http://localhost:8080
+ZBX_SERVER_HOST=192.168.130.170
+```
+
+### Step 5.2: Configure Ollama (VirtualBox Server B) Listening Port
+Ensure the Ollama daemon inside VirtualBox binds to `0.0.0.0` (all interfaces):
+```bash
+# SSH into Server B (192.168.130.161)
+sudo systemctl edit ollama.service
+
+# Add the environment variables:
+[Service]
+Environment="OLLAMA_HOST=0.0.0.0"
+
+# Reload and restart service
+sudo systemctl daemon-reload
+sudo systemctl restart ollama
+```
+
+---
+
+## 6. Zabbix Reconfiguration for Multi-Node SNMPv3 Telemetry
+
+To monitor all distributed deployment environments securely:
+
+### Step 6.1: Deploy SNMPv3 Daemons
+Install and configure SNMPv3 daemons on WSL, Hyper-V Database VM, and VirtualBox AI VM:
+```bash
+sudo apt update && sudo apt install -y snmpd
+```
+Edit `/etc/snmp/snmpd.conf`:
+```
+# Listen on all interfaces
+agentAddress udp:161
+
+# Create secure SNMPv3 User
+createUser securityUser SHA "securityAuthPassword" AES "securityPrivPassword"
+rouser securityUser authpriv
+```
+Restart daemon:
+```bash
+sudo systemctl restart snmpd
+```
+
+### Step 6.2: Configure Zabbix Server Dashboard (Web UI)
+1. Open Zabbix in your browser at `http://192.168.130.170:8081`.
+2. Navigate to **Configuration > Hosts > Create Host**.
+3. Create three distinct hosts:
+   - **WSL-Workstation** (IP: `192.168.1.50`)
+   - **Database-Node** (IP: `192.168.130.170`)
+   - **AI-Node** (IP: `192.168.130.161`)
+4. Add the **SNMP Interface** pointing to Port 161 for each host.
+5. In the **Security Tab**, select SNMPv3, enter Username `securityUser`, select Auth Protocol `SHA` / `securityAuthPassword`, and Privacy Protocol `AES` / `securityPrivPassword`.
+6. Attach the pre-installed **Local Food AI Telemetry** Template.
+
+---
+
+## 7. Verifying Alert Channels
+
+### 7.1: Microsoft Teams / Discord Alert Webhook
+To verify Zabbix is communicating with Discord / Teams:
+1. Trigger a test CPU threshold spike inside WSL:
+   ```bash
+   yes > /dev/null & sleep 10 ; killall yes
+   ```
+2. Verify Zabbix triggers the alert and transmits the notification.
+3. Check your designated channel for the incoming payload:
+   - Expected Output: `[PROBLEM] High CPU Utilization Detected on WSL-Workstation`.
+
+### 7.2: Password Reset Email (SMTP Gateway)
+1. In the Streamlit UI Sidebar, select **Reset Password**.
+2. Trigger a reset link for user `ClinicianA`.
+3. Check the inbox or SMTP system log (`tail -f /var/log/mail.log` on Server A) to verify outbound delivery.
+
+---
+
+## 8. Operator Post-Installation Checklist
+
+Run these test cases to verify the installation:
+
+| TEST CASE ID | ACTIONS TO PERFORM | EXPECTED RESULTS | STATUS |
+| :--- | :--- | :--- | :---: |
+| **TC-OP-01** | Search 'Cheese' on Search Tab | 10+ records returned in <0.04s. Listeria warning flags on unpasteurized. | `[ ]` |
+| **TC-OP-02** | Enter '1.5 cups' in Plate Tab | Parsed and converted to metric grams based on density index. | `[ ]` |
+| **TC-OP-03** | Ask Chat: 'Can I eat sushi?' | Llama3.2:3b retrieves database context and flags raw fish as forbidden for pregnancy. | `[ ]` |
+| **TC-OP-04** | Trigger manual db backup | Timestamped compressed .sql.gz created inside backups/ folder. | `[ ]` |
+| **TC-OP-05** | Terminate Ollama Container | Zabbix PROBLEM active alert generated on dashboard in < 30 seconds. | `[ ]` |
 """
 }
 
@@ -146,4 +562,4 @@ for filename, content in docs.items():
         f.write(content.replace('$Id$', git_id))
     print(f"Generated {filepath}")
 
-print("\nDocs directory perfectly mirrored.")
+print("\nDocs directory perfectly mirrored with operator level runbooks.")

+ 185 - 0
manage_services.sh

@@ -0,0 +1,185 @@
+#!/bin/bash
+# ==============================================================================
+# $Id$
+# File: manage_services.sh
+# Purpose: Comprehensive Service Manager for Local Food AI development.
+#          Allows operators and developers to cleanly start, stop, restart,
+#          and inspect all project elements in sequential order of dependencies
+#          without triggering online data ingestion pipelines.
+# ==============================================================================
+
+# Exit immediately if a command exits with a non-zero status (except when checked)
+set -e
+
+# Sequence priority rules:
+# STARTUP:  1. MySQL -> 2. Ollama & SearXNG -> 3. Streamlit App & Nginx Proxy -> 4. Zabbix & Airflow
+# SHUTDOWN: 1. Zabbix & Airflow -> 2. Nginx & App -> 3. SearXNG & Ollama -> 4. MySQL
+
+COMPOSE_FILE="docker-compose.yml"
+
+# Colors for output logging
+GREEN='\033[0;32m'
+BLUE='\033[0;34m'
+YELLOW='\033[1;33m'
+RED='\033[0;31m'
+NC='\033[0m' # No Color
+
+log_info() {
+    echo -e "${BLUE}[INFO] $(date '+%Y-%m-%d %H:%M:%S') - $1${NC}"
+}
+
+log_success() {
+    echo -e "${GREEN}[SUCCESS] $(date '+%Y-%m-%d %H:%M:%S') - $1${NC}"
+}
+
+log_warn() {
+    echo -e "${YELLOW}[WARNING] $(date '+%Y-%m-%d %H:%M:%S') - $1${NC}"
+}
+
+log_error() {
+    echo -e "${RED}[ERROR] $(date '+%Y-%m-%d %H:%M:%S') - $1${NC}"
+}
+
+# Verify Docker Compose file exists
+if [ ! -f "$COMPOSE_FILE" ]; then
+    log_error "Docker Compose file ($COMPOSE_FILE) not found in the current directory."
+    exit 1
+fi
+
+start_services() {
+    log_info "Initiating sequential startup sequence for development..."
+
+    # Step 1: Start MySQL Database
+    log_info "Step 1/4: Starting MySQL Database Container..."
+    docker compose -f "$COMPOSE_FILE" up -d mysql
+    
+    # Wait for MySQL to become fully ready and accept connections
+    log_info "Waiting for MySQL database socket to be available..."
+    until docker compose -f "$COMPOSE_FILE" exec mysql mysqladmin ping -h"localhost" -u"root" -p"BTSai123" --silent; do
+        sleep 2
+        echo -n "."
+    done
+    echo ""
+    log_success "MySQL is online and healthy."
+
+    # Step 2: Start local AI Engine (Ollama) and Anonymous Search (SearXNG)
+    log_info "Step 2/4: Starting Ollama and SearXNG microservices..."
+    docker compose -f "$COMPOSE_FILE" up -d ollama searxng
+    
+    # Wait briefly for Ollama daemon bind
+    sleep 3
+    log_success "AI and Search infrastructure successfully online."
+
+    # Step 3: Start Core Streamlit Application UI and Nginx Gateway Proxy
+    log_info "Step 3/4: Starting Streamlit UI and Nginx Proxy Gateway..."
+    docker compose -f "$COMPOSE_FILE" up -d app nginx
+    log_success "Frontend and Proxy elements online."
+
+    # Step 4: Start DevOps Orchestrator Stack (Zabbix Monitoring, Zabbix Agent, Airflow)
+    log_info "Step 4/4: Deploying Zabbix Monitoring suite and Airflow Supervisors..."
+    # Note: Airflow scheduler/webserver are started for pipeline supervision
+    docker compose -f "$COMPOSE_FILE" up -d zabbix-server zabbix-web zabbix-agent
+    
+    # Check if Airflow service elements exist in compose file before starting
+    if grep -q "airflow-webserver" "$COMPOSE_FILE"; then
+        docker compose -f "$COMPOSE_FILE" up -d airflow-webserver airflow-scheduler || log_warn "Airflow containers not defined or failed to start."
+    fi
+
+    log_success "All Local Food AI development services started successfully!"
+}
+
+stop_services() {
+    log_info "Initiating sequential graceful shutdown sequence..."
+
+    # Step 1: Stop Monitoring and Supervisor Stack
+    log_info "Step 1/4: Stopping Zabbix suite and Airflow Supervisors..."
+    docker compose -f "$COMPOSE_FILE" stop zabbix-agent zabbix-web zabbix-server
+    if grep -q "airflow-webserver" "$COMPOSE_FILE"; then
+        docker compose -f "$COMPOSE_FILE" stop airflow-scheduler airflow-webserver || true
+    fi
+    log_success "DevOps monitoring stack shut down."
+
+    # Step 2: Stop Streamlit Application and Secure Proxy Gateway
+    log_info "Step 2/4: Shutting down Nginx and App frontend..."
+    docker compose -f "$COMPOSE_FILE" stop nginx app
+    log_success "Application frontend shut down."
+
+    # Step 3: Stop AI Ollama Inference Engine and SearXNG Search Gateway
+    log_info "Step 3/4: Stopping SearXNG and Ollama AI Engine..."
+    docker compose -f "$COMPOSE_FILE" stop searxng ollama
+    log_success "AI services shut down."
+
+    # Step 4: Stop Core MySQL Database Container gracefully (prevent table corruption)
+    log_info "Step 4/4: Stopping MySQL database..."
+    docker compose -f "$COMPOSE_FILE" stop mysql
+    log_success "Database node cleanly shut down."
+
+    log_success "All Local Food AI services stopped gracefully!"
+}
+
+check_status() {
+    log_info "Inspecting status of stack containers..."
+    docker compose -f "$COMPOSE_FILE" ps
+    
+    log_info "Active network sockets check:"
+    echo "---------------------------------------------------------"
+    echo "Port  | Target Service               | Status"
+    echo "---------------------------------------------------------"
+    
+    # Check MySQL on 3307
+    if nc -z localhost 3307 2>/dev/null; then
+        echo -e "3307  | MySQL Database Node          | ${GREEN}ONLINE${NC}"
+    else
+        echo -e "3307  | MySQL Database Node          | ${RED}OFFLINE${NC}"
+    fi
+
+    # Check Ollama on 11434
+    if nc -z localhost 11434 2>/dev/null; then
+        echo -e "11434 | Ollama AI Engine             | ${GREEN}ONLINE${NC}"
+    else
+        echo -e "11434 | Ollama AI Engine             | ${RED}OFFLINE${NC}"
+    fi
+
+    # Check Nginx Gateway on 80
+    if nc -z localhost 80 2>/dev/null; then
+        echo -e "80    | Nginx Gateway (HTTP Proxy)   | ${GREEN}ONLINE${NC}"
+    else
+        echo -e "80    | Nginx Gateway (HTTP Proxy)   | ${RED}OFFLINE${NC}"
+    fi
+
+    # Check Zabbix Web UI on 8081
+    if nc -z localhost 8081 2>/dev/null; then
+        echo -e "8081  | Zabbix Dashboard             | ${GREEN}ONLINE${NC}"
+    else
+        echo -e "8081  | Zabbix Dashboard             | ${RED}OFFLINE${NC}"
+    fi
+    echo "---------------------------------------------------------"
+}
+
+show_help() {
+    echo "Usage: $0 {start|stop|restart|status}"
+    echo "  start   - Deploy and wake up all services sequentially according to priorities."
+    echo "  stop    - Gracefully turn off containers in reverse dependency order."
+    echo "  restart - Perform sequential shutdown followed by sequential boot."
+    echo "  status  - Print container status and inspect physical TCP port sockets."
+    exit 1
+}
+
+case "$1" in
+    start)
+        start_services
+        ;;
+    stop)
+        stop_services
+        ;;
+    restart)
+        stop_services
+        start_services
+        ;;
+    status)
+        check_status
+        ;;
+    *)
+        show_help
+        ;;
+esac

+ 48 - 1
scripts/generate_pdfs.py

@@ -37,9 +37,56 @@ def main():
         
         try:
             pdf = MarkdownPdf(toc_level=2)
-            pdf.add_section(Section(md_content))
+            base_name = os.path.basename(md_file)
+            
+            if base_name == 'project_report.md':
+                print("Splitting project_report.md into Portrait/Landscape/Portrait sections...")
+                parts = md_content.split('## 2. Project File Catalog & Documentation')
+                if len(parts) == 2:
+                    portrait_part1 = parts[0]
+                    remaining = parts[1]
+                    parts2 = remaining.split('## 3. Directory Structure Map')
+                    if len(parts2) == 2:
+                        landscape_part = '## 2. Project File Catalog & Documentation' + parts2[0]
+                        portrait_part2 = '## 3. Directory Structure Map' + parts2[1]
+                        
+                        pdf.add_section(Section(portrait_part1, paper_size="A4"))
+                        pdf.add_section(Section(landscape_part, paper_size="A4-L"))
+                        pdf.add_section(Section(portrait_part2, paper_size="A4"))
+                    else:
+                        print("WARNING: Could not find Directory Structure Map heading. Defaulting to full portrait.")
+                        pdf.add_section(Section(md_content, paper_size="A4"))
+                else:
+                    print("WARNING: Could not find Project File Catalog heading. Defaulting to full portrait.")
+                    pdf.add_section(Section(md_content, paper_size="A4"))
+            else:
+                pdf.add_section(Section(md_content, paper_size="A4"))
+                
             pdf.save(pdf_file)
             print(f"Saved {os.path.basename(pdf_file)}")
+            
+            # Copy to workspace root if applicable
+            import shutil
+            root_dir = os.path.join(os.path.dirname(__file__), '..')
+            if base_name == 'project_report.md':
+                dest = os.path.join(root_dir, 'Project.pdf')
+                try:
+                    if os.path.exists(dest):
+                        os.remove(dest)
+                    shutil.copy2(pdf_file, dest)
+                    print("Successfully updated root Project.pdf")
+                except Exception as copy_err:
+                    print(f"WARNING: Could not copy to root Project.pdf: {copy_err}")
+            elif base_name == 'retro_planning.md':
+                dest = os.path.join(root_dir, 'Retro Planning.pdf')
+                try:
+                    if os.path.exists(dest):
+                        os.remove(dest)
+                    shutil.copy2(pdf_file, dest)
+                    print("Successfully updated root Retro Planning.pdf")
+                except Exception as copy_err:
+                    print(f"WARNING: Could not copy to root Retro Planning.pdf: {copy_err}")
+                    
         except Exception as e:
             print(f"WARNING: Could not save {os.path.basename(pdf_file)}. File might be locked or open in a viewer. Error: {e}")
 

+ 64 - 31
scripts/generate_project_report.py

@@ -6,109 +6,139 @@ import sys
 # Ensure stdout handles UTF-8 correctly
 sys.stdout.reconfigure(encoding='utf-8')
 
-# Dictionary containing static details about files
+# Dictionary containing static details about files with relative paths and detailed purposes
 FILE_DETAILS = {
     "app.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\app.py",
+        "location": "./app.py",
         "purpose": "Core Streamlit Web Application. Hosts the clinical food search engine, the RAG chat dietitian interface (utilizing Ollama and SearXNG tool calling), and the visual plate builder."
     },
     "ingest_csv.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\ingest_csv.py",
+        "location": "./ingest_csv.py",
         "purpose": "High-performance background database loader. Stream-reads and batch-inserts the 3GB OpenFoodFacts dataset into MySQL using Pandas chunking and optimizes indices post-load."
     },
     "unit_converter.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\unit_converter.py",
+        "location": "./unit_converter.py",
         "purpose": "Mathematical converter engine that parses natural recipe volume inputs (e.g. cups, spoons) and converts them to metric weights based on macro density mappings."
     },
     "snmp_notifier.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\snmp_notifier.py",
+        "location": "./snmp_notifier.py",
         "purpose": "Observability SNMP utility. Formulates and transmits raw SNMP trap payloads to the central Zabbix monitoring server on critical application failures."
     },
     "configure_zabbix_alerts.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\configure_zabbix_alerts.py",
+        "location": "./configure_zabbix_alerts.py",
         "purpose": "DevOps provisioning script. Uses the Zabbix API to automatically set up host groups, custom templates, items, triggers, actions, and media types for alerts."
     },
+    "configure_zabbix_email.py": {
+        "location": "./configure_zabbix_email.py",
+        "purpose": "Security & Monitoring. Configures email media types and SMTP server routes for Zabbix alert notifications on system downtime."
+    },
     "zabbix_telemetry.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\zabbix_telemetry.py",
-        "purpose": "Telemetry collector daemon. Scrapes live memory usage, Streamlit active user threads, and query performance to feed the Zabbix dashboard."
+        "location": "./zabbix_telemetry.py",
+        "purpose": "Monitoring agent daemon. Queries active application statistics, memory, and query timers to supply Zabbix telemetry indicators."
     },
     "check_users.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\check_users.py",
+        "location": "./check_users.py",
         "purpose": "Security utility. Verifies user accounts inside the MySQL `users` table and checks password hashing complexity."
     },
     "rotate_passwords.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\rotate_passwords.py",
+        "location": "./rotate_passwords.py",
         "purpose": "Administrative credential utility. Cycles and re-encrypts database passwords within the `.env` secret file."
     },
     "myloginpath.py": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\myloginpath.py",
+        "location": "./myloginpath.py",
         "purpose": "MySQL credential companion helper that simplifies the generation of encrypted login path configuration profiles."
     },
     "data_sync.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\data_sync.sh",
+        "location": "./data_sync.sh",
         "purpose": "Master pipeline coordinator. Supports download fetching in --online mode and local file processing in offline fallback mode."
     },
     "backup_db.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\backup_db.sh",
+        "location": "./backup_db.sh",
         "purpose": "Resiliency backup automation. Runs mysqldump on user tables inside the active container and prunes backups older than 7 days."
     },
     "reset.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\reset.sh",
+        "location": "./reset.sh",
         "purpose": "Teardown script. Wipes local temporary containers and prunes volume locks during crashes."
     },
     "proper_reset.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\proper_reset.sh",
+        "location": "./proper_reset.sh",
         "purpose": "High-level administrative wipe script that brings the entire network stack and repositories back to a pristine state."
     },
     "deploy.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\deploy.sh",
+        "location": "./deploy.sh",
         "purpose": "Naked OS installation guide. Installs necessary system packages, Python venv libraries, and native Ollama."
     },
     "start_batch_ingest.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\start_batch_ingest.sh",
+        "location": "./start_batch_ingest.sh",
         "purpose": "Asynchronous background shell script wrapping the main csv ingestion stream inside a detached session."
     },
     "download_csv.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\download_csv.sh",
+        "location": "./download_csv.sh",
         "purpose": "Downloader helper script that fetches specific smaller subsets of OpenFoodFacts CSV files."
     },
     "master_trigger.sh": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\master_trigger.sh",
+        "location": "./master_trigger.sh",
         "purpose": "Orchestrator script that wakes and verifies multiple secondary subservices in sequence."
     },
+    "manage_services.sh": {
+        "location": "./manage_services.sh",
+        "purpose": "DevOps service manager script. Handles automated, sequential startup, shutdown, restart, and health checking of all container elements in the stack."
+    },
+    "generate_docs.py": {
+        "location": "./generate_docs.py",
+        "purpose": "Dynamic doc generator. Generates and mirrors all markdown manuals under `/docs` with live Git log metadata injection."
+    },
     "docker-compose.yml": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\docker-compose.yml",
+        "location": "./docker-compose.yml",
         "purpose": "Main 10-container Docker orchestration map defining MySQL, App UI, Ollama Engine, SearXNG, Nginx proxy, Airflow stack, and Zabbix server suites."
     },
     "docker-compose_skip.yml": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\docker-compose_skip.yml",
+        "location": "./docker-compose_skip.yml",
         "purpose": "Resilient 8-container offline/local single-node orchestration manifest."
     },
     "alembic.ini": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\alembic.ini",
+        "location": "./alembic.ini",
         "purpose": "Alembic configuration setting routing database connection URIs for versioning schemas."
     },
     "my.cnf": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\my.cnf",
+        "location": "./my.cnf",
         "purpose": "Custom tuned MySQL database performance settings enabling local_infile data loading and index page buffers."
     },
     ".env": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\.env",
+        "location": "./.env",
         "purpose": "Secret storage container holding encrypted MySQL user passwords and active environment flags."
     },
     ".gitattributes": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\.gitattributes",
+        "location": "./.gitattributes",
         "purpose": "Git clean/smudge layout mapping enabling automatic tracking of dynamic $Id$ metadata expansion within version files."
     },
     "requirements.txt": {
-        "location": "c:\\Users\\lanfr144\\Documents\\DOPRO1\\Antigravity\\Food\\requirements.txt",
+        "location": "./requirements.txt",
         "purpose": "Python runtime dependency catalog storing strict library versioning constraints."
+    },
+    "scripts/generate_pdfs.py": {
+        "location": "./scripts/generate_pdfs.py",
+        "purpose": "PDF document builder. Converts all markdown documentation manuals under `/docs` into high-fidelity PDF format with expanded Git version headers."
+    },
+    "scripts/generate_project_report.py": {
+        "location": "./scripts/generate_project_report.py",
+        "purpose": "Technical project report generator. Automatically gathers codebase structure, Git commit metadata, and purpose records to construct the Project.pdf report."
+    },
+    "scripts/setup_deploy.py": {
+        "location": "./scripts/setup_deploy.py",
+        "purpose": "DevOps deployment script. Orchestrates local and VM container sets, verifying network connectivity and system parameters."
+    },
+    "scripts/taiga_sync_final.py": {
+        "location": "./scripts/taiga_sync_final.py",
+        "purpose": "Taiga automated synchronization helper. Pushes bug tickets, fills wiki pages, and assigns unassigned user stories."
     }
 }
 
 def get_git_info(filename):
     try:
-        cmd = ['git', 'log', '-1', '--format=%h|%an|%ad|%s', '--date=format:%Y/%m/%d %H:%M:%S', '--', filename]
+        # Standardize path separators for git command line
+        git_filename = filename.replace('\\', '/')
+        cmd = ['git', 'log', '-1', '--format=%h|%an|%ad|%s', '--date=format:%Y/%m/%d %H:%M:%S', '--', git_filename]
         output = subprocess.check_output(cmd, encoding='utf-8').strip()
         if output:
             parts = output.split('|')
@@ -163,19 +193,22 @@ The **Local Food AI** capstone project has successfully completed all sprint ite
 4. **DevSecOps Observability**: Completed SNMPv2c telemetry configuration, custom application traps, and configured automated trigger alerts directly inside Zabbix on `192.168.130.170`.
 5. **Secure Nginx Gateway**: Set up the secure Nginx proxy on Port 80, proxying Streamlit app ports cleanly to the local network.
 6. **Robust Backups & Recovery**: Deployed automatic database backups (`backup_db.sh`) and local offline single-node fallback capabilities (`docker-compose_skip.yml`).
+7. **Sequential Operations Manager**: Created `manage_services.sh` to allow developers to safely stop, start, and restart all microservices in the proper dependency order without triggering redundant online ingestion sequences.
 
 ---
 
 ## 2. Project File Catalog & Documentation
-Below is an exhaustive description of every critical file in the repository, detailing its absolute location, primary purpose, and active Git version tags.
+Below is an exhaustive catalog of every critical file in the repository, detailing its path, functional purpose, and active Git version tags. 
+
+*Note: This chapter is compiled in landscape layout inside Project.pdf to guarantee complete columns readability.*
 
-| File Name | Absolute Location | Purpose & Core Responsibility | Last Commit | Author | Commit Date | Last Commit Message |
-| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
+| File Path | Purpose & Technical Responsibility | Commit | Author | Commit Date | Last Commit Message |
+| :--- | :--- | :--- | :--- | :--- | :--- |
 """
 
     for filename, details in FILE_DETAILS.items():
         git_info = get_git_info(filename)
-        row = f"| **{filename}** | `{details['location']}` | {details['purpose']} | `{git_info['commit']}` | {git_info['author']} | {git_info['date']} | {git_info['message']} |\n"
+        row = f"| **{filename}**<br>`{details['location']}` | {details['purpose']} | `{git_info['commit']}` | {git_info['author']} | {git_info['date']} | *{git_info['message']}* |\n"
         report_content += row
 
     report_content += """