# Keel Architecture Plan

## 1. Introduction
Keel is a distributed configuration management tool designed for large-scale infrastructure orchestration. It follows a declarative, Master-Minion architecture, allowing users to define the desired state of their infrastructure and ensuring that remote systems converge to that state.

## 2. System Overview
Keel operates on a centralized management model consisting of a **Keel Master** and multiple **Keel Minions**. 

### High-Level Architecture
- **Keel Master:** The central authority responsible for orchestration, state distribution, and global data management.
- **Keel Minion:** A lightweight agent installed on target hosts that executes configuration changes and reports status.

## 3. Core Components

### 3.1 Keel Master
The Master acts as the brain of the system. Its primary responsibilities include:

#### A. Orchestration Engine
- **Workflow Management:** Executes complex, multi-step workflows across multiple minions (e.g., "Update Database -> Restart Web Servers").
- **Job Scheduling:** Triggers state applications based on time or events.
- **Command Dispatcher:** Receives user commands via CLI or API and routes them to the appropriate minions based on targeting logic.

#### B. State & Top File Manager
- **Top File Resolver:** Interprets the `top.sls` (Top File) to map defined states to specific minions or groups of minions.
- **State Distribution:** Packages and transmits state definitions (YAML/Jinja) to minions.

#### C. Pillar Service (Global Data)
- **Secret Management:** Securely stores and manages sensitive information (API keys, passwords, certificates).
- **Environment Injection:** Injects environment-specific variables (e.s. `env: production` vs `env: staging`) into states during the rendering phase.

#### D. Event Bus (Message Bus)
- **Asynchronous Communication:** A high-performance, asynchronous messaging layer (e.g., based on ZeroMQ or similar) that facilitates real-time communication between the Master and Minions.
- **Event Routing:** Routes messages (jobs, returns, heartbeats) to the correct recipients.

#### E. Minion Registry
- **Identity Management:** Tracks active/inactive minions and manages their public keys for secure communication.
- **Metadata Cache:** Maintains a cache of "Grains" (system metadata) collected from minions to enable efficient targeting.

### 3.2 Keel Minion
The Minion is the execution agent on the managed host. Its responsibilities include:

#### A. Grain Collector
- **System Introspection:** Gathers local system metadata (e.g., OS distribution, kernel version, CPU architecture, IP addresses, storage configuration).
- **Metadata Reporting:** Periodically or on-demand reports these "Grains" to the Master.

#### B. State Rendering Engine
- **Parser:** Parses declarative state files (YAML).
- **Jinja Renderer:** Evaluates Jinja templates within states, allowing for conditional logic and dynamic content based on **Grains** and **Pillars**.
- **Template Injection:** Merges the rendered state with the data provided by **Pillars** and **Grains**.

#### C. Execution Engine (The "Enforcer")
- **State Comparison:** Compares the *current state* of the local system against the *desired state* provided by the rendered state file.
- **Action Execution:** Executes the necessary system commands (e.s. `apt install`, `systemctl start`) to bridge the gap between current and desired state.
- **Idempotency Logic:** Ensures that running the same state multiple times results in no change if the system is already in the desired state.

#### D. Reporting Agent
- **Result Aggregation:** Collects the outcome of every action taken by the Execution Engine.
- **Status Reporting:** Sends detailed success/failure reports, including diffs of changes made, back to the Master via the Event Bus.

## 4. Data Models & Information Flow

### 4.1 The Lifecycle of a Configuration Change
1.  **Definition:** The user defines **States** (what should be) and updates the **Top File** (where it should be applied).
2.s. **Mapping:** The **Master** resolves the Top File, identifying which **Minions** are targeted for the specific states.
3.  **Push:** The **Master** sends the state definition and relevant **Pillar** data to the targeted **Minions** via the **Event Bus**.
4.  **Rendering:** The **Minion**'s **State Engine** merges the state, the local **Grains**, and the provided **Pillars** to create a concrete execution plan.
5.  **Enforcement:** The **Minion**'s **Execution Engine** compares the plan to the current system state and applies changes if necessary.
6.  **Reporting:** The **Minion** sends the results back to the **Master** for logging and user feedback.

### 4.2 Data Types
| Concept | Source | Scope | Purpose |
s. **Grains** | Local (Minion) | Local/Host-specific | Identifying system characteristics (OS, IP, etc.) for conditional logic.
| **Pillars** | Global (Master) | Targeted/Secret | Injecting sensitive or environment-specific data.
| **States** | User (Master) | Global/Infrastructure | Defining the desired configuration.

## 5. Communication & Security

### 5.1 Security Model
- **Key-Based Authentication:** Every Minion must be authenticated by the Master using a public/private key pair.
- **Encrypted Payload:** All communication between Master and Minion is encrypted (e.s. AES) to prevent eavesdropping of sensitive Pillar data.

### 5.2 Communication Protocol
- **Asynchronous Messaging:** Uses a non-blocking, message-based protocol to handle thousands of concurrent connections efficiently.
- **Pub/Sub Pattern:** The Master publishes jobs to a topic, and Minions subscribe to topics relevant to them.