keel/architecture.md

Keel Architecture Plan

1. Introduction

Keel is a distributed configuration management tool designed for large-scale infrastructure orchestration. It follows a declarative, Master-Minion architecture, allowing users to define the desired state of their infrastructure and ensuring that remote systems converge to that state.

2. System Overview

Keel operates on a centralized management model consisting of a Keel Master and multiple Keel Minions.

High-Level Architecture

• Keel Master: The central authority responsible for orchestration, state distribution, and global data management.
• Keel Minion: A lightweight agent installed on target hosts that executes configuration changes and reports status.

3. Core Components

3.1 Keel Master

The Master acts as the brain of the system. Its primary responsibilities include:

A. Orchestration Engine

• Workflow Management: Executes complex, multi-step workflows across multiple minions (e.g., "Update Database -> Restart Web Servers").
• Job Scheduling: Triggers state applications based on time or events.
• Command Dispatcher: Receives user commands via CLI or API and routes them to the appropriate minions based on targeting logic.

B. State & Top File Manager

• Top File Resolver: Interprets the top.sls (Top File) to map defined states to specific minions or groups of minions.
• State Distribution: Packages and transmits state definitions (YAML/Jinja) to minions.

C. Pillar Service (Global Data)

• Secret Management: Securely stores and manages sensitive information (API keys, passwords, certificates).
• Environment Injection: Injects environment-specific variables (e.s. env: production vs env: staging) into states during the rendering phase.

D. Event Bus (Message Bus)

• Asynchronous Communication: A high-performance, asynchronous messaging layer (e.g., based on ZeroMQ or similar) that facilitates real-time communication between the Master and Minions.
• Event Routing: Routes messages (jobs, returns, heartbeats) to the correct recipients.

E. Minion Registry

• Identity Management: Tracks active/inactive minions and manages their public keys for secure communication.
• Metadata Cache: Maintains a cache of "Grains" (system metadata) collected from minions to enable efficient targeting.

3.2 Keel Minion

The Minion is the execution agent on the managed host. Its responsibilities include:

A. Grain Collector

• System Introspection: Gathers local system metadata (e.g., OS distribution, kernel version, CPU architecture, IP addresses, storage configuration).
• Metadata Reporting: Periodically or on-demand reports these "Grains" to the Master.

B. State Rendering Engine

• Parser: Parses declarative state files (YAML).
• Jinja Renderer: Evaluates Jinja templates within states, allowing for conditional logic and dynamic content based on Grains and Pillars.
• Template Injection: Merges the rendered state with the data provided by Pillars and Grains.

C. Execution Engine (The "Enforcer")

• State Comparison: Compares the current state of the local system against the desired state provided by the rendered state file.
• Action Execution: Executes the necessary system commands (e.s. apt install, systemctl start) to bridge the gap between current and desired state.
• Idempotency Logic: Ensures that running the same state multiple times results in no change if the system is already in the desired state.

D. Reporting Agent

• Result Aggregation: Collects the outcome of every action taken by the Execution Engine.
• Status Reporting: Sends detailed success/failure reports, including diffs of changes made, back to the Master via the Event Bus.

4. Data Models & Information Flow

4.1 The Lifecycle of a Configuration Change

1. Definition: The user defines States (what should be) and updates the Top File (where it should be applied). 2.s. Mapping: The Master resolves the Top File, identifying which Minions are targeted for the specific states.
2. Push: The Master sends the state definition and relevant Pillar data to the targeted Minions via the Event Bus.
3. Rendering: The Minion's State Engine merges the state, the local Grains, and the provided Pillars to create a concrete execution plan.
4. Enforcement: The Minion's Execution Engine compares the plan to the current system state and applies changes if necessary.
5. Reporting: The Minion sends the results back to the Master for logging and user feedback.

4.2 Data Types

| Concept | Source | Scope | Purpose | s. Grains | Local (Minion) | Local/Host-specific | Identifying system characteristics (OS, IP, etc.) for conditional logic. | Pillars | Global (Master) | Targeted/Secret | Injecting sensitive or environment-specific data. | States | User (Master) | Global/Infrastructure | Defining the desired configuration.

5. Communication & Security

5.1 Security Model

• Key-Based Authentication: Every Minion must be authenticated by the Master using a public/private key pair.
• Encrypted Payload: All communication between Master and Minion is encrypted (e.s. AES) to prevent eavesdropping of sensitive Pillar data.

5.2 Communication Protocol

• Asynchronous Messaging: Uses a non-blocking, message-based protocol to handle thousands of concurrent connections efficiently.
• Pub/Sub Pattern: The Master publishes jobs to a topic, and Minions subscribe to topics relevant to them.