Modular AI Runtime

EnginePool & AgentPool Coordination and Execution

KOLI’s backend orchestration relies on the coordinated operation of two core components: EnginePool and AgentPool. These two pools manage AI model services and intelligent task agents respectively, working together through a unified routing and scheduling system. Below is an overview of how they interact across registration, routing, and task execution.

1 EnginePool: Model Registration & Dynamic Routing

EnginePool serves as the centralized registry and service discovery hub for all AI model engines. When a new model (e.g., LLM, vision, voice clone) is deployed, it registers itself to the pool along with key metadata including:

  • Model name

  • Functionality (e.g., text generation, image recognition)

  • API endpoint

  • Load capacity (e.g., max QPS, health status)

Example pseudocode for engine registration:

EnginePool.register("LLM", endpoint="http://10.0.0.1:8000", max_qps=100)
EnginePool.register("VisionModel", endpoint="http://10.0.0.2:8000", version="v2")
EnginePool.register("VoiceClone", endpoint="http://10.0.0.3:8000", kol_id="Alice")

When an agent needs to invoke a model, EnginePool routes the request to the optimal engine instance based on:

  • Real-time server load

  • Model version compatibility

  • Geolocation proximity

The routing process is fully abstracted from the agents, who only need to specify the functional capability they require. EnginePool handles load balancing and returns the proper endpoint. This architecture ensures system scalability and availability under concurrent load.

2 AgentPool: Task Assignment & Multi-Modal Composition

AgentPool manages the intelligent task agents that encapsulate business workflows. Agents specialize in areas like content generation, market interpretation, DeFi execution, etc.

At system startup, each agent type registers to AgentPool with:

  • Agent name

  • Supported task types

  • Required engine dependencies

When a user or system task is triggered, the scheduler selects the appropriate agent type to process the job based on metadata or AI-based task classification.

Example execution flow inside an agent:

def agent.handle_request(request):
    if request.type == "image_analysis":
        vision = EnginePool.get("VisionModel")
        analysis = vision.analyze(request.image)

        llm = EnginePool.get("LLM")
        result = llm.generate_text(f"Explain the image: {analysis}")
        return result

    elif request.type == "defi_action":
        llm = EnginePool.get("LLM")
        action = llm.plan_defi_action(request.instruction)
        # Forward to blockchain layer...

This example demonstrates how an agent dynamically chains multiple engines (e.g., vision + language) while preserving context across calls.

🧩 EnginePool

The EnginePool manages AI model backends by type:

  • ASR (Speech to Text)

  • LLM (Language Model)

  • TTS (Text to Speech)

Each engine type is registered at startup and exposed via a common interface:

def setup(self, config):
    for asrCfg in config.ASR.SUPPORT_LIST:
        self._pool[ASR][asrCfg.NAME] = ASRFactory.create(asrCfg)
    for llmCfg in config.LLM.SUPPORT_LIST:
        self._pool[LLM][llmCfg.NAME] = LLMFactory.create(llmCfg)
    for ttsCfg in config.TTS.SUPPORT_LIST:
        self._pool[TTS][ttsCfg.NAME] = TTSFactory.create(ttsCfg)

When a user sends a voice message or text input, EnginePool routes the request through the configured inference chain.

🤖 AgentPool

AgentPool maintains and dispatches AI agent instances with on-chain identity and cross-device compatibility.

  • Distributed Agent Runtime: Agents can run in cloud, edge, or consumer environments

  • x402 + ERC-8004 Integration: Each agent has verifiable credentials and payment channels

  • Multi-Interface Dispatching: Web, mobile, plugin, and VR endpoints connect to the same logic layer

  • Health Check + Load Routing: Assign tasks based on availability, latency, and trust score

{
  "agent_id": "0xKOLI123",
  "device": "Oculus VR",
  "availability": true,
  "reputation_score": 96.4
}

🔌 Plugin and Extension Modules

KOLI runtime supports plug-and-play enhancements:

  • Neural Machine Translation (mBART/NLLB)

  • Tool routing & retrieval augmentation (e.g. trading, summarization)

  • On-device inference adapters (LoRA, quantized LLMs)

  • Emotion-aware TTS and prosody stylizers

Together, they deliver a composable and extensible execution backbone for KOLI—one that allows for plug-and-play models, dynamic multi-agent coordination, and scalable real-time AI experiences.

PreviousMultimodal Generation LayerNextSecurity

Last updated