Real-Time Inference Pipelines, JSON-RPC & Streaming API Hooks

1. Overview & Technical Architecture#

SmartERP supports both synchronous streaming inference for interactive chat and asynchronous batch pipelines for bulk document processing.

1. Streaming Server-Sent Events (SSE) Protocol#

When users interact with the floating chat interface (chat_composer.js), the frontend establishes an SSE connection to /ashewa_ai/stream, rendering tokens as they are generated by the model.

Client (OWL) ──> POST /ashewa_ai/stream ──> Odoo Controller ──> vLLM Stream
Client <── Token Chunk 1 <── Yield Chunk 1 <── SSE Stream
Client <── Token Chunk 2 <── Yield Chunk 2 <── SSE Stream
Client <── [DONE] <── Finalize <── Close

2. Python ORM Service Hooks#

Developers can invoke model inference directly from any Odoo model method using self.env['muk_ai.provider']:

# Example ORM Method Invocation
provider = self.env['muk_ai.provider'].search([('code', '=', 'sovereign')], limit=1)
response = provider.generate_text(
    prompt="Audit vendor bill INV/2026/0014 for duplicate line items.",
    temperature=0.1,
    max_tokens=2048,
)

2. Prerequisites & Infrastructure Requirements#

• Runtime Environment: Python 3.12+, PostgreSQL 16 with pgvector extension.
• Security Access Group: Ashewa AI / Administrator (Technical ID: muk_ai.group_ai_manager)
• Hardware Profile: Minimum 16 GB System RAM (CPU inference) or NVIDIA GPU with 8+ GB VRAM.

3. Step-by-Step Configuration & Verification#

Step 1: Navigating to Model Management#

1. Access your SmartERP instance at https://smarterp.et/odoo.
2. Open Settings → Technical → Ashewa AI → Model Registry.
3. Select the custom model profile.

{screenshot:40-inference-pipelines-and-api-hooks-step1-navigation}

Step 2: Testing Endpoint Health & Response Verification#

1. Click the Run Diagnostic Health Check button.
2. Verify token generation speed (tokens/sec) and latency benchmarks.
3. Review sample response in the diagnostic console.

{screenshot:40-inference-pipelines-and-api-hooks-step2-verification}

4. Advanced Configuration Points#

• Automated Tasks: muk_ai.ir_cron_model_health_monitor runs every 15 minutes.
• Multi-Tenant Boundaries: Custom model fine-tuning adapters are strictly segregated per tenant database.

5. Comprehensive Troubleshooting Matrix#