Model Weights, Quantization, and Inference Parameter Topology

Technical parameters, weight quantization (GGUF/AWQ 4-bit/8-bit), attention architectures, and GPU/CPU resource requirements.

1. Overview & Technical Architecture#

The primary sovereign enterprise model is built upon a fine-tuned transformer architecture optimized for low-latency business transaction processing.

1. Model Specifications & Quantization Matrix#

Specification	Primary Sovereign Model	Analytical Cloud Model	Lightweight Edge Model
Base Architecture	LLaMA-3.1 8B Instruct	Gemini 3.5 Flash	LLaMA-3.2 3B Instruct
Parameter Count	8.03 Billion	Proprietary (MoE)	3.21 Billion
Quantization Format	AWQ 4-bit / GGUF Q4_K_M	Full Precision (API)	Q4_0 CPU-Optimized
Context Window	128,000 tokens	1,000,000 tokens	32,000 tokens
VRAM Footprint	5.8 GB VRAM	Managed Cloud	2.2 GB VRAM / CPU RAM
Target Workload	On-premise tax & payroll	Complex cross-module BI	Mobile field service

2. Attention Mechanisms and Tokenizer#

The tokenizer incorporates 16,000 dedicated tokens for the Ethiopic (Ge'ez) script Unicode block (U+1200 to U+137F), ensuring that Amharic text tokenizes with an efficient 1.15 tokens-per-word ratio compared to 4.8 tokens-per-word in standard English-centric tokenizers.

2. Prerequisites & Infrastructure Requirements#

Runtime Environment: Python 3.12+, PostgreSQL 16 with pgvector extension.
Security Access Group: Ashewa AI / Administrator (Technical ID: muk_ai.group_ai_manager)
Hardware Profile: Minimum 16 GB System RAM (CPU inference) or NVIDIA GPU with 8+ GB VRAM.

3. Step-by-Step Configuration & Verification#

Step 1: Navigating to Model Management#

Access your SmartERP instance at https://smarterp.et/odoo.
Open Settings → Technical → Ashewa AI → Model Registry.
Select the custom model profile.

{screenshot:20-model-topology-and-weights-step1-navigation}

Step 2: Testing Endpoint Health & Response Verification#

Click the Run Diagnostic Health Check button.
Verify token generation speed (tokens/sec) and latency benchmarks.
Review sample response in the diagnostic console.

{screenshot:20-model-topology-and-weights-step2-verification}

4. Advanced Configuration Points#

Automated Tasks: muk_ai.ir_cron_model_health_monitor runs every 15 minutes.
Multi-Tenant Boundaries: Custom model fine-tuning adapters are strictly segregated per tenant database.

5. Comprehensive Troubleshooting Matrix#

Symptom / Error	Probable Root Cause	Resolution Action
`Connection refused: Model server unreachable`	Inference daemon stopped or port blocked by firewall.	Restart model service (`systemctl restart vllm`) and verify firewall rules.
`Token limit exceeded on input prompt`	Input context exceeded window size.	Enable context truncation in model profile settings.

መጨረሻ የተሻሻለው: 2026-10-30