orka.orchestrator.dry_run_engine module

Smart Path Evaluator

Intelligent LLM-powered path evaluation system that replaces static mocks with dynamic reasoning about optimal workflow paths.

Uses a two-stage LLM approach: 1. Path Selection LLM: Analyzes agent capabilities and suggests best paths 2. Validation LLM: Validates selections and assesses efficiency

class orka.orchestrator.dry_run_engine.PathEvaluation(node_id: str, relevance_score: float, confidence: float, reasoning: str, expected_output: str, estimated_tokens: int, estimated_cost: float, estimated_latency_ms: int, risk_factors: List[str], efficiency_rating: str)[source]

Bases: object

Result of LLM path evaluation.

node_id: str

relevance_score: float

confidence: float

reasoning: str

expected_output: str

estimated_tokens: int

estimated_cost: float

estimated_latency_ms: int

risk_factors: List[str]

efficiency_rating: str

class orka.orchestrator.dry_run_engine.ValidationResult(is_valid: bool, confidence: float, efficiency_score: float, validation_reasoning: str, suggested_improvements: List[str], risk_assessment: str)[source]

Bases: object

Result of LLM validation.

is_valid: bool

confidence: float

efficiency_score: float

validation_reasoning: str

suggested_improvements: List[str]

risk_assessment: str

class orka.orchestrator.dry_run_engine.SmartPathEvaluator(config: Any)[source]

Bases: object

LLM-powered intelligent path evaluation system.

Replaces static mocks with dynamic reasoning about: - Agent capability matching - Expected output quality - Resource efficiency - Risk assessment

__init__(config: Any)[source]: Initialize smart evaluator with LLM configuration.

async simulate_candidates(candidates: List[Dict[str, Any]], question: str, context: Dict[str, Any], orchestrator: Any) → List[Dict[str, Any]][source]

Evaluate candidates using LLM reasoning based on real agent information.

Parameters:

candidates – List of candidate paths
question – The question being routed
context – Execution context
orchestrator – Orchestrator instance

Returns:

Candidates with LLM evaluation results

orka.orchestrator.dry_run_engine.DryRunEngine: alias of SmartPathEvaluator