Introduction

The document site for light-portal application.

Architecture

Design

Light Portal is an application that connect the providers to the consumers, and it contains many components or applications. Each component will have some API endpoints and a user interface in the portal view single page application.

To allow the users to understand each component in detail in term of design, we have collected all the design documents in this section.

Portal View

Mutliple Environment

This document outlines the necessary changes to configure portal view to work dynamically across different environments (sdx, dev, non-prod, prod) using environment-specific configuration.

1. Environment Variables Setup

Create .env File

Create environment-specific .env files in project root:

# Environment variables
# VITE_BASE_PATH is used as the base URL prefix for API calls.
VITE_BASE_PATH=/bff/admin/
# VITE_PORTAL_URL is the full absolute URL where the frontend static files are served
VITE_PORTAL_URL=https://sdx.lightapi.net/bff

Required Environment Variables

• VITE_BASE_PATH: Defines the sub-path where your application is deployed.
• VITE_PORTAL_URL: The API endpoint base URL.

Benefits of .env Configuration

• Switch environments without code changes
• Maintain a single codebase for all environments

2. Vite Configuration Changes

File: vite.config.js Location: Project root

Required Change:

import { defineConfig, loadEnv } from 'vite';
import react from '@vitejs/plugin-react';

export default defineConfig(({ mode }) => {
  const env = loadEnv(mode, process.cwd(), '');

  return {
    plugins: [react()],
    base: env.VITE_BASE_PATH || "/",
    // ... other configurations
  };
});

Why This Change is Necessary?

The Problem Without base Configuration

When your application is deployed to a sub-path rather than the domain root, all asset references break.

What base Affects

The base configuration controls how Vite prefixes:

• Static asset URLs (JavaScript, CSS, images, fonts)
• Client-side routing paths
• Public folder references

Example: Without vs With base

Without base Configuration:

• App hosted at: https://example.com/portal/
• Vite generates: <script src="/assets/index.js">
• Browser requests: https://example.com/assets/index.js
• Result: 404 Not Found ❌

With base: “/portal/”:

• App hosted at: https://example.com/portal/
• Vite generates: <script src="/portal/assets/index.js">
• Browser requests: https://example.com/portal/assets/index.js
• Result: Success ✅

3. React Router Configuration Changes

File: App.tsx Location: src/App.tsx

Required Change:

import { BrowserRouter } from 'react-router-dom';

function App() {
  const basename = import.meta.env.VITE_BASE_PATH || "/";

  return (
    <BrowserRouter basename={basename}>
      {/* Your app routes and components */}
    </BrowserRouter>
  );
}

export default App;

What basename Does

The basename prop tells React Router the base URL prefix for all routes in your application.

Routing Behavior Comparison

Why It’s Required

When your app is hosted at a sub-path (e.g., https://example.com/portal/), React Router needs to know that /portal is the deployment prefix, not part of your route definitions.

Without basename:

1. You define <Route path="/dashboard" />
2. User visits /portal/dashboard
3. React Router sees /portal/dashboard → no match → Route Not Found ❌

With basename="/portal":

1. React Router strips /portal from the URL
2. Sees /dashboard → matches your route → Success ✅

4. API Call Configuration

Current Behavior Issue

Without a configured base URL, the browser constructs API request URLs relative to the current page origin.

Example:

• App running at: https://example.com/portal/dashboard
• API call: fetch('/api/users')
• Browser sends request to: https://example.com/api/users

This may work in some cases but breaks when:

• API is hosted on a different domain/subdomain
• API has a different base path
• Cross-environment consistency is needed

Solution: Custom Fetch Wrapper

File: src/utils/fetchClient.js

const BASE_URL = import.meta.env.VITE_API_BASE_URL || "";

/**
 * Custom fetch wrapper with automatic base URL prefixing
 * @param {string} endpoint - API endpoint path (e.g., '/api/users')
 * @param {Object} options - Fetch options (method, headers, body, etc.)
 * @returns {Promise} - Response JSON
 */
async function fetchClient(endpoint, options = {}) {
  const url = `${BASE_URL}${endpoint}`;

  const defaultHeaders = {
    "Content-Type": "application/json",
  };

  const config = {
    ...options,
    headers: {
      ...defaultHeaders,
      ...options.headers,
    },
  };

  const response = await fetch(url, config);

  if (!response.ok) {
    throw new Error(`HTTP error! status: ${response.status}`);
  }

  return response.json();
}

export default fetchClient;

Usage Example

import fetchClient from './utils/fetchClient';

// GET request
const users = await fetchClient('/api/users');

// POST request
const newUser = await fetchClient('/api/users', {
  method: 'POST',
  body: JSON.stringify({ name: 'John Doe', email: '[email protected]' }),
});

// With custom headers
const data = await fetchClient('/api/protected', {
  headers: {
    'Authorization': `Bearer ${token}`,
  },
});

Benefits

• Consistency: All API calls use the same base URL
• Environment flexibility: Different API endpoints per environment
• Maintainability: Single place to update API configuration
• Error handling: Centralized response validation

5. Build and Deployment Steps

Step 1: Change the .env variables for specific environment (sdx, dev, prod etc.)

VITE_BASE_PATH=/ (base path)
VITE_PORTAL_URL=https://example.com (endpoint URL)

Step 2: Build

npm run build

MCP Registry Design for AI Gateway

This document outlines the registration and management strategy for Model Context Protocol (MCP) tools within the AI Gateway.

Registration Strategy: The Hybrid Model

For a robust and scalable AI Gateway, the recommended approach is a Hybrid Model: Register the MCP Server as the primary entity, but manage and expose the Tools individually.

This approach balances the technical requirements of connectivity with the operational requirements of governance, security, and performance.

1. Register the Server (The “Connection” Layer)

The MCP Server should be treated as the source of truth and the primary unit of connectivity.

• Centralized Configuration: Authentication (API keys, OAuth), base URLs, transport protocols (SSE or Stdio), and environment variables are defined at the server level.
• Connectivity Management: A single server acts as a wrapper around related APIs. Registering tools individually would create significant overhead and redundant connections.
• Lifecycle & Health Monitoring: If an MCP server goes down, all its tools become unavailable. It is more efficient to monitor health and availability at the server level.
• Dynamic Discovery: The MCP protocol includes a tools/list capability. By registering the server, the gateway can automatically sync and discover new tools when the server is updated, eliminating the need for manual registration of every new function.

2. Expose Tools Individually (The “Governance” Layer)

While the gateway connects to the server, it should expose and manage tools as individual objects. This is crucial for:

• Granular Permissions (RBAC): Access control can be applied at the tool level. For example, a “Finance” team might be granted access to a get-invoice tool but restricted from a modify-ledger tool, even if both reside on the same ERP server.
• Context Window Optimization: Large Language Models (LLMs) have limited context windows. Sending all 50 tools from a large server to an LLM wastes tokens and increases the “lost in the middle” effect. The Gateway should allow for the activation of specific subsets of tools for a given AI session or agent.
• Rate Limiting & Cost Control: High-compute or high-cost tools (e.g., generate-video) can be rate-limited or billed differently compared to lightweight tools (e.g., get-weather).
• Safety & Compliance: Metadata can be attached to individual tools to flag them as Read-Only, Destructive, or Sensitive, enabling specific security flows (like “Human-in-the-loop” approvals) for risky operations.

Recommended Architecture: The “Catalog” Pattern

The implementation should follow a “Catalog” or “App Store” pattern:

1. Provider/Server Registration: An admin registers a server (e.g., “The GitHub MCP Server”) with its credentials.
2. Automated Discovery: The Gateway calls the server’s list_tools method and populates a tool catalog.
3. Governance & Activation: Admins “enable” specific tools for specific model configurations or user groups.
4. Routing Layer: When a model requests a tool, the Gateway resolves the request to the owning Server and handles the underlying communication.

Comparison of Approaches

Data Model & Schema Design

The AI Gateway leverages the existing API registry schema used by light-gateway, with specific enhancements to accommodate the unique requirements of the MCP protocol.

Conceptual Mapping

Core Tables & Enhancements

To support MCP, the following schema adjustments are implemented:

1. API Version (Server Connection)

The api_version_t table is enhanced to store transport-level configurations for stdio or SSE connections.

ALTER TABLE api_version_t ADD COLUMN transport_config TEXT;
-- JSON Example for transport_config: 
-- {"transport": "stdio", "command": "npx", "args": ["-y", "@mcp/server-google"]}

2. API Endpoint (Tool Definition)

The api_endpoint_t table acts as the tool registry. We relax the traditional HTTP method constraints and add fields for MCP tool metadata.

-- Allow 'call' as a valid operation for MCP tools
ALTER TABLE api_endpoint_t DROP CONSTRAINT api_endpoint_t_http_method_check;
ALTER TABLE api_endpoint_t ADD CHECK ( http_method IN ( 'delete', 'get', 'patch', 'post', 'put', 'call' ) );

-- Store the Tool Schema (for LLM validation) and Metadata (for safety flags)
ALTER TABLE api_endpoint_t ADD COLUMN tool_schema TEXT;   -- JSON Schema of the tool inputs
ALTER TABLE api_endpoint_t ADD COLUMN tool_metadata TEXT; -- e.g., {"destructive": true, "read_only": false}

Full Registry Schema Reference

-- API Definition (The MCP Server)
CREATE TABLE api_t (
    host_id                 UUID NOT NULL,
    api_id                  VARCHAR(16) NOT NULL,
    api_name                VARCHAR(128) NOT NULL,
    api_desc                VARCHAR(1024),
    api_status              VARCHAR(32) NOT NULL,
    active                  BOOLEAN NOT NULL DEFAULT TRUE,
    PRIMARY KEY (host_id, api_id)
);

-- API Version (The Connection/Transport)
CREATE TABLE api_version_t (
    host_id                 UUID NOT NULL,
    api_version_id          UUID NOT NULL,
    api_id                  VARCHAR(16) NOT NULL,
    api_version             VARCHAR(16) NOT NULL,
    api_type                VARCHAR(7) NOT NULL,    -- 'mcp', 'openapi', etc.
    transport_config        TEXT,                   -- MCP-specific connection data
    spec                    TEXT,                   -- Full tool manifest (optional)
    active                  BOOLEAN NOT NULL DEFAULT TRUE,
    PRIMARY KEY(host_id, api_version_id),
    FOREIGN KEY(host_id, api_id) REFERENCES api_t(host_id, api_id) ON DELETE CASCADE
);

-- API Endpoint (The Individual Tool)
CREATE TABLE api_endpoint_t (
    host_id              UUID NOT NULL,
    endpoint_id          UUID NOT NULL,
    api_version_id       UUID NOT NULL,
    endpoint             VARCHAR(1024) NOT NULL,  -- Tool Name
    http_method          VARCHAR(10),             -- 'call' for MCP
    endpoint_name        VARCHAR(128) NOT NULL,
    endpoint_desc        TEXT,
    tool_schema          TEXT,                    -- Input parameter validation
    tool_metadata        TEXT,                    -- Safety and cost metadata
    active               BOOLEAN NOT NULL DEFAULT TRUE,
    PRIMARY KEY(host_id, endpoint_id),
    FOREIGN KEY(host_id, api_version_id) REFERENCES api_version_t(host_id, api_version_id) ON DELETE CASCADE
);

Tool Metadata & Synchronization

Populating the api_endpoint_t table involves coordinating data from the MCP Server with operational policies defined within the AI Gateway.

Sources of Metadata

The metadata for each tool is synthesized from three primary sources:

1. Standard MCP Server Response (Automated)

When the Gateway performs a tools/list call, the MCP server provides the baseline technical definition for each tool.

• Source Fields: name, description, inputSchema.
• Mapping: These are mapped directly to endpoint, endpoint_desc, and tool_schema respectively.

2. Gateway Operational Enrichment (Manual/Policy)

Since the standard MCP protocol does not include operational flags (like safety or cost), the AI Gateway manages these in the tool_metadata JSON column.

• Administrative Enrichment: Platform admins use the Gateway UI to tag specific tools. Common tags include:
  • destructive: true: Triggers a warning or confirmation flow.
  • human_approval_required: true: Places the request in a queue for manual sign-off.
  • cost_tier: "high": Used for rate-limiting or internal billing.
• Heuristic Auto-Tagging: The Gateway can automatically infer metadata based on patterns. For example, any tool starting with get_ or list_ is auto-flagged as read_only: true.

3. Protocol Extensions (Custom)

The MCP specification allows for additional properties in the tool object. If a custom MCP server includes an extra metadata or annotations block, the Gateway’s synchronization logic can be configured to capture and store these directly.

Synchronization Workflow

The following lifecycle ensures the Gateway’s registry remains accurate:

1. Connection: The Gateway establishes a connection to the server using the transport_config.
2. Discovery (Sync): The Gateway calls tools/list and performs an “upsert” for all tools found.
   • Existing tools have their tool_schema and endpoint_desc updated.
   • New tools are created with a default active status and baseline tool_metadata.
3. Review: An administrator reviews the newly discovered tools in the Gateway dashboard.
4. Governance Policy: The administrator “enables” the tool for specific roles and configures any required safety metadata (e.g., flagging the drop_table tool as destructive).
5. LLM Execution: When a model calls the tool, the Gateway uses the stored tool_schema for pre-flight validation and the tool_metadata to enforce security policies.

Too Many Pages/Forms

The portal has accumulated many pages, generated forms, custom admin screens, and feature-specific entry points. The sidebar can expose these pages, but it does not help a user understand which pages are required to finish a real business task. The MCP Gateway quick start wizard is a useful experiment, but it also shows the limitation of a rigid linear wizard: real tasks have optional steps, pre-existing data, and multiple valid starting points.

This document proposes a task-oriented navigation layer for portal-view.

Problem

Users currently need to know the portal information architecture before they can complete a task. For example, onboarding an API to MCP Gateway may require some combination of:

• create or select an API
• create or select an API version
• link the API version to a gateway or sidecar instance
• select MCP tools
• configure access control
• revisit instance, API, or role administration later

The same pattern exists across other areas. A task is not a single route; it is a sequence of related pages and forms. The current navigation model makes users pick pages first, then infer the task process themselves.

Current MCP Wizard Observation

The MCP Gateway wizard already has useful building blocks:

• flowConfig.tsx keeps step metadata in one place.
• McpServerForm.tsx renders a generic wizard shell.
• useMcpPrefill.ts can resume from URL context such as apiId, apiVersionId, and instanceApiId.
• Several steps are marked skippable.

However, the wizard is still too rigid:

• Step order is linear even when the task is naturally conditional.
• Initial step selection relies on hard-coded step numbers.
• Optional work is represented as skip buttons instead of task state.
• The wizard duplicates or wraps existing forms instead of treating existing pages/forms as first-class task steps.
• The solution is specific to MCP Gateway and does not help users navigate the rest of the portal.

Design Goals

• Let users start from a task, not a page name.
• Keep existing pages and generated forms as the source of truth.
• Support multiple entry points into the same task.
• Detect what has already been completed and show only relevant next actions.
• Support optional, required, blocked, complete, and skipped steps.
• Preserve role-based visibility and host-specific context.
• Allow users to leave a task, return later, and continue from context.
• Make the approach reusable for MCP, API publishing, access control, deployment, config promotion, migration, and admin workflows.

Non-Goals

• Do not replace every admin page with a wizard.
• Do not create a separate custom form for each task if an existing generated form already works.
• Do not use the sidebar as the only navigation surface.
• Do not force a strict step sequence when the data model allows safe jumping.

Proposed Solution

Add a task-oriented navigation layer above the current pages/forms.

The main pieces are:

1. Task Center
2. Task Registry
3. Task Progress Resolver
4. Task Navigation Shell
5. Global Search and Command Palette
6. Contextual Next Actions

Task Center

The Task Center is a page where users choose what they want to accomplish. It should group work by intent, not by implementation table.

Example task groups:

• API Marketplace
  • Register a new API
  • Add an API version
  • Publish an API
  • Review API details
• MCP Gateway
  • Onboard an existing API to MCP Gateway
  • Register a standalone MCP server
  • Configure MCP tools
  • Configure MCP access control
• Access Control
  • Create role
  • Assign permissions
  • Configure endpoint access
• Platform Operations
  • Register controller/gateway instance
  • Link API version to instance
  • Promote configuration
• Portal Administration
  • Manage host users
  • Export/import portal data
  • Convert migration snapshot

Each task card should show:

• title
• short description
• required role
• common starting object, such as API, instance, host, or client
• progress status when the current context is known
• primary action such as Start, Continue, Review, or Fix Missing Step

Task Registry

Introduce a registry that describes tasks and steps declaratively. This is the generalized version of the current MCP flowConfig.tsx, but it should route to existing pages/forms instead of rendering every step inside one wizard.

Example TypeScript shape:

export type TaskDefinition = {
  id: string;
  title: string;
  description: string;
  category: string;
  roles?: string[];
  keywords: string[];
  entryPoints: TaskEntryPoint[];
  steps: TaskStep[];
};

export type TaskStep = {
  id: string;
  title: string;
  description?: string;
  required: boolean;
  dependsOn?: string[];
  route: (ctx: TaskContext) => string;
  formId?: string;
  completeWhen?: TaskCompletionCheck;
  visibleWhen?: TaskVisibilityCheck;
  blockedWhen?: TaskBlockedCheck;
};

The task registry should live close to portal navigation code, for example:

src/tasks/taskRegistry.ts
src/tasks/taskTypes.ts
src/tasks/resolvers/
src/pages/tasks/TaskCenter.tsx
src/pages/tasks/TaskDetail.tsx

Page And Form Metadata

To make search and tasks work well, pages and generated forms need metadata.

For generated forms, the metadata can come from Forms.json plus a small registry override when the form title is not enough.

For custom pages, add a route/page registry:

export type PageDefinition = {
  route: string;
  title: string;
  description?: string;
  category: string;
  roles?: string[];
  keywords: string[];
  entities?: string[];
};

This registry can feed:

• sidebar sections
• Task Center
• command palette
• page breadcrumbs
• contextual next actions

The important rule is that page/form metadata should be reused, not copied into each wizard.

Task Progress Resolver

A task should not blindly ask users to complete steps that are already done. Each task can have a resolver that checks the current host and entity context.

For MCP Gateway, the resolver can check:

• API exists
• API version exists
• instance API link exists
• MCP tool configuration exists
• access control exists

The UI then marks each step:

• Complete
• Required
• Optional
• Blocked
• Skipped
• Needs review

The resolver should use existing query endpoints where possible. The first implementation can query on page load. Later, it can cache per task/session.

Task Navigation Shell

Instead of a full-screen wizard that owns all steps, use a task shell that can wrap or accompany existing pages.

Recommended behavior:

• A task detail page shows the checklist and current state.
• Selecting a step navigates to the existing page/form with task context in the URL or router state.
• The target page shows a compact “Task” panel or return link.
• After save, the user can return to the checklist or continue to the next recommended step.

Example URL:

/app/form/createService?task=mcp-onboard-api&returnTo=/app/tasks/mcp-onboard-api

This keeps existing page behavior intact while adding guided navigation.

Global Search And Command Palette

The portal should have a global launcher. It should search tasks, pages, forms, and entities.

Examples:

• “onboard mcp”
• “create api”
• “auth client”
• “relation type”
• “instance api”
• “export snapshot”

Search results should be role-aware and host-aware.

Result types:

• Task
• Page
• Form
• Entity
• Recent item

This is the fastest way to help expert users without forcing them through a wizard.

Contextual Next Actions

Detail pages should expose next actions based on the current entity.

Examples:

• API detail
  • Add version
  • Link version to gateway
  • Configure MCP tools
  • Configure access control
• Instance detail
  • Link API version
  • Configure MCP tools
  • View gateway servers
• Auth client detail
  • Assign owner
  • Review sessions
  • Review audit
• Snapshot export
  • Convert snapshot
  • Import snapshot

These actions should come from the same task registry, not from one-off buttons hard-coded on every page.

MCP Gateway Example

The MCP Gateway quick start can be rebuilt as a task:

Task: Onboard API to MCP Gateway

Steps:
1. Select or create API
2. Select or create API version
3. Choose deployment mode
4. Link API version to gateway or sidecar instance
5. Select MCP tools
6. Configure access control

Step behavior:

• API selection is required unless apiId is already provided.
• API version is required unless apiVersionId is already provided.
• Spec upload is optional and only shown when creating a new API/version.
• Deployment mode is required when the version is not linked.
• Gateway selection is required only for centralized deployment.
• Tool selection is optional if users only want to register the server first.
• Access control is optional but should be shown as a recommended final step.

This task can support several entry points:

/app/tasks/mcp-onboard-api
/app/tasks/mcp-onboard-api?apiId=...
/app/tasks/mcp-onboard-api?apiId=...&apiVersionId=...
/app/tasks/mcp-onboard-api?instanceApiId=...

The UI should not rely on fixed step numbers. It should compute visible steps from the task context and completion state.

Task State

Start with client-side state:

• URL query parameters for entity context
• sessionStorage for in-progress task context
• existing backend records for real completion state

Later, add persisted task state if needed:

• user id
• host id
• task id
• context JSON
• skipped step ids
• last active step
• updated timestamp

Persisting task state should not become the source of truth for business data. It should only remember navigation state and user choices. Completion should be derived from actual portal records.

Sidebar Role

The sidebar should become smaller and more stable. It should expose major areas, not every page/form.

Recommended sidebar sections:

• Home
• Tasks
• Marketplace
• MCP Gateway
• Operations
• Administration

Deep links should still exist, but they should be discoverable through search, contextual actions, and task detail pages.

Implementation Plan

Phase 1: Inventory And Metadata

• Create page/form metadata registry.
• Add task registry types.
• Register the most-used pages and forms.
• Add global search over registered tasks/pages/forms.

Phase 2: Task Center

• Add /app/tasks.
• Add task category cards.
• Add task detail checklist page.
• Implement client-side task context with URL parameters and session storage.

Phase 3: MCP Gateway Task

• Convert the current MCP wizard flow into mcp-onboard-api task definition.
• Reuse existing MCP components for the pages that still need custom UI.
• Replace hard-coded step numbers with resolver-driven visible steps.
• Add return-to-task behavior after saves.

Phase 4: Contextual Actions

• Add task actions to API detail and instance detail pages.
• Add task actions to access control and config pages where appropriate.
• Use the task registry to drive action visibility.

Phase 5: Broader Rollout

• Add tasks for API publishing, config promotion, host/user management, and snapshot export/import.
• Reduce sidebar clutter once task/search usage is available.
• Add persisted task state only if session storage is not enough.

Risks And Mitigations

Recommendation

Keep the MCP Gateway wizard as a prototype, but do not build more isolated wizards in the same style. The long-term solution should be:

• a task registry
• a Task Center
• resolver-driven progress
• global search
• contextual next actions
• reuse of existing pages and generated forms

This gives new users guided paths while still letting experienced users jump directly to the page or form they already know.

Config Update Page

The current portal-view configuration admin area is complete but split across many table pages and generated forms. A customer Settings page shows a denser workflow: list applicable config properties in a tree, edit scalar values inline, and open a modal for list/map values. This document proposes a similar page for portal-view that can update config property overrides at the environment, product, product version, instance, API, app, and app-api levels. In this document, API, app, and app-api mean the instance-linked config override scopes represented by instanceApiId, instanceAppId, and instanceApiId + instanceAppId.

Current Implementation

The customer Settings implementation is centered on these files:

• Settings.jsx
• SettingsListView.jsx
• SettingsListMapModal.jsx
• InputForm.jsx
• JsonSchemaForm.jsx

The useful behavior is:

• one page loads applicable config properties and the current custom values
• properties are displayed as a tree under configName
• scalar values are edited inline
• list and map values open a modal with form, raw JSON, and raw YAML tabs
• save chooses create or update based on whether the override exists
• delete removes the override and lets the inherited value show again

The customer implementation currently handles instance, instance API, instance app, and instance app API targets. It chooses the query/write action from instanceId, instanceApiId, and instanceAppId.

portal-view already has separate config override pages:

• src/pages/config/ConfigEnvironment.tsx
• src/pages/config/ConfigProduct.tsx
• src/pages/config/ConfigProductVersion.tsx
• src/pages/config/ConfigInstance.tsx
• src/pages/config/ConfigInstanceApi.tsx
• src/pages/config/ConfigInstanceApp.tsx
• src/pages/config/ConfigInstanceAppApi.tsx

Those pages use Material React Table, fetch one override aggregate at a time, and navigate to generated react-schema-form routes for create/update. The form definitions live in src/data/Forms.json, and the generic form runner is src/components/Form/Form.tsx.

The existing form approach works for CRUD, but it is inefficient for config editing because the user must pick a config, pick a property, leave the list page, edit one value, and return.

Goals

• Provide a single task-oriented config editor for the seven override scopes.
• Show the property catalog and current override values together.
• Preserve the existing config-command write APIs.
• Preserve optimistic concurrency by carrying aggregateVersion for existing override rows.
• Avoid client-side joins across independently paginated result sets.
• Keep existing table pages and generated forms available as admin fallback routes.
• Support scalar editing inline and structured list/map editing in a modal.
• Show inherited/default value and custom override value separately.
• Make delete/reset mean “remove this override”, not “delete the base property”.
• Support read-only and hidden states when the user lacks write permission for a scope or target.

Non-Goals

• Do not replace react-schema-form globally.
• Do not replace the existing config list pages during the first release.
• Do not edit File or Cert property values inline in the first release. Those can continue to use existing generated forms.
• Do not require every config property to have a typed form schema before the page is useful.
• Do not add a bulk transaction command in the first release. The UI can stage multiple changes and then orchestrate the existing single-row commands.

Recommended UX

Add a Config Update page under the configuration task area. The first row is a scope and target selector:

• Scope: Environment, Product, Product Version, Instance, API, App, App API
• Target: the selected scope’s identity, such as environment, productId, productVersionId, instanceId, instanceApiId, instanceAppId, or both app/API ids for app-api
• Optional filters: config phase, config type, property type, resource type, and “show overridden only”
• Save mode: staged changes by default, with optional single-row Apply for quick edits

Below the selectors, render a tree/table:

• group rows by configName
• property leaf rows show propertyName
• columns: value type, inherited value, override value, effective source, required, resource type, config phase, description, status
• row status: inherited, overridden, dirty, saving, conflict, error
• toolbar actions: expand all, collapse all, refresh, reset override, review changes, apply changes
• row action menu: view history, open fallback form, copy identifiers

Editing behavior:

• string: inline text editor or larger popover editor for long values
• boolean: select true/false
• integer and float: numeric editor with validation before save
• list and map: open a structured modal
• unsupported valueType: view-only with a link to the existing form route
• propertyType File or Cert: open the existing create/update form in a drawer or modal overlay

The page should keep the inherited/default value visible while editing an override. If the override is deleted, the row remains visible and falls back to the inherited value.

The staged-change panel should list every pending create, update, and reset before applying. This matters for coordinated changes such as enabling a flag and setting a related URL. The backend commands can still run one by one, but the user gets a review step and can see partial failures without losing the full set of intended changes.

The row action menu should include View History. It should link to the audit log or history page pre-filtered by configId, propertyId, and the selected scope target. The row already shows updateUser and updateTs; history gives operators the deeper trail they need when debugging production configuration changes.

Structured Value Modal

The modal should start with raw JSON and raw YAML tabs. If a schema is available for the property, add a Form tab.

The customer code loads schema assets with:

schemas/<propertyName>/<propertyName>.json
schemas/<propertyName>/config.js

portal-view does not currently have this property-schema convention, so the first implementation should not depend on local schema assets. Use raw JSON/YAML with syntax and JSON validation first. Typed form support should come from the light-portal schema registry through schema-query, schema-command, and schema_t.

If a row has a schemaId and schemaVersion, the dialog should lazily fetch the published schema body from schema-query when the user opens the structured editor. The main getConfigUpdateProperties response should include schema metadata but not schemaBody, so the paginated table does not move large schema documents unnecessarily.

The schema association should key by configId + propertyId. Human-friendly keys such as configName + propertyName can be shown in the UI, but should not be used as the durable validation key.

List/map values should be saved as compact JSON strings because the command APIs store propertyValue as a string.

Schema Registry Validation

The schema registry should be used for structured map and list values once the registry is hardened enough for production validation. The config update page should treat the registry as optional per property: rows with no schema still use valueType validation and raw JSON/YAML editing.

getConfigUpdateProperties should return lightweight schema metadata:

{
  "schemaId": "security-jwt-claim-mapping",
  "schemaVersion": "1.0.0",
  "schemaType": "json",
  "schemaStatus": "P",
  "hasSchema": true
}

The UI should enable the Form tab only when the schema exists, is published, and is compatible with the property value type. Schema documents should be cached by hostId + schemaId + schemaVersion, with host-specific lookup falling back to a global schema.

Validation must run in both places:

• frontend validation gives immediate editor feedback and highlights the JSON path that failed
• backend validation remains authoritative in the config command handlers before a create or update override is accepted

Backend validation should parse propertyValue according to valueType before running JSON Schema validation. The UI should normalize YAML input to compact JSON before sending the command payload, so the command APIs continue to receive string values.

The schema registry needs a config-property binding before this can be enabled. The preferred minimal binding is schemaId + schemaVersion on the base config property definition. If tenant-specific schemas are needed later, schema lookup can resolve the same schema id/version against the selected hostId first and then fall back to the global row.

API Matrix

The write side can reuse the current command APIs.

For existing override rows, the update/delete payload must include the current aggregateVersion so the event persistence layer can enforce the monotonic version check. For new override rows, the page sends the scope identity, configId, propertyId, and propertyValue.

Security And RBAC

The page must not assume that a user who can view configuration can write every override scope. The selected scope and target should be checked against the same permission model used by the existing config admin routes and command handlers.

Recommended behavior:

• hide scopes the user cannot see
• show read-only rows for scopes the user can read but cannot update
• disable apply/reset controls when the selected target is not writable
• show a lock icon or tooltip for read-only rows
• keep backend command authorization authoritative, even when the UI already filtered the control

Unauthorized command responses should be mapped back to the row that triggered the command. The page should not fail the entire table because one row is not writable.

Read Model

The instance-facing scopes already have applicable-property queries:

• getApplicableConfigPropertiesForInstance
• getApplicableConfigPropertiesForInstanceApi
• getApplicableConfigPropertiesForInstanceApp
• getApplicableConfigPropertiesForInstanceAppApi

These queries return property metadata and inherited/effective values, including:

• configId
• configName
• configPhase
• configType
• propertyId
• propertyName
• propertyType
• propertyValue
• propertySource
• propertySourceType
• valueType
• resourceType
• required
• displayOrder

The same page needs current override metadata from:

• getConfigInstance
• getConfigInstanceApi
• getConfigInstanceApp
• getConfigInstanceAppApi

The page should not join applicable rows and override rows across separately paginated API calls. That produces brittle pagination, filtering, sorting, and row-count behavior. Instead, Phase 1 should add a merged backend read model that returns one row per configurable property with inherited value, override value, effective value, override metadata, and permission hints.

Environment, product, and product version currently have list/getFresh queries for existing overrides, but they do not have equivalent applicable-property queries:

• getConfigEnvironment
• getConfigProduct
• getConfigProductVersion

The new merged query should cover these scopes before they are exposed in the new page. A temporary client merge is acceptable only for a local prototype with unpaginated data; it should not be shipped as the production page behavior.

Proposed Generic Query

Add a Phase 1 query such as getConfigUpdateProperties in config-query.

Request:

{
  "hostId": "host uuid",
  "scope": "instance",
  "target": {
    "instanceId": "instance uuid"
  },
  "filters": {
    "configPhases": ["R"],
    "propertyTypes": ["Config"],
    "resourceTypes": ["all"]
  },
  "offset": 0,
  "limit": 1000,
  "active": true
}

Response:

{
  "total": 1,
  "properties": [
    {
      "scope": "instance",
      "hostId": "host uuid",
      "configId": "config uuid",
      "configName": "security.yml",
      "configPhase": "R",
      "propertyId": "property uuid",
      "propertyName": "jwt.clockSkew",
      "propertyType": "Config",
      "valueType": "integer",
      "resourceType": "all",
      "required": false,
      "schemaId": "security-jwt-clock-skew",
      "schemaVersion": "1.0.0",
      "schemaType": "json",
      "schemaStatus": "P",
      "defaultValue": "60",
      "defaultSourceType": "config_property",
      "overrideValue": "120",
      "overrideAggregateVersion": 3,
      "effectiveValue": "120",
      "effectiveSourceType": "config_instance",
      "canUpdate": true,
      "canDeleteOverride": true
    }
  ]
}

This query should be read-only. It does not need new write commands.

The query owns inheritance and candidate selection. The frontend owns presentation, editing state, and calls to the existing command APIs.

Frontend Structure

Recommended files:

src/pages/config/update/ConfigUpdatePage.tsx
src/pages/config/update/ConfigUpdateTable.tsx
src/pages/config/update/ConfigValueEditor.tsx
src/pages/config/update/ConfigStructuredValueDialog.tsx
src/pages/config/update/configUpdateScopes.ts
src/pages/config/update/configUpdateApi.ts
src/pages/config/update/configValue.ts
src/pages/config/update/configUpdateDraft.ts

configUpdateScopes.ts should be the single source of truth for scope metadata:

type ConfigUpdateScope = {
  id: 'environment' | 'product' | 'productVersion' | 'instance' | 'api' | 'app' | 'appApi';
  label: string;
  targetKeys: string[];
  applicableQuery?: string;
  overrideQuery: string;
  overrideResponseKey: string;
  createAction: string;
  updateAction: string;
  deleteAction: string;
  getFreshAction?: string;
  defaultResourceTypes?: string[];
  defaultConfigPhases?: string[];
};

The page should avoid hard-coding create/update/delete branching inside cell handlers. The handler asks the selected scope metadata which action and keys to use.

Draft And Apply Flow

The default edit mode should stage changes locally. A dirty row is not saved until the user chooses Apply for that row or Review & Apply from the toolbar.

The draft model should track:

• operation: create, update, reset
• previous effective value
• next override value
• scope target keys
• configId
• propertyId
• current aggregateVersion
• validation state

The review dialog should group changes by operation and show enough context for operators to catch mistakes before applying. If multiple commands are applied and one fails, the dialog should show which rows succeeded and which rows need attention. The page should refetch or refresh successful rows and leave failed rows dirty with their error state intact.

Save Flow

1. User edits a row.
2. UI validates the value against valueType and the schema registry when a published schema is attached to the property.
3. UI marks the row dirty and stores a draft operation.
4. User applies a row or opens Review & Apply.
5. UI builds payload from selected scope, row configId, row propertyId, and normalized propertyValue.
6. If an active override row exists, call the update action and include aggregateVersion.
7. If no active override row exists, call the create action.
8. On success, update the row with returned aggregate version or refetch that row.
9. On conflict or error, keep the draft value, restore the displayed committed value, and show the row error.

The local override map should store the full override row, not just the string value. At minimum it needs:

• propertyValue
• aggregateVersion
• active
• scope identity fields
• updateUser
• updateTs

Before update or delete, the UI should support the same getFresh* pattern used by the existing admin pages. If the row has been open for a while, the Apply action can fetch the latest row to get the freshest aggregateVersion. At minimum, a version conflict must offer a “Refresh Row & Try Again” action that reloads that row, compares the current backend value with the user’s draft, and lets the user reapply intentionally.

Validation errors should stay close to the edited cell. For example, an invalid integer should keep the cell in edit/error state with a short message. Backend validation, authorization, and conflict errors should be attached to the row that caused them, not only shown as a global toast.

Reset Flow

Reset means delete the override for the selected target and property.

1. User selects an overridden row.
2. UI calls the scope’s delete action with target keys, propertyId, and aggregateVersion.
3. On success, clear overrideValue and overrideAggregateVersion.
4. The displayed effective value reverts to the inherited/default value.

Rows with no override should not allow reset.

Like update, reset should support getFresh* before delete or expose the same “Refresh Row & Try Again” conflict path.

Routing

Add a route such as:

/app/config/update

The route should accept task context and target context through query params:

/app/config/update?scope=api&instanceApiId=...&task=mcp-onboard-api

Existing config table pages can link to it when they already have target context. Existing generated forms should remain available from row overflow actions for advanced edits and File/Cert values.

The View History row action should preserve context by opening the audit trail in a drawer, modal, or task-aware route with filters already applied. The filter payload should include the selected scope, target keys, configId, and propertyId.

For fallback forms, prefer opening the existing react-schema-form experience inside a drawer or modal over navigating away from the table. That keeps the user’s current scope, filters, expansion state, selected row, and staged changes intact. Full-page navigation can remain as a secondary fallback for complex forms that cannot safely render in an overlay.

Implementation Plan

Phase 0: schema registry foundation for config validation

• Harden schema-query, schema-command, and schema_t enough for production JSON Schema lookup.
• Add a durable config-property-to-schema association with schemaId and schemaVersion.
• Make schema lookup tenant-aware: host-specific schema first, global schema second.
• Validate schema bodies on schema create/update.
• Add backend config value validation for create/update override commands.
• Add tests for schema CRUD, tenant/global lookup, version pinning, and invalid config property values.

Phase 1: merged read model and instance-facing MVP

• Add getConfigUpdateProperties or an equivalent merged query in config-query.
• Return candidate properties, inherited values, current override values, override metadata, schema metadata, and permission hints in one paginated/sortable result.
• Build the page for Instance, API, App, and App API.
• Use existing command APIs for create/update/delete.
• Support scalar inline edits.
• Support list/map raw JSON/YAML modal.
• Support staged changes and Review & Apply.
• Support row-level validation/error/conflict states.
• Enable the typed Form tab only for properties with a published schema.

Phase 2: higher-level scopes

• Add Environment, Product, and Product Version selectors.
• Expose each scope only through the merged read model, not a client-side paginated join.
• Ensure product version respects product-version config/property mappings where available.

Phase 3: typed structured forms

• Use the schema registry for list/map config properties.
• Support custom validators by property key.
• Add tests for string array, object array, map, and malformed JSON/YAML values.

Phase 4: task integration

• Link from configuration task panels to /app/config/update.
• Add contextual next actions from instance, API, app, and app-api pages.
• Keep generated create/update forms as drawer/modal fallback actions.

Risks And Open Questions

• Environment inheritance needs a precise target rule. Existing applicable instance queries include environment_property as an inherited source, but the target environment is not selected by the current instance-facing query contract.
• Product and product-version candidate lists can be too broad if they are loaded from all config properties. Product version should eventually use the product version config mappings.
• The customer Settings code stores custom values in a map keyed by propertyId; for portal-view, the key should include scope target plus propertyId to avoid collisions when multiple targets are loaded.
• propertyId is the stable merge key only after the candidate list has been constrained to the selected target. If multiple configs can contain the same property id in unusual imports, use configId + propertyId.
• The page should avoid silently editing File or Cert values as plain text.
• Staged apply is not atomic until a bulk command exists. The UI must show partial success and partial failure clearly.
• Overlaying generated forms in a drawer depends on the form runner handling router state, success/failure navigation, and task context without forcing a full-page transition.
• The schema registry is not fully implemented and tested yet. Schema-backed validation should not be enabled until tenant-aware lookup, version pinning, and backend command validation are in place.

Recommendation

Build the page as a new task-oriented editor, not as a rewrite of the existing config admin tables. Make the merged getConfigUpdateProperties read model a Phase 1 backend requirement so the frontend does not perform brittle pagination-sensitive joins. Start frontend exposure with the four instance-facing scopes, then add environment, product, and product version once the same merged query handles their inheritance and candidate-selection rules.

Implement the minimal schema registry foundation before enabling schema-backed validation in the config update page. The raw JSON/YAML editor and scalar valueType validation can be built in parallel, but the Form tab and backend schema enforcement should wait for the registry work.

Rust Controller Logging

The controller service dashboard already has a logger page for Java runtimes. That page is built around Logback concepts: named loggers, per-logger levels, historical log content, and live streaming through controller-mediated MCP tools. Rust products need a similar operator workflow, but the underlying logging model is different. Rust services use tracing targets and one runtime logging.filter expression instead of mutable Logback logger objects.

This document proposes a Rust-aware logger page for gateway, agent, API, deployer, and workflow runtimes:

• gtw: light-gateway
• agt: light-agent
• api: Rust API services built on light-axum or light-runtime
• dpl: light-deployer
• wf: light-workflow

The goal is to keep the existing controller page entry point while switching the page behavior based on runtime capabilities.

Current State

portal-view has a unified controller logger page at /app/controller/logger. The page receives a runtime instance from the control pane dashboard and uses controller MCP tools:

• get_loggers
• set_loggers
• get_log_content
• start_logs
• stop_logs

Those contracts work for Java services where the runtime can inspect and update Logback logger levels.

Rust services already expose live logging filter control through the light-runtime MCP handler:

• get_logging_filter
• set_logging_filter
• reload_modules with modules: ["runtime/logging"]

The tested config server baseline is:

logging.filter: info,light_pingora::security=debug

This keeps the process at info by default and enables debug only for the gateway security target.

Goals

• Provide one operator page for Rust log filter control, time-based history, and live streaming.
• Reuse the controller-mediated MCP path instead of adding direct browser access to runtime instances.
• Preserve the existing Java logger page behavior.
• Use Rust tracing vocabulary in the UI: target, level, filter expression, and source.
• Let operators build common filters without memorizing module paths.
• Keep an advanced filter input for exact EnvFilter expressions.
• Support time-range log lookup from the running process.
• Support live log streaming through notifications/log.
• Make reset behavior explicit: live filter changes are temporary unless the instance configuration is updated separately.

Non-Goals

• Do not replace Java Logback logger management.
• Do not make the browser connect directly to pods, services, or container runtimes.
• Do not keep historical logs in portal-view or controller memory.
• Do not store full authorization headers, tokens, cookies, request bodies, or other secrets in log files, log responses, or live stream payloads.

Runtime Detection

The logger page should select the Rust experience when either condition is true:

• the selected runtime instance advertises product type gtw, agt, api, dpl, or wf
• the runtime MCP tools/list or controller tool discovery includes get_logging_filter

If detection is uncertain, portal-view can attempt get_logging_filter and fall back to the Java logger page if the response says logging control is not available.

The page should show a capability banner when a selected runtime supports only some features:

Page Layout

Use the current logger page route and high-level structure, but render Rust content when the selected instance is Rust.

Header:

• service label
• runtime instance ID
• service ID
• product type
• address and port
• connection status
• logging capability status

Tabs:

• Filter
• History
• Live Stream

The Java page can keep Config, History, and Live Stream; the Rust page uses Filter instead of Config because the operator edits one tracing filter expression, not a list of Logback logger objects.

Filter Tab

The filter tab controls the active runtime logging.filter.

Controls:

• current effective filter
• filter source, such as values.yml:logging.filter, env:RUST_LOG, or mcp:set_logging_filter
• default level selector
• target rows for common Rust modules
• advanced filter text area
• Apply Live
• Reset From Config

Levels:

• error
• warn
• info
• debug
• trace
• off

Recommended default level is info.

Example generated filter:

info,light_pingora::security=debug

Apply flow:

operator changes target rows
  -> portal-view builds EnvFilter expression
  -> controller calls runtime set_logging_filter
  -> runtime validates and applies the filter immediately
  -> portal-view refreshes get_logging_filter

Reset flow:

operator clicks Reset From Config
  -> controller calls reload_modules with runtime/logging
  -> runtime reloads logging.filter from current resolved values
  -> portal-view refreshes get_logging_filter

Baseline changes are handled outside this page. If an operator wants the filter to survive restart or reset, they should update the selected instance configuration, for example:

logging.filter: info,light_pingora::security=debug

Target Presets

The advanced filter must accept any valid Rust tracing target. The module picker should be backed by reference data so new targets can be added without a portal-view deployment.

Portal-view should load the dropdown from:

/r/data?name=logging_target

Recommended reference table mapping:

The simplest page can load all active targets from /r/data?name=logging_target and group them client-side by product. If product-specific filtering is needed later, add a reference relation such as logging-target-product that links each target to common, gtw, agt, api, dpl, or wf. Operators can still type a custom target if the target is not present in the reference table.

Suggested seed data:

Common targets:

Gateway targets:

Agent targets:

API targets:

Deployer targets:

Workflow targets:

The UI can also learn targets from returned history and live rows. Any target seen in logs can become a temporary suggestion for that browser session, but the authoritative dropdown source is the logging_target reference table.

History Tab

The history tab fetches logs from the running application for a time range.

Controls:

• presets: last 5, 10, 30, and 60 minutes
• required start time
• optional end time
• minimum level
• optional target filter
• text search
• result limit

Request:

{
  "runtimeInstanceId": "019...",
  "startTime": "2026-06-17T21:30:00Z",
  "endTime": "2026-06-17T21:45:00Z",
  "loggerLevel": "debug",
  "loggerName": "light_pingora::security",
  "limit": 1000
}

For compatibility, loggerName maps to the Rust target and loggerLevel maps to the minimum tracing level. The controller can keep the existing get_log_content tool name.

Recommended normalized row shape:

{
  "timestamp": "2026-06-17T21:37:43.147463Z",
  "level": "DEBUG",
  "logger": "light_pingora::security",
  "target": "light_pingora::security",
  "message": "JWT validation failed after JWKS refresh: InvalidSignature",
  "fields": {
    "error": "InvalidSignature"
  }
}

The response can preserve the current grouped shape for compatibility:

{
  "content": {
    "light_pingora::security": {
      "logs": [
        {
          "timestamp": "2026-06-17T21:37:43.147463Z",
          "level": "DEBUG",
          "message": "JWT validation failed after JWKS refresh: InvalidSignature"
        }
      ]
    }
  }
}

Portal-view should flatten the grouped response into rows, as the current Java page already does.

History source selection:

1. If a JSON log file is configured, parse that file first. This should be the preferred source because the same file can be collected by Splunk or another logging system.
2. If no log file is configured, use Kubernetes pod logs or container logs when the controller/runtime environment can access them.
3. If neither source is available, return an explicit unsupported response.

The browser must not read Kubernetes or container logs directly. The controller or runtime-side tool should own that platform access and return the normalized row shape above.

When reading a JSON log file, the reader should filter by timestamp, level, target, and text search. If the file format is line-oriented JSON, each line should contain at least timestamp, level, target, and message.

Live Stream Tab

The live stream tab starts and stops log streaming for the selected runtime instance.

Controls:

• full filter expression
• start
• stop
• clear
• auto-scroll toggle
• bounded client buffer
• stream status

Request:

{
  "runtimeInstanceId": "019...",
  "filter": "info,light_pingora::security=debug"
}

start_logs should accept the full Rust filter expression because this is the syntax Rust operators already use. For backward compatibility, the controller can still accept level and loggerName, then translate them into a filter expression.

The stream filter controls which events are sent to that stream subscription. It must not change the process-wide logging.filter; process-wide changes still go through set_logging_filter. Because tracing filters can suppress events before stream filtering sees them, the UI should warn when the stream filter is more verbose than the current active runtime filter.

Notification:

{
  "method": "notifications/log",
  "params": {
    "runtimeInstanceId": "019...",
    "timestamp": "2026-06-17T21:37:43.147463Z",
    "level": "DEBUG",
    "logger": "light_pingora::security",
    "target": "light_pingora::security",
    "message": "JWT validation failed after JWKS refresh: InvalidSignature"
  }
}

The portal-view live buffer should remain bounded. The current 1000-row FIFO buffer is a good default.

Each browser/controller session must have its own stream subscription. Starting a stream from one operator must not replace another operator’s stream for the same runtime instance.

Runtime Implementation

Add shared Rust logging support to light-runtime, not separately in every product.

Recommended components:

• LoggingControl: existing active EnvFilter control.
• JsonLogWriter: optional line-oriented JSON file writer for services that need historical lookup or Splunk ingestion.
• LogFileReader: reads and filters configured JSON log files.
• PlatformLogProvider: controller-side or runtime-side abstraction for Kubernetes pod logs and container logs when no log file is configured.
• LogStreamHub: per-client subscriptions for live streaming.
• LogRecord: normalized timestamp, level, target, message, fields, and optional span/correlation fields.

Recommended runtime MCP tools:

The JSON log file should be configurable:

logging.file.enabled: true
logging.file.path: /var/log/light-gateway/app.log
logging.file.format: json
logging.file.maxBytes: 104857600
logging.file.maxFiles: 10
logging.stream.maxSubscribers: 20

Defaults should be conservative. If no JSON log file and no platform log provider are available, get_log_content should return a clear unsupported response instead of an empty success that looks like there were no logs.

Controller Changes

The controller should expose Rust logging tools through the same callTool path used by the existing logger page.

Add or pass through these tool names:

• get_logging_filter
• set_logging_filter
• reload_modules
• get_log_content
• start_logs
• stop_logs

For Rust runtimes, get_loggers and set_loggers are not the primary control surface. The UI should use get_logging_filter and set_logging_filter instead. The controller may keep get_loggers and set_loggers for Java compatibility.

The controller should route notifications/log back to the portal-view websocket with the originating runtimeInstanceId so the page can ignore logs from other selected services.

For history, the controller should resolve sources in this order:

1. configured JSON log file
2. Kubernetes or container log provider
3. unsupported response with a clear reason

Portal-View Implementation

Recommended structure:

• keep /app/controller/logger as the route
• keep the existing Logger component as the shell
• split Java and Rust behavior into child panels:
  • JavaLoggerPanel
  • RustLoggerPanel
• reuse the current history and live table rendering where possible
• add a Rust filter builder for logging.filter

Rust filter builder state:

type RustFilterDraft = {
  defaultLevel: "error" | "warn" | "info" | "debug" | "trace" | "off";
  targets: Array<{ target: string; level: string }>;
  advanced: string;
  mode: "builder" | "advanced";
};

In builder mode, portal-view generates the expression:

<defaultLevel>,<target>=<level>,<target>=<level>

In advanced mode, portal-view sends the text exactly as entered and lets the runtime validate it.

The page should show a warning when the current source is mcp:set_logging_filter, because that indicates a live override that can be lost on restart or reset by reloading runtime/logging.

Baseline Configuration

Live debug changes should call set_logging_filter; they should not update config server by default.

To persist a baseline filter, the operator should use the instance configuration page and update:

logging.filter: info,light_pingora::security=debug

After saving the instance configuration, the config update flow can call:

{
  "name": "reload_modules",
  "arguments": {
    "modules": ["runtime/logging"]
  }
}

This makes the saved config the active baseline. Alternatively, the operator can return to the logger page and use Reset From Config to reload only runtime/logging.

The Rust logger page can link to the selected instance configuration, but it should not write baseline config itself.

Security And Safety

• Gate filter changes and log access behind the same controller permissions as the Java logger page.
• Treat logs as sensitive operational data.
• Do not render raw ANSI escape sequences as HTML.
• Truncate very large messages and expose an expand action.
• Mask obvious token and secret fields in JSON log output, history responses, and live stream payloads.
• Rate-limit live streams per runtime instance and per controller client.
• Show a warning before enabling broad trace filters.

Rollout Plan

1. Add controller pass-through for get_logging_filter and set_logging_filter.
2. Add RustLoggerPanel in portal-view with filter control only.
3. Add JSON file logging and a get_log_content reader for Rust services.
4. Add Kubernetes/container log fallback when no log file is configured.
5. Add Rust start_logs and stop_logs backed by per-client stream subscriptions.
6. Seed the logging_target reference data and load dropdown options from /r/data?name=logging_target.
7. Enable product-specific target presets for gtw, agt, api, dpl, and wf.

Resolved Decisions

• Historical logs are not kept in memory. Use a configured JSON log file first; if there is no file, fall back to Kubernetes or container logs when they are available.
• start_logs accepts a full filter expression. Compatibility fields such as level and loggerName can be translated by the controller.
• The module dropdown is backed by the logging_target reference table exposed through /r/data?name=logging_target.
• The logger page does not save a baseline. Baseline changes belong in instance configuration.

Human Task UI

ask is the workflow task type that pauses execution for human input. The runtime can now create task_asst_t and worklist_t rows when an ask task waits, so portal-view needs a generic human-task interface that lets an assigned user open the task, provide the requested answer, and resume the workflow.

This document proposes the portal UI and service contracts for that interface.

Current State

The workflow engine persists waiting ask tasks in task_info_t. The ask configuration is stored in task_info_t.task_output.ask, and the workflow runtime context remains on process_info_t.context_data.

The assignment layer is separate:

• worklist_t represents a user/category worklist.
• task_asst_t represents a concrete task assignment.
• Role assignment is resolved by the workflow runtime into one task assignment per active user in the role.

The Worklist page can show assigned tasks, but it does not yet provide a generic input screen for the user to approve or enter data.

Goals

• Provide one generic page for all human-input workflow tasks.
• Render the input controls from the ask task definition, not from a workflow-specific page.
• Let users open tasks from the Worklist page.
• Keep assignment, claim, completion, and authorization checks on the service side.
• Support role-assigned tasks where multiple users may receive the same work.
• Resume the workflow through the existing completeTask command.
• Keep the UI useful for approval tasks first while leaving room for richer object-input tasks.

Non-Goals

• Do not create a custom approval page for each workflow.
• Do not make every workflow task human-actionable; only ask tasks use this interface.
• Do not expose raw database rows directly to the page.
• Do not overload the engine locked field for human claims; that field is already used by the workflow executor as a worker lease.
• Do not replace the existing Worklist administration page in the first phase.

User Flow

The primary flow is:

Worklist
  -> open assigned task
  -> Human Task detail
  -> render prompt and input controls from ask metadata
  -> submit answer
  -> completeTask command
  -> workflow executor resumes the process

For a simple approval workflow, the user sees the prompt and two action buttons derived from the ask options. For structured input, the same page renders a schema-driven form.

The runtime flow for a role-assigned ask task is:

1. The workflow executor creates one task_asst_t row per assignee and keeps the parent task_info_t row waiting for input.
2. The assigned user opens the row from Worklist.
3. getHumanTask loads the assignment, task state, workflow metadata, and process context into one stable page payload.
4. The user submits an answer through completeTask.
5. completeTask validates assignment ownership, locks the parent task row, records the result, and deactivates or cancels sibling assignments in the same transaction.
6. The workflow executor observes the completed ask task and resumes the process with the submitted answer.

Route Design

Add a human task detail route:

/app/workflow/HumanTask

The route should accept taskAsstId and task context through query parameters or router state:

/app/workflow/HumanTask?taskAsstId=...&taskId=...

The Worklist page should route to this detail page for actionable task rows. Worklist administration actions such as create, update, and delete worklists should remain separate from human task completion.

If a dedicated inbox page is needed later, add:

/app/workflow/HumanTasks

That page can list only actionable assigned tasks, while the existing Worklist page can remain the administrative view of worklist definitions.

Data Model Decisions

task_info_t remains the canonical workflow engine task state. Its locked column must stay reserved for executor leasing. Human task claims must not set task_info_t.locked = 'Y', because that would make the executor treat the row as worker-owned runtime work.

Useful existing task_info_t fields for the human task page are:

• status_code: parent task state. Waiting ask tasks should be open for input; completed ask tasks should be read-only.
• deadline_ts: optional due or expiry timestamp to show in the UI.
• locking_user and locking_role: possible global claim metadata if claim is implemented, but not a replacement for assignment-level authorization.
• task_output: source of the ask metadata.
• result_code: submitted answer envelope after completion.

task_asst_t remains the assignment layer. The current active flag and unassigned_reason can hide completed assignments from Worklist, but they are too loose to represent claim, release, expiry, and reporting states cleanly. Add an explicit assignment status early in development so the query and command contracts are built on the final assignment state model:

ALTER TABLE task_asst_t
  ADD COLUMN status_code VARCHAR(16) NOT NULL DEFAULT 'ASSIGNED',
  ADD COLUMN claimed_by VARCHAR(126),
  ADD COLUMN claimed_ts TIMESTAMP WITH TIME ZONE,
  ADD COLUMN claim_expires_ts TIMESTAMP WITH TIME ZONE;

Recommended assignment statuses:

Keep active as a fast visibility/backward-compatibility flag. Use status_code for business state and audit/reporting semantics.

Query Contract

The UI should not assemble a human task by calling several generic table queries. Add a normalized query action such as getHumanTask.

Request:

{
  "hostId": "...",
  "taskAsstId": "..."
}

Response:

{
  "hostId": "...",
  "taskAsstId": "...",
  "taskId": "...",
  "processId": "...",
  "wfInstanceId": "...",
  "wfTaskId": "requestApproval",
  "assignedTs": "...",
  "assigneeId": "...",
  "assignmentStatusCode": "ASSIGNED",
  "claimedBy": null,
  "claimedTs": null,
  "deadlineTs": "2026-05-23T14:30:00Z",
  "categoryCode": "approval",
  "reasonCode": "human-approval",
  "taskStatusCode": "W",
  "workflow": {
    "wfDefId": "...",
    "namespace": "light-portal",
    "name": "human-approval",
    "version": "1.0.0"
  },
  "ask": {
    "prompt": "Review the workflow request and choose a decision.",
    "mode": "approval",
    "options": [
      {
        "label": "Approve",
        "value": "APPROVED",
        "description": "Continue the request."
      },
      {
        "label": "Reject",
        "value": "REJECTED",
        "description": "Stop the request."
      }
    ],
    "required": true,
    "allowComment": true,
    "contextKeys": ["requestId", "summary"]
  },
  "contextSummary": {
    "requestId": "REQ-001",
    "summary": "..."
  },
  "context": {
    "requestId": "REQ-001",
    "summary": "..."
  }
}

The service should read from task_asst_t, task_info_t, process_info_t, and wf_definition_t, then return a stable task-detail view. The UI should treat this response as the source of truth.

The query should return a curated contextSummary when the ask metadata defines contextKeys. It may also include the raw context object for administrator troubleshooting or for workflows that have not yet declared a curated context shape. The default user view should prefer contextSummary.

For a list page, add getHumanTaskList later. It should return only active assignments for the current user unless the caller has an administrative permission.

Input Rendering

The page renders controls from ask.mode, ask.options, and ask.schema.

If ask.mode is missing, default to text. If approval has no options, the UI may render default APPROVED and REJECTED actions.

Comments should be configurable per ask task. The recommended metadata is:

{
  "allowComment": true,
  "commentRequired": false
}

approval and confirm should default to allowing comments. Other modes can opt in when the workflow author wants users to explain the submitted value.

Answer Shape

Use a consistent answer envelope for completeTask.

{
  "value": "APPROVED",
  "comment": "Looks good.",
  "submittedAt": "2026-05-22T14:30:00Z"
}

For object input, value is the submitted object:

{
  "value": {
    "approvedLimit": 5000,
    "expirationDate": "2026-06-30"
  },
  "comment": "Approved with a reduced limit.",
  "submittedAt": "2026-05-22T14:30:00Z"
}

The workflow receives this object as the ask task output. A workflow that needs only the selected value can export .output.value; a workflow that wants the full audit envelope can export .output.

Completion Command

The detail page submits through completeTask:

{
  "host": "lightapi.net",
  "service": "workflow",
  "action": "completeTask",
  "version": "0.1.0",
  "data": {
    "hostId": "...",
    "taskId": "...",
    "taskAsstId": "...",
    "statusCode": "C",
    "completedTs": "2026-05-22T14:30:00Z",
    "response": {
      "value": "APPROVED",
      "comment": "Looks good.",
      "submittedAt": "2026-05-22T14:30:00Z"
    }
  }
}

The command should verify that:

• the assignment exists and is active
• the assignment status allows submission
• the current user is the assignee or has an administrative permission
• the task is an ask task
• the task is still waiting for input
• the submitted answer matches ask.mode, ask.options, and ask.schema

The browser may send taskAsstId, taskId, and the answer, but it must not be trusted to identify the completing user. The command service should derive the user id and roles from the authenticated token. A client-supplied completedUser value should be ignored for normal human-task completion.

Completion must be atomic. In one database transaction:

1. Load the task_asst_t row and verify it belongs to the current user, unless the caller has an explicit administrative override permission.
2. Lock the parent task_info_t row, for example with SELECT ... FOR UPDATE.
3. Reject the command if the parent task is already completed or no longer waiting for input.
4. Validate the answer against the ask metadata and, for object mode, the JSON schema.
5. Update task_info_t with status C, completed_ts, completed_user, and the answer envelope in result_code.
6. Mark the selected assignment COMPLETED and inactive.
7. Mark sibling active assignments for the same task_id as CANCELLED, inactive, and unassigned_reason = 'completed_by_other_user'.

If another user completes the same parent task first, return a stale-task conflict response, preferably HTTP 409, and leave the duplicate submission unapplied.

Claim And Concurrency

Role assignment can create several active assignments for the same task_id. The first user-input page should use optimistic completion with a server-side final check: only the first valid completion succeeds, and later submissions receive a stale-task conflict response. This proves the core flow before adding the operational complexity of explicit claim/release commands.

For a better user experience, add an optional claimHumanTask command.

Recommended claim behavior:

• claim records the current user on the human-task assignment path with task_asst_t.status_code = 'CLAIMED', claimed_by, and claimed_ts
• claim does not set task_info_t.locked = 'Y'
• claim expires after a short timeout or can be released
• completion still performs the final status check

The engine locked column should remain reserved for executor leasing. Human claims should use either assignment-specific fields added later or locking_user/locking_role without changing the executor lease flag.

When claim is enabled for role-assigned tasks, sibling assignment rows should be visible as claimed or unavailable instead of letting users submit stale answers. Live refresh should use the existing portal notification channel if one is available. If workflow tasks need their own lightweight channel later, prefer server-sent events before adding a separate websocket service.

Assignment Cleanup

When a human task is completed:

• the selected assignment should no longer appear as actionable
• sibling active assignments for the same task_id should also disappear
• the task completion result should remain on task_info_t

With the current table shape, the minimal implementation can deactivate active task_asst_t rows for the task and set unassigned_reason to completed or completed_by_other_user. A later schema iteration can add explicit assignment status fields if the UI needs richer assignment history.

With the recommended status column, cleanup should use structured states:

• selected assignment: status_code = 'COMPLETED', active = false, unassigned_reason = 'completed'
• sibling assignments: status_code = 'CANCELLED', active = false, unassigned_reason = 'completed_by_other_user'

Authorization

Normal users should only query and complete assignments where task_asst_t.assignee_id matches their user id. Administrative users may view all assignments for the host if the workflow task endpoints allow it.

The browser should send taskAsstId, but the service should not trust the browser to identify the assignee. It should resolve the current user from the authenticated token and compare it to the assignment row.

Administrative completion on behalf of another user should require a distinct permission, not just the ability to query workflow tasks. The command should record both the authenticated actor and the effective completed user if override support is added.

Recommended authorization model:

• normal completion requires the endpoint write scope and task_asst_t.assignee_id = authenticated user id
• administrative override requires workflow.task.override
• a host or portal administrator, such as the configured portal.admin, may be treated as satisfying the override permission if that is the established portal authorization convention

Use a broad scope such as workflow.write for access to the write endpoint if the service defines workflow-specific scopes. If the current service only has a portal-level write scope, keep the OAuth scope broad and enforce workflow.task.override as the fine-grained application permission.

Page Layout

The detail page should be compact and task-oriented:

• header with workflow name, task name, status, and due date if deadlineTs is present
• assignment summary with assignee and category
• prompt panel
• context panel with selected workflow/process fields
• input area rendered from ask metadata
• sticky submit actions for long forms
• error or stale-task state

The context panel should show enough data for the user to decide, but it should not dump the full context_data object by default. The first phase can show common fields and provide a collapsible raw context view for administrators.

Timeout handling should be visible as read-only metadata as soon as task_info_t.deadline_ts is available. The UI can show due date or expiry status without implying that automatic runtime timeout processing has already been implemented.

Error States

The page should handle these states explicitly:

• assignment not found
• assignment no longer active
• task already completed
• task is not an ask task
• ask metadata missing or invalid
• validation failed
• submit conflict because another user completed the task first
• workflow resume failed after completion

The submit conflict case should take the user back to the worklist after showing that the task is no longer available.

Implementation Phases

Phase 1:

• Add the task_asst_t.status_code migration before building the query and command handlers.
• Add getHumanTask.
• Add /app/workflow/HumanTask.
• Link actionable Worklist task rows to the detail page.
• Render approval, choice, multiChoice, confirm, and text.
• Submit through completeTask.
• Validate assignment ownership and ask metadata in the command layer.
• Complete the parent task and assignment cleanup in one transaction.
• Return stale-task conflicts for duplicate submissions.
• Hide completed and sibling-cancelled assignments from the worklist.

Phase 2:

• Add schema-driven object input.
• Add JSON schema validation on the command side.
• Add curated context metadata such as ask.contextKeys.

Phase 3:

• Add optional claimHumanTask and releaseHumanTask.
• Add claim expiry handling.
• Add a dedicated human task inbox.
• Add assignment history and richer audit display.
• Add live Worklist refresh for claim/completion events through the existing portal notification channel, with server-sent events as the fallback.
• Add file input if workflow use cases require it.

Resolved Questions

• task_asst_t should gain explicit assignment status fields. active remains useful for filtering but should not be the only state model.
• The human task query should return curated context when workflow metadata defines it, with raw context available for administrative troubleshooting.
• Comments should be configurable. Approval and confirm modes should default to allowing comments.
• Timeout metadata should be visible as read-only UI state before automatic timeout processing is implemented.
• Administrative override should use the fine-grained permission workflow.task.override, with portal.admin as the broad administrator path if the portal authorization layer already uses it.
• Claim/release should remain Phase 3. Phase 1 should rely on optimistic completion and atomic duplicate-submit rejection.
• Live Worklist refresh should use the existing portal notification channel first. If there is no reusable channel, use server-sent events before adding a dedicated websocket service.

User Filter

As more portal users manage their own APIs, clients, instances, schedules, and configuration records, giving every operator a broad admin role becomes too coarse. A broad admin can see and modify records created by other admins on the same host. This document proposes an incremental owner-scoped filtering model for portal-view.

The first step is a UI-side filter based on the user recorded on each row, such as update_user. This is not a complete security boundary. The same rule must eventually be enforced in the query and command services with fine-grained authorization from the rule engine. The UI implementation is still useful because it improves day-to-day user experience and gives us a concrete policy shape to move into the service layer.

Problem

Portal admin pages were originally designed for a small set of trusted operators. Many tables expose all host-scoped records once the user can access the admin page.

That model creates problems as adoption grows:

• application owners need to manage their own APIs, clients, and instances
• broad admin roles expose unrelated records from other teams
• users can accidentally edit or delete records owned by another user or team
• creating one role per page, such as api-admin or instance-admin, still does not solve row ownership
• service-layer fine-grained authorization is not available everywhere yet

The immediate need is to let users use admin-like pages while limiting the rows they see and act on.

Current Experiment

Schedule.tsx is the first experimental page. The idea is:

• users can access the schedule admin surface
• normal users only see schedules where updateUser matches their user id
• global admins or schedule admins can still see all schedules
• the updateUser column can be hidden for normal users
• create/update/delete actions are available only on the visible set

One implementation detail matters: ownership filters must be added before the request payload serializes the filters array.

const apiFilters = [];

if (ownedOnly && userId) {
  apiFilters.push({ id: "updateUser", value: userId });
}

const cmdData = {
  filters: JSON.stringify(apiFilters),
};

Adding the filter after cmdData.filters is built will not send it to the backend.

Design Goals

• Allow regular users to manage records they created or updated.
• Avoid giving every self-service user broad all-record admin visibility.
• Keep the admin table implementation familiar and incremental.
• Centralize the owner filter logic instead of duplicating it page by page.
• Make the UI rule match the future service-layer rule as closely as possible.
• Preserve host scoping and existing role-based page visibility.
• Avoid presenting UI-side filtering as a security boundary.

Non-Goals

• Do not claim UI filtering is sufficient authorization.
• Do not replace service-layer rule-engine enforcement.
• Do not solve full team ownership in the first UI-only pass.
• Do not migrate every admin page in one large change.
• Do not overload update_user as the permanent ownership model if a better owner field exists or can be added.

Ownership Model

There are several possible ownership signals. They should be treated in this order of preference.

update_user is acceptable for the first UI experiment because many tables already have it. However, it has an important semantic problem: ownership moves to whoever last updated the row. If Alice creates an API and Bob updates it, Bob becomes the owner under an update_user rule.

The long-term model should add explicit owner fields where needed:

owner_user_id
owner_position_id

owner_group_id is intentionally deferred. Groups are still useful for flat team membership, but position ownership fits the portal authorization model better when access should follow the organization hierarchy. owner_org_id is also deferred because normal portal records are already scoped by host_id, and host_t links back to org_t through the host domain. Add organization-level ownership only if a future cross-host/global ownership use case requires it.

Do not add created_by and updated_by as authorization fields in Phase 4. The existing update_user and update_ts columns remain the last-updater audit trail. If creator audit becomes important, add create_user and create_ts as audit fields later, not as substitutes for stable ownership.

Until explicit owner columns exist, each page should declare which field is used for interim UI owner filtering.

Role Model

Use one page per entity type, but separate page visibility from row scope.

Do not give every user account access to every admin page. Only pages that are safe for self-service ownership should be exposed to user, and each of those pages must apply the owner filter and action guards.

The admin role can be repurposed as the global all-record role once the sidebar stops using it as a broad menu marker. Role checks must use exact role tokens. A role such as schedule-admin must not match admin through substring checks.

Access Modes

The UI should support three access modes.

Owner-Scoped Admin

This is the default self-service mode. The user can open admin pages, but rows are filtered to records they own.

Example:

roles: user
scope: owned
filter: updateUser = current user id

All-Scope Admin

This is for operators who can see and manage every record on the current host.

Example roles:

admin
schedule-admin

The default all-scope role is admin. Page-specific roles such as schedule-admin can opt a user into all-record visibility for one area. Do not use platform-admin as a global all-scope role because the portal already has a Platform Admin page for deployment platform management.

Read-Only or Support View

Some users may need to see records without modifying them. This can be added later with separate flags:

canReadAll = true
canWriteOwned = true
canWriteAll = false

Proposed UI Architecture

Add a small ownership-scope helper used by admin pages.

Example shape:

type OwnershipScopeOptions = {
  roles?: string | null;
  userId?: string | null;
  ownerField: string;
  allScopeRoles?: string[];
};

type OwnershipScope = {
  ownedOnly: boolean;
  ownerFilter: { id: string; value: string } | null;
  canWriteAll: boolean;
};

Example usage:

import {
  applyOwnershipFilter,
  defaultAllScopeRoles,
  ownershipScope,
} from "../utils/ownershipScope";

const ownership = ownershipScope({
  roles,
  userId,
  ownerField: "updateUser",
  allScopeRoles: [...defaultAllScopeRoles, "schedule-admin"],
});

const apiFilters = applyOwnershipFilter(columnFiltersWithoutActive, ownership);

This helper should live near other portal navigation/task utilities or in a small access utility module, for example:

src/utils/ownershipScope.ts

or:

src/tasks/accessScope.ts

The helper should not call the backend. It only computes the UI filter and UI capabilities from the current user state.

Sidebar Behavior

The sidebar should not use admin as a marker on every admin menu link. That made the whole Administration group disappear for normal users and prevented owner-scoped self-service pages from being reachable.

Recommended behavior:

• admin users see every Administration link.
• non-admin users see only Administration links explicitly marked with user or a matching entity role, such as role: "user schedule-admin".
• only add user to a link after that page has owner-scoped filtering and action guards.
• remove role: "admin" from individual menu links.
• use exact role-token matching instead of string includes, so schedule-admin does not accidentally grant admin.

At the Phase 3 rollout point, the following Administration links are safe to expose to user because the pages apply the shared owner-scope helper and action guards:

• API Admin
• API Detail
• OAuth Auth Client and Client Token
• App Admin
• Instance Admin, Runtime Instance, and instance relationship pages
• Schedule Admin
• Workflow Definition

Configuration, platform admin, user/role admin, workflow process/task/audit pages, and lower-volume metadata pages should remain admin-only until they have the same owner-scope treatment or a separate support/read-only policy.

Admin Page Behavior

For an owner-scoped user:

• add the owner filter before the query payload is serialized
• hide the owner column if it does not add useful information
• show a small scope label such as “My records”
• keep create actions available
• allow update/delete only for rows matching the ownership rule
• preserve normal table sorting, pagination, and global filter behavior

For an all-scope admin:

• do not add the owner filter
• show a scope label such as “All host records”
• show the owner/update columns
• allow existing admin actions

For a user without enough context:

• if userId is missing, do not run an owner-scoped query
• show a clear message that user context is required
• avoid falling back to all-record visibility

Action-Level Guard

List filtering is not enough for a good UI. Row actions should also check the same scope.

Example:

const canUpdateRow =
  ownership.canWriteAll ||
  row.original.updateUser === userId;

For rows the user cannot modify:

• hide destructive actions, or
• disable them with a tooltip explaining the scope

Even after service-layer authorization is implemented, the UI should keep these guards so users understand why an action is unavailable.

Phase 4 Ownership Columns

For high-value entity tables, add canonical owner columns directly on the entity row:

owner_user_id UUID NULL
owner_position_id VARCHAR(128) NULL

Recommended constraints where the table has host_id:

FOREIGN KEY (host_id, owner_user_id)
  REFERENCES user_host_t(host_id, user_id)

FOREIGN KEY (host_id, owner_position_id)
  REFERENCES position_t(host_id, position_id)

Both owner columns should be nullable during migration. New records should get owner_user_id from the authenticated user on the service side by default. Do not trust a browser-submitted owner user id unless the caller has permission to assign ownership.

owner_position_id should be optional on create. The UI can show a host position dropdown populated from the user’s allowed positions. If the user has exactly one effective position and the page is configured for position ownership, the UI can default to that position. If the user has multiple positions, require an explicit choice when position ownership is desired.

For portal forms, the optional position owner field should be exposed as ownerPositionId and backed by the existing position label dynaselect query. The form action uses the position/getPositionLabel endpoint, which is backed by the queryPositionLabel persistence method and returns the id/label pairs needed by the select control.

Do not expose ownerUserId as a normal create/update form field. The command path must derive owner_user_id from the authenticated user in the event context. If an owner-transfer use case is needed later, implement it as a separate command with explicit authorization and audit behavior.

Normal update forms may update owner_position_id when the page allows the caller to choose or clear the owning position. update_user changes on every update and remains audit metadata. owner_user_id should not change on normal update; it changes only through an explicit owner-transfer action restricted to the current owner, admin, or the relevant entity-admin role.

Existing rows should be migrated conservatively:

• if update_user can be resolved to a user in the host, it can be used as an initial owner_user_id
• leave owner_position_id null unless there is a reliable source for the owning position
• rows with no owner columns populated should be treated as unassigned legacy rows, visible only to all-scope admins until an owner is assigned

Service-Layer Target

The UI filter is an interim step. The durable solution belongs in the query and command services.

The service layer should eventually:

• derive user id, roles, host id, and scopes from JWT claims
• ignore client-supplied owner filters as an authorization source
• inject owner predicates into query handlers based on the authenticated user
• reject update/delete commands when the user does not own the row and lacks all-scope permission
• use rule-engine policies for exceptions and domain-specific ownership

Once service-side owner enforcement is implemented, the UI should no longer be the source of authorization predicates. The service should inject the ownership predicate from authenticated user context and rule-engine decisions.

The UI should still keep owner-aware behavior for usability:

• show “My records” or “Admin View” scope labels
• hide or show owner columns based on the user’s scope
• disable update/delete actions that the current user cannot take
• optionally send a simple view hint such as scope=owned or scope=all

The service must treat any UI-supplied scope or owner filter as a hint only. It must ignore, override, or reject filters that would expand the caller’s authorized scope.

For owner-scoped users, the service-side predicate should be an OR condition:

owner_user_id = current_user_id
OR owner_position_id IN current_user_effective_positions

For all-scope admins, such as admin or the relevant entity-admin role, the service should omit this owner predicate and return all rows within the normal host scope.

The UI and backend should share the same policy concepts:

host scope
entity type
owner field
owned-only permission
all-record permission
read vs write capability

Position hierarchy must be resolved by the service layer or rule engine. A JWT claim such as pos=ai-engineer only grants exact-position access unless the service expands it to effective positions from position_t and user_position_t. If hierarchy is enabled, the effective position set should include inherited positions according to the existing position inheritance rules.

Rows with owner_position_id IS NULL are not position-owned. A user can still see the row if owner_user_id matches their user id. Rows where both owner_user_id and owner_position_id are null are unassigned legacy rows and should not be visible to normal owner-scoped users by default.

Rule Engine Direction

The rule engine can express policies such as:

user can read API when api.owner_user_id == user.user_id
user can update API when api.owner_user_id == user.user_id
admin can read all APIs on host
admin can update all APIs on host
api-admin can read all APIs on host
api-admin can update all APIs on host
support can read all APIs but cannot update

For tables that do not yet have explicit ownership fields, the policy can temporarily map ownership to update_user.

Rollout Plan

Phase 1: Fix Schedule Experiment

• Fix filter ordering so updateUser is included in the request.
• Use roles plus user id to decide owner-scoped vs all-scope mode.
• Add action-level guards for update/delete.
• Keep the current route behavior unchanged.

Phase 2: Add Reusable UI Helper

• Create a shared ownership-scope helper.
• Add unit-level coverage if the repo has a practical test pattern.
• Document default all-scope roles.
• Keep owner field configurable per page.

Phase 3: Apply To High-Value Admin Pages

Start with pages where users commonly manage their own records:

• API admin
• API detail/version admin
• OAuth clients
• client apps
• instances
• instance API links
• schedules
• workflow definitions

Then expand to lower-volume metadata pages.

Current implementation status:

• src/utils/ownershipScope.ts centralizes exact role matching, owner-scope calculation, owner filter injection, and owner-column hiding.
• Sidebar access now exposes only scoped links to user or matching entity-admin roles, while exact admin continues to see all Administration links.
• API pages use admin and api-admin for all-record scope, with user limited by updateUser.
• OAuth client pages use admin and oauth-client-admin for all-record scope, with user limited by updateUser.
• Client app pages use admin and app-admin for all-record scope, with user limited by updateUser.
• Instance pages use admin and instance-admin for all-record scope, with user limited by updateUser.
• Schedule pages use admin and schedule-admin for all-record scope, with user limited by updateUser.
• Workflow Definition uses admin and workflow-admin for all-record scope, with user limited by updateUser.
• Task/page search registries use exact role-token checks so schedule-admin or another entity-admin role does not accidentally match global admin, while exact admin still has global visibility.

Deferred from this phase:

• Workflow Process, Task, Worklist, Work, Audit, and Trace remain admin-only until their ownership rules are defined and implemented.
• Configuration and platform pages remain admin-only because their ownership model is not yet defined.
• User and role administration remain admin-only because exposing them to self-service users would require a separate delegated-administration model.

Phase 4: Add Explicit Ownership Fields

Where update_user is too weak, add proper owner fields through the database and services.

Candidate fields:

owner_user_id
owner_position_id

Apply these first to the high-value tables that already have owner-scoped admin pages. Keep the fields nullable during migration, default owner_user_id from the authenticated user on create, and make owner transfer explicit.

Current implementation status:

• portal-db adds nullable owner_user_id and owner_position_id columns to the high-value portal tables used by the owner-scoped admin pages.
• The migration backfills owner_user_id from update_user only when update_user is already a UUID. Non-UUID audit values remain unassigned instead of blocking the migration.
• A database insert trigger defaults owner_user_id from update_user for new rows when the command path writes the authenticated user id into update_user.
• Query projections for the scoped UI pages now return ownerUserId and ownerPositionId, and UUID filtering recognizes ownerUserId.
• portal-view now uses ownerUserId for ownership checks on action controls. The UI no longer sends an owner filter for service-enforced pages because service-side scope must include both direct user ownership and position ownership.
• Owner-aware create/update forms expose optional ownerPositionId with a host-scoped position dynaselect backed by queryPositionLabel.
• Command schemas allow optional ownerPositionId for the owner-aware create and update commands. They do not accept ownerUserId; owner_user_id comes from the authenticated event user.
• light-portal persistence writes owner_user_id from the event user on create and writes owner_position_id from ownerPositionId on create/update.
• Schedule query is the first service-enforced owner-scope path. Non all-scope users are filtered by owner_user_id = current_user_id OR owner_position_id IN effective positions based on authenticated audit context.

Remaining rollout work:

• Add explicit owner-transfer commands instead of changing ownership through normal update forms.

Phase 5: Enforce In Services

• Add query-side owner predicates.
• Add command-side ownership checks.
• Move policy decisions into rule-engine configuration.
• Keep the UI filters as usability hints, not authorization.

Current implementation status:

• Query-side owner predicates are implemented for Schedule, API, API Version, App, OAuth Client, Client Token, Instance, Instance API, Instance API Path Prefix, Instance App, Instance App API, Runtime Instance, and Workflow Definition.
• Query handlers derive scope from the authenticated audit attachment. Users with the global admin role or the entity-specific all-scope role bypass the owner predicate; other users are scoped by user id or effective positions.
• The UI keeps owner-aware action guards, but it does not send the owner filter as a request filter for service-enforced pages. That keeps position-owned rows visible when the service grants access by owner_position_id.
• The db-provider keeps backward-compatible query methods and adds owner-aware overloads so query services can roll forward independently.

Remaining service rollout work:

• Add command-side ownership checks before update/delete actions.
• Add explicit owner-transfer commands and audit events.
• Move the all-scope role and position hierarchy decisions from Java guards into rule-engine policy once the service-side rule context is ready.

Future Improvement: Entity Access Grants

Do not introduce a generic ownership table in Phase 4. It adds query joins, pagination complexity, and weaker referential integrity before we have a clear sharing use case.

A generic table can be added later for secondary grants, sharing, and delegated administration. It should supplement the canonical owner columns rather than replace them.

Possible future shape:

entity_access_t
  host_id
  entity_type
  entity_id
  principal_type   -- user, position, group, role
  principal_id
  access_level     -- owner, maintainer, viewer

Use this only when we need use cases such as:

• share one API with another position or group
• give support read-only access to a selected set of records
• delegate maintenance without transferring the canonical owner
• manage record-specific exceptions from an Access Admin page

Risks And Mitigations

Recommendation

Use owner-scoped filtering as the first UI step, but centralize it immediately. Do not copy the Schedule.tsx logic into every page by hand.

The recommended path is:

1. fix the schedule filter ordering
2. introduce a reusable ownership-scope helper
3. apply it to the most common self-service admin pages
4. add explicit owner fields where update_user is not good enough
5. enforce the same rules in query and command services through the rule engine

This gives users a safer admin experience now while creating a clear migration path to real fine-grained authorization.

Contextual Help Links

portal-view has many pages, generated forms, task flows, and admin tables. Even with the task-oriented navigation work, users still need page-specific and form-specific help when they are making a decision or filling a field. This document proposes a contextual help-link model for pages and forms.

Problem

Users often need help at the exact point where they are working:

• what this page is for
• when to use this form
• what required fields mean
• which optional fields matter
• what permissions or ownership rules apply
• what happens after submit
• how this page fits into a larger task

Today, help is usually outside the UI context. Users must know where to look, which document applies, and which page or form name maps to the screen in front of them.

Design Goals

• Add a clear help entry point to every major page and generated form.
• Keep help content close to the product documentation source of truth.
• Avoid bloating the portal-view application bundle with documentation.
• Allow documentation-only updates without rebuilding portal-view.
• Make help links declarative so page, form, and task metadata can drive them.
• Keep link identifiers stable even if routes or component names change.
• Support future documentation search, related topics, and task-specific help.
• Preserve the ability to run the app locally with a configurable docs base URL.

Non-Goals

• Do not build a full documentation authoring system inside portal-view.
• Do not duplicate long user guides in component source files.
• Do not block a page or form rollout because full documentation is missing.
• Do not use contextual help as a replacement for better labels, validation, or field-level error messages.

Documentation Location Decision

The help content should live in light-portal-doc. portal-view should store only metadata that points to the relevant help page.

Recommended split:

light-portal-doc
  src/help/portal-view/
    pages/
    forms/
    tasks/
    concepts/

portal-view
  page registry, task registry, and form metadata with help ids or help paths

Why light-portal-doc

Pros:

• Keeps user-facing documentation in the documentation repo.
• Allows documentation changes without rebuilding or redeploying portal-view.
• Avoids increasing the app bundle with markdown content.
• Supports documentation search, navigation, publishing, and review workflows.
• Allows the same help content to be linked from support tickets, onboarding, release notes, and external docs.
• Fits the existing pattern where portal-view design docs already live in light-portal-doc.

Cons:

• Requires stable published URLs.
• Requires a configurable docs base URL for local and deployed environments.
• Can drift from UI behavior unless we add link validation and ownership rules.

Why Not portal-view/docs

Pros:

• Easy to review UI and docs in one PR.
• Help content can be tightly coupled to the component version.
• Local development does not need a separate docs deployment.

Cons:

• Documentation-only changes require app rebuilds and deployments.
• Large markdown content can bloat the frontend repo and build context.
• It is harder to provide a proper documentation navigation/search experience.
• It encourages implementation notes and user help to mix in the same repo.

Recommendation: use light-portal-doc for content and keep portal-view limited to stable link metadata.

Help Content Structure

Create a user-facing help tree separate from design docs:

src/help/portal-view/
  pages/
    api-admin.md
    api-detail.md
    instance-admin.md
    schedule-admin.md
  forms/
    create-api.md
    update-api.md
    create-client.md
    update-instance.md
  tasks/
    mcp-onboard-api.md
    register-standalone-mcp-server.md
  concepts/
    ownership-and-positions.md
    hosts-and-user-hosts.md
    api-versioning.md

Use page-level help for screen orientation and form-level help for submission semantics. Use concept help for reusable explanations that should not be copied into many page/form documents.

URL Strategy

Help URLs should be stable and human-readable.

Recommended public URL shape:

/help/portal-view/pages/api-admin
/help/portal-view/forms/create-api
/help/portal-view/tasks/mcp-onboard-api
/help/portal-view/concepts/ownership-and-positions

Do not make the public URL depend on React route internals or component names. If a route changes from /app/api to another route later, the help URL should not need to change.

portal-view should build the absolute link from a runtime config value:

PORTAL_DOC_BASE_URL=https://doc.lightapi.net

or for Vite:

VITE_PORTAL_DOC_BASE_URL=https://doc.lightapi.net

Local development can point to a local docs server:

VITE_PORTAL_DOC_BASE_URL=http://localhost:3000

Metadata Contract

Use a stable help id or help path in the app metadata. A help path is more direct and easier to validate.

Page registry example:

{
  id: "api-admin",
  title: "API Admin",
  route: "/app/apis",
  helpPath: "/help/portal-view/pages/api-admin",
}

Task registry example:

{
  id: "mcp-onboard-api",
  title: "Onboard API to MCP Gateway",
  helpPath: "/help/portal-view/tasks/mcp-onboard-api",
}

Form metadata example:

{
  "formId": "createApi",
  "helpPath": "/help/portal-view/forms/create-api",
  "actions": []
}

If we need indirection later, we can change to helpId and resolve it through a small registry:

{
  helpId: "forms.create-api"
}

Start with helpPath because it is simple, transparent, and works well with static documentation.

Portal UI Behavior

Each page and generated form should have a small help action in a predictable location.

Recommended behavior:

• open help in a new browser tab
• use an external-link icon or help icon with an accessible label
• keep the help action near the page title or form title
• if a form is opened inside a task shell, prefer form-specific help first and show task help as a secondary link
• if no specific help exists yet, fall back to the nearest page or concept help

Example resolution order for a form opened from a task:

1. form helpPath
2. current task helpPath
3. current page helpPath
4. generic portal help landing page

Do not render a broken link. If a help path is missing, hide the action or show the fallback help link.

Generated Forms

Generated forms should support a top-level helpPath field in Forms.json. The renderer can read it and show a help action in the form header.

For example:

{
  "formId": "createSchedule",
  "helpPath": "/help/portal-view/forms/create-schedule",
  "schema": {},
  "form": []
}

Field-level help can be added later, but it should not be the first step. Many field descriptions can stay in the JSON schema title/description. Use field-level help only for fields where a short description is not enough, such as security, ownership, deployment, or advanced configuration fields.

Possible future field shape:

{
  "key": "ownerPositionId",
  "helpPath": "/help/portal-view/concepts/ownership-and-positions"
}

Task-Aware Help

The task-oriented navigation layer should support task help separately from page or form help. A user working on the same form may need different context depending on the task.

Example:

• createApi opened from “Register a new API” links to create API form help.
• createApi opened from “Onboard API to MCP Gateway” can also link to MCP onboarding task help.

The UI should pass task context through existing task URL parameters and layout state, then render both links when useful:

Help: Create API
Related: Onboard API to MCP Gateway

Authoring Guidelines

Each page help document should include:

• what the page is used for
• who can access it
• what records are visible
• common actions
• links to related forms and tasks

Each form help document should include:

• when to use the form
• what happens after submit
• required fields
• important optional fields
• ownership and permission behavior
• validation or troubleshooting notes

Keep help content user-facing. Do not put implementation details, class names, or database internals in the main help body unless they are truly needed for an operator.

Validation

To prevent link drift, add a lightweight validation step once the first help docs exist.

Validation should check:

• every helpPath in portal-view points to a markdown source in light-portal-doc
• every high-value page has page help
• every high-value form has form help
• no help path uses a route-specific or component-specific unstable name

This can start as a script in light-portal-doc or a shared CI check that accepts both repo paths.

Rollout Plan

Phase 1: Documentation Structure

• Create src/help/portal-view/pages.
• Create src/help/portal-view/forms.
• Create src/help/portal-view/tasks.
• Create src/help/portal-view/concepts.
• Add placeholder help pages for the high-value admin pages and forms.

Phase 2: App Metadata

• Add optional helpPath to pageRegistry.ts.
• Add optional helpPath to taskRegistry.ts.
• Add optional top-level helpPath to generated form metadata.
• Add a docs base URL runtime config.

Phase 3: UI Components

• Add a reusable help-link component.
• Render page help near page titles.
• Render form help in the generated form header.
• Render task help in the task navigation shell.
• Add fallback behavior when a specific help link is missing.

Phase 4: Coverage And Validation

• Add help paths for all self-service owner-scoped admin pages.
• Add help paths for all high-value create/update forms.
• Add a validation script for help path coverage and broken links.
• Add missing docs over time as pages move into the task-oriented model.

Initial Scope

Start with the pages and forms most likely to be used by self-service users:

• API Admin and API Detail
• create/update API
• create/update API Version
• App Admin
• create/update App
• OAuth Client and Client Token
• create/update Client
• create Client Token
• Instance Admin and relationship pages
• create/update Instance
• create Instance API
• create/update Instance API Path Prefix
• create Instance App
• create Instance App API
• Schedule Admin
• create/update Schedule
• Workflow Definition
• create/update Workflow Definition

Then expand to admin-only pages after their ownership and access model is clear.

MVP Decisions

Use these decisions for the first implementation.

Missing Help Links

Do not hide the help action when a specific page, form, or task help path is missing. Fall back to the generic portal-view help landing page:

/help/portal-view/index

This keeps the UI consistent. A missing specific help page should degrade to general help instead of making the help affordance disappear.

Help Presentation

Open help in a new browser tab for the MVP. Do not build an embedded markdown viewer, side drawer, or iframe-based documentation panel in the first version.

This keeps portal-view small and avoids adding documentation rendering, iframe, routing, and panel-state complexity to the app. A side panel can be revisited later if users need in-page help while editing long forms.

JSON Schema Descriptions

Do not auto-generate full form help pages from JSON schema descriptions. Schema titles and descriptions are best used for inline labels, helper text, or field-level tooltips.

Form-level help should explain why the form exists, when to use it, what happens after submit, and how the form fits into a larger workflow. It should not simply repeat field types and required flags.

Documentation Versioning

Use latest documentation URLs for the MVP. Do not introduce release-versioned help URLs in the first implementation.

The portal will likely support both cloud SaaS deployments and enterprise on-premise deployments. SaaS users normally interact with the latest deployed portal, but enterprise customers may run older portal versions for a longer period. Versioned docs are therefore a good future requirement, but they should not block the first help-link rollout.

Keep helpPath values relative and version-neutral:

/help/portal-view/forms/create-api

Then versioning can be introduced later by changing only the configured docs base URL:

PORTAL_DOC_BASE_URL=https://doc.lightapi.net/v2.0

This keeps the app metadata stable while allowing SaaS to use latest docs and on-premise builds to point at version-specific documentation.

Future Enhancements

In-Page Help Drawer

Add an optional in-page help drawer after the helpPath metadata is stable and the first new-tab implementation has proven useful.

The drawer should be opt-in, not the default for every form. Long or complex configuration forms can declare:

{
  "helpPath": "/help/portal-view/forms/update-instance",
  "inPageHelp": true
}

When enabled, the UI can render a right-side drawer that displays the help document through an iframe or a lightweight markdown renderer. This avoids constant tab switching for complex forms while keeping the MVP simple.

Field-Level Help Paths

Add field-level help paths sparingly for complex fields and architectural concepts. Standard fields should continue to use JSON schema titles, descriptions, helper text, or tooltips.

Example future field metadata:

{
  "key": "ownerPositionId",
  "helpPath": "/help/portal-view/concepts/ownership-and-positions"
}

The UI can render a small help icon next to the field label when a field-level helpPath exists. Good candidates include ownership, security, OAuth token exchange, deployment target, transport configuration, and workflow definition fields.

Versioned Documentation

Add release-versioned documentation when multiple portal versions must be supported at the same time, especially for on-premise enterprise deployments.

The relative helpPath values should remain unchanged. The deployment or build configuration should select the versioned docs base URL:

SaaS/latest:
PORTAL_DOC_BASE_URL=https://doc.lightapi.net

On-premise v2.0:
PORTAL_DOC_BASE_URL=https://doc.lightapi.net/v2.0

This gives cloud deployments a simple latest-docs experience and gives enterprise deployments a path to version-matched help without changing portal-view metadata.

Recommendation

Store user-facing help content in light-portal-doc and add declarative helpPath metadata in portal-view. This keeps documentation maintainable and publishable while allowing every page, form, and task to provide context-aware help from the UI.

Event Processing Notifications

Portal commands are event driven. After a command is submitted, one or more CloudEvents are written to event_store_t and outbox_message_t. The hybrid-query event consumer later processes the outbox rows and updates the projection tables used by portal-view.

The notification page in the user profile is intended to show the user the recent processing result for those events. Today the table and read path exist, but notification_t is not populated consistently, so the page cannot provide meaningful status.

Current State

The command path already writes events through the common command handler:

1. The command handler validates and enriches the request.
2. It builds one or more CloudEvents.
3. It inserts those events into event_store_t and outbox_message_t.
4. The command returns before the query-side projection has necessarily run.

The query side can run through either event-processing pipeline, selected by configuration:

• Pg-notify pipeline: DbEventConsumerStartupHook polls outbox_message_t, uses the table’s gapless c_offset, groups rows by transaction_id, and writes failed transactions to the database dead_letter_queue.
• Kafka pipeline: a connector publishes rows from outbox_message_t to Kafka. PortalEventConsumerStartupHook consumes those records, groups records by the command-side transaction_id, and produces failed transactions to the Kafka DLQ topic when DLQ is enabled.

Both pipelines eventually call PortalDbProvider.handleEvent(conn, event). handleEvent dispatches the event to the projection method for that event type. Because both pipelines process the same outbox-backed events, they should share the same user-facing notification status model.

The notification pieces are partially present:

• notification_t exists in portal-db.
• NotificationDataPersistenceImpl can query notification_t.
• NotificationServiceImpl can insert a notification row.
• user-query exposes getNotification.
• portal-view has a notification table page.

The current gap is that notification rows are not created at the central event processing boundary.

There is also a separate UI error in MailMenu: it calls getPrivateMessage, whose handler currently returns an empty response. That explains the browser error Unexpected end of JSON input, but it is separate from the notification status design.

Goals

• Show the current user the latest event processing results in the profile notification page.
• Record both successful and failed projection processing.
• Preserve event processing correctness even if notification insertion fails.
• Keep notification creation centralized instead of adding calls to every projection method.
• Support commands that emit multiple events.
• Make the read API filter by host and user by default.
• Keep enough diagnostic data to debug failed projections.
• Keep notification writes idempotent so event replay is safe.

Non-Goals

• Do not replace event_store_t, outbox_message_t, or dead_letter_queue.
• Do not use notifications as the source of truth for projection state.
• Do not build a real-time push channel in the first phase.
• Do not add notification logic manually to every projection method.
• Do not expose other users’ processing history to non-admin users.

Recommended Design

Use notification_t as an operational projection-status table. The command side creates PENDING rows at the central event publication boundary, and the hybrid-query event consumer updates those rows with the processing result.

The primary processing-result write point should be the centralized outbox consumer path, around the call to PortalDbProvider.handleEvent(conn, event).

Recommended lifecycle:

command handler
  -> event_store_t
  -> outbox_message_t
  -> notification_t PENDING row
  -> response to caller

hybrid-query consumer
  -> read outbox_message_t
  -> handleEvent(conn, event)
  -> projection table write
  -> notification_t status row

For command-side publication, insert or update one notification row for each CloudEvent with status PENDING in the same transaction that writes event_store_t and outbox_message_t. Leave event_partition and event_offset null for this first insert, because the consumer has not observed the event position yet.

For successful projection processing, update the notification row for the CloudEvent to status SUCCEEDED and populate event_partition and event_offset from the active processor’s outbox position.

For failed projection processing, insert or update one notification row for each failed CloudEvent with status FAILED or DLQ, and store the exception message. Populate event_partition and event_offset when the processor has that information.

Status Model

Use one explicit status field. Do not keep is_processed; this feature is being implemented for the first time, and a boolean cannot distinguish pending, success, retry, DLQ, and skipped outcomes.

Recommended statuses:

The UI should show the status labels, not the underlying event pipeline. The same status meanings apply to both pg-notify and Kafka processing.

Schema

The existing table is close, but it is too small for operational status and has nonce as INTEGER while event tables use BIGINT.

Recommended table shape:

CREATE TABLE notification_t (
    id                  UUID NOT NULL,
    host_id             UUID NOT NULL,
    user_id             UUID NOT NULL,
    nonce               BIGINT NOT NULL,
    event_class         VARCHAR(255) NOT NULL,
    event_json          TEXT NOT NULL,
    event_ts            TIMESTAMP WITH TIME ZONE NULL,
    process_ts          TIMESTAMP WITH TIME ZONE NOT NULL,
    status              VARCHAR(16) NOT NULL,
    error               VARCHAR(2048) NULL,
    aggregate_id        VARCHAR(255) NULL,
    aggregate_type      VARCHAR(255) NULL,
    aggregate_version   BIGINT NULL,
    event_partition     INTEGER NULL,
    event_offset        BIGINT NULL,
    transaction_id      UUID NULL,
    read_ts             TIMESTAMP WITH TIME ZONE NULL,
    PRIMARY KEY (host_id, id),
    FOREIGN KEY (host_id) REFERENCES host_t(host_id) ON DELETE CASCADE
);

user_id is intentionally not a foreign key to user_t. PENDING rows are inserted on the command side before projection tables are updated, so enforcing that projection FK would break commands such as user creation before the projection catches up.

Recommended indexes:

CREATE INDEX idx_notification_user_process_ts
    ON notification_t (host_id, user_id, process_ts DESC);

CREATE INDEX idx_notification_status_process_ts
    ON notification_t (host_id, status, process_ts DESC);

CREATE INDEX idx_notification_transaction
    ON notification_t (host_id, transaction_id);

CREATE INDEX idx_notification_event_position
    ON notification_t (host_id, event_partition, event_offset);

CREATE INDEX idx_notification_unread_failure
    ON notification_t (host_id, user_id, process_ts DESC)
    WHERE read_ts IS NULL AND status IN ('FAILED', 'DLQ');

event_partition and event_offset are intentionally generic processing position fields. They are useful for operator diagnostics, but the UI should not label them as pg-notify or Kafka details. In the pg-notify processor, event_partition is the configured logical consumer partition and event_offset is outbox_message_t.c_offset. In the Kafka processor, event_partition and event_offset are the consumed Kafka record partition and offset.

Both columns are nullable. PENDING rows should leave them empty at initial insert time. They are filled later by the pg-notify or Kafka processor when the processing result changes the row to SUCCEEDED, FAILED, DLQ, or SKIPPED.

transaction_id remains a UUID because it is generated by the command side and used by both event processors.

Do not store pipeline name, source topic/channel name, or DLQ destination in notification_t. Those are implementation details of the configured event pipeline. Operators can use service configuration and logs when they need pipeline-specific diagnostics.

For existing installations, ship this as a patch:

ALTER TABLE notification_t ALTER COLUMN nonce TYPE BIGINT;
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS status VARCHAR(16);
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS event_ts TIMESTAMP WITH TIME ZONE;
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS aggregate_id VARCHAR(255);
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS aggregate_type VARCHAR(255);
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS aggregate_version BIGINT;
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS event_partition INTEGER;
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS event_offset BIGINT;
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS transaction_id UUID;
ALTER TABLE notification_t ADD COLUMN IF NOT EXISTS read_ts TIMESTAMP WITH TIME ZONE;

ALTER TABLE notification_t DROP CONSTRAINT IF EXISTS notification_t_user_id_fkey;
ALTER TABLE notification_t DROP COLUMN IF EXISTS is_processed;
ALTER TABLE notification_t ALTER COLUMN status SET NOT NULL;

CREATE INDEX IF NOT EXISTS idx_notification_unread_failure
    ON notification_t (host_id, user_id, process_ts DESC)
    WHERE read_ts IS NULL AND status IN ('FAILED', 'DLQ');

Write Path

Notification writes need an explicit transaction policy. A single rule cannot cover every status:

• Failure and DLQ notifications must be durable even when projection writes are rolled back.
• Success notifications must not claim success until the projection write has committed.
• Notification write failures should not break projection processing.

Use a REQUIRES_NEW style helper for notification writes that must survive a projection rollback. In plain JDBC, this means opening a separate connection with its own commit/rollback boundary.

For success rows, there are two safe options:

1. Commit the projection transaction first, then write SUCCEEDED in a separate notification transaction.
2. Write SUCCEEDED inside the projection transaction, but wrap it in a savepoint and treat notification insert failure as non-fatal.

The first option is the recommended default because notification failures cannot roll back projection updates. The tradeoff is a small window where projection has committed but the success notification is missing. That is acceptable because event_store_t remains the source of truth and success notifications are user feedback, not projection correctness.

Recommended service methods:

void recordPending(Map<String, Object> event, UUID transactionId);

void recordSuccess(Map<String, Object> event, EventMetadata metadata);

void recordFailure(Map<String, Object> event, EventMetadata metadata, String error, String status);

recordPending should participate in the command-side transaction that writes event_store_t and outbox_message_t. It may store transaction_id, because that value is generated by the command side, but it must leave event_partition and event_offset null. recordSuccess and recordFailure should use the event-processing transaction policy described below.

EventMetadata should carry only pipeline-neutral data that is not inside the CloudEvent map:

• eventPartition: the active processor’s partition value. For pg-notify this is the configured logical consumer partition; for Kafka this is the consumed Kafka record partition.
• eventOffset: the active processor’s offset value. For pg-notify this is outbox_message_t.c_offset; for Kafka this is the consumed Kafka record offset.
• transactionId: the command-side transaction UUID used by both processors.

Both consumers should build this metadata before calling handleEvent, so the failure path still has offset and transaction context after the projection transaction is rolled back.

Use an idempotent upsert:

INSERT INTO notification_t (...)
VALUES (...)
ON CONFLICT (host_id, id) DO UPDATE SET
    process_ts = EXCLUDED.process_ts,
    status = EXCLUDED.status,
    error = EXCLUDED.error,
    event_partition = EXCLUDED.event_partition,
    event_offset = EXCLUDED.event_offset,
    transaction_id = EXCLUDED.transaction_id;

This makes replay and fallback processing safe.

Success Handling

In the normal batch path:

begin projection transaction
for each event from the active pipeline:
  parse CloudEvent
  handleEvent(conn, event)
commit projection transaction

for each successfully committed event:
  recordSuccess(event, metadata) in separate notification transaction

Do not write SUCCEEDED before the projection transaction commits unless it is part of the same transaction. If it is written in the same transaction, a projection rollback must roll back the success row too.

The implementation can keep an in-memory list of successfully applied events while processing the batch. After commit, loop through that list and upsert the success notifications. If a success notification write fails, log it and continue; do not retry the projection.

Failure Handling

Failure rows must be written outside the failed projection transaction.

In fallback mode, processing is retried per transaction. For a failed transaction:

begin projection transaction
savepoint projection_attempt
  process transaction events
on exception:
  rollback to projection_attempt or rollback projection transaction
  write failed events to database DLQ or Kafka DLQ topic
  recordFailure(event, metadata, error, "DLQ") in separate notification transaction

For pg-notify, the DLQ destination is the database dead_letter_queue table. For Kafka, the DLQ destination is the configured Kafka DLQ topic. The failure notification can be committed with the database DLQ transaction for pg-notify, or in a separate notification transaction immediately after the Kafka DLQ produce request is accepted. The key requirement is that it must not be part of the projection work that is being rolled back.

If the database connection enters an unrecoverable error state, close it and open a fresh connection for the DLQ and failure notification writes.

If the payload cannot be parsed as a CloudEvent, the consumer may not know the CloudEvent id or event type. In that case, the DLQ remains the primary failure record. If the consumer metadata still has host, user, partition, offset, and transaction id, the consumer can create a diagnostic notification with a generated id, but this should be treated as a best-effort operational row.

Pending Handling

PENDING is part of phase one. Add pending rows at the central command-side publication boundary that writes event_store_t and outbox_message_t.

The pending notification should be written in the same command-side transaction as the event-store and outbox rows. If the command rolls back, the pending notification must roll back too. Do not add pending writes to individual command handlers.

At this stage, the notification row should contain command-known fields only: CloudEvent id, host id, user id, nonce, event class, event JSON, event timestamp, and transaction id. The processor-owned event_partition and event_offset fields remain null until event processing updates the row.

Read API

Keep getNotification as the main query endpoint, but tighten its contract.

Recommended request fields:

{
  "hostId": "uuid",
  "userId": "uuid",
  "offset": 0,
  "limit": 25,
  "status": "SUCCEEDED",
  "eventClass": "ClientCreatedEvent",
  "nonce": "123",
  "fromTs": "2026-05-08T00:00:00Z",
  "toTs": "2026-05-08T23:59:59Z",
  "error": "duplicate key"
}

Recommended response:

{
  "total": 1,
  "notifications": [
    {
      "id": "uuid",
      "hostId": "uuid",
      "userId": "uuid",
      "nonce": 123,
      "eventClass": "ClientCreatedEvent",
      "status": "SUCCEEDED",
      "processTs": "2026-05-08T16:12:00Z",
      "aggregateId": "host|client",
      "aggregateType": "Client",
      "aggregateVersion": 2,
      "transactionId": "uuid",
      "eventPartition": 0,
      "eventOffset": 1001,
      "error": null,
      "eventJson": "{...}"
    }
  ]
}

eventPartition and eventOffset are intentionally displayed as generic position fields regardless of which event pipeline is configured. The main list can hide them by default and show them in the detail view.

userId is a filter on getNotification, not a separate endpoint contract. The profile notification page should always send the logged-in user’s userId. The admin notification page can omit userId to request host-wide results, or pass a specific userId to narrow the host-wide view to one user.

Authorization rules:

• Normal users can query only their own token user_id within the selected hostId. If the request omits userId, the backend should apply the token user_id; if the request supplies another userId, the backend should reject it or override it with the token user_id.
• Admin users can query all users for the host by omitting userId, or filter to a specific user by providing userId.
• The backend should enforce this using token claims, not only UI filters.

Portal View

The profile notification page should become a processing-status view.

Recommended columns:

• Time
• Status
• Event
• Aggregate
• Nonce
• Error
• Details

Recommended default filters:

• hostId from the selected host.
• userId from the logged-in user.
• No default status filter.
• Most recent first.
• Last 25 rows.

The UI should display concise summaries and keep full eventJson behind an expandable detail row or dialog.

Show all events associated with the user, including successful, failed, and derived events. Derived events should be visible as their own rows instead of being collapsed under the original command transaction.

The current processFlag filter should be replaced by status. No is_processed compatibility mapping is needed because this feature has not yet started populating notification_t.

The header MailMenu should not call getPrivateMessage unless that handler is restored. For notification status, add a small notification badge endpoint or reuse getNotification with limit = 5.

The header badge should count only unread failure notifications, such as FAILED and DLQ, and display the count in red when the count is greater than zero.

In the list, FAILED and DLQ status badges should also use red styling.

Phase 2 adds two narrow user-query RPCs:

• getUnreadNotificationCount: returns unread FAILED and DLQ notifications for the current hostId and userId.
• markFailureNotificationsRead: sets read_ts on unread FAILED and DLQ notifications for the current hostId and userId.

The header uses the count endpoint for its badge and marks failures read when the user opens the notification menu. The notification page also marks failures read when it is opened.

Admin Notification Page

Phase 3 should add a separate admin notification page instead of overloading the profile notification page. The recommended location is:

• Route: /app/event/notifications
• Menu: Administration -> Event Admin -> Notifications

This page should reuse the same notification table and getNotification read API, but with admin defaults:

• hostId from the selected host.
• No default userId filter, so admins see host-wide results.
• Default status filter for FAILED and DLQ, with an option to show all statuses.
• Filters for userId, eventClass, status, transactionId, aggregateId, processing position, time range, and error text.
• No unread badge behavior and no call to markFailureNotificationsRead.

The page should clearly identify itself as an admin view, such as “Admin View: Host Notifications”. Host-wide access must still be enforced by the backend using token roles.

Operational Cleanup

Notifications are operational history. They should not grow forever.

Recommended retention:

• Keep successful notifications for 30 to 90 days.
• Keep failed and DLQ notifications longer, such as 180 days.
• Allow host-level configuration later if needed.

Cleanup should be implemented as a generic operational cleanup process, not as notification-specific UI or command-handler logic. The first cleanup target is notification_t, but the same framework should also support other operational tables such as message_t for private messages.

Recommended implementation:

• Add an OperationalCleanupStartupHook on the query side.
• Run cleanup on a fixed interval, such as daily, with config-driven enablement, interval, batch size, and per-target retention days.
• Use a single cleanup coordinator that owns multiple cleanup targets. Each target defines its table, timestamp column, status/type conditions if needed, retention duration, and batch delete SQL.
• Use a database lock, such as a PostgreSQL advisory lock or a dedicated cleanup lock row, so only one service instance performs cleanup at a time.
• Delete in bounded batches to avoid long table locks and large transactions.
• Use a separate database connection and transaction for cleanup work.
• Log cleanup failures and continue service startup; cleanup failure must not block query APIs or event processing.

Do not use schedule_t directly for this cleanup. That scheduler is business workflow infrastructure that emits events into event_store_t and outbox_message_t. Operational cleanup is local maintenance and should stay out of the event-processing path.

Example notification cleanup:

WITH doomed AS (
    SELECT host_id, id
    FROM notification_t
    WHERE (status IN ('SUCCEEDED', 'SKIPPED') AND process_ts < ?)
       OR (status IN ('FAILED', 'DLQ') AND process_ts < ?)
    ORDER BY process_ts
    LIMIT ?
)
DELETE FROM notification_t n
USING doomed d
WHERE n.host_id = d.host_id
  AND n.id = d.id;

Private-message cleanup can be another target using message_t.send_time:

WITH doomed AS (
    SELECT host_id, from_id, nonce
    FROM message_t
    WHERE send_time < ?
    ORDER BY send_time
    LIMIT ?
)
DELETE FROM message_t m
USING doomed d
WHERE m.host_id = d.host_id
  AND m.from_id = d.from_id
  AND m.nonce = d.nonce;

Recommended default cleanup targets:

Do not delete recent PENDING notifications. Old PENDING rows should be treated as an operational signal first because they may indicate that the event consumer is stopped or lagging. If a hard cap is needed later, make it a separate, longer retention policy.

Snapshot and Promotion

notification_t should be treated as an operational table, not a promoted projection table.

It should be excluded from global snapshot export and conversion alongside event_store_t, outbox_message_t, dead_letter_queue, log_counter, and consumer_offsets.

Rollout Plan

Phase 1: Make Notifications Useful

• Add status and diagnostic columns to notification_t.
• Add pipeline-neutral event_partition, event_offset, and transaction_id metadata.
• Change NotificationService to support separate notification transactions.
• Insert PENDING rows at the central command-side outbox publication boundary.
• Insert SUCCEEDED rows after successful handleEvent.
• Insert DLQ rows in fallback failure handling.
• Update getNotification to support status and correct timestamp fields.
• Update portal-view to use status, default to the current user, and show all user-associated events including derived events.

Phase 2: Improve User Feedback

• Add an unread marker with read_ts.
• Add a small header badge query for unread FAILED and DLQ notifications and render the badge in red.
• Mark unread failure notifications as read when the user opens the header menu or the notification page.

Phase 3: Operations

• Add a generic operational cleanup startup hook with retention targets for notification_t and message_t.
• Make cleanup configurable by enablement, interval, batch size, and per-target retention days.
• Add a database lock so only one service instance runs cleanup at a time.
• Add an admin notification page under Event Admin that uses getNotification without a userId filter for host-wide failures.
• Add dashboards or alerts for repeated DLQ statuses.

Risks and Mitigations

API Marketplace Catalog

Context

The portal already has a Marketplace navigation group and an api-marketplace page registry entry. The current API administration page is table-oriented and is useful for owners, but it is not a consumer catalog. A Marketplace API catalog should let users discover APIs by business category, capability, protocol, lifecycle status, and governance metadata.

API create and update forms already use the standardized taxonomy fields:

• categoryIds for selected category identifiers.
• tagIds for selected tag identifiers.
• getCategoryLabelByType with entityType = "api" for category options.
• getTagLabelByType with entityType = "api" for tag options.

The service query layer also returns categoryIds, categories, tagIds, and tags for API rows. The catalog should use those fields for display and filtering instead of reintroducing the legacy apiTags string field.

Goals

• Add a Marketplace menu entry for an API catalog.
• Use database-backed categories and tags, not hard-coded UI lists.
• Keep categories and tags reusable across future catalog pages.
• Keep API create/update forms as the source of truth for taxonomy assignment.
• Give consumers a browse-first experience instead of an admin table.
• Support deep links from a catalog listing to API detail, versions, endpoints, runtime bindings, and owner actions.
• Preserve host scope and ownership rules already used by API administration.

Non-Goals

• Do not replace API administration pages with the catalog.
• Do not store display names in API rows when they can be resolved from category_t, tag_t, entity_category_t, and entity_tag_t.
• Do not use the old apiTags field for catalog filtering.
• Do not make taxonomy values static frontend constants.
• Do not expose private tenant APIs through a public catalog without an explicit visibility and authorization decision.

Current Building Blocks

User Experience

The first screen under Marketplace should be the usable catalog, not a landing page. The recommended route is:

/app/marketplace/api

The sidebar can keep the existing Marketplace group, but its children should move from API-type-only links to intent-based entries:

• API Catalog
• API Clients
• Schema Catalog
• YAML Rule
• Schema Form

The API Catalog page should provide:

• Search across API id, name, description, business group, line of business, capability, platform, git repository, categories, and tags.
• Category tabs or a category rail based on getCategoryLabelByType.
• Grouped tag filters based on getTagLabelByType.
• Filter chips for active category and tag selections.
• A compact card or list row per API with name, description, status, categories, tags, owner, business group, and latest version summary.
• Actions to view details, review versions, create a new version, update the API metadata, and open related runtime or access-control pages when the user has permission.

The catalog should support an Uncategorized bucket for active APIs without category assignments. This avoids hiding incomplete data and gives admins an easy cleanup target.

Categories And Tags

Categories should be stable browse buckets. Tags should be flexible facets. Both are stored with entityType = "api" so the same tag names can be reused for other entity types without forcing cross-catalog semantics.

Recommended initial API categories:

The stored category_name must stay lower-case and URL-friendly. The display labels above are UI labels derived from those values.

Recommended initial API tag groups:

Stored tag names must stay lower-case and URL-friendly. If a display label needs capitalization, the UI should format it or the label endpoint should provide a separate display field later.

Tags without tag_group_code or tag_group_label should be shown under a General filter group in the catalog UI. Configured groups should appear first by group_sort_order; the General group should appear after configured groups, matching the current label query behavior where null group sort values sort last.

Data Flow

Catalog filter option loading:

portal-view
  -> category/getCategoryLabelByType(hostId, entityType = "api")
  -> tag/getTagLabelByType(hostId, entityType = "api")

Catalog result loading:

portal-view
  -> service/getApi(hostId, offset, limit, active, filters, globalFilter, sorting)
  -> api rows with categoryIds, categories, tagIds, tags

The catalog should prefer server-side pagination and filtering. Client-side filtering is acceptable only for a small first pass because it breaks as soon as the API count exceeds one fetched page.

Query Contract

The existing getApi contract already supports filters, globalFilter, sorting, offset, limit, hostId, and active. To make the catalog work well at scale, add first-class filter support for taxonomy fields:

{
  "hostId": "01964b05-552a-7c4b-9184-6857e7f3dc5f",
  "offset": 0,
  "limit": 20,
  "active": true,
  "categoryIds": ["..."],
  "tagIds": ["..."],
  "tagMatch": "all",
  "globalFilter": "payment"
}

Recommended semantics:

• categoryIds uses OR semantics by default. An API in any selected category is returned.
• tagIds should support tagMatch = "all" and tagMatch = "any".
• Category and tag filters should use EXISTS against entity_category_t and entity_tag_t with entity_type = 'api' and active = TRUE.
• Display arrays should continue to be returned as categories and tags.
• Form update payloads should continue to submit identifiers only through categoryIds and tagIds.

Page Design

The API Catalog page can be implemented as a dedicated page rather than trying to stretch the current API admin table.

Proposed files:

src/pages/marketplace/ApiCatalog.tsx
src/pages/marketplace/components/ApiCatalogFilters.tsx
src/pages/marketplace/components/ApiCatalogCard.tsx
src/pages/marketplace/hooks/useApiCatalog.ts

Page state:

• search text
• selected category ids
• selected tag ids
• tag match mode
• active status
• pagination
• sorting
• view mode, either compact list or card grid

Catalog state should be URL-driven from Phase 1. Search text, selected categories, selected tags, tag match mode, active status, sorting, and pagination should be encoded in the query string so users can refresh the page, use browser navigation, and share filtered catalog URLs. Example:

/app/marketplace/api?q=payment&category=public-api&tag=oauth2&tag=mtls&tagMatch=all&page=1

The page should still reuse existing infrastructure:

• fetchClient for portal query calls.
• useUserState for host and user context.
• buildTaskAwareRoute for deep links.
• ownership utilities for update/delete action visibility.
• TaskActionPanel for publisher/admin next actions.
• pageRegistry and contextual help metadata.

Routing And Navigation

Add or update these portal-view entries:

The existing /app/marketplace route can redirect to /app/marketplace/api or remain a broader Marketplace landing page later. For the first API catalog implementation, redirecting keeps the behavior simple.

Backend Changes

The backend already persists API category and tag relationships. The main backend change is query filtering:

1. Extend service-query spec for optional categoryIds, tagIds, and tagMatch.
2. Update GetApi to pass those optional fields to the DB provider.
3. Update PortalDbProvider#getApi and ApiServicePersistenceImpl#getApi.
4. Add SQL predicates over entity_category_t and entity_tag_t.
5. Verify or add compound indexes for taxonomy filtering.
6. Add tests for category-only, tag-any, tag-all, combined taxonomy filters, and APIs with no taxonomy assignments.

The existing join-table indexes are useful for entity lookups and label resolution, but catalog filtering also needs indexes that start with filter fields. Before implementing Phase 2, verify the query plan and add indexes if needed:

CREATE INDEX idx_entity_tag_filter
ON entity_tag_t (entity_type, tag_id, entity_id)
WHERE active = TRUE;

CREATE INDEX idx_entity_category_filter
ON entity_category_t (entity_type, category_id, entity_id)
WHERE active = TRUE;

For tagMatch = "all", prefer a single grouped subquery over generating one EXISTS predicate per selected tag when the selected tag set can grow. A common shape is to filter entity_tag_t by selected tag ids, group by entity_id, and require COUNT(DISTINCT tag_id) = selectedTagCount.

The query response should continue to include both identifiers and labels:

{
  "apiId": "0001",
  "apiName": "Petstore",
  "categoryIds": ["..."],
  "categories": ["public-api"],
  "tagIds": ["..."],
  "tags": ["openapi", "oauth2"]
}

Implementation Phases

Phase 1: Catalog Page

• Add the API Catalog route and Marketplace menu entry.
• Load category and tag options from the existing label endpoints.
• Load APIs with service/getApi.
• Render search, category filter, grouped tag filter, and API list/card results.
• Store catalog filters, search text, sorting, and pagination in the URL query string.
• Use current query response labels for display.
• Deep-link to existing API detail and update forms.

Phase 2: Server-Side Taxonomy Filters

• Add categoryIds, tagIds, and tagMatch to service-query.
• Implement SQL filtering in ApiServicePersistenceImpl.
• Keep current table filtering support for admin use.
• Add DB provider and handler tests.

Phase 3: Catalog Polish

• Add API detail summary panels with versions, endpoint count, runtime exposure, and access-control hints.
• Add help docs and task links.
• Add optional counts per category and tag if the catalog needs faceted counts.

Open Questions

• Should Marketplace API Catalog show only active APIs by default? The recommendation is yes, with an admin-visible inactive filter.
• Should unauthenticated users ever see catalog data? The recommendation is no until a separate public visibility model is designed.
• Should category selection be single-select or multi-select? The recommendation is multi-select OR semantics for flexibility.
• Should tags use all-match or any-match by default? The recommendation is all for precision, with a visible toggle if users need broader searches.
• Should OpenAPI tags imported from specs automatically create API catalog tags? The recommendation is no for the first pass. Spec tags are often endpoint-level groupings and should not automatically become curated catalog taxonomy.

AI Agent Registration In Task Center

Status

Initial Task Center implementation is available. The first version uses the existing API-version and agent-definition commands with backend validation guardrails. A dedicated composite registration command remains a later automation enhancement.

Context

Light Portal treats an AI agent as an API. The API record provides the stable catalog identity, ownership, display name, marketplace metadata, and lifecycle. The API version record provides the deployable version identity. The agent definition record is an agent-specific profile extension for the same API version.

The current data model already reflects this relationship:

• api_t owns the logical API and display name.
• api_version_t owns the API version identity.
• agent_definition_t.agent_def_id stores the same UUID as api_version_t.api_version_id.
• agent_definition_t stores model and runtime profile fields such as model_provider, model_name, api_key_ref, temperature, and max_tokens.
• Agent query paths join agent_definition_t to api_version_t and api_t to expose the effective agent metadata.

The registration UX should make this model explicit. Operators should not have to understand the table split. They should see one task: register an AI agent.

Goals

• Add a focused Task Center flow named Register AI Agent.
• Register the agent first as an API and API version.
• Create the agent definition profile using the same ID as the API version.
• Keep event sourcing and replay clean by using domain events instead of direct table writes.
• Avoid duplicating mutable display name fields between api_t and agent_definition_t.
• Allow skills, tools, memory, access control, and deployment links to be added after the base agent is registered.

Non-Goals

• Do not create a second standalone agent registry independent from APIs.
• Do not make AgentDefinitionCreatedEvent create api_version_t.
• Do not make ApiVersionCreatedEvent directly write agent_definition_t unless the event schema is intentionally expanded later.
• Do not require all skill and tool assignments during the initial registration.
• Do not replace the existing Manage GenAI Assets task. That task remains the broader maintenance flow.

Identity Model

The agent identity is the API version identity.

api_t
  host_id
  api_id
  api_name              # canonical agent display name

api_version_t
  host_id
  api_version_id        # canonical agent definition id
  api_id
  api_version
  api_type = "agt"      # or accepted legacy value "agent"

agent_definition_t
  host_id
  agent_def_id          # same value as api_version_t.api_version_id
  model_provider
  model_name
  api_key_ref
  temperature
  max_tokens

agent_definition_t should remain a profile extension. It should not duplicate the agent name. Reads can continue to expose agentName, but the value should come from api_t.api_name.

API Type

Use agt as the canonical API type for AI agents if the reference data uses the short code model. During migration, command handlers and queries can accept both agt and agent to avoid breaking existing test data or early records.

The Portal UI should display this as Agent and submit the canonical value. Database columns use snake case such as api_type; event payloads and command requests use camel case such as apiType. The mapper must preserve this translation and normalize agent type values consistently.

When registering an agent against an existing API, the backend must validate the existing API-version family. A logical API should not mix unrelated version types. If the selected api_t already has active versions, they must all be agent versions before an agt version can be added. The reverse should also be enforced: once an API has an active agent version, non-agent API versions should not be added under the same api_id.

Event Model

The Task Center flow should produce two domain events for the required base registration:

1. ApiVersionCreatedEvent
2. AgentDefinitionCreatedEvent

The two-event design is preferred because these are two separate domain facts:

• an API version exists and can participate in the API catalog;
• that API version has an agent runtime profile.

This should not be modeled as two direct table writes from one handler. The event processor should continue to populate projection tables during normal processing and replay.

Event Order

ApiVersionCreatedEvent must be persisted and projected before AgentDefinitionCreatedEvent, because agent_definition_t has a foreign key to api_version_t.

Register AI Agent
  -> ApiCreatedEvent, if the logical API does not already exist
  -> ApiVersionCreatedEvent
  -> AgentDefinitionCreatedEvent
  -> optional AgentSkillCreatedEvent events
  -> optional access-control events

The minimum required sequence for an existing API is:

ApiVersionCreatedEvent
AgentDefinitionCreatedEvent

Aggregate IDs

ApiVersionCreatedEvent keeps the API version aggregate identity:

{
  "aggregateType": "ApiVersion",
  "subject": "<apiVersionId>",
  "data": {
    "hostId": "<hostId>",
    "apiId": "<apiId>",
    "apiVersionId": "<apiVersionId>",
    "apiVersion": "1.0.0",
    "apiType": "agt"
  }
}

AgentDefinitionCreatedEvent uses the same UUID for its aggregate identity:

{
  "aggregateType": "AgentDefinition",
  "subject": "<apiVersionId>",
  "data": {
    "hostId": "<hostId>",
    "agentDefId": "<apiVersionId>",
    "apiVersionId": "<apiVersionId>",
    "modelProvider": "openai",
    "modelName": "gpt-4.1",
    "apiKeyRef": "secret://openai/default",
    "temperature": 0.7,
    "maxTokens": 4096
  }
}

The event utility should continue to accept either agentDefId or apiVersionId for AgentDefinition aggregate ID calculation, but the canonical payload should include both during migration and treat them as equal.

Task Center Flow

Add a Task Center definition:

id: register-ai-agent
title: Register AI Agent
category: API Marketplace or Portal Administration
roles: user, admin
keywords: agent, ai, genai, model, skill, tool

The task should guide the operator through a narrow registration path. It is different from Manage GenAI Assets, which is a broad maintenance task for agents, skills, tools, memory, and session history.

Steps

Task Context

The task context should carry IDs from one step to the next:

{
  "hostId": "<hostId>",
  "apiId": "<apiId>",
  "apiVersionId": "<apiVersionId>",
  "agentDefId": "<apiVersionId>",
  "serviceId": "<serviceId>",
  "providerId": "<modelProvider>",
  "apiType": "agt"
}

When the API version step completes, apiVersionId should be copied to agentDefId automatically before launching the agent definition step.

Incomplete Registration Handling

If the UI calls createApiVersion and then fails before createAgentDefinition is processed, the system can contain an agent API version without an agent definition. This is an incomplete registration, not a valid runnable agent.

The UI and query layer should treat these rows explicitly:

• Agent list views should be able to detect agent API versions missing matching agent_definition_t rows by left joining api_version_t to agent_definition_t.
• The row should be shown as Incomplete or Profile missing, not as a ready agent.
• The primary action should be Complete profile, prefilled with agentDefId = apiVersionId.
• A secondary action can delete or deactivate the orphaned API version if the operator abandons the registration.
• Runtime catalog reads should not expose incomplete agents as executable.

This requirement makes the UI-orchestrated implementation safe enough for the first Task Center version. The long-term backend command should still create the version and profile in one ordered command to reduce orphan creation.

Frontend Design

Phase 1: Task Registry Only

The first implementation can add a Task Center entry that reuses existing pages and forms:

• createApi
• createApiVersion
• createAgentDefinition
• AgentSkill
• SkillTool
• rolePermission
• InstanceApi

This is low risk and aligns with the current task-oriented navigation model.

The createApiVersion form should support prefilled apiType=agt from the task route. The form completion handler should save returned apiVersionId into the task context.

The createAgentDefinition form should accept apiVersionId or agentDefId from task context and submit both values, with agentDefId equal to apiVersionId.

Phase 2: Dedicated Registration Wizard

After the flow is validated, add a dedicated wizard route such as:

/app/genai/register-agent

The wizard can reduce clicks by combining API version and agent profile fields on one page while still submitting separate commands or a composite command.

Recommended sections:

• API identity: API name, API ID, status, owner.
• Version identity: version, service ID, environment tag, target host.
• Model profile: provider, model, API key reference, temperature, max tokens.
• Optional skills: selected skill IDs.
• Optional deployment: instance or gateway link.

Secret Reference Selection

apiKeyRef is a secret reference, not a secret value. The UI should not ask operators to paste raw provider keys into the agent definition form.

The preferred control is a selector populated from the configured secret catalog, config-server reference data, or vault integration available to the current host. The selected value should be stored as a reference such as:

secret://openai/default

If manual entry is temporarily supported, it should be an advanced path with validation. The command should reject values that look like raw API keys and should accept only approved reference schemes.

Secure Default Access

The access-control step is optional for registration completeness, but runtime execution must be secure by default. A newly registered agent should not be publicly invokable just because the API version and profile exist.

Default behavior:

• management is limited to the creator, owner, or admin roles according to the existing ownership model;
• runtime invocation is denied until an explicit role, scope, policy, or runtime assignment grants access;
• skill and tool assignment does not override access control;
• if no access policy exists, the gateway or agent runtime should treat the effective execution policy as deny-all.

Backend Command Options

Option 1: UI-Orchestrated Existing Commands

The Task Center flow calls existing commands in sequence:

1. createApi, if a new API is needed.
2. createApiVersion.
3. createAgentDefinition.
4. Optional createAgentSkill events.

This is the recommended initial implementation. It avoids changing command handler infrastructure and uses existing event types.

This option must include incomplete-registration handling. Without that, a browser failure or second-command validation error can leave an agent API version without an agent definition. That state is repairable, but the UI must surface it clearly and runtime catalog reads must ignore it.

Option 2: Composite Register Command

Add a composite command such as:

lightapi.net/genai/registerAiAgent/0.1.0

The command would validate the combined request and emit ordered events:

1. ApiVersionCreatedEvent.
2. AgentDefinitionCreatedEvent.
3. Optional AgentSkillCreatedEvent events.

This improves user experience for automation and API consumers, but it requires the command layer to support a multi-event result in one request. The command must not bypass event processing or write projection tables directly.

The initial composite command should require an existing apiId. Keeping API creation as a separate command keeps the backend contract smaller and preserves the existing API ownership workflow. A later full registration command can add ApiCreatedEvent if automation needs to create the logical API and agent version in one request.

Recommendation

Start with Option 1 only if incomplete registrations are visible and repairable. Prioritize Option 2 before exposing a one-click production registration wizard, because it gives the backend one validation boundary for the API version and agent profile.

Validation Rules

Command handlers should enforce these rules server-side:

• Agent API versions must use apiType = agt or an accepted compatible value.
• New writes should use canonical agt. Legacy agent should be accepted only for migration, import, or replay compatibility.
• A logical API should not mix active agent and non-agent API versions.
• agentDefId must equal apiVersionId when both are present.
• The referenced API version must exist before creating the agent definition.
• The referenced API version must belong to the same hostId.
• The referenced API version must have agent API type.
• modelProvider and modelName are required for creation.
• apiKeyRef, when present, must be a secret reference and not a raw provider key.
• temperature, when provided, must be in the supported provider range.
• maxTokens, when provided, must be positive.
• Optional skill IDs must reference active skills in the same host scope.

The UI should guide the user, but the command and persistence layers should remain authoritative.

Query And Display

Agent list and detail views should display a joined projection:

The Agent Definition page should make the API identity read-only once selected. Mutable profile fields should remain editable through AgentDefinitionUpdatedEvent.

Delete And Update Semantics

Updating the API name should update the visible agent name because the display name comes from api_t.api_name.

Updating the API version should not implicitly update model settings. Model profile changes should use AgentDefinitionUpdatedEvent.

Deleting or deactivating the API version should cascade or hide the agent definition through the existing API-version relationship. Explicit AgentDefinitionDeletedEvent remains useful when the operator wants to disable the agent profile while keeping the API version.

Migration Notes

• Existing rows that use api_type = agent can remain readable while the UI moves toward canonical agt.
• Projection builders can normalize legacy agent events to agt in api_version_t after the migration window. Event streams remain immutable, but new command writes should use only agt.
• Existing task contexts may carry either apiVersionId or agentDefId. Task utilities should normalize both values to the same ID.
• Documentation and form labels should say Agent API version id where the ID is exposed.
• Import/export and event replay should preserve event order for agent registration bundles.

Implementation Plan

1. Add Register AI Agent to portal-view/src/tasks/taskRegistry.ts.
2. Add help content under src/help/portal-view/tasks/register-ai-agent.md.
3. Ensure createApiVersion can be launched with apiType=agt.
4. Ensure form completion stores apiVersionId into task context.
5. Ensure createAgentDefinition can prefill agentDefId from apiVersionId.
6. Add server-side validation that agentDefId == apiVersionId.
7. Add compatibility handling for agt and agent API type values.
8. Add incomplete-registration detection and repair actions for agent API versions that do not have a matching agent definition.
9. Add secure-by-default invocation checks for agents with no explicit access policy.
10. Add integration tests for the two-event registration sequence.
11. Add a composite registerAiAgent command for API-version plus profile creation.

Resolved Recommendations

• Persist agt as the canonical API type after migration. Keep agent readable for replay, import, and old data, but reject new command writes using agent after the migration window.
• Put Register AI Agent under API Marketplace initially because the agent is registered as an API and should be discoverable through the API catalog. If a dedicated GenAI Assets category is added later, the task can move there without changing the backend model.
• Keep skill assignment optional. An agent can be useful as an LLM-only worker, and required skill assignment would block simple conversational agents.
• The first composite command should require an existing API and emit ApiVersionCreatedEvent plus AgentDefinitionCreatedEvent. Keep ApiCreatedEvent separate until automation needs a full create-everything command.

Decision Summary

Register AI agents through a Task Center flow that starts from API and API version registration. Create api_version_t first, then create agent_definition_t with agentDefId equal to apiVersionId. Use two domain events for the two required facts, keep projection writes behind event processing, reject mixed API-type families, treat incomplete version-only registrations as repairable but non-runnable, default runtime invocation to deny-all, and make the broader skill/tool/access setup optional follow-up steps.

OAuth Kafka

Token Exchange

This document outlines the design decisions and implementation details for supporting multiple token exchange flows in the oauth-kafka module.

Comparison of Detection Methods

When implementing token exchange (RFC 8693), the server must determine which identity provider (IdP) issued the subject_token to verify it correctly and map claims.

Implementation Strategy

Our implementation in ProviderIdTokenPostHandler uses Option 4: Client Context as the primary strategy:

1. Database-Driven Configuration: A new column token_ex_type has been added to the auth_client_t table to specify the supported exchange type for each client.

   ALTER TABLE auth_client_t ADD COLUMN token_ex_type VARCHAR(64);
   

2. Supported Exchange Types:
   • msal: Microsoft Authentication Library based exchange.
   • ccac: Client Credentials to Authorization Code exchange.
3. Flow Determination: Instead of relying on client-supplied parameters like requested_token_type, the server retrieves the token_ex_type from the client context in the database to decide which handler to use. This ensures that only authorized exchange types are performed for each specific client.

Recommendation

For the light-portal ecosystem:

• Option 4: Client Context is the selected method. It provides the highest level of security by ensuring that token exchange flows are explicitly configured and restricted on a per-client basis in the database.
• token_ex_type should be populated for any client that requires token exchange functionality. Clients without this configuration will not be allowed to perform token exchange.

Future Considerations

• Implement automated issuer discovery if the number of external providers grows.
• Support “opaque” token exchange by integrating with introspection endpoints of external IdPs.
• Extend the auth_client_t configuration to support multiple allowed exchange types per client if needed.

Light OAuth

Light OAuth IPv6 Support

Problem

The Rust light-oauth service binds its HTTPS listener to 0.0.0.0. That is correct for IPv4, but it does not accept connections sent to an IPv6 address.

In a dual-stack container network, Docker DNS can return the IPv6 address for light-oauth before the IPv4 address. A client that does not retry the next address can fail even though the service is healthy on IPv4. One observed failure is the gateway proxying /oauth2/{providerId}/code to https://light-oauth:6881 and receiving ECONNREFUSED on the IPv6 address.

Goals

• Allow light-oauth to bind IPv4, IPv6, or a specific interface from config.
• Keep the current default behavior as IPv4 wildcard binding.
• Avoid breaking existing deployments whose server.yml does not contain the new property.
• Build the listener address with SocketAddr so IPv6 addresses are parsed correctly.

Non-Goals

• Do not enable IPv6 for every deployment by default.
• Do not change TLS, OIDC, token, or database behavior.
• Do not change gateway upstream retry behavior in this change.

Configuration

light-oauth adds a server bind IP property:

ip: ${server_ip:0.0.0.0}

The default value remains:

server_ip: "0.0.0.0"

To listen on IPv6 wildcard:

server_ip: "::"

To listen on a specific IPv4 or IPv6 address:

server_ip: "172.16.1.3"

server_ip: "fdd0:0:0:1::3"

Implementation

The Rust config model includes an ip field with a default of 0.0.0.0. The default keeps old external server.yml files working.

The listener address is built as:

#![allow(unused)]
fn main() {
let ip = config.ip.parse::<IpAddr>()?;
let addr = SocketAddr::new(ip, config.port);
}

This avoids string formatting problems with IPv6 addresses. For example, :: plus port 6881 must become [::]:6881, not :::6881.

Deployment Guidance

Use IPv6 binding only when the runtime network is intentionally dual-stack and other services can reach the IPv6 address. In a container environment, confirm that:

• the container network has IPv6 enabled;
• the service has an IPv6 address;
• clients resolve or connect to the same address family;
• health checks cover the chosen address family.

For local or single-stack deployments, keep the default IPv4 binding.

Verification

For IPv4 default:

curl -k https://light-oauth:6881/oauth2/<providerId>/keys

For IPv6 wildcard binding in a dual-stack network, verify from another container:

curl -k -g https://[<light-oauth-ipv6>]:6881/oauth2/<providerId>/keys

If the client uses service DNS, verify that the first returned address family is reachable:

getent ahosts light-oauth
curl -k -v https://light-oauth:6881/oauth2/<providerId>/keys

Light Controller

Light Controller IPv6 Support

Problem

The Rust controller-rs service is the control-plane endpoint for runtime registration, discovery, MCP admin traffic, and portal event streams. In dual-stack deployments, clients can resolve the controller hostname to either IPv4 or IPv6. The controller listener and its TLS configuration must therefore support IPv6 without changing the existing IPv4 deployment defaults.

The important distinction is between socket addresses and service metadata:

• the controller listener uses a socket address such as 0.0.0.0:8438 or [::]:8438;
• registered runtime instances publish a host address string plus a separate port, such as fdd0:0:0:1::3 and 8443.

Current Behavior

controller-rs already stores its bind address as a Rust SocketAddr. The default remains IPv4 wildcard binding:

0.0.0.0:8438

The listener address can be overridden with CONTROLLER_ADDR. For IPv6, the value must use bracketed socket-address syntax:

CONTROLLER_ADDR='[::]:8438'

This value is used directly by both server modes:

• HTTPS/WSS mode uses axum_server::bind(settings.listen_addr).
• HTTP/WS mode uses tokio::net::TcpListener::bind(settings.listen_addr).

There is no string concatenation of IP and port in the controller listener path, so the common IPv6 failure form :::8438 is avoided.

Goals

• Support IPv4, IPv6, and dual-stack controller listener binding.
• Keep the current default as IPv4 wildcard binding.
• Keep the existing CONTROLLER_ADDR configuration contract.
• Accept runtime registration metadata that uses IPv4 literals, IPv6 literals, or DNS hostnames.
• Reject unspecified runtime registration addresses such as 0.0.0.0 and ::, because those are bind addresses, not reachable service addresses.

Non-Goals

• Do not enable IPv6 by default.
• Do not change controller WebSocket paths or JSON-RPC contracts.
• Do not change the runtime registration metadata shape.
• Do not add client-side IPv4 fallback in this change.

Listener Configuration

Default IPv4 listener:

CONTROLLER_ADDR='0.0.0.0:8438'

IPv6 wildcard listener:

CONTROLLER_ADDR='[::]:8438'

Specific IPv6 interface:

CONTROLLER_ADDR='[fdd0:0:0:1::10]:8438'

Specific IPv4 interface:

CONTROLLER_ADDR='172.16.1.10:8438'

The brackets are required only because CONTROLLER_ADDR is a full socket address. They separate the IPv6 literal from the port.

TLS Configuration

The controller starts with TLS enabled by default. IPv6 listener support does not remove the normal TLS hostname requirements.

For production, provide a certificate whose Subject Alternative Name covers the DNS name or IP address clients use to reach the controller:

CONTROLLER_TLS_CERT_PATH=/config/server.pem
CONTROLLER_TLS_KEY_PATH=/config/server.key
CONTROLLER_TLS_TRUST_CERT_PATH=/config/ca.pem

For generated local self-signed certificates, include any IPv6 literal that clients will use:

CONTROLLER_TLS_SERVER_NAME=localhost
CONTROLLER_TLS_ALT_NAMES='localhost,127.0.0.1,::1'

If clients connect by DNS name, prefer adding that DNS name to the certificate SAN and keep clients using the name instead of a raw IP literal.

Runtime Registration Metadata

Runtime services connect outbound to Light Controller and send service/register metadata. The registration address is not a socket address; the address and port are separate fields.

IPv6 registration metadata should use the raw IPv6 literal:

{
  "serviceId": "com.networknt.light-gateway-1.0.0",
  "protocol": "https",
  "address": "fdd0:0:0:1::3",
  "port": 8443
}

Do not use brackets in the address field:

{
  "address": "[fdd0:0:0:1::3]"
}

Brackets are only used when constructing URL authorities or socket addresses. The controller validates registration addresses as IP literals or DNS hostnames and rejects unspecified bind addresses such as 0.0.0.0 and ::.

Discovery Behavior

Discovery returns the registered address and port separately. Downstream clients, such as light-gateway, are responsible for building a reachable upstream authority. For IPv6 literals, clients must bracket the address when they construct a URL or host:port authority:

address = fdd0:0:0:1::3
port    = 8443
target  = [fdd0:0:0:1::3]:8443

Deployment Guidance

Only configure the controller with CONTROLLER_ADDR='[::]:8438' when the host, container network, Kubernetes Service, and ingress path are intended to accept IPv6 traffic.

In a dual-stack environment, verify all of these:

• the controller process is listening on IPv6;
• DNS returns the expected address family;
• TLS SANs cover the hostname or IP clients use;
• runtime services can open outbound WebSocket connections to the controller;
• registered service metadata publishes reachable addresses, not wildcard bind addresses.

Verification

From a peer in the same network:

getent ahosts <controller-host>
curl -k -g https://[<controller-ipv6>]:8438/health
curl -k -v https://<controller-host>:8438/health

For WebSocket clients, verify the same address family through the real endpoint:

wss://<controller-host>:8438/ws/microservice
wss://<controller-host>:8438/ws/discovery
wss://<controller-host>:8438/ctrl/mcp

If a TLS client connects by IPv6 literal and fails certificate validation, check the certificate SANs before changing the controller listener.

Agent Skill And API Endpoint Discovery

Problem

The GenAI chat flow has two separate concepts that are easy to confuse:

• The light-gateway MCP endpoint is the runtime server that lists and executes tools. An agent should call the gateway for tools/list and tools/call. A listed tool may be backed by a downstream MCP server or by a gateway-routed HTTP/OpenAPI endpoint.
• Portal-query is the catalog service for skills, tools, and agent assignments. The agent should read this catalog through the genai-query API, cache it locally, and search it during chat.
• The controller registry remains a runtime control-plane service for registration, discovery, and cache-management commands. It should not own the portal skill/tool catalog and should not execute downstream MCP or REST calls.

During chat, light-agent should use its local catalog cache to find relevant skills, then call tools/list on the gateway to verify executable tools. Tool execution still goes through the gateway. If the catalog cache is empty or stale, the agent should refresh it from portal-query. If portal-query is temporarily unavailable, the agent should still be able to use the gateway tool list directly.

The missing piece is a portal-managed catalog that explains which API endpoints exist, which endpoint projections are invokable by agents, which skills they belong to, and which agents are allowed or expected to use those skills. Without that catalog, the agent can list executable gateway tools, but it has no domain guidance beyond each tool description.

Goals

• Keep the gateway as the runtime source of truth for MCP tool execution.
• Keep direct gateway tools/list and tools/call working even when no skills have been authored.
• Treat API endpoints as the generic capability unit. MCP tools, OpenAPI operations, JSON-RPC methods, and future protocol operations should all become endpoint-level capabilities before they are exposed to agents.
• Populate a portal endpoint and tool catalog from API version parsing, LightAPI descriptions, gateway-discovered MCP tools, manually pasted MCP tools/list payloads, and gateway-routed REST tools.
• Let portal users create skills that contain instructions and curated tool selections.
• Let portal users assign skills to agent definitions.
• Use the genai-query API and spec as the portal-query access surface for skills, tools, and agent assignments.
• Let the agent cache the effective catalog locally and reload it when controller cache-management invalidation is triggered.
• Make skills useful for progressive disclosure without requiring every MCP tool to be wrapped before it can be called.
• Store semantic routing metadata for endpoint capabilities so the agent or portal-query can perform macro-filtering, keyword search, vector ranking, context viability checks, and safety filtering.

Non-Goals

• Do not move MCP request routing or downstream REST calls into the controller.
• Do not implement skill/search in controller-rs. Controller-rs can invalidate the agent cache, but portal-query owns catalog reads.
• Do not use config-server as the first delivery path for the skills/tools catalog. The agent can fetch from portal-query and cache locally.
• Do not require every gateway tool to have a skill before it is executable.
• Do not replace the existing MCP Gateway registry design. This design extends it with agent-facing skill curation.
• Do not implement embeddings in the first phase. Keyword search is enough for the initial local catalog search.
• Do not limit the catalog to MCP tools. The UI may use “tool” when referring to LLM tool-calling, but the persistent capability model should be endpoint first.
• Do not use skill assignments as the only authorization control. Gateway policy and downstream authorization still apply at execution time.

Concepts

Target Architecture

The target flow keeps runtime execution and control-plane metadata separate.

Portal UI
  -> writes api_endpoint_t, tool_t, tool_param_t, skill_t, skill_tool_t, skill_workflow_t, agent_skill_t

light-gateway /mcp
  -> lists executable tools from mcp-router.tools and downstream MCP servers
  -> executes tools/call against downstream MCP or REST services

light-workflow
  -> owns deterministic multi-step workflow execution, task state, and audit events

portal-query genai-query API
  -> serves skill/tool/agent-skill catalog reads from portal data

controller-rs portal registry
  -> registers agents and sends cache-management invalidation commands

light-agent
  -> loads assigned skills and mapped tools from portal-query
  -> caches the effective catalog locally
  -> searches cached skills during chat
  -> lists executable tools from light-gateway
  -> calls selected tools through light-gateway

For the account-agent example:

1. The gateway exposes account tools such as getAccounts and getAccountByNo.
2. Portal stores the canonical endpoint rows in api_endpoint_t.
3. Portal publishes selected endpoint rows into tool_t as agent-invokable capabilities.
4. An operator creates an “Account Management” skill in skill_t.
5. Portal links that skill to the account tools through skill_tool_t.
6. Portal assigns the skill to the account agent through agent_skill_t.
7. At startup or cache reload, the agent reads the assigned catalog through genai-query and caches it locally.
8. At chat time, the agent searches its local catalog cache.
9. The agent combines matched skill instructions with the gateway tool definitions.
10. Any tool call still goes to light-gateway tools/call.

Source Of Truth

The gateway is the runtime source of truth for executable tools. If a tool is not available from the gateway, the agent should not be able to execute it just because it exists in the portal database.

api_endpoint_t is the canonical portal endpoint catalog. It stores the endpoint identity, protocol method, path, logical tool schema, endpoint description, and raw tool metadata for one API version.

tool_t is the agent-facing projection of an endpoint. It stores the tool name, agent description, implementation type, optional endpoint reference, response schema, active flag, semantic routing fields, and semantic embedding. The full metadata object should still be preserved in api_endpoint_t.tool_metadata for import/export and agent cache payloads.

The portal database is the control-plane catalog. It stores:

• operator-friendly descriptions,
• skill instructions,
• agent assignments,
• governance metadata,
• cached or imported tool schemas.

Tool sync should be idempotent. The recommended unique identity is:

host_id + api_version_id + endpoint

Gateway exposure is a separate deployment selection. The catalog should sync all endpoint rows for an API version, then let the user choose which endpoint/tool projections are deployed to a specific gateway instance.

For runtime-executable projections, the gateway identity is:

hostId + serviceId + envTag

The access token used for portal catalog or gateway deployment APIs should carry matching host, sid, and env claims. Portal-query must verify those claims against the requested hostId, serviceId, and envTag before returning or changing catalog data.

Runtime verification means checking whether an endpoint projection is actually listed by a deployed gateway through tools/list. This should be done against the selected gateway instance when an operator is preparing or reviewing a gateway deployment. A later host-wide diagnostics view can aggregate all registered gateways, but phase 2 does not need host-wide verification as the default.

Runtime verification is not part of the persistence projection. The persistence layer should store catalog state, endpoint/tool metadata, and inactive drift state, but it should not call a live gateway. The portal UI, deployment review flow, or a diagnostics endpoint should call the selected gateway’s tools/list with the operator or service credential, compare the returned tool names and schemas with the catalog, and surface the result as deployment drift.

If a previously imported endpoint or tool disappears from the gateway, the sync process should mark the catalog projection inactive instead of deleting it immediately. This preserves skill mappings and gives operators a clear drift signal.

Current Data Model

The database already has the main tables needed for this design:

• skill_t: skill name, description, content_markdown, embedding placeholder, version, and active flag.
• tool_t: agent-facing tool catalog with name, description, implementation metadata, endpoint reference, and response schema.
• tool_param_t: parameter-level metadata and validation schema.
• agent_skill_t: maps agent definitions to skills.
• skill_tool_t: maps skills to tools for progressive disclosure.
• api_endpoint_t: MCP or REST endpoint metadata, including tool_schema and tool_metadata.
• wf_definition_t: stores workflow definitions as YAML for the light-workflow runtime.

Phase 3.5 should add a skill-to-workflow mapping table rather than storing workflow YAML inside skill_t. The recommended table is:

The current phase 2 persistence path can preserve semantic metadata in api_endpoint_t.tool_metadata before dedicated routing columns exist. That is acceptable for import/export compatibility and for small catalogs searched from the agent’s local cache. It should not be treated as the final indexed search shape. Before portal-query performs database-side macro-filtering over large catalogs or before vector ranking becomes a production dependency, promote the high-use routing fields to first-class columns or indexed relationships and backfill them from tool_metadata.

The existing MCP Registry design already maps MCP tools into api_endpoint_t. OpenAPI parsing also creates endpoint rows. This design uses tool_t as the agent-facing catalog row and links it back to api_endpoint_t when the tool originates from an API endpoint.

Recommended mapping for gateway-imported tools:

tool_t.implementation_type should be a standardized enum aligned with the LightAPI Description execution model. Endpoint-backed tools should use a LightAPI endpoint implementation type rather than preserving every downstream transport as a different tool implementation. The downstream protocol remains in the endpoint and LightAPI request metadata.

Recommended first enum values:

For lightapi_endpoint, execution still goes through gateway tools/call when the endpoint is exposed to a gateway. The source protocol, such as MCP, OpenAPI, JSON-RPC, OpenRPC, or gRPC, belongs in api_endpoint_t, tool_metadata, and the LightAPI request description.

Endpoint-First Capability Model

Agents and skills should operate over endpoint capabilities, not only over MCP tools. MCP remains the runtime protocol for tool-calling through the gateway, but the catalog should support any endpoint that can be represented as an agent-invokable capability.

Recommended capability layers:

1. api_endpoint_t: canonical endpoint row for the API version.
2. tool_t: agent-facing executable projection of the endpoint.
3. tool_param_t: normalized top-level input parameters derived from the endpoint’s JSON Schema.
4. skill_tool_t: curated relationship between a skill and a tool projection, including per-skill overrides such as priority, examples, or approval notes.
5. agent_skill_t: assignment of skills to agent definitions.

This model supports these source types:

tool_param_t should be generated from the logical input schema, not from wire transport details alone. For OpenAPI, the logical input schema should merge path parameters, query parameters, and request body into one object. For MCP, the logical input schema is the MCP inputSchema. For JSON-RPC, it is the logical params schema.

Semantic Routing Metadata

The customer-required semantic routing fields should be first-class indexed catalog data, not only JSON metadata. They are used for macro-filtering before expensive keyword, vector, or LLM ranking, so the common filter fields must be queryable through normal portal-query indexes.

Recommended indexed fields or relationships:

• domain and semantic namespace,
• sensitivity tier,
• semantic weight,
• target personas,
• active state,
• source protocol and implementation type,
• portal category and tag relationships.

Recommended phase 2 column names for endpoint and tool projections:

The full structured payload should still be preserved in api_endpoint_t.tool_metadata so LightAPI import/export, gateway config generation, and agent cache payloads have one portable metadata object.

Recommended api_endpoint_t.tool_metadata shape:

{
  "routing": {
    "domain": "finance.accounts",
    "category": "account-management",
    "semanticNamespace": "prod.accounts.core",
    "targetPersonas": ["account-agent", "customer-support-agent"],
    "semanticDescription": "Retrieves account profile and status information when a user asks about an existing account.",
    "semanticKeywords": ["account lookup", "customer account", "balance", "status"],
    "contextRequirements": {
      "requiredInputs": ["accountNo"],
      "requiredContext": ["host_id"]
    },
    "dependencies": [
      {
        "endpoint": "/v1/accounts/{accountNo}@get",
        "relation": "frequently_chained_after"
      }
    ],
    "semanticWeight": 0.75,
    "sensitivityTier": "Internal-Only",
    "fallbackEndpoint": "/v1/accounts@get",
    "embedding": {
      "model": "tool-description-embedding",
      "source": "semanticDescription"
    }
  },
  "safety": {
    "read_only": true,
    "destructive": false,
    "humanApprovalRequired": false
  }
}

Recommended ownership:

The first semantic search implementation can work from the agent’s local cache:

1. Filter by host, active flag, assigned skill, domain, namespace, target persona, and sensitivity tier.
2. Exclude endpoints whose required context is not available in the current workflow or chat state.
3. Rank by keyword matches over skill text, endpoint name, tool name, description, semantic keywords, and LightAPI capability text.
4. When embeddings are populated, combine vector similarity with the keyword score and multiply by semanticWeight.
5. Call gateway tools/list and intersect the ranked set with currently executable tools before exposing schemas to the LLM.

Embedding Recommendation

Keep the first production embedding dimension at 384 because the current Postgres vector column is already VECTOR(384) and the first catalog use case is routing over short endpoint descriptions, not long document retrieval.

Recommended model strategy:

• Use a provider abstraction with configured embedding_model, embedding_dimension, and embedding_source.
• For OpenAI-hosted embeddings, use text-embedding-3-small with the dimensions parameter set to 384.
• For on-prem or firewall-restricted deployments, use a local embedding service that is configured to emit 384-dimensional vectors.
• Store enough metadata to know how a vector was created: model, dimension, source text hash, source field, and generated timestamp.
• Re-embed when the semantic description, keywords, domain, or model config changes.

The portal catalog write path should remain in the portal service layer that owns api_endpoint_t and tool_t persistence. Because the current portal command/query services are Java, the Java side should own transactions, versioning, and persistence of embedding results. A Rust service or worker can still generate embeddings behind an internal API or queue consumer, especially if local model performance is better there. In that model, Java requests or consumes the vector and writes it through the normal portal persistence path.

LightAPI Description Enrichment

LightAPI Description should be the preferred enrichment source for endpoint capabilities. OpenAPI and MCP tools/list are good at initial extraction, but LightAPI adds the agent-oriented context needed for high-accuracy routing:

• endpoint identity and stable endpointId
• domain, tags, lifecycle, visibility, and capability group
• logical input schema and request mapping
• result schema and result cases
• examples and behavior notes
• progressive disclosure metadata
• agent-facing descriptions, personas, keywords, context requirements, and guardrails

Recommended merge priority for endpoint metadata:

1. Portal operator overrides.
2. Endpoint-level LightAPI Description.
3. API-level inherited LightAPI Description context.
4. OpenAPI/OpenRPC/protobuf/MCP source extraction.
5. Gateway runtime tools/list discovery.

This keeps runtime discovery useful while letting curated LightAPI descriptions provide richer semantic routing without hand-authoring every endpoint as an independent skill.

Phase 2 persistence should be treated as the receiver for this metadata, not as the extractor. The openapi-parser, a LightAPI Description parser, or a dedicated ingestion worker must emit the enriched endpoint payload on the API version event. At minimum, the event payload for each endpoint should include:

• endpointId, endpoint identity, protocol, method, path, name, and description,
• logical toolSchema generated from the LightAPI operation input contract,
• toolMetadata.routing with namespace, domain, capability group, personas, keywords, context requirements, sensitivity tier, and semantic weight where present,
• toolMetadata.safety from LightAPI safety, visibility, idempotency, and destructive-operation hints,
• response schema or result metadata when it is available for the tool projection.

If the parser only emits the base OpenAPI or MCP fields, the catalog remains valid but only has low-enrichment metadata. The phase 2 implementation should record that as an ingestion gap, not as a persistence defect.

Portal Catalog Contract

The agent should read skills and tools through the genai-query API in portal-query. The source spec is:

genai-query/src/main/resources/spec.yaml

The current spec already includes catalog endpoints for the main entities:

• getAgentSkill and getFreshAgentSkill
• getSkill and getFreshSkill
• getSkillTool and getFreshSkillTool
• getSkillDependency and getFreshSkillDependency
• getTool and getFreshTool
• getToolParam and getFreshToolParam

Phase 2 should add a dedicated effective catalog endpoint instead of forcing the agent to compose many generic query endpoints. The endpoint should still live in genai-query, not controller-rs.

Recommended endpoint behavior:

• verify the caller’s token claims before reading catalog rows,
• require request host_id, service_id, and env_tag,
• match token host, sid, and env claims to those request values,
• return only endpoint/tool projections valid for that host, service, and environment,
• include active endpoint metadata, tool schemas, safety metadata, routing metadata, and skill mappings relevant to the agent,
• support a freshness or version field so the agent can cache the result.

The agent should cache the returned structure locally:

{
  "host_id": "00000000-0000-0000-0000-000000000000",
  "agent_def_id": "00000000-0000-0000-0000-000000000000",
  "catalog_version": 42,
  "skills": [
    {
      "skill_id": "00000000-0000-0000-0000-000000000000",
      "name": "Account Management",
      "description": "Use account tools to inspect and manage customer accounts.",
      "content_markdown": "Prefer read-only tools before create or update tools.",
      "tools": [
        {
          "tool_id": "00000000-0000-0000-0000-000000000000",
          "endpoint_id": "00000000-0000-0000-0000-000000000000",
          "name": "getAccounts",
          "endpoint": "/v1/accounts@get",
          "api_type": "openapi",
          "description": "List account summaries.",
          "input_schema": {
            "type": "object",
            "properties": {}
          },
          "routing_metadata": {
            "domain": "finance.accounts",
            "semanticNamespace": "prod.accounts",
            "semanticKeywords": ["account list", "customer accounts"],
            "sensitivityTier": "Internal-Only"
          },
          "safety": {
            "read_only": true,
            "destructive": false
          }
        }
      ]
    }
  ]
}

For phase 2, the agent definition identity is the agent API version identity. agent_definition_t.agent_def_id stores the same UUID as api_version_t.api_version_id; the table is an agent-specific profile extension for model and runtime settings, not a second standalone agent registry. The agent display name comes from api_t.api_name, so agent_definition_t does not duplicate the API name. API Admin continues to own the API/API-version lifecycle, Instance Admin continues to own deployed instances, and the Agent Definition page edits the profile for that API version.

The previous registry skill/search response shape was:

{
  "skills": [
    {
      "skill_id": "00000000-0000-0000-0000-000000000000",
      "name": "Account Management",
      "description": "Use account tools to inspect and manage customer accounts.",
      "tool_name": "getAccounts",
      "input_schema": {
        "type": "object",
        "properties": {}
      }
    }
  ]
}

That flattened shape can remain as an internal compatibility DTO while the agent is migrated, but it should not be the long-term external contract. The target cache shape should support a skill with multiple tools:

{
  "skills": [
    {
      "skill_id": "00000000-0000-0000-0000-000000000000",
      "name": "Account Management",
      "description": "Use account tools to inspect and manage customer accounts.",
      "content_markdown": "Prefer read-only tools before create or update tools.",
      "tools": [
        {
          "name": "getAccounts",
          "description": "List account summaries.",
          "input_schema": {
            "type": "object",
            "properties": {}
          }
        }
      ]
    }
  ]
}

Migration rule:

• Remove the controller-rs skill/search placeholder.
• The agent can temporarily accept both the flattened shape and the nested tools shape while its portal-query client is being migrated.
• After migration, the nested effective catalog shape becomes the preferred internal cache contract.

Agent identity can come from token claims, configured agent definition, or request fields. If inference is not enough, pass explicit fields to the portal-query catalog call:

{
  "agent_def_id": "00000000-0000-0000-0000-000000000000",
  "host_id": "00000000-0000-0000-0000-000000000000",
  "service_id": "com.networknt.account-agent-1.0.0",
  "env_tag": "dev"
}

Runtime Behavior

The agent should treat the portal catalog as helpful guidance, not as a hard dependency for basic tool use.

Recommended behavior:

1. At startup, call the genai-query API to load the effective agent catalog.
2. Cache the catalog locally under host_id, agent identity, and catalog version.
3. During chat, search the local catalog with the user prompt.
4. If matched skills are returned, add skill instructions to the prompt context.
5. If matched skills include tool mappings, prefer those tools for the LLM tool list.
6. Call gateway tools/list to verify executable tools and obtain the current runtime schemas.
7. Intersect skill-selected tool names with gateway-listed tools.
8. If no skills match, or the local catalog is unavailable, fall back to gateway tools/list.
9. Execute all LLM tool calls through gateway tools/call.

When portal data changes, controller cache management can invalidate the agent’s local catalog cache. Reload behavior should match the agent’s initial loading strategy:

• if the agent loads the catalog during startup, invalidation should trigger an eager reload so the next chat request sees current metadata;
• if the agent loads the catalog on the first request, invalidation can clear the cache and let the next request reload lazily.

This keeps the account-agent usable before the portal skill catalog is fully populated and avoids making controller-rs part of the catalog query or execution path.

Portal UI

Endpoint Catalog And Tool Projection

The catalog UI should be endpoint-first but still show the tool projection that agents will see. It should let operators:

• browse api_endpoint_t rows by API, API version, endpoint, method, source, and active state,
• import or resync endpoint capabilities from OpenAPI, MCP tools/list, manually pasted MCP tools payloads, LightAPI descriptions, and selected gateway runtime surfaces,
• publish selected endpoint rows into tool_t as agent-invokable tools,
• generate or refresh tool_param_t rows from the logical input schema,
• see tool name, description, input schema, downstream endpoint, API type, semantic namespace, domain, personas, sensitivity tier, and runtime executable state,
• compare catalog metadata against source specs and current gateway tools/list,
• mark missing endpoint projections inactive,
• override operator-facing descriptions without changing gateway config,
• review and edit semantic routing metadata such as keywords, context requirements, fallback endpoint, priority weight, read-only, destructive, sensitive, or human-approval-required.

The first implementation should not depend only on live gateway access. It can import from the endpoint rows produced by API version parsing, including manual MCP tools/list JSON pasted into the API version spec field. Gateway tools/list should then be used to verify which imported projections are currently executable by a deployed gateway.

Skill Editor

The Skill Editor should let operators:

• create and update skill_t rows,
• write content_markdown instructions,
• link tools through skill_tool_t,
• set tool access level and per-skill config,
• preview which tools the skill would expose for a sample prompt,
• optionally link the skill to one or more workflow definitions,
• activate or deactivate skills.

Skill content should be short and operational. It should describe when to use the skill, how to interpret the tools, and any sequencing rules. It should not contain secrets.

Workflow-backed Skills

Some skills are only guidance plus a curated tool set. Other skills need a repeatable process that calls several tools, branches on results, waits for human input, runs assertions, or leaves an audit trail. Those skills should use light-workflow as the orchestration layer.

The boundary is:

Workflow-backed skills should be optional. Use a workflow when the skill represents a durable or regulated process, such as API onboarding, approval, validation, remediation, scheduled live testing, or a multi-step operation with clear checkpoints. Do not require workflow backing for simple skills that only guide an agent toward one tool call or open-ended exploration.

The workflow definition remains canonical in wf_definition_t.definition as YAML. The skill workspace should link to the definition through skill_workflow_t and should reuse the generic workflow editor described in Workflow Editor. The skill workspace can constrain the editor with skill context, but it should not implement its own workflow runtime.

For workflow-backed skills, skill_tool_t becomes the allowed tool set. A save-time validator should reject workflow steps that reference a gateway tool not linked to the skill, unless the step is explicitly marked as a future or external dependency. This keeps progressive disclosure, operator review, and workflow execution aligned.

Recommended Skill Workspace tabs:

Agent Skill Assignment

The Agent Skill Assignment UI should let operators:

• select an agent definition,
• assign one or more active skills through agent_skill_t,
• set priority and sequence,
• preview the final skill list for that agent,
• verify that each assigned skill still has at least one executable gateway tool.

Portal-query And Agent Cache Implementation

Catalog lookup should be implemented through the genai-query API. The agent should fetch the assigned active catalog, cache it locally, and run progressive disclosure search against the cache.

Phase 5 implements this for the Rust light-agent only. Other agent runtimes can adopt the same genai-query contract later, but they are not part of the Phase 5 implementation scope.

Initial algorithm:

1. Resolve host_id from the agent runtime configuration and the catalog request token. Resolve agent_def_id from LIGHT_AGENT_AGENT_DEF_ID or LIGHT_AGENT_API_VERSION_ID. Resolve service_id and env_tag from the registered Rust agent service config.
2. Call genai-query getEffectiveAgentCatalog.
3. The endpoint loads active agent_skill_t rows and linked active skill_t, skill_tool_t, tool_t, tool_param_t, and skill_workflow_t rows.
4. Build a nested effective catalog grouped by skill, with each skill carrying its mapped tools, schemas, endpoint identity, safety flags, and routing metadata.
5. Cache the effective catalog locally with catalogVersion and catalogHash.
6. During chat, macro-filter cached entries by agent persona, domain, namespace, sensitivity tier, active state, and available workflow context.
7. Rank cached entries by simple text matching over skill_t.name, skill_t.description, skill_t.content_markdown, tool_t.name, tool_t.description, endpoint name, endpoint description, and semantic keywords.
8. Intersect the final candidate list with gateway tools/list before exposing tool schemas to the LLM.

Controller cache management should invalidate this local cache when portal catalog data changes. After invalidation, the agent reloads from portal-query.

Later algorithm:

• Add vector search over skill_t.description_embedding and tool_t.description_embedding.
• Add vector search over endpoint semantic descriptions and LightAPI capability text.
• Include skill dependency expansion from skill_dependency_t.
• Use dependency mappings and fallback endpoints for chain planning, prefetch, and failure repair.
• Include inactive or missing-tool diagnostics for portal admin views, not for normal agent search.

Gateway Implementation

The gateway should keep the MCP data-plane contract stable:

• tools/list returns the executable tool set for the caller.
• tools/call routes by tool name to downstream MCP servers or REST services.
• Gateway policy remains authoritative at execution time.
• Gateway does not depend on skill_t or agent_skill_t to execute tools.

The gateway can expose an administrative sync endpoint later, but the first portal sync can call the existing MCP tools/list endpoint with an operator or service credential.

mcp-router.tools in values.yml should stay a runtime execution projection, not the full semantic registry. It should include the fields the gateway needs to list and call tools, plus safety metadata that must be enforced at runtime. Richer semantic routing metadata should stay in portal-query and the agent cache unless the gateway needs it for a concrete runtime policy decision.

Security Rules

• Skill assignment narrows what the agent should offer to the LLM, but it does not grant runtime authorization by itself.
• Gateway access control, endpoint scopes, OAuth token claims, and downstream service authorization still decide whether a tool call is allowed.
• Tool schemas and descriptions are not trusted input. They should be validated before storing and escaped when rendered.
• Skill content must not contain secrets, tokens, private keys, or passwords.
• A stale catalog row must not make a removed gateway tool executable.
• A stale local agent cache must be intersected with gateway tools/list before exposing tools to the LLM.
• Controller cache invalidation only forces reload; it does not grant access to catalog rows or executable tools.
• Sensitive or destructive tool metadata should be enforced by the gateway or a policy layer, not only by prompt instructions.
• Sensitivity tier must be checked before catalog disclosure. An agent without clearance for Restricted-PII should not receive the endpoint description or schema even if a skill references it.
• Context requirements are not only prompt hints. If required context is missing, the endpoint should be excluded or routed to an ask/workflow step that obtains the missing value.

Failure Handling

Phased Implementation

Phase 1: Preserve Direct MCP Baseline

• Keep agent tool execution through gateway tools/call.
• Remove the controller-rs skill/search placeholder before it becomes a dependency.
• Ensure agent falls back to gateway tools/list when no catalog cache is available.
• Keep direct gateway tools/list and tools/call working without portal skills.

Phase 2: API Endpoint Catalog Sync

• Add portal UI for endpoint-first import and resync.
• Use existing API version parsing to populate api_endpoint_t for OpenAPI and MCP tools, including manual MCP tools/list payloads accepted in the API version spec field.
• Sync all endpoint rows for the API version into the endpoint catalog. Do not limit the catalog to the endpoints currently selected for one gateway instance.
• Import or refresh LightAPI Description metadata for endpoint enrichment.
• Publish selected endpoint rows into tool_t as agent-facing tool projections.
• Generate tool_param_t from each endpoint’s logical input schema.
• Link every API-origin tool projection back to api_endpoint_t.endpoint_id.
• Store semantic routing metadata in indexed endpoint/tool fields and preserve the full metadata payload in api_endpoint_t.tool_metadata.
• If the first code slice only writes tool_metadata, keep that as a compatibility step and add the indexed routing-column migration before database-side macro-filtering or production vector ranking is enabled.
• Let users select which endpoint projections should be exposed to a specific gateway instance. This deployment selection is separate from endpoint catalog sync.
• Verify runtime executability outside persistence with gateway tools/list for the selected gateway instance when a gateway is reachable.
• Mark disappeared or non-executable projections inactive instead of deleting them.
• Add drift indicators for schema, description, safety metadata, and semantic routing metadata changes.

Phase 3: Skill Authoring

• Keep the existing skill_t CRUD page as the phase 3 authoring surface.
• Add skill-scoped category and tag assignment to the create/update skill forms. The UI should use dropdowns populated from the existing portal taxonomy where entity_type = 'skill'.
• Persist skill categories through entity_category_t and skill tags through entity_tag_t; do not add tags or categories columns to skill_t.
• Implement skill save as a composite command: one event updates the skill row and one taxonomy event replaces the selected category/tag associations for the same skill.
• Keep content_markdown as the instruction body. YAML or JSON skill files are import/export envelopes; if full structured skill authoring is introduced later, add a nullable JSONB skill-spec column beside content_markdown instead of replacing it.
• Keep embeddings optional.

Phase 3.5: Skill Workspace And Structured Authoring

• Add a richer Skill Workspace with Overview, Tools, Workflow, Preview, and Test tabs.
• Add tool linking workflows for skill_tool_t and formalize skill_tool_t.config for per-skill tool overrides.
• Add workflow-backed skill support through skill_workflow_t, with wf_definition_t.definition kept as the canonical workflow YAML.
• Reuse the generic Workflow Editor in the Workflow tab for YAML editing, step preview, validation, and test runs.
• Add validation that workflow tool-call steps reference tools linked to the skill through skill_tool_t.
• Add “create skill from LightAPI/tool” flows that can generate a draft skill, link relevant tools, and optionally create a starter workflow definition.
• Add YAML/JSON import/export for structured skill documents. Normalize YAML to JSON for storage when a persisted structured payload is needed, while keeping Markdown instructions in content_markdown.

Phase 4: Agent Assignment

• Add portal UI for agent_skill_t.
• Let operators assign active skills to agent definitions.
• Add an Agent Definition assignment entry point in addition to the existing agent_skill_t table page, so operators can manage assigned skills from the agent context.
• Add a batch assignment composite command that emits one AgentSkillCreatedEvent per selected skill.
• Add validation that assigned skills have at least one active direct skill_tool_t link. A workflow-backed skill does not satisfy this by having only skill_workflow_t; the workflow must use the skill’s linked tools.
• Enforce assignment validation in command handlers and mirror the same checks as UI preflight feedback.
• Treat sequence_id as the deterministic effective prompt/display order and priority as a ranking weight for later catalog/search behavior.

Phase 5: Real Skill Search

• Add the dedicated genai-query getEffectiveAgentCatalog endpoint with token verification against host, sid, and env claims.
• The endpoint returns the active nested catalog for one hostId + agentDefId + serviceId + envTag: agent metadata, assigned skills, tags, categories, skill config, mapped tools, tool params, routing/safety fields, workflow references, catalogVersion, and catalogHash.
• Implement the Rust light-agent portal-query client using that endpoint.
• Build and cache the nested effective catalog inside the Rust agent.
• Start with local macro-filtering and keyword matching over cached skills, endpoint metadata, and tool projections.
• Intersect selected catalog tool names with gateway tools/list; execute only through gateway tools/call.
• Wire controller cache-management invalidation to clear the Rust agent catalog cache. The next chat request lazily reloads from portal-query.
• If portal-query is unavailable or no agent definition ID is configured, the Rust agent falls back to direct gateway tools/list without portal catalog filtering.
• Add vector ranking after 384-dimensional embeddings are populated and combine it with semanticWeight.

Phase 6: Semantic Routing And Governance

• Support the Rust light-agent only. Other agent runtimes can adopt the same catalog and diagnostics contracts later.
• Use the normalized sensitivity tiers public, internal, confidential, and restricted. Treat missing or unknown tool tiers as internal.
• Enforce sensitivity-tier disclosure before portal-query returns the effective catalog to the agent. Tools blocked by policy are omitted from the returned tools list and surfaced as diagnostics for admin review.
• Block destructive or approval-required tools unless the skill/tool policy names an approval workflow. Until workflow-owned approval state exists, the current active row plus aggregate version remains the catalog versioning authority.
• Keep gateway tools/list and tools/call as the runtime source of truth. The Rust agent must still intersect catalog-selected tools with live gateway tools/list.
• Add Rust-agent diagnostics that compare the effective catalog against gateway tools/list at /diagnostics/tools, showing catalog tools missing from the gateway, gateway tools outside the catalog, and policy-blocked catalog tools.
• Enforce the same destructive, approval-required, and sensitivity metadata at the gateway before tools/call execution. A blocked call should include auditInfo fields and gateway debug/warn logs with the tool name, endpoint, tier, policy reason, and approval state.
• Do not write catalog-disclosure audit records into audit_log_t; it is reserved for workflow. Phase 6 uses auditInfo so the existing audit log file path captures blocked gateway decisions. A generic audit table can be added in a later governance phase if file logging is not enough.

Resolved Phase 2 Decisions

• Phase 2 endpoint catalog sync covers all endpoint rows for an API version. Gateway exposure is a separate step where users select which endpoint/tool projections to deploy to a specific gateway instance.
• Runtime verification means checking the selected gateway instance’s tools/list response to confirm that a deployed endpoint projection is executable there. It is not the same as endpoint catalog sync and should be implemented in the portal UI, deployment review flow, or diagnostics layer, not inside the persistence projection.
• Gateway exposure identity is hostId + serviceId + envTag. The token used for portal APIs must carry matching host, sid, and env claims.
• tool_t.implementation_type should be standardized and aligned with the LightAPI Description execution model. Endpoint-backed tools should use the standardized endpoint implementation type, with downstream protocol stored in endpoint and LightAPI metadata.
• High-use semantic routing fields should be indexed columns or indexed relationships, with the full structured payload preserved in api_endpoint_t.tool_metadata. JSON-only persistence is only an interim import/export-compatible shape for small catalogs or local-cache search.
• LightAPI Description enrichment requires an upstream parser or ingestion worker to emit enriched endpoint payloads. The persistence layer can store tool_schema, tool_metadata.routing, and tool_metadata.safety, but it does not derive those fields from the raw LightAPI document by itself.
• Endpoint category and tag classification should reuse the existing portal tag and category system.
• Embeddings should start at 384 dimensions to match the current VECTOR(384) schema. Use a provider abstraction so hosted OpenAI embeddings or local embedding services can be swapped without changing the catalog schema.
• genai-query should expose a dedicated effective catalog endpoint. Its token verification must match request host_id, service_id, and env_tag against token host, sid, and env claims.
• Cache reload behavior depends on the loading strategy. Startup-loaded catalogs should eagerly reload after invalidation. First-request-loaded catalogs can reload lazily on the next request.
• Phase 2 focuses on tool and endpoint metadata. Skill-specific metadata and per-skill tool config should be designed later with the skill authoring phase.
• Phase 3 uses the existing taxonomy join tables for skill tags and categories. Skill files may be YAML or JSON, but the database should keep content_markdown for the instruction body; a structured JSONB skill-spec column belongs in a later full authoring/import phase if it becomes needed.

Resolved Phase 3.5 Decisions

• Use light-workflow for workflow-backed skills that need durable multi-tool orchestration, approvals, assertions, retries, scheduled tests, or audit history.
• Do not force every skill into a workflow. Skills remain the discovery and guidance layer, and simple skills can stay instruction-and-tool based.
• Keep light-gateway as the runtime tool execution path. Workflow tasks that call tools should still use gateway-visible tool identities and should not bypass gateway policy.
• Keep workflow definitions in wf_definition_t.definition as YAML. Link skills to workflow definitions through skill_workflow_t instead of embedding workflow definitions in skill_t.
• Treat skill_tool_t as the allowed tool set for workflow-backed skills. Save-time validation should flag workflow tool calls that are not linked to the skill.
• Build the workflow authoring UI as a generic reusable editor first, then embed it inside the Skill Workspace with skill-aware reference filtering and validation.

Resolved Phase 4 Decisions

• An assignable skill must be active and must have at least one active direct tool link through skill_tool_t. Active skill_workflow_t rows are useful orchestration metadata, but they do not replace the direct allowed-tool set.
• Workflow-backed skill assignment should also rely on the Phase 3.5 validator: workflow tool-call references must resolve to tools linked through skill_tool_t.
• Validation must be enforced server-side by createAgentSkill, updateAgentSkill, and the batch assignment composite command. The Portal UI should run the same checks as preflight feedback, but UI checks are not authoritative.
• Keep the existing AgentSkill table page and add an Agent Definition assignment context so operators can assign and inspect skills from the agent they are configuring.
• Batch assignment should be a composite command that creates multiple AgentSkillCreatedEvent events from one request.
• sequence_id controls deterministic ordering when building the agent’s effective skill prompt/catalog. priority is reserved as a ranking weight for later effective-catalog and search behavior.
• Live gateway runtime executability checks are not part of Phase 4 persistence validation. Keep them as a diagnostics or governance item that compares cataloged/assigned tools with the selected gateway instance’s tools/list response before deployment or runtime enablement.

Recommendation

Implement this as a progressive control-plane enhancement. The gateway remains the execution path, and portal-authored skills become the agent guidance layer served by portal-query. The agent should cache the effective catalog locally and reload it after controller cache-management invalidation. This lets MCP tools work immediately through tools/list and tools/call, while still giving portal operators a clean path to organize tools into skills, assign those skills to agents, and improve retrieval over time.

Agent API Compose And Multi-Agent Workflow

Problem

portal-config-loc and portal-config-dev are being updated so local Docker Compose stacks can run light-agent beside the portal services, demo APIs, and gateway. The first implementation adds one light-agent directly to the main compose file and points the gateway direct registry at http://light-agent:8083.

That works for a single account agent, but it does not scale cleanly for the next phase:

• the base portal stack should remain usable without demo APIs or agents,
• demo APIs and agents should be startable as an optional local package,
• all services must still share the same Docker network,
• multiple light-agent instances need unique runtime identities,
• each agent needs a different effective skill/tool/workflow catalog,
• workflows need to orchestrate API access across multiple specialized agents.

The design goal is to split deployment concerns without splitting the runtime network or the control-plane model.

Goals

• Move the two demo APIs and local light-agent services into a separate Docker Compose overlay file.
• Keep the overlay on the same Docker network as the portal stack, gateway, controller, config-server, Postgres, and hybrid services.
• Support multiple light-agent containers from the same image with different service ids, advertised addresses, ports, model settings, agent definitions, skills, tools, and workflows.
• Keep light-gateway as the MCP runtime path for tools/list and tools/call.
• Keep portal-query as the source for the effective agent catalog.
• Use skill_workflow_t and wf_definition_t to connect skills to executable workflows.
• Use light-workflow for deterministic orchestration, retries, human tasks, assertions, audit, and multi-agent coordination.

Non-Goals

• Do not move the demo APIs or agent services into a different Compose project by default.
• Do not create a second Docker network for the demo APIs and agents.
• Do not make one agent container host multiple unrelated agent definitions.
• Do not move MCP tool execution into controller-rs or portal-query.
• Do not require every gateway tool to be wrapped by a skill before baseline gateway tool execution works.
• Do not store workflow definitions inside skill_t.

Compose File Split

Use the main compose files for platform services and a separate overlay for demo APIs plus agents.

Recommended files:

The base stack should own shared infrastructure:

• Postgres,
• config-server,
• controller,
• hybrid-query,
• hybrid-command,
• light-gateway,
• light-workflow,
• OAuth and other platform services.

The overlay should own optional local workloads:

• demo-customer-profile-api,
• demo-offer-decision-api,
• light-agent-account,
• light-agent-offer,
• future specialized agents.

The overlay is intended to be started with the base files in the same Compose command. In that mode Docker Compose creates or reuses one project default network, and every service can resolve every other service by service name.

Example for portal-config-dev:

docker compose \
  -f docker-compose.yml \
  -f docker-compose.agent-api.yml \
  up -d

Example for portal-config-loc/all-in-pg:

docker compose \
  -f docker-compose.yml \
  -f docker-compose-rust.yml \
  -f docker-compose.agent-api.yml \
  up -d

If the overlay must be started separately, it must still use the same Compose project name as the base stack. Otherwise Docker will create a second default network and the gateway will not resolve the agent and demo API service names.

Network Contract

The preferred local contract is the Compose default network for the active project. Do not declare a separate network in the overlay when the overlay is run with the base stack.

Service-to-service URLs should use Compose service DNS names:

http://light-agent-account:8083
http://light-agent-offer:8083
http://demo-customer-profile-api:8080
http://demo-offer-decision-api:8080

Host port mappings are only for browser or curl access from the developer machine. They should not be used by gateway, agents, workflows, or demo APIs to call each other.

For agent containers, use a stable service name and advertised address:

server.advertisedAddress: ${LIGHT_AGENT_ADVERTISED_ADDRESS:light-agent-account}
server.httpPort: ${LIGHT_AGENT_HTTP_PORT:8083}

The internal port can stay 8083 for every agent because each agent is a different container. Only host-published ports must be unique.

Agent Service Identity

Each agent instance needs a unique runtime identity. The identity is not just the Docker service name.

Recommended identity fields:

The same image can run multiple agents. Compose injects different environment variables and config-server startup values into each service.

Example overlay shape:

services:
  light-agent-account:
    image: ${LIGHT_AGENT_IMAGE:-networknt/light-agent:latest}
    ports:
      - ${ACCOUNT_AGENT_PORT:-8083}:8083
    volumes:
      - ./light-agent-rust/config:/config:ro
      - ./light-controller-rust/ca.pem:/keystore/ca.pem:ro
    environment:
      LIGHT_RS_CONFIG_DIR: /config
      DATABASE_URL: postgres://postgres:secret@postgres:5432/configserver
      LIGHT_PORTAL_AUTHORIZATION: "${LIGHT_AGENT_LIGHT_PORTAL_AUTHORIZATION:-}"
      LIGHT_AGENT_HOST_ID: "${LIGHT_AGENT_HOST_ID:-01964b05-552a-7c4b-9184-6857e7f3dc5f}"
      LIGHT_AGENT_AGENT_DEF_ID: "${ACCOUNT_AGENT_DEF_ID:-}"
      LIGHT_AGENT_SERVICE_ID: com.networknt.agent.account-1.0.0
      LIGHT_AGENT_ADVERTISED_ADDRESS: light-agent-account
      LIGHT_AGENT_MODEL: "${ACCOUNT_AGENT_MODEL:-gpt-5.5}"
      CODEX_API_KEY: "${ACCOUNT_AGENT_CODEX_API_KEY:-}"
      CODEX_ACCOUNT_ID: "${ACCOUNT_AGENT_CODEX_ACCOUNT_ID:-}"
      CODEX_REASONING_EFFORT: "${ACCOUNT_AGENT_CODEX_REASONING_EFFORT:-low}"
      RUST_LOG: "${ACCOUNT_AGENT_RUST_LOG:-info}"
      AGENT_LOG_ANSI: "false"

  light-agent-offer:
    image: ${LIGHT_AGENT_IMAGE:-networknt/light-agent:latest}
    ports:
      - ${OFFER_AGENT_PORT:-8084}:8083
    volumes:
      - ./light-agent-rust/config:/config:ro
      - ./light-controller-rust/ca.pem:/keystore/ca.pem:ro
    environment:
      LIGHT_RS_CONFIG_DIR: /config
      DATABASE_URL: postgres://postgres:secret@postgres:5432/configserver
      LIGHT_PORTAL_AUTHORIZATION: "${LIGHT_AGENT_LIGHT_PORTAL_AUTHORIZATION:-}"
      LIGHT_AGENT_HOST_ID: "${LIGHT_AGENT_HOST_ID:-01964b05-552a-7c4b-9184-6857e7f3dc5f}"
      LIGHT_AGENT_AGENT_DEF_ID: "${OFFER_AGENT_DEF_ID:-}"
      LIGHT_AGENT_SERVICE_ID: com.networknt.agent.offer-1.0.0
      LIGHT_AGENT_ADVERTISED_ADDRESS: light-agent-offer
      LIGHT_AGENT_MODEL: "${OFFER_AGENT_MODEL:-gpt-5.5}"
      CODEX_API_KEY: "${OFFER_AGENT_CODEX_API_KEY:-}"
      CODEX_ACCOUNT_ID: "${OFFER_AGENT_CODEX_ACCOUNT_ID:-}"
      CODEX_REASONING_EFFORT: "${OFFER_AGENT_CODEX_REASONING_EFFORT:-low}"
      RUST_LOG: "${OFFER_AGENT_RUST_LOG:-info}"
      AGENT_LOG_ANSI: "false"

The example deliberately avoids container_name. Compose service names already provide stable DNS on the project network, and omitting container_name avoids cross-project name collisions.

Gateway Registry

The gateway should route to agent services through Docker DNS names, not host addresses. For the local direct registry:

direct-registry.directUrls:
  com.networknt.agent.account-1.0.0: http://light-agent-account:8083
  com.networknt.agent.offer-1.0.0: http://light-agent-offer:8083

The same rule applies to demo APIs. Gateway route targets should be service names on the shared Compose network.

When an agent is registered through controller, its runtime identity should match the config-server tuple used by the container:

host + serviceId + envTag

The agent should keep server.enableRegistry: true so controller can discover it and send catalog cache invalidation notifications.

Effective Agent Catalog

Each agent loads its effective catalog from portal-query with:

hostId + agentDefId + serviceId + envTag

The effective catalog includes:

• the agent definition,
• assigned skills from agent_skill_t,
• tool projections from skill_tool_t and tool_t,
• tool parameters from tool_param_t,
• workflow mappings from skill_workflow_t,
• workflow definitions from wf_definition_t,
• policy diagnostics for tools that should not be exposed.

The agent caches the effective catalog locally. Controller cache-management messages should clear that cache when skills, tools, workflows, or assignments change. On the next chat turn, the agent refreshes the catalog from portal-query.

Gateway execution stays separate from catalog reads:

portal-query
  -> effective catalog, skills, tools, workflows, policies

light-gateway
  -> tools/list
  -> tools/call

If portal-query is temporarily unavailable, the direct gateway tool list can remain usable for baseline tool execution. If a tool is in the catalog but is not returned by gateway tools/list, the agent must not execute it.

Capability Model

Agents should be specialized by catalog assignment rather than by image build.

Recommended specialization:

The capability boundary is the effective catalog:

• agent_skill_t assigns skills to the agent,
• skill_tool_t controls which tools a skill can expose,
• skill_workflow_t controls which workflows a skill can start or reference,
• workflow and gateway policy still enforce runtime access.

This keeps the runtime image generic while making each agent instance purpose-built.

Workflow Orchestration

light-workflow should orchestrate multi-step API and agent flows. Agents provide reasoning and tool selection inside their assigned domain, while workflow provides deterministic control flow.

Recommended orchestration responsibilities:

Example advisory flow:

1. A user starts an advisory request.
2. The coordinator agent or portal UI starts a workflow in light-workflow.
3. The workflow calls the account agent with the customer-profile skill.
4. The account agent reads its effective catalog and calls customer profile tools through light-gateway.
5. The workflow validates the profile response with an assert task.
6. The workflow calls the offer agent with the offer-decision skill.
7. The offer agent calls offer decision tools through light-gateway.
8. The workflow applies policy checks, optional human approval, and final response shaping.

The workflow is the durable orchestration record. Agent chat history and memory can support reasoning, but they should not be the only source of orchestration state.

Skill To Workflow Mapping

Use skill_workflow_t to link a skill to one or more workflow definitions:

The effective catalog should include these mappings so the agent can decide whether a user request should be answered directly, routed to a tool, or handed to a workflow.

For destructive or externally visible operations, the skill should prefer a workflow mapping over direct tool execution. The workflow can add approval, assertions, idempotency keys, retries, and audit events.

API Access Pattern

Agents should not call downstream business APIs directly. They should use the gateway data plane:

light-agent
  -> light-gateway /mcp tools/list
  -> light-gateway /mcp tools/call
  -> downstream MCP server or REST/OpenAPI-backed tool

Workflows should also use gateway-backed calls when invoking API operations:

do:
  - get-profile:
      call: mcp
      with:
        session: gateway
        tool: customer_profile_get
        arguments:
          customerId: "${ .customerId }"

When workflow needs reasoning, it should call an agent task:

do:
  - review-offer:
      call: agent
      with:
        agent: com.networknt.agent.offer-1.0.0
        skill: offer-decision
        input:
          customerId: "${ .customerId }"
          profile: "${ .profile }"

The exact agent invocation transport can evolve, but the logical contract is stable: workflow names the agent and skill, and the called agent uses its effective catalog to constrain tools and workflow options.

Local Configuration Layout

Use one shared config template when agents differ only by environment variables:

light-agent-rust/
  config/
    startup.yml
    client.yml
    values.yml

Use per-agent config folders only when the bootstrap or runtime config needs to diverge beyond service id, advertised address, model, or agent definition:

light-agent-rust/
  account/config/
  offer/config/
  advisor/config/

The recommended first phase is the shared template plus per-service Compose environment overrides. This avoids copying the same config files for every agent.

Keep secrets outside git:

• portal bearer token,
• provider API keys,
• provider account ids,
• customer CA material,
• database credentials outside local defaults.

Rollout Plan

Phase 1: Compose Overlay

• Add docker-compose.agent-api.yml beside the current base compose files.
• Move demo APIs and local light-agent services into the overlay.
• Rename the first agent service to light-agent-account.
• Remove container_name from agent services.
• Update gateway direct registry entries to service DNS names.
• Verify the rendered compose model with the base and overlay files together.

Phase 2: Multiple Agent Instances

• Add one overlay service per specialized agent.
• Assign unique service ids and host ports.
• Add portal agent definitions for each service id and env tag.
• Assign skills through agent_skill_t.
• Assign tools through skill_tool_t.
• Assign workflows through skill_workflow_t.
• Verify each agent can load a distinct effective catalog.

Phase 3: Workflow Orchestration

• Create workflow definitions for cross-agent API access.
• Add workflow mappings to skills.
• Let coordinator or portal UI start workflows for multi-step tasks.
• Use gateway MCP calls for API operations.
• Use agent tasks only where domain reasoning is required.
• Add policy checks for destructive tools and approval-required workflows.

Phase 4: Operational Hardening

• Add health checks for every agent and demo API.
• Add startup validation that each agent has a non-empty effective catalog when LIGHT_AGENT_AGENT_DEF_ID is configured.
• Add gateway drift diagnostics comparing catalog tools with gateway tools/list.
• Add cache invalidation verification after skill, tool, or workflow changes.
• Add docs for common local run commands and expected service URLs.

Design Decisions

• Overlay Scope: The first overlay will include both the account and offer agents, together with the two demo APIs.
• Port Publishing: Every agent will publish its own UI port and register with the control plane independently. Chat clients will discover agents via controller registration, and light-gateway will discover them via explicit direct-registry entries.
• Workflow Triggers: Workflow start requests will go through a dedicated workflow command API for the first implementation.
• Agent Orchestration: Agent-to-agent calls will not be exposed as direct gateway tools. Multi-agent flows are orchestrated exclusively via light-workflow. Currently, call: agent tasks are native, catalog-backed model calls executed directly by the workflow engine (bypassing the containerized light-agent tool loops). The containerized agents are primarily used by chat clients. Future implementations may choose to invoke the containerized agent services from workflow.

Agent Memory Event Refactor

Problem

GlobalSnapshotPersistenceImpl currently skips these Hindsight memory tables for both snapshot export and snapshot-to-event conversion:

agent_memory_bank_t
agent_memory_doc_t
agent_memory_unit_t
agent_memory_entity_t
agent_memory_unit_entity_t
agent_memory_entity_cooccur_t
agent_memory_link_t
agent_memory_directive_t
agent_memory_reflection_t
agent_session_history_t

The skip is intentional for the current implementation. These tables are not currently populated from portal events as the source of truth. They are runtime state written directly by light-agent or memory client code, so exporting them as portable portal domain state or converting them into generic created events would be unsafe.

The current implementation also has a schema drift risk:

• light-agent writes directly to the current Hindsight tables: agent_memory_bank_t, agent_session_history_t, and agent_memory_unit_t.
• light-fabric/crates/hindsight-client writes directly to agent_memory_unit_t.
• light-portal Java db-provider has event replay methods for AgentSessionHistory and AgentMemory, but those methods do not cover the current Hindsight table family. AgentMemory writes agent_memory_t, and AgentSessionHistory expects the older session_history_id, process_id, role, and content shape rather than the current (host_id, bank_id, session_id, messages) schema.
• The Rust importer also skips the same agent_memory_* and agent_session_history_t tables, so the Java and Rust conversion paths are aligned around the current non-event-backed behavior.

Goal

Refactor the agent memory persistence path so that memory state has a clear owner:

command/event path -> event_store_t -> db-provider replay -> Hindsight tables

Once that contract is in place, snapshot export and conversion can safely include the event-backed memory state where appropriate.

Non-Goals

• Do not promote existing direct-write memory rows into snapshots before a backfill/migration event strategy exists.
• Do not convert derived caches into authoritative state unless a product decision requires exact cache promotion.
• Do not require every chat token or partial model response to become an event.
• Do not remove the current direct PostgreSQL path until light-agent has a stable command-backed memory store and operational validation.

Current State

Snapshot Export And Conversion

GlobalSnapshotPersistenceImpl excludes the memory tables from export and conversion. This prevents two bad outcomes:

• exporting user/session memory into another environment without an explicit promotion contract
• converting rows into events that no replay handler can faithfully apply

The Rust importer has the same conversion skip list. Any future change must update both Java and Rust paths.

light-agent

light-agent currently owns some memory writes directly:

ensure_session_memory_bank
  INSERT INTO agent_memory_bank_t

session history persistence
  INSERT INTO agent_session_history_t ... ON CONFLICT DO UPDATE

hindsight retain
  INSERT INTO agent_memory_unit_t

This is operationally simple and gives the agent read-your-writes behavior, but it bypasses portal command validation, event persistence, replay, and snapshot conversion.

Java db-provider

The Java db-provider already has event handler plumbing for many GenAI tables. For memory, however, the existing methods are not aligned with the Hindsight schema:

AgentMemoryCreatedEvent -> agent_memory_t
AgentSessionHistoryCreatedEvent -> old session-history row shape

There are no current event handlers for:

agent_memory_bank_t
agent_memory_doc_t
agent_memory_unit_t
agent_memory_entity_t
agent_memory_unit_entity_t
agent_memory_entity_cooccur_t
agent_memory_link_t
agent_memory_directive_t
agent_memory_reflection_t

Recommended Design

Use events for durable memory state, and treat pure caches as rebuildable projection state.

Recommended ownership:

agent_memory_entity_cooccur_t should stay projection-owned unless exact co-occurrence counts are considered business state. It can be rebuilt from memory units and unit-entity links during replay.

agent_session_history_t needs an explicit decision. It contains conversation content and may be high volume. The recommended first phase is to make it event-backed for correctness, but keep snapshot export opt-in because it can contain sensitive user text.

Event Model

Add explicit event constants and aggregate constants for the Hindsight schema. Use aggregate ids that include enough context to avoid cross-bank collisions.

Suggested aggregate ids:

AgentMemoryBank:        hostId|bankId
AgentMemoryDoc:         hostId|bankId|docId
AgentMemoryUnit:        hostId|bankId|unitId
AgentMemoryEntity:      hostId|bankId|entityId
AgentMemoryUnitEntity:  hostId|bankId|unitId|entityId
AgentMemoryLink:        hostId|bankId|fromUnitId|toUnitId|linkType
AgentMemoryDirective:   hostId|bankId|directiveId
AgentMemoryReflection:  hostId|bankId|reflectionId
AgentSessionHistory:    hostId|bankId|sessionId

Suggested events:

AgentMemoryBankCreatedEvent
AgentMemoryBankUpdatedEvent
AgentMemoryBankDeletedEvent

AgentMemoryDocCreatedEvent
AgentMemoryDocUpdatedEvent
AgentMemoryDocDeletedEvent

AgentMemoryUnitRetainedEvent
AgentMemoryUnitUpdatedEvent
AgentMemoryUnitDeletedEvent

AgentMemoryEntityCreatedEvent
AgentMemoryEntityUpdatedEvent
AgentMemoryEntityDeletedEvent

AgentMemoryUnitEntityLinkedEvent
AgentMemoryUnitEntityUnlinkedEvent

AgentMemoryLinkCreatedEvent
AgentMemoryLinkUpdatedEvent
AgentMemoryLinkDeletedEvent

AgentMemoryDirectiveCreatedEvent
AgentMemoryDirectiveUpdatedEvent
AgentMemoryDirectiveDeletedEvent

AgentMemoryReflectionCreatedEvent
AgentMemoryReflectionUpdatedEvent
AgentMemoryReflectionDeletedEvent

AgentSessionHistoryCreatedEvent
AgentSessionHistoryAppendedEvent
AgentSessionHistoryCompactedEvent
AgentSessionHistoryDeletedEvent

Do not reuse the current AgentMemoryCreatedEvent name for agent_memory_unit_t. That name already maps to legacy agent_memory_t and would create ambiguity. Either deprecate the legacy event family or keep it separate with a clear LegacyAgentMemory name in documentation and tests.

For session history, avoid Upserted as the long-term event name. The underlying table may use INSERT ... ON CONFLICT DO UPDATE, but the event log should express intent. Use AgentSessionHistoryCreatedEvent to start a session, AgentSessionHistoryAppendedEvent to add one or more messages, and AgentSessionHistoryCompactedEvent only when the retained JSON history is summarized or truncated.

db-provider Refactor

Add a dedicated Hindsight persistence component, for example:

HindsightMemoryPersistence
HindsightMemoryPersistenceImpl

Responsibilities:

• replay Hindsight memory events into the current tables
• preserve aggregate_version ordering on every mutable table
• handle JSONB, vector(384), and UUID[] fields explicitly
• maintain foreign-key order during replay
• rebuild or incrementally update derived agent_memory_entity_cooccur_t

Update:

PortalConstants
EventTypeUtil
PortalDbProvider.handleEvent
PortalDbProviderImpl
GlobalSnapshotPersistenceImpl table-to-event overrides
GlobalSnapshotPersistenceImpl skip lists
importer/src/snapshot/table_rules.rs

The replay order must satisfy foreign keys:

agent_memory_bank_t
agent_memory_doc_t
agent_memory_unit_t
agent_memory_entity_t
agent_memory_unit_entity_t
agent_memory_link_t
agent_memory_directive_t
agent_memory_reflection_t
agent_session_history_t

If agent_memory_entity_cooccur_t remains derived, rebuild it after replay or update it from AgentMemoryUnitEntityLinkedEvent.

light-agent Refactor

Introduce a memory persistence abstraction:

MemoryStore
  DirectPgMemoryStore
  PortalCommandMemoryStore

DirectPgMemoryStore preserves the current local behavior during migration. It should be marked as a local/runtime compatibility mode and should not be considered portable event state.

PortalCommandMemoryStore should be the enterprise/default target once the command path is stable. It sends memory commands through the portal command API using the agent’s service token. This gives memory writes the same validation, event persistence, replay, and audit behavior as the rest of the portal.

Configuration:

memory:
  writeMode: portal-command # portal-command | direct-pg
  retainSessionHistory: true
  exportableMemory: false

Initial implementation uses environment variables in light-agent:

LIGHT_AGENT_MEMORY_WRITE_MODE=portal-command # portal-command | direct-pg
LIGHT_AGENT_PORTAL_COMMAND_URL=https://...   # optional; defaults from portal config

exportableMemory should default to false until privacy and environment promotion rules are finalized.

DirectPgMemoryStore should be phased out after PortalCommandMemoryStore is stable. Keeping two permanent write paths would reintroduce schema drift and make local development behave differently from production.

Read-Your-Writes

The agent currently reads directly from PostgreSQL after direct writes. Moving writes behind command/event processing creates a read-your-writes requirement. For Phase 1, the command endpoint should apply the projection synchronously before returning. This keeps light-agent simple and avoids session-local buffer race conditions.

Other options can be evaluated later if latency requires them:

• agent keeps a small session-local memory buffer until replay catches up
• agent reads through a query endpoint that can merge persisted memory with the session-local buffer

Snapshot Policy

After the event-backed path is implemented:

1. Remove event-backed Hindsight tables from CONVERSION_SKIP_TABLES.
2. Keep export opt-in for memory tables because they may contain private user content.
3. Keep agent_memory_entity_cooccur_t skipped if it remains derived.
4. Add explicit table-to-event overrides for each event-backed Hindsight table.
5. Keep Java GlobalSnapshotPersistenceImpl and Rust importer skip lists in sync.

Suggested export behavior:

default snapshot export: skip memory content
entityTypes=agent_memory: include event-backed memory tables
entityTypes=agent_session_history: include session history only when explicitly requested

Production session history export should be blocked by default even when the entity type is requested. Allow production export only with an explicit administrative override and a masking/scrubbing step. Lower environments may allow opt-in export for debugging, but the export response should record that memory/session content was included.

Migration Plan

Phase 1: Align db-provider With Current Schema

• Add HindsightMemoryPersistenceImpl.
• Add constants and event dispatch for the current Hindsight schema.
• Deprecate or rename legacy AgentMemory and old AgentSessionHistory methods that do not match the current tables.
• Add db-provider tests for replaying bank, unit, session history, and one association table.

Phase 2: Add Command APIs

• Add command schemas for Hindsight memory operations.
• Validate hostId, bankId, and optional agentDefId ownership.
• Generate events through the normal command path.
• Add authorization checks so an agent can only write memory for its host and allowed bank.

Phase 3: Refactor light-agent

• Introduce MemoryStore.
• Move direct SQL writes behind DirectPgMemoryStore.
• Add PortalCommandMemoryStore.
• Default local development to direct mode if needed, but document it as non-portable.
• Deprecate direct mode after the command path is stable and make PortalCommandMemoryStore the only supported production write path.
• Validate service-token host, sid, and env before writing through command APIs.

Phase 4: Snapshot And Import

• Add table-to-event overrides and conversion tests.
• Remove event-backed tables from conversion skip lists.
• Keep export of memory content opt-in.
• Update Rust importer table rules and dependency graph.
• Add replay-order tests for the FK chain.

Phase 5: Backfill Existing Rows

• Build a one-time backfill tool that reads existing direct-write rows and emits synthetic Hindsight events in dependency order.
• Preserve aggregate_version where possible.
• Mark backfilled events with metadata such as:

{
  "source": "agent-memory-backfill",
  "backfilled": true
}

Do not remove skip rules for production exports until backfill has been run or the deployment has no legacy direct-write rows.

Testing

Add focused tests:

• GlobalSnapshotPersistenceImplTest: memory tables remain skipped before event support; event-backed tables are included after the event-backed path is enabled.
• db-provider replay tests for each Hindsight event family.
• EventTypeUtil aggregate-id tests.
• Rust importer table-rule parity tests.
• light-agent MemoryStore tests using a mock command client.
• end-to-end test: light-agent retain memory -> command event -> replay -> recall reads the memory.

Resolved Decisions

• agent_session_history_t is exportable only as an explicit opt-in. Production export is blocked unless an administrative override and data masking/scrubbing step are provided.
• agent_memory_entity_cooccur_t remains derived. Store the underlying facts as events and rebuild or update co-occurrence counts as projection state.
• Direct PostgreSQL writes are a migration bridge only. They should be removed after the command-backed memory path is stable.
• Memory vectors should not be stored in events. Events store source text, metadata, and embedding model metadata when needed. Projection rebuilds should generate vectors, preferably through the embedding task pipeline, so the platform can re-embed after model upgrades.

Tool Description Embedding Population

Problem

The GenAI Tool page lets users update tool_t.description through the updateTool form. Endpoint-backed tools are also projected into tool_t when api_endpoint_t is populated from OpenAPI, MCP tools/list, or LightAPI Description input.

The schema already has tool_t.description_embedding VECTOR(384), but the current write paths only populate the plain text description:

• ApiServicePersistenceImpl.syncEndpointToolProjections(...) inserts or updates endpoint-backed tool_t rows from api_endpoint_t.
• GenAIPersistenceImpl.createTool(...) inserts manually authored tools.
• GenAIPersistenceImpl.updateTool(...) updates the Tool page edit form.
• genai-command create/update tool contracts do not accept an embedding field, and the Portal UI should not expose raw vectors to users.

As a result, new endpoint-backed tool rows start with a null description_embedding, and user edits can leave any future vector stale unless the write path marks it for regeneration.

Goals

• Populate tool_t.description_embedding for endpoint-backed and manually authored tools.
• Regenerate the embedding whenever the effective embedding source text changes.
• Keep Tool create/update latency independent from external embedding provider latency.
• Avoid trusting browser-submitted vectors.
• Preserve keyword search and normal CRUD behavior when embedding generation is disabled or temporarily failing.
• Keep the first implementation aligned with the existing VECTOR(384) schema.

Non-Goals

• Do not require every tool to have an embedding before it can be listed, edited, linked to a skill, or executed through the gateway.
• Do not move MCP execution into portal-query or the controller.
• Do not store API keys or provider secrets in tool metadata.
• Do not expose raw embedding vectors in the Tool page by default.

Recommended Design

Use asynchronous server-side embedding generation. Tool writes should save the description immediately, mark the embedding stale or pending, and record the embedding task in the same database transaction as the tool_t update. A worker then picks up committed tasks, generates a 384-dimensional vector from a normalized source string, and updates tool_t.description_embedding only if the tool row still matches the source that was embedded.

For phase 1, this should use a transactional work table or transactional outbox pattern. Do not call the external embedding provider inside the command transaction, but do insert or update the work item before that transaction commits. If a later implementation publishes tasks to Kafka or another queue, the database transaction should still write an outbox row first, and a dispatcher should publish after commit. This avoids a failure mode where the tool row commits successfully but the embedding task is never queued.

This keeps command handling reliable and makes the embedding field a derived read-model value, not user-authored command input.

API version import/update
  -> api_endpoint_t rows
  -> endpoint-backed tool_t projection
  -> upsert embedding_task_t in the same transaction

Tool create/update form
  -> ToolCreatedEvent or ToolUpdatedEvent
  -> tool_t row update
  -> upsert embedding_task_t in the same transaction

embedding worker
  -> poll committed pending tasks
  -> load current tool row
  -> build source text
  -> call configured embedding provider
  -> update tool_t.description_embedding with compare-and-set guard

Embedding Source Text

The vector should be generated from stable semantic fields, not audit fields or IDs. The default source can be:

name: <tool_t.name>
description: <tool_t.description>
endpoint: <tool_t.api_method> <tool_t.api_endpoint>
domain: <tool_t.routing_domain>
namespace: <tool_t.semantic_namespace>
protocol: <tool_t.source_protocol>
personas: <tool_t.target_personas>

For endpoint-backed tools, the projection can enrich the source with api_endpoint_t.endpoint_desc and semantic keywords from api_endpoint_t.tool_metadata.routing.semanticKeywords when available. The LLM-facing description remains tool_t.description; enrichment only improves semantic retrieval.

Staleness Tracking

The current table only has the vector. To make regeneration safe and auditable, add lightweight metadata beside it:

If the first implementation avoids schema expansion, it should at least set description_embedding = NULL whenever the description or semantic routing fields change. That prevents stale vector search, but it gives weaker operational visibility than explicit status and source-hash columns.

The metadata can live in tool_t beside the vector for simple read-heavy queries. If row width becomes a concern, move the vector and metadata to a 1:1 table such as tool_embedding_t or a generic entity_embedding_t; keep the same source-hash and status contract either way. The work table should not be the only durable location for ready-state metadata because completed work rows may be retried, compacted, or purged.

Write Path Hooks

The persistence hooks should be narrow:

1. When syncEndpointToolProjections(...) inserts or updates a tool row, compute the source hash from the projected values. If it differs from the stored hash, store the new description_embedding_source_hash, mark embedding status pending, and upsert an embedding task in the same transaction.
2. When createTool(...) writes a new row, store the source hash, mark the embedding pending, and upsert an embedding task in the same transaction unless the normalized source text is blank. For blank source text, clear the vector and mark the status blank without creating a task.
3. When updateTool(...) changes description, name, endpoint, routing domain, namespace, source protocol, target personas, endpoint description, or semantic keywords, store the new source hash, mark the embedding pending, and upsert an embedding task in the same transaction.
4. When a tool is deactivated, no embedding work is needed. Existing vectors can remain stored, but vector queries must filter active = TRUE.

The command contract should not add a descriptionEmbedding property. If a future admin API needs a manual vector load, it should be a separate privileged maintenance action, not part of the normal Tool form.

Embedding writes are read-model maintenance, not user-authored tool changes. The preferred implementation should not emit a normal ToolUpdatedEvent and should not advance the business aggregate version used for user edits. If the local persistence framework requires a row-level version for every physical update, store it separately on the embedding row or task row so embedding maintenance does not interfere with Tool form optimistic concurrency.

Endpoint Sync And Manual Overrides

Endpoint-backed tools need an explicit description ownership contract. Without one, a user can improve the Tool page description and later lose the edit when the API version is synced again from OpenAPI, MCP tools/list, or LightAPI Description input.

Recommended behavior:

• api_endpoint_t remains the source of imported endpoint metadata.
• tool_t.description is the user-facing LLM description.
• When endpoint projection first creates a tool, copy the endpoint description into tool_t.description.
• When a user edits tool_t.description for an endpoint-backed tool, mark the tool description as a manual override.
• Later endpoint syncs should update generated endpoint fields and api_endpoint_t.endpoint_desc, but should not overwrite tool_t.description while the manual override is active.
• Provide a later admin action to reset the description to the imported source.

Suggested columns:

If a deployment wants endpoint sync to be the absolute source of truth, the Tool page must make that clear before allowing edits, because later syncs will overwrite user-authored descriptions. The default portal behavior should favor manual overrides to avoid surprising users.

Work Queue Options

Three implementation options are viable:

Recommended phase 1 is a database work table or polling worker. It is simpler than putting provider calls inside the command request and safer than calling an external model from inside a database transaction.

Use a generic work table from the start so the same worker can later populate skill_t.description_embedding and other platform embeddings without adding one queue per entity type. The table can be named embedding_task_t.

Use a unique key such as (host_id, entity_type, entity_id, source_hash) so the transactional upsert is idempotent.

The worker should claim tasks with row locking, for example FOR UPDATE SKIP LOCKED, so multiple workers can run safely. The final tool update should use the source hash as the primary compare-and-set guard:

UPDATE tool_t
SET description_embedding = ?,
    description_embedding_model = ?,
    description_embedding_dimension = 384,
    description_embedding_ts = CURRENT_TIMESTAMP,
    description_embedding_status = 'ready',
    description_embedding_error = NULL
WHERE host_id = ?
  AND tool_id = ?
  AND active = TRUE
  AND description_embedding_source_hash = ?;

If the row no longer matches, the worker should drop that result and let the newer pending job win. This prevents stale vectors from overwriting a newer description.

Avoid using aggregate_version as a hard CAS requirement unless it is truly needed for local event-sourcing rules. The version may change because of fields that are not part of the embedding source, causing spurious worker failures even when the source hash is still valid. If aggregate_version must be checked, a CAS failure should reload the row; if the stored source hash is unchanged, retry the embedding update using the current version. If the source hash changed, drop the stale result.

Embedding Provider

Add a small server-side provider abstraction:

EmbeddingProvider.embed(model, dimension, inputText) -> float[384]

Configuration should include:

For hosted providers, configure a model that can emit 384 dimensions, such as an OpenAI-compatible embedding endpoint with an explicit dimensions parameter. For restricted deployments, use a local embedding service that emits the same dimension.

Search And Indexing

Vector search should only use ready embeddings:

WHERE host_id = ?
  AND active = TRUE
  AND description_embedding IS NOT NULL
  AND description_embedding_status = 'ready'
ORDER BY description_embedding <=> ?

Add a pgvector index when catalog size makes sequential vector scans too slow:

CREATE INDEX idx_tool_description_embedding
    ON tool_t USING hnsw (description_embedding vector_cosine_ops)
    WHERE active = TRUE
      AND description_embedding IS NOT NULL
      AND description_embedding_status = 'ready';

genai-query can continue keyword search while embeddings are being populated. When vector ranking is enabled, combine vector distance with existing macro filters such as host, active flag, assigned skill, routing domain, semantic namespace, sensitivity tier, source protocol, and semantic_weight.

Vector nearest-neighbor search should run in genai-query against PostgreSQL with pgvector, not inside the agent’s local catalog cache. Database-side search scales better because it can apply tenant, active-state, RBAC, assigned-skill, domain, and sensitivity filters before returning a small top-K result. The agent can still keep a lightweight local cache for fallback keyword matching and gateway intersection, but it should not need to download every catalog vector to rank tools.

Backfill

Existing rows need a one-time backfill:

1. Scan active tools with a non-blank description and null or stale embedding.
2. Queue embedding work in batches per host.
3. Generate and persist vectors with retry/backoff.
4. Report counts: total tools, ready, pending, failed, disabled, blank source.

Backfill should be restartable and idempotent. It should not block portal startup or the Tool page.

Portal UI

The first UI change should be optional diagnostics, not vector editing:

• Do not show description_embedding in create/update forms.
• Optionally show read-only status columns on the Tool page: Embedding Status, Embedding Model, and Embedding Updated.
• After a user updates the description, show the saved description immediately. The embedding can move from pending to ready asynchronously.
• Add an admin action later for “Refresh Embedding” if operators need manual repair.

Failure Behavior

• If embedding is disabled, save descriptions normally and mark status disabled.
• If provider calls fail, keep the tool active and searchable by keyword.
• Failed rows should retry with backoff and surface diagnostics.
• A stale worker result must not overwrite a newer description’s embedding.
• If the source text is blank, clear the embedding and mark the status disabled or blank.

Implementation Phases

Phase 1: Safe Population

• Add embedding metadata columns to tool_t, or add a 1:1 embedding table, and add a generic embedding_task_t for queued work.
• Add description manual-override metadata for endpoint-backed tools.
• Add write-path hooks in endpoint projection and Tool create/update persistence. The hooks must upsert embedding work in the same database transaction as the tool row change.
• Add a polling or queue-backed embedding worker.
• Add a backfill command for existing active tools.
• Add focused tests that endpoint projection and updateTool mark embeddings pending when descriptions change.

Phase 2: Diagnostics

• Expose read-only embedding status through getTool and getFreshTool.
• Add Tool page status columns or a diagnostics view.
• Add retry and refresh operations for failed rows.

Phase 3: Retrieval

• Add the pgvector index.
• Add vector ranking to genai-query or the effective catalog path.
• Combine vector score with keyword score, macro filters, and semantic_weight.
• Keep gateway tools/list intersection as the runtime executability check.

Design Decisions

• Use a transactional work table or outbox for phase 1. The provider call is asynchronous, but task creation must be committed atomically with the tool row change.
• Use source hash as the primary stale-result guard. Treat aggregate_version as diagnostic or optional unless local persistence rules require it.
• Make the task table generic with entity_type and entity_id, so skills and future entities can share the same worker.
• Preserve manual Tool page description edits with a manual override flag for endpoint-backed tools.
• Reuse the same worker for skill_t.description_embedding when skill semantic search is enabled. The task shape should already support entity_type = 'skill'.
• Run vector ranking in genai-query with pgvector and return top-K results to the agent. Keep local agent ranking as a fallback or small-cache optimization, not the primary scalable path.

Workflow Editor

Purpose

The Workflow Editor is the generic Portal authoring surface for light-workflow definitions. It should replace the raw textarea-only workflow definition experience with a structured editor that still preserves YAML as the canonical workflow definition stored in wf_definition_t.definition.

The editor is reusable. It can be opened from the Workflow Definition page, embedded in the Skill Workspace, or used by future task-specific authoring flows such as API onboarding, scheduled live tests, and remediation playbooks.

Design Boundary

light-workflow owns workflow execution, task state, retries, waiting human tasks, and audit events. The Portal editor authors definitions and starts test runs, but it must not implement its own workflow runtime.

The gateway remains the runtime tool execution path. Workflow steps that invoke tools should reference gateway-visible tools or endpoint descriptions and then execute through the same runtime path used by agents.

The editor should not duplicate endpoint contracts. API, MCP, JSON-RPC, gRPC, and other endpoint details belong in LightAPI descriptions, OpenAPI/OpenRPC documents, protobuf metadata, or the portal endpoint catalog. Workflow tasks reference those descriptions and provide step-level wiring, guards, exports, and error handling.

Current State

The current Portal implementation already has the persistence and generic CRUD surface needed for a first editor:

• wf_definition_t stores namespace, name, version, and definition.
• workflow-command exposes create, update, delete, and start workflow commands.
• workflow-query exposes workflow definition reads.
• portal-view has a Workflow Definition table and generic create/update forms whose definition field is a YAML textarea.

The first Workflow Editor can therefore be an incremental UI improvement over the existing definition CRUD and start workflow command.

Goals

• Keep workflow YAML as the canonical persisted artifact.
• Provide a readable step outline or graph next to the YAML editor.
• Validate definitions before save and before test runs.
• Let users discover and reference endpoint descriptions, gateway tools, skills, rules, and human task types from a side panel.
• Support workflow definition create, update, import, export, and start-test flows.
• Make the editor embeddable so skill authoring can use the same workflow authoring component with skill-specific constraints.
• Preserve owner scoping and existing Portal command/query conventions.

Non-Goals

• Do not execute workflow logic in Portal View.
• Do not make skills the workflow runtime.
• Do not store workflow YAML in skill_t.
• Do not require a visual drag-and-drop graph before the editor is useful.
• Do not copy full API contracts into workflow steps when endpoint descriptions can be referenced.
• Do not fork or embed the Apache KIE Serverless Logic Web Tools as the first implementation path. They are useful reference material for CNCF Serverless Workflow concepts, but they are tightly coupled to the strict upstream spec and would be expensive to adapt for Light-Fabric agentic extensions.

Authoring Model

The editor should maintain two synchronized representations:

All saves should serialize from the YAML source or from a parsed model that round-trips to the same specification format. If the visual editor changes a step, it should update the YAML and keep the YAML visible.

The editor should support progressive enhancement:

1. YAML editor plus parsed step outline.
2. Step palette and property panel that edit YAML safely.
3. Read-only graph preview.
4. Drag-and-drop graph editing once round-trip behavior is reliable.

Implementation Architecture

The recommended implementation is a custom React editor built from focused building blocks:

The workflow YAML or JSON document remains the source of truth. CodeMirror edits parse into the editor store. The parsed workflow model is then projected into React Flow nodes and edges. React Flow edits update the same model and then serialize back to the YAML document.

This avoids adding a second large browser editor runtime to portal-view, which already uses CodeMirror for Markdown and OpenAPI JSON/YAML editing. It also avoids fighting a visualizer that only understands the strict CNCF Serverless Workflow schema, while still letting Portal define first-class visual treatments for Light-Fabric task types such as agent, mcp, ask, assert, rule, switch, and future LLM or approval-oriented steps.

CodeMirror should use a custom JSON Schema derived from the CNCF Serverless Workflow schema plus Light-Fabric agentic extensions. For JSON definitions, use a CodeMirror 6 JSON Schema integration such as codemirror-json-schema to provide linting, autocomplete, and hover details. For YAML definitions, reuse the existing portal-view CodeMirror YAML setup where possible and add schema validation through a YAML language-server bridge or equivalent worker-backed integration. The goal is Monaco-like schema assistance without Monaco’s bundle cost.

React Flow should not own the persisted shape. It owns layout, selection, edge creation, and node interaction. The persisted workflow definition should remain independent of the canvas library so a future editor or CLI can read the same definitions.

Recommended sync behavior:

1. Parse CodeMirror content into a typed workflow model when the YAML is valid.
2. Preserve text edits and show problems when YAML is invalid; do not destroy the user’s in-progress text.
3. Project valid workflow models to React Flow nodes and edges.
4. Let graph edge changes update transition targets in the model.
5. Let property-panel changes update the model through schema-aware controls.
6. Serialize model changes back into the YAML document using stable formatting.
7. Keep conflict handling explicit when source edits and graph edits race.

Mermaid can be used for documentation or a lightweight read-only preview, but it is not the long-term authoring surface. JSONForms can be useful inside property panels, but it should not replace the graph/source editor combination.

Layout

Recommended first layout:

The generic Workflow Definition page can use the full layout. The Skill Workspace can embed the same editor with a narrower reference scope and a skill-aware validation profile.

Step Palette

The editor should understand the task types defined by the Light-Fabric agentic workflow design:

The palette should create minimal valid YAML fragments. Users can then edit the full YAML when advanced options are needed.

Reference Panel

The editor should help authors reference existing catalog objects instead of typing fragile identifiers by hand:

• workflow definitions and versions,
• LightAPI endpoint descriptions,
• API endpoints and tool projections,
• gateway-visible MCP tools,
• rule definitions,
• agent definitions,
• skills and skill-linked tools when the editor is embedded in the Skill Workspace.

For generic workflow authoring, the reference panel can show all objects the current user is allowed to read. For skill authoring, it should filter tools to the skill’s linked tools and flag references outside that set.

Validation

Validation should run in layers:

Runtime diagnostics should be separate from persistence validation. A workflow definition can be saved before a gateway is reachable, but the editor should make missing runtime executability visible before test or deployment.

Test Runner

The editor should support a test panel that starts a workflow instance through the existing workflow start command and then reads instance events and task state through the workflow query APIs.

The test panel should support:

• JSON workflow input,
• start run,
• event stream or polling view,
• current context and output preview,
• waiting task completion for ask or approval steps,
• assertion and rule failure display,
• gateway or endpoint call failure display,
• rerun with the same input.

The test runner is a client of light-workflow; it does not execute workflow steps in the browser.

Skill Workspace Integration

Phase 3.5 skill authoring should embed the Workflow Editor rather than create a second skill-specific workflow UI.

Recommended integration:

1. The Skill Workspace has a Workflow tab.
2. The tab lets the user choose none or workflow-backed.
3. In workflow-backed mode, the user can select an existing workflow definition or create a draft definition.
4. The link is stored in skill_workflow_t.
5. The editor reference panel filters tool references to the tools linked by skill_tool_t.
6. Validation rejects or warns on workflow tool calls not present in the skill’s allowed tool set.
7. The Test tab starts the linked workflow with sample JSON input and displays the same workflow events used by the generic editor.

This keeps the skill as a discovery and guidance artifact while light-workflow owns deterministic orchestration.

Data And API Changes

The first generic editor can reuse existing workflow definition APIs. Later phases should add editor-friendly endpoints only when they remove real UI complexity.

Phase B adds the validation endpoint and keeps the reference catalog composed from existing read models. A single combined catalog endpoint remains optional if the multiple list queries become noisy or slow.

Server-side validation should be authoritative. Client-side validation is useful for responsiveness but should not be the only guard before saving or testing a workflow definition.

Phased Implementation

Phase A: Structured YAML Editor

• Add a generic Workflow Editor component and route.
• Replace create/update workflow definition textarea navigation with the editor where practical.
• Keep YAML visible and canonical.
• Reuse the existing portal-view CodeMirror editor stack with the Light-Fabric workflow schema for YAML/JSON validation, autocomplete, hover help, folding, and parse markers.
• Parse YAML client-side to render a step outline and problems panel.
• Add import/export and basic validation before save.

Phase B: Catalog-Aware Authoring

• Add a reference panel for endpoint descriptions, tools, rules, agents, and workflow definitions.
• Add a step palette that inserts valid YAML snippets.
• Add schema-backed property panels for selected steps. Use dropdowns for catalog references and constrained enums instead of free-text fields where Portal already has authoritative labels.
• Add server-side validation through validateWfDefinition.
• Add runtime diagnostics that compare MCP tool references with gateway tools/list or the Rust agent /diagnostics/tools endpoint when a gateway target is selected.

Phase C: Test And Worklist Integration

• Add a test runner panel backed by light-workflow start and query APIs.
• Show workflow events, current task state, waiting human tasks, assertions, and final output.
• Let users complete ask tasks from the test panel.
• Link failed test runs to remediation tasks or worklist entries.

Phase C uses the existing Portal workflow command/query boundary. The editor starts a test run through workflow/startWorkflow, then refreshes getProcessInfo, getTaskInfo, getTaskAsst, getWorklist, and getAuditLog for the returned wfInstanceId. The test panel completes a waiting human task through workflow/completeTask, which preserves the structured response in the event data and materializes the task as completed through the existing TaskInfoUpdatedEvent projection.

The panel should expose remediation links instead of silently creating production work. Failed process or task rows can open a prefilled remediation task form, and task assignments can jump to the workflow worklist with the current workflow instance context.

Phase D: Visual Graph Editing

• Add a React Flow graph preview after the outline is stable.
• Represent Light-Fabric task types with custom React Flow nodes and explicit transition edges.
• Add drag-and-drop graph editing only after YAML/model round-trip behavior is reliable.
• Keep YAML as the source of truth even when visual editing is enabled.

Phase D adds the graph as a projection of the parsed YAML model, not a separate persisted representation. The graph reads steps, tasks, states, or do containers and renders one custom React Flow node per detected step. Node styling reflects the Light-Fabric task type, and the graph can overlay runtime task status from the Phase C test-run read models when the workflow task id matches a graph step id.

Explicit transition fields such as next, then, to, and transition become solid graph edges. Ordered fallback edges are shown as dashed edges so authors can distinguish model transitions from inferred sequence. Creating an edge in React Flow updates the source step’s transition in YAML, and deleting an explicit edge removes that transition target from YAML. Dragging nodes changes only the authoring layout in the browser session; it does not mutate the saved workflow definition.

The graph must continue to tolerate partial or invalid authoring states. If the YAML cannot be parsed into a known workflow container, the editor keeps the source editor and validation panels usable and shows an empty graph state rather than blocking authoring.

Recommendation

Build the generic Workflow Editor before the Skill Workspace embeds workflow authoring. The skill UI should provide context and constraints, while the workflow editor provides YAML editing, step preview, validation, and test runs for every workflow authoring use case in Portal.

Portal Catalog Scope

Problem

Light Portal supports multiple tenants through host_id and can also host multiple runtime environments in one portal instance. A common deployment shape is:

Within an organization or a cloud deployment, operators need a catalog for APIs, API endpoints, tools, skills, schemas, rules, workflows, categories, and tags. Some catalog entries are reusable platform knowledge. Other entries are tenant-owned, environment-bound, or tied to a concrete gateway deployment.

The main design question is whether Light Portal should clone catalog rows into every host/tenant, or maintain one shared catalog per portal instance and expose it through a separate single page application and virtual host.

The recommended answer is neither full cloning nor a UI-only split. The portal should model catalog scope explicitly:

• shared catalog definitions use global scope,
• tenant-specific definitions and overrides use host scope,
• environment-specific runtime bindings use host plus environment scope,
• a separate SPA may expose the same backend catalog, but it should not become the catalog authority.

Goals

• Avoid duplicating the full catalog for every tenant.
• Prevent catalog drift between tenants and between portal instances.
• Preserve tenant isolation for private APIs, private skills, secrets, access control, and runtime bindings.
• Let dev and sit share one portal instance while still keeping their runtime endpoint targets separate.
• Let stg and prd share another portal instance while keeping production controls stricter.
• Support an effective catalog query that combines global definitions with host-specific rows and environment-specific bindings.
• Reuse existing portal-query APIs and the genai-query catalog direction for agent-facing skills and tools.
• Keep light-gateway as the runtime MCP execution path for tools/list and tools/call.
• Support promotion or import/export between portal instances instead of relying on ad hoc row copies.

Non-Goals

• Do not clone every global catalog row into every tenant by default.
• Do not make a separate SPA the source of truth for catalog data.
• Do not bypass host-scoped authorization just because a catalog item is global.
• Do not put secrets, client credentials, runtime tokens, or deployment state in global catalog rows.
• Do not move MCP tool execution from light-gateway into portal-query, controller-rs, or the catalog UI.
• Do not require every MCP or API endpoint to be wrapped in a skill before the gateway can expose it as a runtime tool.

Current Model

The database already contains both global-capable and host-scoped patterns.

category_t and tag_t have nullable host_id. A null host_id means the category or tag is global. A non-null host_id means the row belongs to one host. Their unique indexes already separate global uniqueness from host-specific uniqueness.

The query behavior for category and tag labels returns both host-specific rows and global rows for a host. This is the right shape for taxonomy and catalog organization metadata.

Other catalog entities are currently host-scoped:

• api_t
• api_version_t
• api_endpoint_t
• agent_definition_t
• skill_t
• tool_t
• tool_param_t
• agent_skill_t
• skill_tool_t
• skill_dependency_t

Those tables use host_id NOT NULL and most query paths filter by host_id = ?. This is correct for private tenant data and runtime-bound data, but it is too narrow for reusable platform catalog definitions if the only sharing mechanism is row replication.

Design Decision

Use a scoped catalog inside Light Portal.

The portal backend remains the source of truth. The catalog UI can be part of the existing portal SPA or exposed through another SPA/virtual host, but both UI surfaces must read and write through the same portal-query and command APIs.

The durable model is:

global catalog definition
  -> host enablement or host override
    -> environment runtime binding

This model allows one shared definition for reusable knowledge and separate tenant or environment controls where isolation matters.

Scope Types

Global rows are reusable definitions. Host rows are ownership and isolation. Environment rows are runtime selection.

Catalog Entity Guidance

Effective Catalog

Consumers should not need to manually merge global and host rows. Portal-query should expose an effective catalog read model for each host and runtime context.

The effective catalog request should include:

• hostId
• serviceId when the catalog is for a gateway, agent, or runtime service
• envTag when the result is environment-specific
• optional agentDefId when the result is for an agent
• optional filters for entity type, category, tag, protocol, routing domain, or capability

The effective catalog response should include:

• global definitions visible to the caller,
• host-specific definitions visible to the caller,
• host overrides that shadow global defaults,
• environment bindings for the requested envTag,
• active state and catalog version or freshness metadata,
• category and tag labels from both global and host-specific taxonomy rows,
• enough provenance to show whether a row came from global scope, host scope, or an environment binding.

Recommended precedence:

environment binding > host override > global definition

This keeps shared definitions stable while allowing host and environment customization.

Data Model Direction

For tables that already support nullable host_id, keep the current pattern:

host_id IS NULL  -> global/shared row
host_id = ?      -> host-specific row

For strictly host-scoped catalog tables, do not simply make every host_id nullable without checking foreign keys and runtime assumptions. Some tables are correctly host-scoped because they point to tenant-owned APIs, credentials, gateway endpoints, or agent assignments.

Use one of these patterns per entity:

1. Nullable host_id on the definition table when the entity can safely be global and all references can resolve global plus host rows.
2. Separate template and binding tables when the definition is global but enablement is tenant-specific.
3. Keep the current host-scoped table when the entity is inherently tenant or runtime bound.

For reusable skills and tools, the safest long-term shape is template plus binding:

catalog_skill_template_t
  -> host_skill_t or skill_t host override
    -> agent_skill_t assignment

catalog_tool_template_t
  -> host tool projection
    -> skill_tool_t mapping
    -> gateway runtime tools/list verification

If the implementation starts smaller, it can add nullable global scope to selected catalog definition tables first, but the query contract must still return the effective catalog and indicate scope provenance.

Separate SPA Or Virtual Host

A separate SPA deployed with LightAPI and sign-in as another BFF virtual host is useful as a catalog presentation surface. It can provide a marketplace-style view for shared APIs, tools, skills, schemas, rules, and workflows.

It should not own separate catalog state.

Recommended use:

• browse global catalog definitions,
• request enablement for a host,
• compare host overrides with global definitions,
• review environment bindings,
• publish or promote catalog versions between portal instances.

Avoid using the separate SPA to bypass tenant-aware portal APIs. The BFF should still pass authenticated requests to portal-query or command APIs, and those APIs must enforce host, service, environment, and role checks.

Environment Handling

Within one portal instance, environments should be runtime bindings, not cloned catalog universes.

For a dev/sit instance:

• one shared catalog can describe a capability,
• dev and sit get separate env_tag bindings,
• runtime endpoints can differ through target_host, service_id, instance, deployment, or gateway registration,
• a tool can be visible in both environments but executable only where the gateway lists it.

For a stg/prd instance:

• stg and prd can share approved global definitions,
• production enablement should require stricter workflow or authorization,
• secrets, tokens, OAuth clients, runtime instances, and deployment state remain environment-specific,
• catalog promotion into prd should preserve stable IDs and versions.

Promotion Between Portal Instances

The boundary between dev/sit and stg/prd is an instance boundary. Treat it as a promotion boundary, not as live replication between tenants.

Recommended promotion flow:

1. Author or import catalog definitions in the lower portal instance.
2. Review and approve the global or host-scoped definitions.
3. Export selected catalog rows with their versions and dependencies.
4. Import into the target portal instance.
5. Resolve environment bindings for stg or prd.
6. Verify runtime exposure through the selected light-gateway tools/list.
7. Activate the target bindings.

Promotion should be idempotent. A repeated import of the same catalog version should update or confirm the same target definition instead of creating duplicates.

Security And Authorization

Global catalog visibility does not mean global execution permission.

Authorization must be checked at these layers:

• portal UI and BFF authentication,
• portal-query read authorization,
• command API write authorization,
• host and environment claim matching,
• category/tag visibility when private taxonomy is used,
• gateway tools/list availability,
• gateway tools/call policy,
• downstream service authorization.

For runtime catalog reads used by gateways and agents, the token should include host, sid, and, when environment-specific data is requested, env. The query handler should compare those claims with the requested hostId, serviceId, and envTag.

UI Guidance