OpenSpace/docs/codex-desktop-sidecar-evolution.md

Codex Desktop Sidecar Evolution Integration

Goal

This integration keeps the normal Codex Desktop workflow unchanged while moving OpenSpace skill capture and self-evolution onto a separate provider-backed sidecar path.

The target user experience is:

• Main coding still happens in Codex Desktop with the user's normal subscription login.
• OpenSpace does not take over the main task loop.
• Sidecar evolution can be invoked explicitly after a task and spend provider API tokens instead of the main Codex Desktop session.

Short Answer: Was this mainly an API-level dual routing change?

No.

The final effect does not come from a simple in-process "dual route" inside one OpenSpace runtime where:

• coding uses Codex Desktop subscription auth, and
• evolution uses a provider API

That approach is not viable because Codex Desktop subscription login is not exposed to the Python process as a reusable API credential.

Instead, the final implementation uses process-level split routing:

• the main coding session remains in Codex Desktop
• self-evolution runs through an isolated OpenSpace sidecar with its own provider-backed MCP server

API compatibility work was still necessary, but it is only one part of the solution.

Embedding Split Routing

The sidecar now also supports a separate skill-embedding route from the main LLM.

Recommended setup:

OPENSPACE_MODEL=gpt-5.4
OPENSPACE_LLM_API_KEY=sk-xxx
OPENSPACE_LLM_API_BASE=http://127.0.0.1:8080/v1

OPENSPACE_SKILL_EMBEDDING_BACKEND=local
OPENSPACE_SKILL_EMBEDDING_MODEL=BAAI/bge-small-en-v1.5

If you want a dedicated remote endpoint for skill embeddings instead of local fastembed, set:

OPENSPACE_SKILL_EMBEDDING_BACKEND=remote
OPENSPACE_SKILL_EMBEDDING_API_KEY=sk-embed-xxx
OPENSPACE_SKILL_EMBEDDING_API_BASE=https://example.com/v1
OPENSPACE_SKILL_EMBEDDING_MODEL=openai/text-embedding-3-small

What Was Implemented

1. OpenAI-compatible provider bridge for OpenSpace

File:

• openspace/llm/client.py

Why it was needed:

• The third-party relay worked with Codex's /responses path.
• OpenSpace uses LiteLLM / OpenAI-style chat completion flows.
• The relay was not reliable enough for OpenSpace's normal streaming path.

What changed:

• Added an OpenAI-compatible streaming fallback that talks directly to /chat/completions.
• Reconstructed streamed text, reasoning content, and tool calls into the shape OpenSpace already expects.
• Enabled this path through OPENSPACE_LLM_OPENAI_STREAM_COMPAT.

Effect:

• OpenSpace can use the relay provider for evolution workloads.

2. Evolution-only MCP sidecar

File:

• openspace/evolution_mcp_server.py

Why it was needed:

• The user wanted OpenSpace to handle only post-task evolution and skill capture.
• The main coding loop had to stay outside OpenSpace.

What changed:

• Added a separate MCP server exposing only evolve_from_context.
• This server builds context from the current workspace, conversation summary, and git diff.
• It captures reusable skills without becoming the main task executor.

Effect:

• OpenSpace now has a narrow sidecar role instead of replacing the host coding agent.

3. Sidecar-capable skill engine without full task recording

File:

• openspace/tool_layer.py

Why it was needed:

• The original skill evolution path assumed a fuller OpenSpace task/recording pipeline.
• The new sidecar path needed to create skills without enabling the normal OpenSpace recording flow.

What changed:

• Added enable_skill_engine_without_recording.
• Kept execution analysis tied to recording.
• Allowed skill evolution and skill store initialization in sidecar mode without enabling full task recordings.

Effect:

• Sidecar capture can work independently without creating full OpenSpace task sessions.

4. Isolated Desktop launcher overlay

File:

• scripts/codex-desktop-evolution

Why it was needed:

• The main Codex Desktop session had to keep the user's normal login and defaults.
• The sidecar config had to be added without polluting ~/.codex.

What changed:

• Created an overlay CODEX_HOME at ~/.codex-openspace-desktop.
• Copied the primary Desktop auth and config base into the overlay.
• Added only one extra MCP server: openspace_evolution.
• Scrubbed OPENSPACE_* variables before launching the main Codex process.
• Avoided inheriting arbitrary shell state or leaking sidecar credentials into the main coding session.

Effect:

• Main Codex Desktop remains normal.
• The sidecar is available only in the isolated overlay profile.

5. Agent instruction trigger for sidecar capture

File:

• AGENTS.md

Why it was needed:

• The sidecar should be callable naturally from the Desktop workflow.
• The user should not need to restate the full MCP call every time.

What changed:

• Added a repo-level instruction that maps phrases like sidecar 自进化一下 to openspace_evolution.evolve_from_context.
• Limited the default behavior to:
  • no code changes
  • no main-task takeover
  • at most one high-reuse skill by default

Effect:

• The sidecar behaves like a narrow post-task tool integrated into the normal Desktop workflow.

Other Supporting Changes

MCP stdout flush fix

File:

• openspace/mcp_server.py

What changed:

• Avoided a final stdout flush crash when the MCP stdio transport closes before Python exit.

Missing dependency for MCP backend

Files:

• pyproject.toml
• requirements.txt

What changed:

• Added websockets>=15.0.0
• Added openspace-evolution-mcp as a console entrypoint

Frontend dependency refresh

File:

• frontend/package-lock.json

What changed:

• Updated lodash-es
• Updated vite

This was a maintenance fix and is not part of the sidecar architecture itself.

Architecture Summary

The final architecture is:

1. Codex Desktop remains the main coding agent.
2. Codex Desktop keeps using the user's normal subscription login.
3. A separate overlay profile adds an openspace_evolution MCP server.
4. That MCP server runs OpenSpace with provider-backed credentials.
5. OpenSpace uses the provider only for post-task evolution and skill capture.

This means the practical "dual routing" exists at the workflow/process boundary, not as a single shared in-process auth router.

Usage

Launch the Desktop profile that includes the sidecar:

cd /Users/admin/PycharmProjects/openspace
./scripts/codex-desktop-evolution app

Inside that Desktop session, trigger sidecar capture with:

sidecar 自进化一下

or the longer explicit form:

对当前这轮工作做一次 sidecar 自进化。不要改代码，不要接管任务。请调用 openspace_evolution.evolve_from_context，基于当前对话、git diff 和关键改动，自动提炼 task/summary，最多生成 1 个高复用 skill，并告诉我 skill 名称、路径、为什么值得保留。

Result

The implemented effect is:

• normal Codex Desktop coding stays unchanged
• OpenSpace self-evolution is available on demand
• provider token spend is isolated to the sidecar path
• the sidecar does not silently take over the main workflow