PhysDesignBench
GitHub
Design task · v2

Quantum-8

Eight-Mode Quantum Projector Electrothermal Design

Design an eight-mode quantum projector with a tree of seven MZIs, 15 electrothermal phase controls, and trusted fidelity scoring.

Runs
13
Scored finals
6 of 13
Best final
S = 0.48 GPT-6 Astra · Codex
Score over timeBest-so-far score S for each run · small dots are probes, large dots final submissions
Score S from 0 to 1 against elapsed minutes. Final scores: GPT-6 Astra (Codex) 0.479; Opus 5.5 (Claude Code) 0.380; GPT-6.1 Sol (Codex) 0.345; GPT-6.1 Sol (mini-SWE-agent) 0.318; GPT-5.6 Sol (Codex) 0.122; Fable 5.1 (Claude Code) 0.098; Gemini 4 Argon (mini-SWE-agent) Invalid; GLM 5.3 Flash (high) (Claude Code) Invalid; GLM 5.3 Flash (max) (mini-SWE-agent) Invalid; Kimi K3.1 Pro (Claude Code) Invalid; Muse Wonder (high) (Muse Code) Invalid; Sonnet 5.5 (mini-SWE-agent) Invalid; Sonnet 5.5 (high) (Claude Code) Invalid.0.000.250.500.751.000306090120150180Elapsed time (min)Score SSonnet 5.5 · mini-SWE-agent: final Invalid at 121 minGPT-6.1 Sol · mini-SWE-agent: final S = 0.318 at 152 minGemini 4 Argon · mini-SWE-agent: final Invalid at 121 minGLM 5.3 Flash (high) · Claude Code: final Invalid at 130 minGLM 5.3 Flash (max) · mini-SWE-agent: final Invalid at 58 minKimi K3.1 Pro · Claude Code: final Invalid at 121 minMuse Wonder (high) · Muse Code: final Invalid at 140 minSonnet 5.5 (high) · Claude Code: final Invalid at 110 minFable 5.1 · Claude Code: final S = 0.098 at 121 minGPT-5.6 Sol · Codex: final S = 0.122 at 154 minGPT-6.1 Sol · Codex: final S = 0.345 at 122 minOpus 5.5 · Claude Code: final S = 0.380 at 119 minGPT-6 Astra · Codex: final S = 0.479 at 123 min

Showing 13 of 13 runs. Hover a line or a model to isolate it; click a model to hide or show it.

Final layouts

Every run’s final GDS, at one scale.

The design each run ended with, drawn from its evaluated GDS. Every layout uses the same physical scale, so sizes compare directly. Hatched layouts failed a validity gate and score zero. Select a layout to zoom and toggle its layers.

  • Optical mask
  • Heater
  • Metal
  • Pad opening
  • Phase marker
  • MMI marker

  • Quantum-8 · Codex · GPT-6 Astra

    S = 0.479 · scale bar 200 µm

    Fit
    GPT-6 AstraCodexS = 0.48

  • Quantum-8 · Claude Code · Claude Opus 5.5

    S = 0.380 · scale bar 200 µm

    Fit
    Opus 5.5Claude CodeS = 0.38

  • Quantum-8 · Codex · GPT-6.1-sol

    S = 0.345 · scale bar 200 µm

    Fit
    GPT-6.1 SolCodexS = 0.34

  • Quantum-8 · mini-SWE-agent · GPT-6.1-sol

    S = 0.318 · scale bar 200 µm

    Fit
    GPT-6.1 Solmini-SWE-agentS = 0.32

  • Quantum-8 · Codex · GPT-5.6-sol

    S = 0.122 · scale bar 200 µm

    Fit
    GPT-5.6 SolCodexS = 0.12

  • Quantum-8 · Claude Code · Claude Fable 5.1

    S = 0.098 · scale bar 200 µm

    Fit
    Fable 5.1Claude CodeS = 0.10
  • No final GDS: the evaluator rejected the final source before producing a layout
    Gemini 4 Argonmini-SWE-agentInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    GLM 5.3 Flash (high)Claude CodeInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    GLM 5.3 Flash (max)mini-SWE-agentInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    Kimi K3.1 ProClaude CodeInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    Muse Wonder (high)Muse CodeInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    Sonnet 5.5mini-SWE-agentInvalid · S = 0

  • Quantum-8 · Claude Code · Claude Sonnet 5.5

    Invalid final · S = 0 · scale bar 200 µm

    Fit
    Sonnet 5.5 (high)Claude CodeInvalid · S = 0