PhysDesignBench
GitHub
Design task · v2

MZI-v2

Broadband 6x6 Rectangular MZI Mesh

Design a broadband 6×6 rectangular MZI mesh with 36 quantized controls and trusted optical, electrical, and electrothermal verification.

Runs
13
Scored finals
7 of 13
Best final
S = 0.82 GPT-6.1 Sol · mini-SWE-agent
Score over timeBest-so-far score S for each run · small dots are probes, large dots final submissions
Score S from 0 to 1 against elapsed minutes. Final scores: GPT-6.1 Sol (mini-SWE-agent) 0.818; GPT-6 Astra (Codex) 0.786; Sonnet 5.5 (mini-SWE-agent) 0.635; Fable 5.1 (Claude Code) 0.532; GPT-6.1 Sol (Codex) 0.412; Sonnet 5.5 (high) (Claude Code) 0.388; Opus 5.5 (Claude Code) 0.367; Gemini 4 Argon (mini-SWE-agent) Invalid; GLM 5.3 Flash (high) (Claude Code) Invalid; GLM 5.3 Flash (max) (mini-SWE-agent) Invalid; GPT-5.6 Sol (Codex) Invalid; Kimi K3.1 Pro (Claude Code) Invalid; Muse Wonder (high) (Muse Code) Invalid.0.000.250.500.751.000306090120150180210240Elapsed time (min)Score SSonnet 5.5 · mini-SWE-agent: final S = 0.635 at 177 minGPT-6.1 Sol · mini-SWE-agent: final S = 0.818 at 159 minGemini 4 Argon · mini-SWE-agent: final Invalid at 121 minGLM 5.3 Flash (high) · Claude Code: final Invalid at 113 minGLM 5.3 Flash (max) · mini-SWE-agent: final Invalid at 91 minGPT-5.6 Sol · Codex: final Invalid at 120 minKimi K3.1 Pro · Claude Code: final Invalid at 121 minMuse Wonder (high) · Muse Code: final Invalid at 96 minOpus 5.5 · Claude Code: final S = 0.367 at 104 minSonnet 5.5 (high) · Claude Code: final S = 0.388 at 147 minGPT-6.1 Sol · Codex: final S = 0.412 at 220 minFable 5.1 · Claude Code: final S = 0.532 at 144 minGPT-6 Astra · Codex: final S = 0.786 at 176 min

Showing 13 of 13 runs. Hover a line or a model to isolate it; click a model to hide or show it.

Final layouts

Every run’s final GDS, at one scale.

The design each run ended with, drawn from its evaluated GDS. Every layout uses the same physical scale, so sizes compare directly. Hatched layouts failed a validity gate and score zero. Select a layout to zoom and toggle its layers.

  • Optical mask
  • Heater
  • Metal
  • Pad opening
  • Phase marker
  • MMI marker

  • MZI-v2 · mini-SWE-agent · GPT-6.1-sol

    S = 0.818 · scale bar 500 µm

    Fit
    GPT-6.1 Solmini-SWE-agentS = 0.82

  • MZI-v2 · Codex · GPT-6 Astra

    S = 0.786 · scale bar 500 µm

    Fit
    GPT-6 AstraCodexS = 0.79

  • MZI-v2 · mini-SWE-agent · Claude Sonnet 5.5

    S = 0.635 · scale bar 500 µm

    Fit
    Sonnet 5.5mini-SWE-agentS = 0.64

  • MZI-v2 · Claude Code · Claude Fable 5.1

    S = 0.532 · scale bar 500 µm

    Fit
    Fable 5.1Claude CodeS = 0.53

  • MZI-v2 · Codex · GPT-6.1-sol

    S = 0.412 · scale bar 500 µm

    Fit
    GPT-6.1 SolCodexS = 0.41

  • MZI-v2 · Claude Code · Claude Sonnet 5.5

    S = 0.388 · scale bar 500 µm

    Fit
    Sonnet 5.5 (high)Claude CodeS = 0.39

  • MZI-v2 · Claude Code · Claude Opus 5.5

    S = 0.367 · scale bar 500 µm

    Fit
    Opus 5.5Claude CodeS = 0.37
  • No final GDS: the evaluator rejected the final source before producing a layout
    Gemini 4 Argonmini-SWE-agentInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    GLM 5.3 Flash (high)Claude CodeInvalid · S = 0

  • MZI-v2 · mini-SWE-agent · GLM 5.3 Flash

    Invalid final · S = 0 · scale bar 500 µm

    Fit
    GLM 5.3 Flash (max)mini-SWE-agentInvalid · S = 0

  • MZI-v2 · Codex · GPT-5.6-sol

    Invalid final · S = 0 · scale bar 500 µm

    Fit
    GPT-5.6 SolCodexInvalid · S = 0

  • MZI-v2 · Claude Code · Kimi K3.1 Pro

    Invalid final · S = 0 · scale bar 500 µm

    Fit
    Kimi K3.1 ProClaude CodeInvalid · S = 0

  • MZI-v2 · Muse Code · Muse Wonder

    Invalid final · S = 0 · scale bar 500 µm

    Fit
    Muse Wonder (high)Muse CodeInvalid · S = 0