PhysDesignBench
GitHub
Design task · v1

RAFT-Spec

Microring-Assisted Fourier Spectrometer Electrothermal Design

Design a microring-assisted Fourier spectrometer with a smooth add-drop resonator, thermally scanned MZI, physical heater routing, and trusted robust sparse-spectrum reconstruction scoring.

Runs
13
Scored finals
10 of 13
Best final
S = 0.44 GPT-6.1 Sol · Codex
Score over timeBest-so-far score S for each run · small dots are probes, large dots final submissions
Score S from 0 to 1 against elapsed minutes. Final scores: GPT-6.1 Sol (Codex) 0.443; GPT-6 Astra (Codex) 0.442; Gemini 4 Argon (mini-SWE-agent) 0.439; GPT-6.1 Sol (mini-SWE-agent) 0.437; Opus 5.5 (Claude Code) 0.434; Fable 5.1 (Claude Code) 0.423; GPT-5.6 Sol (Codex) 0.422; Sonnet 5.5 (high) (Claude Code) 0.412; Kimi K3.1 Pro (Claude Code) 0.409; Sonnet 5.5 (mini-SWE-agent) 0.408; GLM 5.3 Flash (high) (Claude Code) Invalid; GLM 5.3 Flash (max) (mini-SWE-agent) Invalid; Muse Wonder (high) (Muse Code) Invalid.0.000.250.500.751.000153045607590105120135Elapsed time (min)Score SSonnet 5.5 · mini-SWE-agent: final S = 0.408 at 38 minGPT-6.1 Sol · mini-SWE-agent: final S = 0.437 at 121 minGLM 5.3 Flash (high) · Claude Code: final Invalid at 121 minGLM 5.3 Flash (max) · mini-SWE-agent: final Invalid at 114 minMuse Wonder (high) · Muse Code: final Invalid at 68 minKimi K3.1 Pro · Claude Code: final S = 0.409 at 63 minSonnet 5.5 (high) · Claude Code: final S = 0.412 at 37 minGPT-5.6 Sol · Codex: final S = 0.422 at 121 minFable 5.1 · Claude Code: final S = 0.423 at 63 minOpus 5.5 · Claude Code: final S = 0.434 at 51 minGemini 4 Argon · mini-SWE-agent: final S = 0.439 at 121 minGPT-6 Astra · Codex: final S = 0.442 at 121 minGPT-6.1 Sol · Codex: final S = 0.443 at 121 min

Showing 13 of 13 runs. Hover a line or a model to isolate it; click a model to hide or show it.

Final layouts

Every run’s final GDS, at one scale.

The design each run ended with, drawn from its evaluated GDS. Every layout uses the same physical scale, so sizes compare directly. Hatched layouts failed a validity gate and score zero. Select a layout to zoom and toggle its layers.

  • Optical mask
  • Heater
  • Metal
  • Pad opening
  • Primitive marker

  • RAFT-Spec · Codex · GPT-6.1-sol

    S = 0.443 · scale bar 200 µm

    Fit
    GPT-6.1 SolCodexS = 0.44

  • RAFT-Spec · Codex · GPT-6 Astra

    S = 0.442 · scale bar 200 µm

    Fit
    GPT-6 AstraCodexS = 0.44

  • RAFT-Spec · mini-SWE-agent · Gemini-4-Argon

    S = 0.439 · scale bar 200 µm

    Fit
    Gemini 4 Argonmini-SWE-agentS = 0.44

  • RAFT-Spec · mini-SWE-agent · GPT-6.1-sol

    S = 0.437 · scale bar 200 µm

    Fit
    GPT-6.1 Solmini-SWE-agentS = 0.44

  • RAFT-Spec · Claude Code · Claude Opus 5.5

    S = 0.434 · scale bar 200 µm

    Fit
    Opus 5.5Claude CodeS = 0.43

  • RAFT-Spec · Claude Code · Claude Fable 5.1

    S = 0.423 · scale bar 200 µm

    Fit
    Fable 5.1Claude CodeS = 0.42

  • RAFT-Spec · Codex · GPT-5.6-sol

    S = 0.422 · scale bar 200 µm

    Fit
    GPT-5.6 SolCodexS = 0.42

  • RAFT-Spec · Claude Code · Claude Sonnet 5.5

    S = 0.412 · scale bar 200 µm

    Fit
    Sonnet 5.5 (high)Claude CodeS = 0.41

  • RAFT-Spec · Claude Code · Kimi K3.1 Pro

    S = 0.409 · scale bar 200 µm

    Fit
    Kimi K3.1 ProClaude CodeS = 0.41

  • RAFT-Spec · mini-SWE-agent · Claude Sonnet 5.5

    S = 0.408 · scale bar 200 µm

    Fit
    Sonnet 5.5mini-SWE-agentS = 0.41
  • No final GDS: the evaluator rejected the final source before producing a layout
    GLM 5.3 Flash (high)Claude CodeInvalid · S = 0

  • RAFT-Spec · mini-SWE-agent · GLM 5.3 Flash

    Invalid final · S = 0 · scale bar 200 µm

    Fit
    GLM 5.3 Flash (max)mini-SWE-agentInvalid · S = 0
  • No final GDS: the evaluator rejected the final source before producing a layout
    Muse Wonder (high)Muse CodeInvalid · S = 0