PhysDesignBench
GitHub
Reimplementation task · v1-canonical-j-two-image

SRSN Grating-v0

SRSN Grating Coupler Implementation/Optimization

Implement and optimize a silicon-rich silicon nitride grating above a thin SiN waveguide for vertical fiber-to-chip coupling at 1550 nm. Compare evaluator-owned coupling loss with the paper’s 3D-FDTD baseline.

Runs
11
Scored finals
11 of 11
Best final
S = 0.59 GPT-6 Astra · Codex
Score over timeBest-so-far score S for each run · small dots are probes, large dots final submissions
Score S from 0 to 1 against elapsed minutes. Final scores: GPT-6 Astra (Codex) 0.591; GPT-6.1 Sol (Codex) 0.574; Fable 5.1 (Claude Code) 0.556; Muse Wonder (high) (Muse Code) 0.532; Sonnet 5.5 (high) (Claude Code) 0.525; Opus 5.5 (Claude Code) 0.525; GPT-5.6 Sol (Codex) 0.523; Kimi K3.1 Pro (Claude Code) 0.523; GLM 5.3 Flash (max) (mini-SWE-agent) 0.522; GLM 5.3 Flash (high) (Claude Code) 0.520; Gemini 4 Argon (mini-SWE-agent) 0.519.0.000.250.500.751.000153045607590105120135Elapsed time (min)Score SGemini 4 Argon · mini-SWE-agent: final S = 0.519 at 121 minGLM 5.3 Flash (high) · Claude Code: final S = 0.520 at 48 minGLM 5.3 Flash (max) · mini-SWE-agent: final S = 0.522 at 73 minKimi K3.1 Pro · Claude Code: final S = 0.523 at 121 minGPT-5.6 Sol · Codex: final S = 0.523 at 120 minOpus 5.5 · Claude Code: final S = 0.525 at 85 minSonnet 5.5 (high) · Claude Code: final S = 0.525 at 28 minMuse Wonder (high) · Muse Code: final S = 0.532 at 121 minFable 5.1 · Claude Code: final S = 0.556 at 92 minGPT-6.1 Sol · Codex: final S = 0.574 at 120 minGPT-6 Astra · Codex: final S = 0.591 at 120 min

Showing 11 of 11 runs. Hover a line or a model to isolate it; click a model to hide or show it.

Final layouts

Every run’s final GDS, at one scale.

The design each run ended with, drawn from its evaluated GDS, with the task’s ports, modes and band as the evaluator defines them, the run’s own length, and the published result that S = 0.5 stands for. Every layout uses the same physical scale, so sizes compare directly. Hatched layouts failed a validity gate and score zero. Select a layout to zoom and toggle its layers.

  • Optical mask

  • SRSN Grating-v0 · Codex · GPT-6 Astra

    S = 0.591 · scale bar 5 µm

    Fit
    GPT-6 AstraCodexS = 0.59

  • SRSN Grating-v0 · Codex · GPT-6.1-sol

    S = 0.574 · scale bar 5 µm

    Fit
    GPT-6.1 SolCodexS = 0.57

  • SRSN Grating-v0 · Claude Code · Claude Fable 5.1

    S = 0.556 · scale bar 5 µm

    Fit
    Fable 5.1Claude CodeS = 0.56

  • SRSN Grating-v0 · Muse Code · Muse Wonder

    S = 0.532 · scale bar 5 µm

    Fit
    Muse Wonder (high)Muse CodeS = 0.53

  • SRSN Grating-v0 · Claude Code · Claude Sonnet 5.5

    S = 0.525 · scale bar 5 µm

    Fit
    Sonnet 5.5 (high)Claude CodeS = 0.53

  • SRSN Grating-v0 · Claude Code · Claude Opus 5.5

    S = 0.525 · scale bar 5 µm

    Fit
    Opus 5.5Claude CodeS = 0.52

  • SRSN Grating-v0 · Codex · GPT-5.6-sol

    S = 0.523 · scale bar 5 µm

    Fit
    GPT-5.6 SolCodexS = 0.52

  • SRSN Grating-v0 · Claude Code · Kimi K3.1 Pro

    S = 0.523 · scale bar 5 µm

    Fit
    Kimi K3.1 ProClaude CodeS = 0.52

  • SRSN Grating-v0 · mini-SWE-agent · GLM 5.3 Flash

    S = 0.522 · scale bar 5 µm

    Fit
    GLM 5.3 Flash (max)mini-SWE-agentS = 0.52

  • SRSN Grating-v0 · Claude Code · GLM 5.3 Flash

    S = 0.520 · scale bar 5 µm

    Fit
    GLM 5.3 Flash (high)Claude CodeS = 0.52

  • SRSN Grating-v0 · mini-SWE-agent · Gemini-4-Argon

    S = 0.519 · scale bar 5 µm

    Fit
    Gemini 4 Argonmini-SWE-agentS = 0.52