Benchmark

RTL Generation, Understanding, and Repository Context

• ChipVerilog (2026) — Builds multilevel RTL generation tasks from OpenCores designs, combining module equivalence checks with system-level simulation. Paper
• NotSoTiny (2025) — Builds contextual module-completion tasks from ongoing Tiny Tapeout projects, using formal equivalence checks to address limitations of static benchmarks. Paper Code
• RTL-BenchMT (2026) — Uses agents and formal verification to identify specification ambiguities, revise problems, and expand RTL generation benchmarks.
• RTLLM (2023) — Evaluates RTL generation from natural-language specifications, with reference designs, testbenches, and synthesis-quality measurements. Paper
• VerilogEval (2023) — Evaluates Verilog completion and specification-to-RTL generation, using simulation to check functionality and analyze common failures. Paper
• Vhdl-RepoBench (2026) — Preserves VHDL hierarchy and cross-file dependencies to evaluate completion, repository summarization, and specification-to-module generation.

Verification, Assertions, and Debugging

• AssertionBench (2025) — Evaluates LLM-generated RTL assertions using OpenCores designs and formally checked reference assertions. Paper
• AssertLLM2 (2026) — Evaluates how SVA prevents and detects bugs using specifications, complete RTL, and injected mutations. Reproduction requires JasperGold. Paper
• FIXME (2025) — Covers specification understanding, reference models, testbenches, SVA generation, and RTL repair with textual and multimodal specifications. The current flow depends on VCS and VC Static. Paper
• FVEval (2024) — Evaluates formal verification capabilities through NL2SVA and Design2SVA assertion-generation tasks. Full evaluation requires Cadence Jasper. Paper
• HierSVA (2026) — Builds reference SVA for hierarchical RTL and separately evaluates provability, vacuity, specification alignment, mutation detection, and coverage. Reproduction requires VC Formal. Code Data
• ICCAD 2026 — Regression Failure Bucketing (2026) — Synopsys’ verification regression failure-bucketing task, with problem descriptions, test materials, and interim evaluation results.

PPA Optimization and HLS

• Bench4HLS (2026) — Evaluates simulation correctness, synthesizability, and PPA of LLM-generated HLS C++, from kernels to accelerators. The current toolchain uses Vitis HLS/Vivado. Paper
• CktEvo (2026) — Targets multi-file IP repositories to optimize RTL PPA while preserving functional correctness. Paper
• MetRex (2025) — Pairs RTL with synthesized area, delay, static power, and natural-language reasoning to evaluate metric estimation. Paper Data
• Pluto (2025) — Evaluates the efficiency of LLM-generated hardware against reference implementations optimized separately for area, delay, and power. Paper Data
• RTL-OPT (2026) — Provides pairs of suboptimal RTL and manually optimized references to evaluate PPA improvements that preserve functionality. Paper

Netlists and Physical Design

• FloorSet Challenge (2026) — Intel provides synthetic SoC floorplanning data with design constraints, optimizer templates, and evaluation scripts for comparing data-driven floorplanning methods.
• ICCAD 2025 — Incremental Placement Optimization (2025) — NVIDIA’s joint gate-sizing, buffer-insertion, and cell-movement task evaluates incremental optimization after detailed placement. Paper Page
• ICCAD 2026 — LLM-Assisted Netlist Exploration and Transformation (2026) — Cadence’s LLM-assisted netlist exploration and transformation task, with problem descriptions and test materials.

Integrated Design Tasks and Long-Running Workflow Evaluation

• Build-A-Chip-Bench (2026) — Evaluates agents from workload specifications to GDS, checking functionality, timing, routing, and DRC before scoring PPA. The live leaderboard is forthcoming.
• ChipBench (2026) — Covers hierarchical Verilog generation, RTL debugging, and generation of Python, SystemC, and CXXRTL reference models. Paper
• CLOSER-Bench (2026) — Compares spec-to-RTL, RTL-to-GDS, and spec-to-GDS workflows across 10 experimental tasks, studying budget allocation and cross-stage repair.
• CVDP (2025) — Engineer-written chip design and verification tasks spanning single modules and repositories, with Docker environments. Some tasks use commercial EDA tools. Paper Data

Analog, Mixed-Signal, and Multimodal Evaluation

• AMSbench (2025) — A multimodal benchmark for analog and mixed-signal schematic perception, analysis, and design. Data
• AnalogGym (2024) — Provides multiple analog topologies, specifications, and simulation tests for comparing automated sizing methods.
• CircuitSense (2026) — Evaluates circuit-image perception, symbolic equation derivation, and design reasoning, with a synthetic data generation pipeline. Data
• Long-Horizon Analog Design Bench (2026) — Evaluates analog and mixed-signal agents on long-running SKY130/ngspice design tasks, using schematic-level electrical specifications for acceptance.
• NetlistBench (2026) — Uses deterministic structural oracles to assess SPICE netlist recognition, editing, hierarchy operations, and equivalence judgments.