This work was conducted by students at UCSD to help enable fundamental research in EDA using the ASAP7 open-source PDK. When the ASAP7 research enablement was first released [1], only Cadence enablement was provided. As a result, researchers have not been able to perform post-route extraction and timing, or timing-driven P&R, using Synopsys tools. To address these inconveniences, Nishizawa et al. released the Synopsys Enablement for ASAP7 [2]. They provide a TF file, ITF, and TLU+. However, there are two limitations. First, the released TF file corresponds only to the x4 of ASAP7, and therefore cannot be used for x1. Second, the released ITF file estimates wire capacitances to be larger than those in the official QRC tech file. Therefore, to obtain consistent design results between the two tools, a TF file for x1, along with TLU+ and NXTGRD files that match the official QRC file, is required. Our work develops and makes public TF(x1), TLU+ and other necessary design enablement so that post-route extraction and timing, as well as timing-driven P&R, are possible for Synopsys tool users. We provide testcases and example scripts showing “reasonable” correlation of our new Synopsys enablement to the previously-existing Cadence enablement. Importantly, please note that indicators of correlation that we present are not, and should not be construed as, "benchmarking" of any kind. We make no value judgments regarding the respective merits of commercial EDA tools. We welcome suggestions for improvement or improved materials to be included in this repository; please communicate these by email or use GitHub issues, pull requests, etc. Our contact information is provided below. Last, please note and read carefully the headers of all TCL files in this repository that invoke commercial EDA tools. We thank both Cadence and Synopsys for allowing their copyrighted IP to be made publicly available for use by researchers in this manner.
Additionally, the Synopsys enablement for NanGate45 is available here [3].
- Andrew B. Kahng (abk@ucsd.edu)
- Sayak Kundu (sakundu@ucsd.edu)
- Jakang Lee (jal216@ucsd.edu)
- Design Enablement
- Validation of Design Enablement
- Experimental Results
- Usage and Running Instructions
- References
- Appendix
The files required for design implementation are as follows.
| Cadence | Synopsys | |
|---|---|---|
| Physical library | LEF | LEF |
| Technology library | LEF | TF |
| Liberty library | LIB | DB |
| Interconnect File | ICT | ITF |
| RC Lookup Table | CAPTABLE | TLU+ |
| Parasitic Extraction | QRC | NXTGRD |
All enablements, except for Synopsys enablement, are taken from [1]. The TF file was generated by converting the official technology file (.lef) using Synopsys Milkyway, and the ITF was created based on layer information extracted from conditionally released official Calibre DRC rules file. The TLU+ and NXTGRD files were then obtained by converting the ITF file using Synopsys StarRC.
To validate the Synopsys enablement that we provide, we conduct experiments comparing the results from the flows of two EDA companies (i.e., Synopsys and Cadence) under various conditions. The evaluation metrics we use are as follows:
- Number of Matching Timing Paths. This metric is based on the endpoint timing information output by both tools. It measures the consistency of PEX and STA engines between the two tools.
- Net Capacitance. The capacitance of all nets in the design. This metric can be further categorized based on whether coupling capacitance is present.
- Timing Details for Each Endpoint. The characteristics of all timing endpoints in the design, based on setup analysis. These values can be further categorized based on whether the options for coupling capacitance extraction and signal integrity analysis are enabled.
- Actual Arrival Time
- Worst Slack
- Each Cell Delay
- Each Wire Delay
- Each Cell Output Slew
- Each Wire Output Slew
We use the following three experimental flows to measure the consistency of wire delays between the two tools. The flows below are sorted alphabetically by the PEX tool names. If the PEX tool names are the same, they are further sorted alphabetically by the STA tool names.
Flow 1: This is the standard flow for sign-off, where the timing details of the two tools are compared. A representative example is:
- 1-1: Quantus & Tempus vs. StarRC & Primetime
Flow 2: To isolate the effect of SPEF files, the STA engine is fixed, and timing details are compared as follows:
- 2-1: Quantus & Primetime vs. StarRC & Primetime
- 2-2: Quantus & Tempus vs. StarRC & Tempus
Flow 3: To isolate the effect of STA engines, the PEX engine is fixed, and timing details are compared as follows:
- 3-1: Quantus & Primetime vs. Quantus & Tempus
- 3-2: StarRC & Primetime vs. StarRC & Tempus
- 3-3: None & Primetime vs. None & Tempus
Here, None indicates that no SPEF file is provided.
For all flows (1–3), coupled capacitance extraction and signal integrity analysis are enabled (true). If you would like to explore other cases, you can adjust the CC and SI parameters and extract data accordingly. However, this repository does not provide analysis for those experiments. In addition, please note that the results of these flows are not referenced against any particular tool. For example, the experimental result of FLOW 1-1 is reported as the average of Quantus & Tempus vs. StarRC & Primetime and StarRC & Primetime vs. Quantus & Tempus, since the R² score can vary depending on the order of the data.
We must note that when extracting timing paths from different tools, the timing paths reported for given endpoints and/or for given analysis requests do not always exactly match. Therefore, for the purpose of comparing the characteristics of the timing paths in our experiment, we define ‘matching timing path types’ as follows:
- Case 1: The timing paths from both tools have the same cells and transition types (i.e., rise, fall).
- Case 2: The timing paths from both tools consist of the same cells but have different transition types.
- Case 3: At least one cell in the timing paths differs between the two tools.
In this repository, Case 1 and Case 2 are used to compare wire parasitic consistency. Additionally, to analyze the timing of paths not included in Case 1 and Case 2, we match all timing endpoints on a one-to-one basis and compare their timing details. While this does not reveal delay differences caused by wire parasitics, it is sufficient to demonstrate the consistency of timing details.
The figure below shows the path matching types in our repository:
We summarize the experimental results using tables. All data tables are available in “benchmark/total.xlsx”. Data and plots can be directly examined, either by following usage instructions below to extract the data yourself, or by checking the files that have been uploaded in the "benchmark" directory.
We use “absolute difference” for evaluation because the scale of the data we handle is very small, and use of relative difference would lead to data distortion. However, since absolute difference is expressed in absolute terms, its value varies depending on the range of the data. Therefore, when reporting our results here, we use “Normalized Absolute difference.” This is simply the mean or the maximum absolute difference, divided by the data range. Here, data range is defined as the difference between the maximum and minimum values across all data points from both groups being compared.
The formula is as follows:
Normalized Absolute Difference = Absolute Difference / (Maximum - Minimum)
The abbreviations used in the experimental report have the following meanings:
- DESIGN: The name of the top module in the benchmark.
- CC: Whether or not to include and extract the “Coupling Capacitance” (CC) during parasitic extraction.
- SI: Whether or not to enable “Signal Integrity” (SI) analysis during static timing analysis.
- INVS: Innovus
- FC: Fusion Compiler
- QTS: Quantus
- TPS: Tempus
- STRC: StarRC
- PT: PrimeTime
- PEX: Parasitic Extraction Engine
- STA: Static Timing Analysis Engine
-
Number of Matching Timing Paths
Statistics for paths that match both Case 1 and Case 2.-
Flow 1-1
- Lowest matching rate: 35.71 % (ARIANE)
- Average matching rate: 52.66 %
- Highest matching rate: 67.69 % (JPEG)
-
Flow 2-1
- Lowest matching rate: 33.51 % (ARIANE)
- Average matching rate: 53.12 %
- Highest matching rate: 68.75 % (JPEG)
-
Flow 2-2
- Lowest matching rate: 23.94 % (ARIANE)
- Average matching rate: 50.02 %
- Highest matching rate: 68.39 % (JPEG)
-
Flow 3-1
- Lowest matching rate: 66.22 % (BSG)
- Average matching rate: 69.92 %
- Highest matching rate: 74.64 % (JPEG)
-
Flow 3-2
- Lowest matching rate: 47.99 % (ARIANE)
- Average matching rate: 73.89 %
- Highest matching rate: 87.64 % (JPEG)
-
Flow 3-3
- Lowest matching rate: 35.18 % (ARIANE)
- Average matching rate: 68.42 %
- Highest matching rate: 83.99 % (JPEG)
-
Summary: Summarizing the results, we achieved an average path matching rate of 52.66% across the four designs. When fixing the STA engine, the average matching rate was 51.57%, while fixing the PEX engine yielded an average matching rate of 70.74%. Since our TLU+ does not perfectly align with the original QRC file, the PEX engine appears to have a greater impact on the results than the STA engine. However, considering that the path matching rate between the two tools remains around 70% on average even when using the exact same SPEF, it can be inferred that there is also a certain degree of inconsistency between the STA engines.
-
-
Net Capacitance
The average net capacitance comparison results across all designs are as follows:- R2 score: 0.8914
- mean normalized absolute difference: 0.72 %
- maximum normalized absolute difference: 37.3 %
-
Timing Details for Each Endpoint
The comparison results for Actual Arrival Time (AAT) and Slack. Overall, across all designs:-
AAT:
- R2 score: 0.9107
- Mean normalized absolute difference: 5.49 %
- Maximum normalized absolute difference: 31.95 %
-
Slack:
- R2 score: 0.9454
- Mean normalized absolute difference: 3.26 %
- Maximum normalized absolute difference: 13.08 %
-
If you want to run the predefined studies, please use:
Usage) ./run.sh
If you want to run with a custom combination:
Example) make DESIGN=aes_cipher_top PEX_TOOL1=quantus STA_TOOL1=tempus CC1=true SI1=true PEX_TOOL2=starRC STA_TOOL2=primetime CC2=true SI2=true CASE=1
The execution results are stored in the "benchmark/(your_design_name)" directory.
Our repository is composed of the following files and directories:
- ASAP7: Directory containing design enablement for Cadence and Synopsys.
- benchmark: The design directory that we used for evaluation:
- Input files for Extracting Design Data (extract the split files simply by using "cat" command together with "tar" command.)
- DEF file with routing
- Flattened netlist file after routing stage
- SDC file with all input/output port constraints
- Output files
- SPEF files
- CSV files of net capacitance
- CSV files of timing details
- summary files
- plot
- done: A directory that collects files used to verify the completion of a particular step in the Makefile. In particular, it deals with generating data tables and plots.
- Input files for Extracting Design Data (extract the split files simply by using "cat" command together with "tar" command.)
- scripts: This directory contains EDA tool scripts that allow users to run PEX and STA according to their preferences.
- Makefile: Users can process data from various tools they wish to compare using the make command. This Makefile performs the following functions:
- It executes the user-selected PEX tool and STA tool to generate CSV files containing net capacitance and timing details. Since two types of data are required for comparison, users must define two flow types (please refer to the example parameters below).
- It reads the generated CSV files and produces evaluation metrics such as plots, R2 scores, and absolute differences.
- Parameters
- DESIGN: The top-level block name defined in the DEF and netlist.
- PEX_TOOL1 / PEX_TOOL2: The PEX Engine for extracting the SPEF file. The allowed values are "none", "innovus", "quantus", "fusion_compiler", and "starRC". Innovus utilizes TQuantus which is the built-in PEX engine.
- STA_TOOL1 / STA_TOOL2: The STA engine for extracting timing details. The allowed values are "innovus", "tempus", "fusion_compiler", and "PrimeTime".
- CC1 / CC2: Whether to extract coupling capacitance along with the SPEF when using the PEX Engine. The allowed values are "true", "false".
- SI1 / SI2: Whether to extract timing details with signal integrity analysis in the STA engine. The allowed values are "true", "false".
- NWORST: The maximum number of worst paths to extract at each endpoint. This is typically set to 1. The allowed values are the number over one.
- CASE: Which matched path type to assume when extracting timing details (please refer to the aforementioned "path matching type"). The allowed values are "1", "2", "all".
- run.sh: The file that defines the presets used in the experiment.
For commercial EDA tools, we used the versions below.
[Cadence]
- Innovus: 21.1
- Quantus: 21.1
- Tempus: 21.1
[Synopsys]
- Fusion Compiler: 22.3
- PrimeTime: 18.6
- StarRC: 18.6
[1] Official ASAP7 repository from ASAP7
[2] ASAP7-SNPS repository from ASAP7_SNPS
[3] Nangate45-Synopsys-Enablement from NanGate45-Synopsys-Enablement
- Flow 1-1
| DESIGN | # of total timing paths | # of matched timing paths [Case 1 + Case 2] |
|---|---|---|
| AES | 1,041 | 618 (59.37 %) |
| ARIANE | 23,215 | 8,290 (35.71 %) |
| BSG | 253,138 | 121,204 (47.63 %) |
| JPEG | 4,515 | 3,056 (67.69 %) |
- Flow 2-1
| DESIGN | # of total timing paths | # of matched timing paths [Case 1 + Case 2] |
|---|---|---|
| AES | 1,041 | 668 (64.17 %) |
| ARIANE | 23,215 | 7,780 (33.51 %) |
| BSG | 253,138 | 116,544 (46.04 %) |
| JPEG | 4,515 | 3,104 (68.75 %) |
- Flow 2-2
| DESIGN | # of total timing paths | # of matched timing paths [Case 1 + Case 2] |
|---|---|---|
| AES | 1,041 | 626 (60.13 %) |
| ARIANE | 23,215 | 5,558 (23.94 %) |
| BSG | 253,138 | 120,558 (47.63 %) |
| JPEG | 4,515 | 3,088 (68.39 %) |
- Flow 3-1
| DESIGN | # of total timing paths | # of matched timing paths [Case 1 + Case 2] |
|---|---|---|
| AES | 1,041 | 721 (69.26 %) |
| ARIANE | 23,215 | 16,150 (69.57 %) |
| BSG | 253,138 | 167,623 (66.22 %) |
| JPEG | 4,515 | 3,370 (74.64 %) |
- Flow 3-2
| DESIGN | # of total timing paths | # of matched timing paths [Case 1 + Case 2] |
|---|---|---|
| AES | 1,041 | 883 (84.82 %) |
| ARIANE | 23,215 | 11,140 (47.99 %) |
| BSG | 253,138 | 190,112 (75.10 %) |
| JPEG | 4,515 | 3,957 (87.64 %) |
- Flow 3-3
| DESIGN | # of total timing paths | # of matched timing paths [Case 1 + Case 2] |
|---|---|---|
| AES | 1,041 | 857 (82.32 %) |
| ARIANE | 23,215 | 8,168 (35.18 %) |
| BSG | 253,138 | 182,740 (72.19 %) |
| JPEG | 4,515 | 3,792 (83.99 %) |
| DESIGN | Cadence's PEX ↔ Synopsys's PEX | R2 | Normalized Absolute Difference | |
|---|---|---|---|---|
| Mean | Max | |||
| AES | QTS ↔ STRC | 0.8910 | 0.99% | 21.57% |
| ARIANE | QTS ↔ STRC | 0.8663 | 1.05% | 50.91% |
| JPEG | QTS ↔ STRC | 0.9456 | 0.19% | 23.22% |
| BSG | QTS ↔ STRC | 0.9134 | 0.67% | 53.60% |
| DESIGN | TOOLS | CASE | Actual Arrival Time | Slack | ||||
|---|---|---|---|---|---|---|---|---|
| R2 | Normalized Absolute Diff. | R2 | Normalized Absolute Diff. | |||||
| Mean | Max | Mean | Max | |||||
| AES | QTS-TPS ↔ STRC-PT | 1 | 0.947 | 5.21% | 15.32% | 0.9765 | 2.81% | 15.61% |
| 2 | 0.3641 | 16.03% | 25.30% | 0.5386 | 15.13% | 25.34% | ||
| all | 0.9677 | 5.28% | 14.99% | 0.9834 | 3.04% | 14.76% | ||
| QTS-PT ↔ STRC-PT | 1 | 0.9535 | 5.60% | 15.13% | 0.9818 | 2.93% | 13.80% | |
| 2 | -0.202 | 21.40% | 31.70% | 0.0116 | 21.46% | 31.17% | ||
| all | 0.9624 | 5.85% | 14.90% | 0.9829 | 3.33% | 14.72% | ||
| QTS-TPS ↔ STRC-TPS | 1 | 0.9581 | 4.64% | 14.67% | 0.9778 | 2.80% | 15.44% | |
| 2 | 0.4227 | 14.66% | 24.50% | 0.5272 | 14.56% | 25.18% | ||
| all | 0.972 | 4.91% | 14.35% | 0.9832 | 3.22% | 15.00% | ||
| QTS-TPS ↔ QTS-PT | 1 | 0.9988 | 0.82% | 3.33% | 0.9986 | 0.85% | 3.66% | |
| 2 | 0.9963 | 1.19% | 3.25% | 0.9961 | 1.37% | 2.66% | ||
| all | 0.9987 | 0.93% | 3.96% | 0.9986 | 0.96% | 3.64% | ||
| STRC-TPS ↔ STRC-PT | 1 | 0.9996 | 0.58% | 3.34% | 0.9998 | 0.31% | 4.42% | |
| 2 | 0.9911 | 2.47% | 4.58% | 0.9998 | 0.36% | 1.36% | ||
| all | 0.9995 | 0.59% | 3.52% | 0.9997 | 0.34% | 4.33% | ||
| NONE-PT ↔ NONE-TPS | 1 | 0.9978 | 1.24% | 3.66% | 0.9989 | 0.76% | 3.18% | |
| 2 | 0.9897 | 2.18% | 6.01% | 0.9947 | 1.78% | 2.76% | ||
| all | 0.9976 | 1.37% | 4.57% | 0.9986 | 0.93% | 3.18% | ||
| ARIANE | QTS-TPS ↔ STRC-PT | 1 | 0.8178 | 7.84% | 18.66% | 0.8496 | 7.01% | 18.17% |
| 2 | -0.0276 | 10.25% | 19.09% | 0.2449 | 9.28% | 19.03% | ||
| all | 0.7169 | 6.87% | 16.47% | 0.7243 | 6.74% | 17.25% | ||
| QTS-PT ↔ STRC-PT | 1 | 0.7932 | 8.56% | 21.07% | 0.8572 | 6.91% | 19.14% | |
| 2 | 0.6134 | 11.45% | 21.33% | 0.7254 | 9.95% | 19.50% | ||
| all | 0.6641 | 7.43% | 17.50% | 0.7149 | 6.92% | 18.06% | ||
| QTS-TPS ↔ STRC-TPS | 1 | 0.8794 | 6.76% | 19.41% | 0.9069 | 5.86% | 18.29% | |
| 2 | 0.2507 | 9.81% | 16.60% | 0.4523 | 8.45% | 15.53% | ||
| all | 0.7173 | 6.87% | 16.45% | 0.7284 | 6.68% | 17.19% | ||
| QTS-TPS ↔ QTS-PT | 1 | 0.9964 | 0.71% | 5.22% | 0.9978 | 0.53% | 6.36% | |
| 2 | 0.9945 | 0.91% | 2.28% | 0.9979 | 0.60% | 1.97% | ||
| all | 0.9959 | 0.71% | 5.22% | 0.9974 | 0.52% | 6.36% | ||
| STRC-TPS ↔ STRC-PT | 1 | 0.9986 | 0.41% | 9.14% | 0.9983 | 0.53% | 10.13% | |
| 2 | 0.9996 | 0.28% | 1.31% | 0.9993 | 0.43% | 1.62% | ||
| all | 0.9988 | 0.31% | 9.14% | 0.9984 | 0.38% | 10.03% | ||
| NONE-PT ↔ NONE-TPS | 1 | 0.9952 | 1.15% | 3.55% | 0.9956 | 1.21% | 4.33% | |
| 2 | 0.9939 | 2.18% | 3.68% | 0.9962 | 1.90% | 2.74% | ||
| all | 0.9904 | 1.39% | 3.31% | 0.9896 | 1.40% | 3.90% | ||
| BSG | QTS-TPS ↔ STRC-PT | 1 | 0.9191 | 4.29% | 81.00% | 0.9668 | 1.03% | 6.29% |
| 2 | 0.7143 | 6.69% | 17.72% | 0.8726 | 1.30% | 3.66% | ||
| all | 0.9034 | 4.00% | 74.21% | 0.953 | 1.02% | 6.09% | ||
| QTS-PT ↔ STRC-PT | 1 | 0.8395 | 6.56% | 21.05% | 0.9592 | 1.16% | 5.65% | |
| 2 | 0.3271 | 11.08% | 21.36% | 0.8364 | 4.83% | 12.28% | ||
| all | 0.7935 | 6.09% | 19.30% | 0.9376 | 1.21% | 5.44% | ||
| QTS-TPS ↔ STRC-TPS | 1 | 0.9264 | 4.29% | 16.15% | 0.9696 | 0.97% | 3.99% | |
| 2 | 0.6376 | 6.71% | 22.82% | 0.8942 | 1.27% | 5.03% | ||
| all | 0.9027 | 4.13% | 16.70% | 0.9539 | 1.02% | 4.72% | ||
| QTS-TPS ↔ QTS-PT | 1 | 0.966 | 2.26% | 71.28% | 0.9972 | 0.24% | 3.72% | |
| 2 | 0.8984 | 4.04% | 12.53% | 0.9881 | 0.91% | 6.92% | ||
| all | 0.9628 | 2.31% | 71.28% | 0.9958 | 0.27% | 3.72% | ||
| STRC-TPS ↔ STRC-PT | 1 | 0.9914 | 0.37% | 78.21% | 0.9987 | 0.13% | 4.11% | |
| 2 | 0.9969 | 0.52% | 5.89% | 0.9976 | 0.14% | 1.38% | ||
| all | 0.9923 | 0.42% | 76.78% | 0.9984 | 0.13% | 4.07% | ||
| NONE-PT ↔ NONE-TPS | 1 | 0.9858 | 0.89% | 85.17% | 0.9962 | 0.29% | 1.09% | |
| 2 | 0.986 | 1.63% | 3.53% | 0.9974 | 0.77% | 1.89% | ||
| all | 0.9882 | 0.95% | 84.39% | 0.9958 | 0.29% | 1.08% | ||
| JPEG | QTS-TPS ↔ STRC-PT | 1 | 0.959 | 4.62% | 12.83% | 0.9885 | 2.19% | 12.27% |
| 2 | 0.9584 | 4.03% | 11.99% | 0.9849 | 2.68% | 8.02% | ||
| all | 0.958 | 4.57% | 18.53% | 0.9882 | 2.08% | 14.44% | ||
| QTS-PT ↔ STRC-PT | 1 | 0.9489 | 5.09% | 12.72% | 0.9857 | 2.38% | 11.73% | |
| 2 | 0.9196 | 8.22% | 14.29% | 0.9793 | 4.50% | 8.84% | ||
| all | 0.9491 | 5.06% | 17.68% | 0.9857 | 2.31% | 13.96% | ||
| QTS-TPS ↔ STRC-TPS | 1 | 0.9646 | 4.29% | 12.85% | 0.9896 | 2.00% | 12.49% | |
| 2 | 0.8807 | 5.83% | 15.82% | 0.9675 | 3.41% | 7.34% | ||
| all | 0.9622 | 4.31% | 18.85% | 0.9888 | 1.93% | 14.66% | ||
| QTS-TPS ↔ QTS-PT | 1 | 0.9981 | 0.88% | 3.21% | 0.9965 | 1.25% | 4.39% | |
| 2 | 0.9853 | 2.53% | 5.62% | 0.9941 | 1.55% | 3.80% | ||
| all | 0.9978 | 0.90% | 4.91% | 0.9969 | 1.15% | 4.39% | ||
| STRC-TPS ↔ STRC-PT | 1 | 0.9996 | 0.43% | 2.37% | 0.9994 | 0.48% | 3.00% | |
| 2 | 0.9928 | 2.15% | 4.02% | 0.9989 | 0.72% | 3.10% | ||
| all | 0.9993 | 0.51% | 3.82% | 0.9994 | 0.48% | 3.00% | ||
| NONE-PT ↔ NONE-TPS | 1 | 0.9931 | 1.82% | 4.90% | 0.9938 | 1.72% | 5.67% | |
| 2 | 0.9354 | 4.81% | 10.01% | 0.9877 | 2.00% | 5.73% | ||
| all | 0.9906 | 2.07% | 7.46% | 0.9936 | 1.77% | 5.67% | ||
In the case of BSG’s AAT, there were outliers among the results of different STA tools. This can be interpreted, as mentioned in [3], as arising from differences in how the STA tools operate.
Our main interest here is QTS-TPS ↔ STRC-PT (i.e., FLOW1-1), so we provide an additional plot for Case 1 only. The plots below show the slack of Flow1-1 for the four designs.
