Stats

  • Locked Locked
  • Replies 24
  • Subscribers 125
  • Views 27799
  • Members are here 0
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

ADE-XL: How to force individual result directories for each Monte Carlo point?

dontpanic

Hi! I am having an issue with the way ADE-XL (6.1.6-64b.500.11) is treating the results directories when running Monte Carlo simulations.

I have in my testbench a custom VA component that samples and dumps signals to the disk. For example, if I run a simulation in "Single run, sweeps and corners" mode with 3 corners, my dumped results are stored in the following directories and files:

[...]/[my_lib]/[my_cell]/adexl/results/data/Interactive.7/1/sch__noiseless/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/Interactive.7/2/sch__noiseless/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/Interactive.7/3/sch__noiseless/psf/myDumpedSimData/vOUTdiff.csv

The problem is when I switch to "Monte Carlo Sampling" mode. For instance, if I run 48 MC points for only 1 corner (and setting in my Job Setup options "Max. Jobs"=8), I see the following directories are being created:

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/1/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/1/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/7/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/7/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/13/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/13/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/19/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/19/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/25/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/25/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/31/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/31/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/37/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/37/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/43/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv
[...]/[my_lib]/[my_cell]/adexl/results/data/MonteCarlo.3/43/sch__noiseless/groupRunDataDir/psf/myDumpedSimData/vOUTdiff.csv~

So even though all the 48 MC points run without problems, ADE-XL is arranging/processing the results directories in a way that the files with the dumped signals are being overwritten, and in the end I am only left with 8 individual "vOUTdiff.csv" files (+8 old "vOUTdiff.csv~").

The root of the problem seems to be in the way the simulations are being launched, since in the individual spectre.out logs I see the warnings coming from my VA code messages about the files being overwritten (see attached excerpt below), and also that the MC points are being treated as "iterations".

How can I avoid this behavior, and force ADE-XL/spectre to keep the individual results directories for every MC point, while being able to run many MC points in parallel?

Thanks in advance for any help!

Cheers,

Jorge

  • Andrew Beckett

    My colleague Ioannis who worked on Frank's case has written up his solution and I've published it here.

    Regards,

    Andrew.

  • dontpanic
    I've just tried it; works like a charm!!!

    Thanks again to everybody; I really appreciate your kind help!

    Cheers,

    Jorge.
  • Frank Wiedmann

    CCR 1483939 has now been submitted asking to fix this problem, so that attaching a dummy pre-run script will no longer be required for correct operation. Anyone who would like to see this fixed as well should ask Cadence Customer Support to submit a duplicate of this CCR on his behalf.

  • Andrew Beckett

    Frank,

    I'm not convinced that this is a bug anyway. The same thing happens when using any kind of built-in sweep in the simulator (and monte carlo is just one example of that) - the VerilogA model is just overwriting the same file each time.

    All the the pre-run script does is to force spectre to be run with single-point monte carlo runs. You don't want that in general, because it's less efficient because you end up with more invocations of the simulator, more time spent reading the netlist and checking out licenses.

    Now, if there was a way to generate a unique filename in each monte carlo point run, then that would be the best solution, in my opinion. You ought to be able to use something like this (I think):

    fp=$fopen("./fixedname%A.txt","w");

    but from my quick experiment, I get an error saying that too many files have been open:

    Fatal error found by spectre at iteration = 2 during DC analysis `mc-002_dcOp',
    during Monte Carlo analysis `mc'.
    FATAL (ASL-3246): "monte.va" 9: MON1: Failed to run the simulation because
    one Verilog-A $fopen statement has been used to open more than one
    file. The number of $fopen statements should be more than the number of
    files to be opened. Specify the correct number of $fopen statements and
    rerun the simulation.

    I do have a $fclose in my case - so I'm not really sure why it is reporting this. I'll ask R&D.

    Regards,

    Andrew.

  • Frank Wiedmann

    The problem does not only exist for files written by VerilogA models but also for other files written by the simulator (nodeset files like spectre.dc etc.), where such a solution might be more difficult to implement. In my opinion, it should be possible to instruct the simulator to use a separate directory for each sample without having to use the current workaround with pre-run scripts.

  • Andrew Beckett

    That's what the %A percent code should effectively solve though - it is the analysis name, which would be unique for each analysis (and sub-analysis). That way you can have a single directory, but generate multiple files. Given that a relative path is relative to the run directory, changing to have a separate directory for each point in the run would be a fairly hefty change in behaviour - and you don't really want the simulator changing working directory throughout the run. So I'm sure there are other alternatives that can be explored...

    You can use %A in any filename written by the simulator - it's not just VerilogA.

    I did also look at doing this:

    string fileName;
    ...
    $sformat(fileName,"prefix%d%s",$cds_get_mc_trial_number(),".txt");
    fp=$fopen(fileName,"w");

    However, this has exactly the same errors as my earlier Verilog-A attempt. I think it's just Verilog-A getting confused about open files.

    Regards,

    Andrew.

  • Frank Wiedmann

    Using separate directories seems to be no problem for the simulation results that go to the psf directory. I don't really see why the same thing should be such a problem for the files that are written to the netlist directory.

  • Andrew Beckett

    FYI - as a follow up from this.

    From MMSIM14.1 ISR15 onwards (because of the fix to CCR 1483939) you can now use $cds_get_mc_trial_number() in a VerilogA module to ensure that you get a unique filename. This is described in solution 20426366.

    Regards,

    Andrew.

  • Frank Wiedmann
    Thank you for the information, Andrew. This looks like an acceptable workaround for files written by VerilogA modules. However, other files written to the netlist directory (like spectre.dc etc.) will still overwrite each other unless the workaround with the pre-run script is used.
  • Andrew Beckett
    Agreed. I'd hoped that the %A percent code could be used in the filename, but this is not unique per iteration, so it doesn't help. That would be the best solution in my opinion - to have a percent keyword that would generate a unique filename. That way the working directory doesn't have to be disrupted to achieve this. Of course, it may need a different percent keyword - but something along those lines.

Community Guidelines

The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.