IMC tool is not mergeing properly already merged coverage files

As I have large number of test cases, I cannot keep coverage directory for all sims at a time. So I am running in groups and merging them and again deleting older folder and running with new group of tests so fianlly trying to merge all of them.

In principle this should work fine, so we'll need to figure out what's special about your case that makes the numbers go down. Firstly, what do you mean when you mention "coverage .icc"? Do you mean a UCD file?

Secondly, if your numbers go down, it's most likely that you are getting some kind of merge conflict between your batches of runs, meaning that some cover bins can't be merged and therefore only take the hit counts from your "primary" test. If your primary test is one of the new runs instead of the last merged result, then you'll get only the hits from the latest run(s) and not from the merged runs. Thus your grades could go down as well as up. To make that more concrete, here's an example:

1st regression generates you 3 tests: A1, B1, C1, and you merge these to create M1 as a merged coverage file.

2nd regression gives you tests A2, B2, C2, and you try to merge them with M1 such that M2 = (A2 + B2 + C2 + M1).

Now by default IMC will take the first coverage file as the baseline, and it'll try to project the remaining files on top of the structural model for the first run. If you give A2 as the first run when creating M2, then A2's coverage bins will define the structure in M2. If for some reason your coverage model has changed between the 1st and 2nd regressions, any bins in M1 that don't match the bins of A2 will simply be ignored.

You should see warnings in the IMC log for any such "conflicts" in the coverage model.

IMC has an option to the merge command, which forces it to create a union of all the coverage models; this would resolve the problem to some extent, but often it simply masks the more fundamental problem that your coverage model changes between regressions. Really you need to review the merge conflict warnings carefully first to be sure that you're not merging incompatible RTL versions.

Thanks Sephen for detailed explanation. .icc means UCD files. I didn't change the RTL between all regression because if there is any RTL change, it won't merge.

When I have created merged coverage files M1, M2, M3 , M4 and then again I am trying to merge M1, M2, M3 , M4 into M_Final to get complete coverage for all sims. I am seeing that upto 3-4 merge, I didn't see any issue but If I am trying to merge many merged UCD, then there I am seeing coverage (bith functional and code coverage) is getting reduced. 

Will merge command works fine even though if we merge in any order means older first and newer later and vice-varsa.

One more things in addition to above, there are some common test in some of merged coverage, Will that be having any impact?

Thanks

Sushant

Hi Sushant.

I would expect the merge to work in any order regardless of the age of the files. If it's failing to merge, it might be necessary for you to open a support ticket on support.cadence.com so that one of the hotline team can webex with you to investigate.

Did you check that IMC doesn't report any warnings or errors when doing the merge to M_Final? Are you saying that:

M_Final = (M1 + M2 + M3 + M4) in one step is OK, but

M_Final = (M1 + M2 + M3 + M4 + M5) would cause the grades in M_Final to go down? That would be very strange and assuming all the UCM files are compatible, would suggest it might be a tool bug. That said, the 17.10 version is quite old now, if there was a bug it might have been fixed already so it would be good if you can try a more recent IMC; you can try the 19.03 release of IMC, it should be able to merge the exact same coverage files without you having to upgrade simulator version or re-run any tests.

Last thing to check for now: if you do an "ls" on the UCM files in your merged directories, take a look at the file name, it'll be something like "icc_00000000_6b171939.ucm"; the two hex numbers represent a checksum on the RTL and the covergroups. Make sure these are identical for all the files you're trying to merge.

Sure. I will check this and if there is still issue persist, I will raise support ticket.

Thanks a lot.

Hi Stephen,

When I have generated coverage dump for 3000 tests and trying to merge them, it is failing with below error :

# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002b846436533e, pid=31471, tid=0x00002b8462e5f700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_121-b13) (build 1.8.0_121-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x92233e]User defined signal 2

When I am trying to merge less number of coverage dumo, it is working fine. Do we have any constraint on merging regarding number of sims?

Thanks

Sushant

Hi Sushant. The error message contains the words "User defined signal 2", indicating that something external to IMC has interfered and caused IMC to exit. Usually this would be your compute cluster (LSF, Grid Engine etc). I suspect that you haven't requested enough memory when submitting the job to your cluster, and IMC is hitting the limit.