Speeding up SystemC compilation with Incisive SystemC

If you’re a C++ and SystemC programmer you know that when you’ve spent all day tracking down a nasty bug, nothing can bum your trip more than having to wait around for a long recompile. Compile time is a bottleneck for SystemC development. Long compile times can come as a particular surprise for HDL programmers who aren’t used to the lag time caused by template instantiation, dependency checking, and the mysterious thinking that goes on the in the guts of g++ as it creates object code from source.

Incisive SystemC offers three tool-based methodologies for easing the burden of long compile times: precompiled headers, catcxx, and distcomp. We’ll explore all three in this posting.

Precompiled headers can dramatically speed up compilation for some projects. There are two ways to gain the advantages of precompiled headers in NC-SC. One is free (if you are using GCC 4.1 – GCC 3.2.3 does not support precompiled headers). By default, ncsc_run, irun, and the example makefiles get the main SystemC and SCV header files from a special directory that contains precompiled versions. In many cases, this speeds up compilation from 10 to 40%.

If you’re working with your own Makefiles you can still use the precompiled versions of the SystemC headers. Just keep the following rules in mind. A precompiled header is either a file or directory that exists in the same directory as the base header file and contains the suffix .gch. So, systemc_h.h.gch is a precompiled header version of systemc_h.h. Your source files that include either systemc.h, systemc, scv.h, and so on, as the very first line can use the precompiled version with the following caveats:

• Only one precompiled header can be used in a particular compilation.
• A precompiled header cannot be used once the first C token is seen. You can have preprocessor directives before a precompiled header; you can even include a precompiled header from inside another header, as long as there are no C tokens before the #include.
• The precompiled header file must be produced for the same language as the current compilation. You cannot use a C precompiled header for a C++ compilation.
• The precompiled header file must be produced by the same compiler version and configuration as the current compilation is using. The easiest way to guarantee this is to use the same compiler binary for creating and using precompiled headers.
• Any macros defined before the precompiled header (including with -D) must either be defined in the same way as when the precompiled header was generated, or must not affect the precompiled header, which usually means that they do not appear in the precompiled header at all.
• Certain command-line options must be defined in the same way as when the precompiled header was generated. At present, it is not clear which options are safe to change and which are not; the safest choice is to use exactly the same options when generating and using the precompiled header.

The second and preferred method for using precompiled headers is to carefully organize your project so that large header files that seldom change are all included (with #include) into a single header file that you can then precompile along with systemc.h, scv.h, and so on. This might require code reorganization in your project, but usually it’s worth it since you can gain significant compile time improvement.

An example shipped based on the OSCI simple_bus (found at www.systemc.org) unit test in:

`ncroot`/tools/systemc/examples/precompiled_headers

demonstrates how to organize your code. The header files have all been included in project_h.h. Note especially that the file systemc.h is also included. ncsc_run is used to compile the header file:

 Another file, project.h, is introduced that includes (with #include) project_h.h. This extra layer is necessary, so we can use #ifdef guards around the inclusion of project_h.h. Then each .cpp file in the project includes (with #include) project.h as the first line in the source. This code arrangement leads to roughly a 20% improvement in compile-time performance for this example.

Moving on from precompiled headers, ncsc_run offers a more exotic approach to compile time reduction. When you specify the –catcxx option, i.e:

 ncsc_run creates a temporary file that includes (with the #include directive) all of the .cpp files in your model. Instead of compiling each file, it compiles the single "catcxx" file. I’ve seen compile time speedup with catcxx on the order of 80%, so in certain cases it will make your compiles sizzle.

It is important that you prepare the files for compilation first. In particular, to avoid multiple definitions of symbols caused by inclusion of header files in multiple source files, protect the header files with the #ifndef and #endif directives. Also, be aware that the #define directive will affect all of the .cpp files in the model that are included in the temporary file. This situation can cause unexpected results (for example redefinition warnings that can be suppressed by the use of the #ifndef macros in front of any #define). Also, include files are passed forward to every .cpp file in the model. If, for example, you need to define NCSC_INCLUDE_TASK_CALLS for one .cpp file that is not the first file in the list, then if systemc.h is already included before the #define comes into effect, you get a compilation failure. In cases like this, you should use the -D option on the command line, instead of #define in the .cpp file. Catcxx works best if the number of files in your model is between two and ten. When the number of files exceeds ten, the compilation improvement starts to diminish.

The third tool Incisive SystemC offers for compilation speed-up is LSF support for distributed compilation of SystemC/C++ source files on multiple machines. You have to use ncsc_run -distcomp to get this support and you must have LSF installed on your local. First point the path environment variable to the LSF bin directory. LSF provides various commands, but ncsc_run uses only bsub, LSF's batch job facility used by ncsc_run to run compile commands on remote machines.

The bsub command can take optional arguments that give you more flexibility: -x for exclusive execute mode, -c to set maximum cpu time, -m to specify host machines. You can pass these arguments to bsub using the -distcompargs option to ncsc_run. Because these options are supported by bsub and not ncsc_run, you should refer to the documentation that came with the distributed platform to resolve any issues related to using these options. By default, ncsc_run passes the -K option to bsub (to indicate waiting for the command to finish), and the -o option to log the bsub results into the bsub_results.log file located in the working directory.

ncsc_run uses make –j num to spawn concurrent bsubjobs; the number of jobs is controlled by the - distcompjobs option, the default being 20 parallel bsub jobs. For example, for a design containing 40 C++ source files, a new bsub call will be issued when one of the first 20 bsub calls finishes up, until all 40 are compiled. The purpose of the - distcompjobs option is to prevent flooding of bsub calls for very large designs. The other options used by ncsc_run to control distributed compilation are: -distplat, -distcomp, and nodistcomp.

Here’s an example:

With this option applied, file1 and file2 are compiled locally, file3 and file4 are distributed.

So the next time you’re feeling the blues of a long build, consider one of Incisive SystemC’s strategies for speeding things up. Then get back to debugging.

George Frazier