For users evaluating Testwell CTC++ or just starting to use this Code Coverage Analyzer, the first steps are described.Getting StartedGetting StartedFor users evaluating Testwell CTC++ or just starting to use this Code Coverage Analyzer, the first steps are described. Testwell CTC++ is a Code Coverage Analyzer for C and C++. With two language add-ons, also Java and C# are supported. It measures code coverage for application software as well as embedded software. Testwell CTC++ provides all typical coverage levels required by industrial safety standards.About Testwell CTC++About Testwell CTC++ Testwell CTC++ is a Code Coverage Analyzer for C and C++. With two language add-ons, also Java and C# are supported. It measures code coverage for application software as well as embedded software. Testwell CTC++ provides all typical coverage levels required by industrial safety standards.Testwell CTC++ can be installed on Windows, Linux and macOS with the following instructions.InstallationInstallationTestwell CTC++ can be installed on Windows, Linux and macOS with the following instructions.The following example illustrates a concrete use case for the operation of Testwell CTC++. This example assumes that you use Testwell CTC++ in the command line. Tutorial ExampleTutorial ExampleThe following example illustrates a concrete use case for the operation of Testwell CTC++. This example assumes that you use Testwell CTC++ in the command line. With a small example program to decide if a number is a prime, we describe the first iteration of coverage measurement with Testwell CTC++. Basic Use of Testwell CTC++ Basic Use of Testwell CTC++ With a small example program to decide if a number is a prime, we describe the first iteration of coverage measurement with Testwell CTC++. Example code consisting of five files of the tutorial example. Code Example Code Example Example code consisting of five files of the tutorial example. The text report of the basic tutorial example. Execution Profile Listing Execution Profile Listing The text report of the basic tutorial example. With the small example from the basic tutorial, we describe the workflow with Testwell CTC++ when the code is changed. Program Changes Program Changes With the small example from the basic tutorial, we describe the workflow with Testwell CTC++ when the code is changed. With the small example from the basic tutorial, we describe one way how to use Testwell CTC++ with makefiles. Use Testwell CTC++ with makefiles Use Testwell CTC++ with makefiles With the small example from the basic tutorial, we describe one way how to use Testwell CTC++ with makefiles. With the small example from the basic tutorial, we describe a convenient way how to use Testwell CTC++ with makefiles and ctcwrap. Use ctcwrap for makefiles Use ctcwrap for makefiles With the small example from the basic tutorial, we describe a convenient way how to use Testwell CTC++ with makefiles and ctcwrap. With the small example from the basic tutorial, we describe what to do if the compilation command is not used for linking the program. Separate Linking Separate Linking With the small example from the basic tutorial, we describe what to do if the compilation command is not used for linking the program. Testwell CTC++ consists of several independent command-line based tools. Their purpose and usage is described in details here.ToolsToolsTestwell CTC++ consists of several independent command-line based tools. Their purpose and usage is described in details here. Measuring code coverage or timing of a program takes place with an instrumented version of the program. The instrumentation is done with the preprocessor ctc. The Preprocessor ctcThe Preprocessor ctc Measuring code coverage or timing of a program takes place with an instrumented version of the program. The instrumentation is done with the preprocessor ctc. The use of the Testwell CTC++ preprocessor utility ctc is connected to the command for compiling/linking. Starting ctc Starting ctc The use of the Testwell CTC++ preprocessor utility ctc is connected to the command for compiling/linking. ctc can be called with the following command-line options.Options of ctcOptions of ctc ctc can be called with the following command-line options.In productive use, the instrumenting tool ctc is usually not used directly, but has to integrate with an existing build process. ctclaunch is, for Windows, the recommended tool to be used for that purpose. The alternative tool is ctcwrap.The Build Launcher ctclaunchThe Build Launcher ctclaunchIn productive use, the instrumenting tool ctc is usually not used directly, but has to integrate with an existing build process. ctclaunch is, for Windows, the recommended tool to be used for that purpose. The alternative tool is ctcwrap.It is easy to integrate Testwell CTC++ with the following build systems using ctclaunch. An exemplary generic description how to do that is given for them.Positive List of Build Systems Suitable for ctclaunchPositive List of Build Systems Suitable for ctclaunchIt is easy to integrate Testwell CTC++ with the following build systems using ctclaunch. An exemplary generic description how to do that is given for them.To distinguish compiler and linker calls from other programs executed during a build, ctclaunch uses the list provided in wrapcmds.txt.Configuration of ctclaunchConfiguration of ctclaunchTo distinguish compiler and linker calls from other programs executed during a build, ctclaunch uses the list provided in wrapcmds.txt. In productive use, the instrumenting tool ctc is often not used directly, but has to integrate with an existing build process. ctcwrap is, for Linux, the tool that can be used for that purpose. On Windows, ctclaunch is recommended first.The Build Wrapper ctcwrapThe Build Wrapper ctcwrap In productive use, the instrumenting tool ctc is often not used directly, but has to integrate with an existing build process. ctcwrap is, for Linux, the tool that can be used for that purpose. On Windows, ctclaunch is recommended first. With the tool ctcpost, a text report and several exchange formats can be generated from the raw coverage data. It can also be used to combine symbol files or data files. Further, ctcpost can be used to display a listing of the contents of symbol files or data files. The Postprocessor ctcpostThe Postprocessor ctcpost With the tool ctcpost, a text report and several exchange formats can be generated from the raw coverage data. It can also be used to combine symbol files or data files. Further, ctcpost can be used to display a listing of the contents of symbol files or data files. Symbol files (MON.sym) and data files (MON.dat) are the input for the report generation with ctcpost. The Input Files of ctcpostThe Input Files of ctcpost Symbol files (MON.sym) and data files (MON.dat) are the input for the report generation with ctcpost. The structural coverage level can be chosen with these options (Function, Decision, Condition and MC/DC). Coverage Levels set with ctcpostCoverage Levels set with ctcpost The structural coverage level can be chosen with these options (Function, Decision, Condition and MC/DC). As report formats, text and XML are available. For text reports, different content is provided by three different options. The text report generated by ctcpost is technically used as input for generating an HTML coverage report with ctc2html.Report Formats of ctcpostReport Formats of ctcpost As report formats, text and XML are available. For text reports, different content is provided by three different options. The text report generated by ctcpost is technically used as input for generating an HTML coverage report with ctc2html. Help, additional information, configuration parameters and the restriction of source files are defined with these options. General Options of ctcpostGeneral Options of ctcpost Help, additional information, configuration parameters and the restriction of source files are defined with these options. Options for specific needs. Special Options of ctcpostSpecial Options of ctcpost Options for specific needs. Apart from generating reports, ctcpost offers options to combine existing files of one type into a new file of the same type. Options are also available to view the contents of symbol/data files. Combining and Reading symbol/data files with ctcpostCombining and Reading symbol/data files with ctcpost Apart from generating reports, ctcpost offers options to combine existing files of one type into a new file of the same type. Options are also available to view the contents of symbol/data files. The two options -f and -nf constrain the output reports of ctcpost: With -f, the output is restricted to the files given. With -nf, the list of files is excluded from the output. Report Generation with ctcpost for Specified FilesReport Generation with ctcpost for Specified Files The two options -f and -nf constrain the output reports of ctcpost: With -f, the output is restricted to the files given. With -nf, the list of files is excluded from the output. With ctc2html, an HTML report about the coverage data is generated from an execution profile listing and the original source code. The HTML Report Generator ctc2htmlThe HTML Report Generator ctc2html With ctc2html, an HTML report about the coverage data is generated from an execution profile listing and the original source code. The following command-line options are available with ctc2html. They can be given in any order. Options of ctc2htmlOptions of ctc2html The following command-line options are available with ctc2html. They can be given in any order. The summary pages of the HTML report shows coverage data per source code file, per function or per directory in a tabular overview. HTML Report: SummariesHTML Report: Summaries The summary pages of the HTML report shows coverage data per source code file, per function or per directory in a tabular overview. The Execution Profile of the HTML report shows the source code per file, annotated with coverage information.HTML Report: Execution ProfileHTML Report: Execution ProfileThe Execution Profile of the HTML report shows the source code per file, annotated with coverage information. The tool ctc2html uses several configuration parameters determining the appearance of the HTML report. They are defined in the form --enable-XXX=value, either as command-line option or in a configuration file. Configuration Settings for ctc2htmlConfiguration Settings for ctc2html The tool ctc2html uses several configuration parameters determining the appearance of the HTML report. They are defined in the form --enable-XXX=value, either as command-line option or in a configuration file. If HOTA is used, the coverage data is written in textualized form, usually to a text file MON.txt. With ctc2dat, this input is converted to the standard data file format ("MON.dat").The Data File Converter ctc2datThe Data File Converter ctc2datIf HOTA is used, the coverage data is written in textualized form, usually to a text file MON.txt. With ctc2dat, this input is converted to the standard data file format ("MON.dat"). With the tool ctcdiff, two text reports from different test runs can be compared. The Text Report Comparator ctcdiffThe Text Report Comparator ctcdiff With the tool ctcdiff, two text reports from different test runs can be compared. ctcdiff can be used with the following command line options.Using ctcdiffUsing ctcdiff ctcdiff can be used with the following command line options. ctcdiff compares two Execution Profile Listings source file by source file. The coverage differences are marked based on the second Execution Profile Listing. The Behavior of ctcdiffThe Behavior of ctcdiff ctcdiff compares two Execution Profile Listings source file by source file. The coverage differences are marked based on the second Execution Profile Listing. XML reports for different build variants of the same code base can be merged with the tool ctcxmlmerge. The XML-Merging Tool ctcxmlmergeThe XML-Merging Tool ctcxmlmergeXML reports for different build variants of the same code base can be merged with the tool ctcxmlmerge. Executed as a hook script during instrumentation, ctc2static initializes the BITCOV setup without a runtime library and (typically) bits as counters.The BITCOV Initiator ctc2staticThe BITCOV Initiator ctc2staticExecuted as a hook script during instrumentation, ctc2static initializes the BITCOV setup without a runtime library and (typically) bits as counters.The auxiliary tool any2mem calculates the required size of the CTC_array used for BITCOV.The Memory Calculator any2memThe Memory Calculator any2memThe auxiliary tool any2mem calculates the required size of the CTC_array used for BITCOV.The tool dmp2txt converts the dumped BITCOV array to HOTA toolchain’s MON.txt format. The Memory Dump Converter dmp2txtThe Memory Dump Converter dmp2txtThe tool dmp2txt converts the dumped BITCOV array to HOTA toolchain’s MON.txt format. Depending on the customer specific environment, Testwell CTC++ is used for many different testing scenarios, in several build environments, IDEs and tool chains. In this section, generic instructions are found as well as specific instructions for these kind of third party software.How To...?How To...?Depending on the customer specific environment, Testwell CTC++ is used for many different testing scenarios, in several build environments, IDEs and tool chains. In this section, generic instructions are found as well as specific instructions for these kind of third party software.Measuring Code Coverage with Testwell CTC++ always works in three independent steps: instrumentation - testing - report generation. Depending on the environment, different ways to use the components of Testwell CTC++ are advantageous. They are described in these chapters.WorkflowsWorkflowsMeasuring Code Coverage with Testwell CTC++ always works in three independent steps: instrumentation - testing - report generation. Depending on the environment, different ways to use the components of Testwell CTC++ are advantageous. They are described in these chapters. The instrumented program is tested without any change necessary. The coverage data is written out during the execution. Test Runs with the Instrumented ProgramTest Runs with the Instrumented Program The instrumented program is tested without any change necessary. The coverage data is written out during the execution. The coverage data is written out from main memory by default at the end of the program execution. There are several alternatives how to trigger this. Saving of Counters: WhenSaving of Counters: When The coverage data is written out from main memory by default at the end of the program execution. There are several alternatives how to trigger this. When the instrumented program is tested, the coverage data is written out at the end of the program execution. This is the default behavior and does not require any action of the user. At the End of the Program ExecutionAt the End of the Program Execution When the instrumented program is tested, the coverage data is written out at the end of the program execution. This is the default behavior and does not require any action of the user. With the configuration parameter EMBED_FUNCTION_NAME, a list of functions can be defined to trigger the saving of coverage data. Using Selected FunctionsUsing Selected Functions With the configuration parameter EMBED_FUNCTION_NAME, a list of functions can be defined to trigger the saving of coverage data. For long-running programs, the saving of coverage data can be triggered during the termination of the program by a kill signal like Control-C. With Kill SignalsWith Kill Signals For long-running programs, the saving of coverage data can be triggered during the termination of the program by a kill signal like Control-C. With #pragma CTC APPEND, the saving of coverage data is triggered. With PragmasWith Pragmas With #pragma CTC APPEND, the saving of coverage data is triggered. The saving of coverage data can be off-loaded to a separate thread, which wakes up periodically and activates the saving. If the program runs forever or if it crashes, the coverage data up to the last saving point is available. From a Separate ThreadFrom a Separate Thread The saving of coverage data can be off-loaded to a separate thread, which wakes up periodically and activates the saving. If the program runs forever or if it crashes, the coverage data up to the last saving point is available. By default, for host programs the coverage data is saved as a data file at the folder containing the symbol file. This can be changed by several configuration parameters. Saving of Counters: WhereSaving of Counters: Where By default, for host programs the coverage data is saved as a data file at the folder containing the symbol file. This can be changed by several configuration parameters. With test cases made known to Testwell CTC++, the counter data can be written in separate data files. Test Case ConceptTest Case ConceptWith test cases made known to Testwell CTC++, the counter data can be written in separate data files. An instrumented program on host uses the configuration in ctc.ini files. Configuration File Access During TestConfiguration File Access During Test An instrumented program on host uses the configuration in ctc.ini files. When a data file is reused in new tests, the counter data written out is added up to the existing counter data if the source file was not changed. Appending vs. Overriding of Counter DataAppending vs. Overriding of Counter Data When a data file is reused in new tests, the counter data written out is added up to the existing counter data if the source file was not changed. The Host-Target Add-On called HOTA provides coverage measurement with Testwell CTC++ for (embedded) targets.Target Testing: The HOTA WorkflowTarget Testing: The HOTA WorkflowThe Host-Target Add-On called HOTA provides coverage measurement with Testwell CTC++ for (embedded) targets.CTC Runtime is build by modifying the delivered C-Code within the HOTA add-on package according to the target's capabilities.Constructing the CTC Runtime for the TargetConstructing the CTC Runtime for the TargetCTC Runtime is build by modifying the delivered C-Code within the HOTA add-on package according to the target's capabilities.Two prerequisites are important when code is instrumented using a cross-compiler and HOTA: The compiler with its options has to be configured for Testwell CTC++, and the runtime library compiled by the user has to be linked to the instrumented code.Instrumenting Code for the TargetInstrumenting Code for the TargetTwo prerequisites are important when code is instrumented using a cross-compiler and HOTA: The compiler with its options has to be configured for Testwell CTC++, and the runtime library compiled by the user has to be linked to the instrumented code. The instrumented program is downloaded to the target and run there in the same way as the original program.Running at the Target Running at the Target The instrumented program is downloaded to the target and run there in the same way as the original program.BITCOV is a further specialized variant of HOTA. It is meant for very small targets where the HOTA arrangement can not be used, usually because of small memory size. To use BITCOV, the HOTA add-on has to be installed.Target Testing: The BITCOV WorkflowTarget Testing: The BITCOV WorkflowBITCOV is a further specialized variant of HOTA. It is meant for very small targets where the HOTA arrangement can not be used, usually because of small memory size. To use BITCOV, the HOTA add-on has to be installed.BITCOV instrumentation is put into effect by configuring the ctc.ini.Instrumenting Code for the TargetInstrumenting Code for the TargetBITCOV instrumentation is put into effect by configuring the ctc.ini.To run a program instrumented with one of the BITCOV variants on the target, perform the following steps.Running at the TargetRunning at the TargetTo run a program instrumented with one of the BITCOV variants on the target, perform the following steps.To generate reports when one of the BITCOV variants are used, an auxiliary tool dmp2txt is used to convert the dumped array to the text version (MON.txt) of the coverage data file. Generating ReportsGenerating ReportsTo generate reports when one of the BITCOV variants are used, an auxiliary tool dmp2txt is used to convert the dumped array to the text version (MON.txt) of the coverage data file. In several situations, parts of the source code shall not be instrumented for coverage measurement, but still be compiled as a part of the project. Testwell CTC++ offers three ways to do that: On source file level, on function level, or inside the code.Partial InstrumentationPartial InstrumentationIn several situations, parts of the source code shall not be instrumented for coverage measurement, but still be compiled as a part of the project. Testwell CTC++ offers three ways to do that: On source file level, on function level, or inside the code. With the three configuration parameters EXCLUDE, NO_EXCLUDE and NO_INCLUDE it is defined which source code files are instrumented by ctc. Partial Instrumentation on File LevelPartial Instrumentation on File Level With the three configuration parameters EXCLUDE, NO_EXCLUDE and NO_INCLUDE it is defined which source code files are instrumented by ctc. With the configuration parameter SKIP_FUNCTION_NAME, single functions can be excluded from instrumentation. Partial Instrumentation on Function LevelPartial Instrumentation on Function LevelWith the configuration parameter SKIP_FUNCTION_NAME, single functions can be excluded from instrumentation. With two Testwell CTC++ specific pragmas, parts of code can be skipped during instrumentation. Skipping of CodeSkipping of Code With two Testwell CTC++ specific pragmas, parts of code can be skipped during instrumentation. Assume we have an instrumented program, with which we have done some test runs and got some coverage reports. Now we change some files and want to continue the testing. How to handle this in Testwell CTC++? Reinstrumentation after Changing Some FilesReinstrumentation after Changing Some Files Assume we have an instrumented program, with which we have done some test runs and got some coverage reports. Now we change some files and want to continue the testing. How to handle this in Testwell CTC++? The build target is not necessarily a complete executable, it can also be a (static) library. Testwell CTC++ can be used to instrument source files of a library. Building an Instrumented Static LibraryBuilding an Instrumented Static Library The build target is not necessarily a complete executable, it can also be a (static) library. Testwell CTC++ can be used to instrument source files of a library. Descriptions how to integrate Testwell CTC++ with third party tools can be found here.IntegrationsIntegrationsDescriptions how to integrate Testwell CTC++ with third party tools can be found here.For Visual Studio, a graphical integration of Testwell CTC++ is provided for instrumentation as well as for report generation.Integration in Visual StudioIntegration in Visual StudioFor Visual Studio, a graphical integration of Testwell CTC++ is provided for instrumentation as well as for report generation.On Windows, an integration of Testwell CTC++ is available for IAR Embedded Workbench. It includes an integration with the IAR build system, several compiler settings, and a graphical user interface integrated in the IDE.Integration in IAR Embedded WorkbenchIntegration in IAR Embedded WorkbenchOn Windows, an integration of Testwell CTC++ is available for IAR Embedded Workbench. It includes an integration with the IAR build system, several compiler settings, and a graphical user interface integrated in the IDE.The IAR Embedded Workbench Integration for Testwell CTC++ is available for download at the customer area of Verifysoft, and can be installed on Windows.InstallationInstallationThe IAR Embedded Workbench Integration for Testwell CTC++ is available for download at the customer area of Verifysoft, and can be installed on Windows.The IAR add-on of Testwell CTC++ offers a full integration with IAR Embedded Workbench, for all steps from building to report generation.Workflow in the IDEWorkflow in the IDEThe IAR add-on of Testwell CTC++ offers a full integration with IAR Embedded Workbench, for all steps from building to report generation.To build an instrumented version with iarbuild.exe from command-line, ctclaunch is used.Builds from command-lineBuilds from command-lineTo build an instrumented version with iarbuild.exe from command-line, ctclaunch is used.The toolchain of Testwell CTC++ can be integrated in Eclipse CDT by configuration. After that, it is possible to instrument code directly from Eclipse, and to generate coverage reports from it.Integration in Eclipse (CDT)Integration in Eclipse (CDT)The toolchain of Testwell CTC++ can be integrated in Eclipse CDT by configuration. After that, it is possible to instrument code directly from Eclipse, and to generate coverage reports from it.As Eclipse is a multi IDE platform, it is necessary to install a plugin to be able to create and build C/C++- projects.Prerequisites and General ConsiderationsPrerequisites and General ConsiderationsAs Eclipse is a multi IDE platform, it is necessary to install a plugin to be able to create and build C/C++- projects.For setting up the managed build system of Eclipse to use Testwell CTC++, it is recommended to create a special configuration for a project, using the release configuration as a basis. Configuration of Instrumented Compiler and Linker CallsConfiguration of Instrumented Compiler and Linker CallsFor setting up the managed build system of Eclipse to use Testwell CTC++, it is recommended to create a special configuration for a project, using the release configuration as a basis. To generate the reports after a test run directly from Eclipse, an external tool configuration can be done. This configuration will be available for the entire workspace.Report GenerationReport GenerationTo generate the reports after a test run directly from Eclipse, an external tool configuration can be done. This configuration will be available for the entire workspace.The following two ways are described how to use Testwell CTC++ with CMake: By defining a switch in one of the CMakeLists.txt files, or by using ctclaunch (on Windows).Integration with CMakeIntegration with CMakeThe following two ways are described how to use Testwell CTC++ with CMake: By defining a switch in one of the CMakeLists.txt files, or by using ctclaunch (on Windows).Background information about Code Coverage and all kind of settings like configuration parameters, pragmas etc. are found here.Reference and DetailsReference and DetailsBackground information about Code Coverage and all kind of settings like configuration parameters, pragmas etc. are found here. Code coverage is a measure on how thoroughly the program code has been exercised during test. The definitions of different coverage levels Testwell CTC++ is using are described here.Code Coverage in Testwell CTC++Code Coverage in Testwell CTC++ Code coverage is a measure on how thoroughly the program code has been exercised during test. The definitions of different coverage levels Testwell CTC++ is using are described here. Function coverage measures which functions are called during test. Function CoverageFunction Coverage Function coverage measures which functions are called during test. Statement Coverage measures how many statements in the source code are executed during tests. Statement CoverageStatement Coverage Statement Coverage measures how many statements in the source code are executed during tests. Decision coverage includes function coverage and measures additionally if decisions are evaluated to TRUE and to FALSE. Decision CoverageDecision Coverage Decision coverage includes function coverage and measures additionally if decisions are evaluated to TRUE and to FALSE. Condition coverage includes decision coverage and additionally records if all subconditions inside decisions are evaluated as true and as false. Condition Coverage Condition Coverage Condition coverage includes decision coverage and additionally records if all subconditions inside decisions are evaluated as true and as false. MC/DC includes condition coverage. Additionally, for each atomic condition an independence pair of TRUE-FALSE-combinations has to be tested, showing that the condition can independently determine the value of the decision. Modified Condition/Decision Coverage (MC/DC)Modified Condition/Decision Coverage (MC/DC) MC/DC includes condition coverage. Additionally, for each atomic condition an independence pair of TRUE-FALSE-combinations has to be tested, showing that the condition can independently determine the value of the decision. Multicondition Coverage includes condition coverage. Additionally, it measures if all TRUE-FALSE-combinations for the evaluation of the atomic conditions inside the decision have been tested. Multicondition Coverage Multicondition Coverage Multicondition Coverage includes condition coverage. Additionally, it measures if all TRUE-FALSE-combinations for the evaluation of the atomic conditions inside the decision have been tested. In Testwell CTC++, line coverage is a visual indication in the HTML report if a source code line was executed or not. Line Coverage Line Coverage In Testwell CTC++, line coverage is a visual indication in the HTML report if a source code line was executed or not. With the different instrumentation modes for coverage and for timing, different structures of the source code are instrumented. The Instrumentation Modes of ctcThe Instrumentation Modes of ctc With the different instrumentation modes for coverage and for timing, different structures of the source code are instrumented. Function coverage provides the lowest code instrumentation overhead, only the beginning of function bodies are enriched with probes. Function Coverage InstrumentationFunction Coverage Instrumentation Function coverage provides the lowest code instrumentation overhead, only the beginning of function bodies are enriched with probes. Decision coverage instrumentation includes function coverage. Additionally, all conditional and unconditional branches and control transfers are instrumented, leading to the use of different counter types. Decision Coverage InstrumentationDecision Coverage Instrumentation Decision coverage instrumentation includes function coverage. Additionally, all conditional and unconditional branches and control transfers are instrumented, leading to the use of different counter types. With multicondition instrumentation, the same parts of the code are instrumented in general as with decision instrumentation. Additionally, assignments are instrumented. Multicondition Coverage InstrumentationMulticondition Coverage Instrumentation With multicondition instrumentation, the same parts of the code are instrumented in general as with decision instrumentation. Additionally, assignments are instrumented. Beside structural coverage, Testwell CTC++ is also able to measure the time or similar cost during code execution. Timing (Execution Cost) InstrumentationTiming (Execution Cost) Instrumentation Beside structural coverage, Testwell CTC++ is also able to measure the time or similar cost during code execution. Configuration of Testwell CTC++Configuration of Testwell CTC++ Configuration files (ctc.ini) store compiler-specific settings and a connection string to the license used by Testwell CTC++.Configuration FileConfiguration File Configuration files (ctc.ini) store compiler-specific settings and a connection string to the license used by Testwell CTC++. Configurations files are looked up from several locations, and determine the settings in a certain order. Hierarchy of Configuration Files and Command BlocksHierarchy of Configuration Files and Command Blocks Configurations files are looked up from several locations, and determine the settings in a certain order. Configuration parameters are used with Testwell CTC++ to achieve three general goals: Setting of user/ project depending decisions; configuring a specific compiler; adapting to rare language and environment restrictions. Configuration ParametersConfiguration Parameters Configuration parameters are used with Testwell CTC++ to achieve three general goals: Setting of user/ project depending decisions; configuring a specific compiler; adapting to rare language and environment restrictions. The following parameters might typically set by a user depending on the project or the specific user setup.User and Project Specific ParametersUser and Project Specific ParametersThe following parameters might typically set by a user depending on the project or the specific user setup. These three configuration parameters are used to exclude source files from instrumentation (but not from compilation).. They define matching rules consecutively. EXCLUDE, NO_EXCLUDE and NO_INCLUDEEXCLUDE, NO_EXCLUDE and NO_INCLUDE These three configuration parameters are used to exclude source files from instrumentation (but not from compilation).. They define matching rules consecutively. With this configuration parameter, a list of function names can be defined triggering the saving of counter data. EMBED_FUNCTION_NAMEEMBED_FUNCTION_NAME With this configuration parameter, a list of function names can be defined triggering the saving of counter data. This configuration parameter determines the name and the path of the data file collecting the counter data during the test run.DATAFILEDATAFILEThis configuration parameter determines the name and the path of the data file collecting the counter data during the test run. This configuration parameter decides if const declarations are instrumented. Possible values are OFF and ON.CONST_INSTRCONST_INSTR This configuration parameter decides if const declarations are instrumented. Possible values are OFF and ON. This setting defines a string to be printed to the timing listing bctcpost. It informs the reader about the type of execution cost values. EXECUTION_COST_TYPEEXECUTION_COST_TYPE This setting defines a string to be printed to the timing listing bctcpost. It informs the reader about the type of execution cost values. With this configuration parameter, a list of functions is defined that are not instrumented. SKIP_FUNCTION_NAMESKIP_FUNCTION_NAME With this configuration parameter, a list of functions is defined that are not instrumented. With this configuration parameter, a list of functions is defined whose parameters shall not be instrumented. SKIP_PARAMETER_LISTSKIP_PARAMETER_LIST With this configuration parameter, a list of functions is defined whose parameters shall not be instrumented. The TICK value is used to scale timer values in reports. TICKTICK The TICK value is used to scale timer values in reports. TIMER defines the name of the cost function, used for execution cost measurement, if timing instrumentation has been selected. TIMERTIMER TIMER defines the name of the cost function, used for execution cost measurement, if timing instrumentation has been selected. This configuration parameter sets the directory to store temporary files during instrumentation. TMP_DIRECTORYTMP_DIRECTORY This configuration parameter sets the directory to store temporary files during instrumentation. The warning level for ctc is set with this parameter. Possible values are none, warn and info. WARNING_LEVELWARNING_LEVEL The warning level for ctc is set with this parameter. Possible values are none, warn and info. These settings shall be usually made once to introduce a new compiler or linker. Verifysoft provides a lot of these compiler configurations on request, and they do not have to be changed typically by a user.Compiler Specific ParametersCompiler Specific ParametersThese settings shall be usually made once to introduce a new compiler or linker. Verifysoft provides a lot of these compiler configurations on request, and they do not have to be changed typically by a user. Compiler and linker commands are made known to Testwell CTC++ with this parameter. COMMANDCOMMAND Compiler and linker commands are made known to Testwell CTC++ with this parameter. With TYPE, the compiler or linker command given by COMMAND is specified. Possible values are compiler, compiler_linker, linker and borland_linker. TYPETYPE With TYPE, the compiler or linker command given by COMMAND is specified. Possible values are compiler, compiler_linker, linker and borland_linker. This configuration parameter specifies if the compiler output file option is different from the linker output file option. Possible values are ON and OFF. DIFF_COMP_AND_LINK_OPTSDIFF_COMP_AND_LINK_OPTS This configuration parameter specifies if the compiler output file option is different from the linker output file option. Possible values are ON and OFF. This configuration parameter sets the extension for object files OBJECT_EXTENSIONOBJECT_EXTENSION This configuration parameter sets the extension for object files This configuration parameter defines a list of options and other arguments ctc shall add when invoking the compilation.OPT_ADD_COMPILEOPT_ADD_COMPILE This configuration parameter defines a list of options and other arguments ctc shall add when invoking the compilation. This configuration parameter may be used to extend the preprocessing command invoked by ctc with some additional parameters. OPT_ADD_PREPROCOPT_ADD_PREPROC This configuration parameter may be used to extend the preprocessing command invoked by ctc with some additional parameters. This parameter sets the options used for specifying the object output file of the compilation. OPT_COMP_OUTOPT_COMP_OUT This parameter sets the options used for specifying the object output file of the compilation. The configuration parameter OPT_COMPILE_ONLY specifies the option(s) to compile without linking. OPT_COMPILE_ONLYOPT_COMPILE_ONLY The configuration parameter OPT_COMPILE_ONLY specifies the option(s) to compile without linking. This configuration parameter defines a list of options triggering ctc to execute the original compilation command first without modification.. After that, normal instrumentation processing is done. OPT_DO_2COMPOPT_DO_2COMP This configuration parameter defines a list of options triggering ctc to execute the original compilation command first without modification.. After that, normal instrumentation processing is done. This setting advises ctcctc which options are used for specifying the linkage output file. OPT_LINK_OUTOPT_LINK_OUT This setting advises ctcctc which options are used for specifying the linkage output file. This configuration parameter defines a list of command options, which must not be used by ctc during the compilation of the instrumented source files. OPT_NO_COMPILEOPT_NO_COMPILE This configuration parameter defines a list of command options, which must not be used by ctc during the compilation of the instrumented source files. This configuration parameter defines a list of options for which ctc will do nothing except emitting the original command with its original parameters.OPT_NO_CTCOPT_NO_CTCThis configuration parameter defines a list of options for which ctc will do nothing except emitting the original command with its original parameters. This configuration parameter defines a list of command options, which must not be used by ctcduring the linking of the instrumented object files. OPT_NO_LINKOPT_NO_LINK This configuration parameter defines a list of command options, which must not be used by ctcduring the linking of the instrumented object files. This configuration parameter defines a list of compiler/ linker options that must not be put in a response file by ctc. OPT_NO_OPTFILEOPT_NO_OPTFILE This configuration parameter defines a list of compiler/ linker options that must not be put in a response file by ctc. This configuration parameter defines a list of compiler/ linker options which must not be used during the pure preprocessing phase ctc invokes. OPT_NO_PREPROCOPT_NO_PREPROCThis configuration parameter defines a list of compiler/ linker options which must not be used during the pure preprocessing phase ctc invokes. OPTCHARS is a list defining the characters the compiler/ linker command allows to specify an option (typically "-" or "/"). OPTCHARSOPTCHARS OPTCHARS is a list defining the characters the compiler/ linker command allows to specify an option (typically "-" or "/"). OPTFILE is a list defining the characters a response file (options file, command file) for the compiler/ linker command is given with. Usually, the supported character is "@". OPTFILEOPTFILE OPTFILE is a list defining the characters a response file (options file, command file) for the compiler/ linker command is given with. Usually, the supported character is "@". The configuration parameter PARAMS is one of four lists of compiler and linker options. PARAMS contains all options having one mandatory argument with an optional blank space between the option and the argument. PARAMSPARAMS The configuration parameter PARAMS is one of four lists of compiler and linker options. PARAMS contains all options having one mandatory argument with an optional blank space between the option and the argument. The configuration parameter PARAMS_LIST is one of four lists of compiler and linker options. PARAMS_LIST contains all options having a list of arguments. PARAMS_LISTPARAMS_LIST The configuration parameter PARAMS_LIST is one of four lists of compiler and linker options. PARAMS_LIST contains all options having a list of arguments. The configuration parameter PARAMS_LIST_SEPARATOR sets one or more separator characters that are used in argument lists for compiler and linker options specified in PARAMS_LIST. PARAMS_LIST_SEPARATORPARAMS_LIST_SEPARATOR The configuration parameter PARAMS_LIST_SEPARATOR sets one or more separator characters that are used in argument lists for compiler and linker options specified in PARAMS_LIST. The configuration parameter PARAMS_OPTIONAL is one of four lists of compiler and linker options. PARAMS_OPTIONAL contains all options having one optional argument. PARAMS_OPTIONALPARAMS_OPTIONAL The configuration parameter PARAMS_OPTIONAL is one of four lists of compiler and linker options. PARAMS_OPTIONAL contains all options having one optional argument. The configuration parameter PARAMS_SEPARATE is one of four lists of compiler and linker options. PARAMS_SEPARATE contains all options having one mandatory argument with a mandatory blank space between the option and the argument. PARAMS_SEPARATEPARAMS_SEPARATE The configuration parameter PARAMS_SEPARATE is one of four lists of compiler and linker options. PARAMS_SEPARATE contains all options having one mandatory argument with a mandatory blank space between the option and the argument. With PREPROC_C, the invocation of the preprocessing stage is specified for C files. PREPROC_CPREPROC_C With PREPROC_C, the invocation of the preprocessing stage is specified for C files. With PREPROC_CXX, the invocation of the preprocessing stage is specified for C++ files. PREPROC_CXXPREPROC_CXX With PREPROC_CXX, the invocation of the preprocessing stage is specified for C++ files. These parameters are usually set once by a customer or a user, if needed, to adapt Testwell CTC++ to their environment.Customer/ Environment Specific ParametersCustomer/ Environment Specific ParametersThese parameters are usually set once by a customer or a user, if needed, to adapt Testwell CTC++ to their environment. For server based (floating) licenses, this parameter specifies the port and the host name of the license server. For file based licenses (evaluation or node-locked), it specifies the location of the license file. FLEXLM_LICENSE_FILEFLEXLM_LICENSE_FILE For server based (floating) licenses, this parameter specifies the port and the host name of the license server. For file based licenses (evaluation or node-locked), it specifies the location of the license file. This configuration parameter sets the C++ dialect used to recognize additional keywords and for the syntax of inline assembly code. Possible values are ANSI, MICROSOFT and BORLAND. DIALECT_CXXDIALECT_CXXThis configuration parameter sets the C++ dialect used to recognize additional keywords and for the syntax of inline assembly code. Possible values are ANSI, MICROSOFT and BORLAND. This configuration parameter sets the C dialect used to recognize additional keywords and for the syntax of inline assembly code. Possible values are ANSI, MICROSOFT and BORLAND. DIALECT_CDIALECT_C This configuration parameter sets the C dialect used to recognize additional keywords and for the syntax of inline assembly code. Possible values are ANSI, MICROSOFT and BORLAND. This configuration parameter sets the characters allowed by a compiler for line-end comments of inline assembly code. ASSEMBLY_COMMENT_CHARSASSEMBLY_COMMENT_CHARS This configuration parameter sets the characters allowed by a compiler for line-end comments of inline assembly code. This configuration parameter defines a list of file extensions for C source files.EXT_CEXT_CThis configuration parameter defines a list of file extensions for C source files.This configuration parameter defines a list of file extensions for C# source files.EXT_CSHARPEXT_CSHARPThis configuration parameter defines a list of file extensions for C# source files.This configuration parameter defines a list of file extensions for C++ source files.EXT_CXXEXT_CXXThis configuration parameter defines a list of file extensions for C++ source files.This configuration parameter defines a list of file extensions for Java source files.EXT_JAVAEXT_JAVAThis configuration parameter defines a list of file extensions for Java source files.This configuration parameter defines a list of file extensions for Objective-C source files.EXT_OBJCEXT_OBJCThis configuration parameter defines a list of file extensions for Objective-C source files.This configuration parameter defines a list of file extensions for files that ctc shall not instrument.EXT_OTHEREXT_OTHERThis configuration parameter defines a list of file extensions for files that ctc shall not instrument. This configuration parameter sets the Testwell CTC++ run-time library to be linked into the instrumented executable. Beside that, other libraries and command line options can be defined in this list.LIBRARYLIBRARY This configuration parameter sets the Testwell CTC++ run-time library to be linked into the instrumented executable. Beside that, other libraries and command line options can be defined in this list.This configuration parameter sets a threshold for ctc how long commands may become. Beyond that threshold, ctc uses a response file to store parts of the command.MAX_CMDLINE_LENGTHMAX_CMDLINE_LENGTHThis configuration parameter sets a threshold for ctc how long commands may become. Beyond that threshold, ctc uses a response file to store parts of the command. With the configuration parameters RUN_BEFORE_ALL, RUN_AFTER_CPP, RUN_AFTER_INSTR and RUN_AFTER_COMP, the execution of additional programs or scripts can be triggered at certain steps during the execution of ctc. Each of these parameters may contain a list of program or script file names.Parameters for Hook Scripts Parameters for Hook Scripts With the configuration parameters RUN_BEFORE_ALL, RUN_AFTER_CPP, RUN_AFTER_INSTR and RUN_AFTER_COMP, the execution of additional programs or scripts can be triggered at certain steps during the execution of ctc. Each of these parameters may contain a list of program or script file names. This configuration parameter determines how source files are identified during instrumentation, testing and reporting by Testwell CTC++.SOURCE_IDENTIFICATIONSOURCE_IDENTIFICATION This configuration parameter determines how source files are identified during instrumentation, testing and reporting by Testwell CTC++. The examination of the run-time behavior of a program is called dynamic analysis. Beyond code coverage, Testwell CTC++ can analyze the execution time of functions. About Dynamic Analysis in Testwell CTC++ About Dynamic Analysis in Testwell CTC++ The examination of the run-time behavior of a program is called dynamic analysis. Beyond code coverage, Testwell CTC++ can analyze the execution time of functions. There is a special option, -hard, that can be used with ctcwrap in some situations, changing the behavior of ctcwrap completely.Special "-hard" Mode of ctcwrapSpecial "-hard" Mode of ctcwrapThere is a special option, -hard, that can be used with ctcwrap in some situations, changing the behavior of ctcwrap completely.The error and warning messages of the different tools of Testwell CTC++ can be found here.Error MessagesError MessagesThe error and warning messages of the different tools of Testwell CTC++ can be found here.Error messages are written to stderr. When ctc is aborted by an error, exit code 1 is set instead of 0.Error Messages of ctcError Messages of ctcError messages are written to stderr. When ctc is aborted by an error, exit code 1 is set instead of 0.During test runs with the instrumented program, warning and error messages are written by the Testwell CTC++ run-time library to stderr.Error Messages during Program ExecutionError Messages during Program ExecutionDuring test runs with the instrumented program, warning and error messages are written by the Testwell CTC++ run-time library to stderr.When reports are generated with the postprocessor ctcpost, error and warning messages are written to stderr. Also, an exit value 1 is returned whenever postprocessor execution is aborted by an error.Error Messages of ctcpostError Messages of ctcpostWhen reports are generated with the postprocessor ctcpost, error and warning messages are written to stderr. Also, an exit value 1 is returned whenever postprocessor execution is aborted by an error.The messages are written to stderr. Each message is considered fatal. i.e. the tool run is aborted with exit code -1 after displaying the message. In a successful run the tool returns with exit code 0. Error Messages of ctcdiffError Messages of ctcdiffThe messages are written to stderr. Each message is considered fatal. i.e. the tool run is aborted with exit code -1 after displaying the message. In a successful run the tool returns with exit code 0. Error messages and warnings of ctc2html are written to stderr. An exit value zero is returned if the ctc2html run was successful, and a value greater than zero if not.Error Messages of ctc2htmlError Messages of ctc2htmlError messages and warnings of ctc2html are written to stderr. An exit value zero is returned if the ctc2html run was successful, and a value greater than zero if not.The error messages are written to stderr, containing a error number, an internal error code, and a description.Error Messages of ctcxmlmergeError Messages of ctcxmlmergeThe error messages are written to stderr, containing a error number, an internal error code, and a description.The error messages are written to stderr.Error Messages of ctc2datError Messages of ctc2datThe error messages are written to stderr.In general, no change in the source code is necessary to use Testwell CTC++. In some use cases, specific #pragma instructions can be used to customize the behavior in detail.Reference of PragmasReference of PragmasIn general, no change in the source code is necessary to use Testwell CTC++. In some use cases, specific #pragma instructions can be used to customize the behavior in detail.Testwell CTC++ uses several dedicated environment variables, and some generic ones. They can be setup system wide, user specific or temporarily in the current session as supported by the underlying operating system.Environment VariablesEnvironment VariablesTestwell CTC++ uses several dedicated environment variables, and some generic ones. They can be setup system wide, user specific or temporarily in the current session as supported by the underlying operating system.With the Testwell CTC++ approach, there are a few language elements that cannot be instrumented in general. This collection provides the reasons for these limitations as well as some ways to cope with them.Conceptual Limitations of InstrumentationConceptual Limitations of InstrumentationWith the Testwell CTC++ approach, there are a few language elements that cannot be instrumented in general. This collection provides the reasons for these limitations as well as some ways to cope with them.The right hand side of declarations with qualifier const must not be instrumented if the variable shall be compile-time constant. Declarations with Qualifier constDeclarations with Qualifier constThe right hand side of declarations with qualifier const must not be instrumented if the variable shall be compile-time constant. C++ functions with the specifier constexpr are not instrumented by Testwell CTC++. In general, this is not possible as the instrumentation would destroy key properties of these functions and their return values.Functions with Specifier constexprFunctions with Specifier constexprC++ functions with the specifier constexpr are not instrumented by Testwell CTC++. In general, this is not possible as the instrumentation would destroy key properties of these functions and their return values.Testwell CTC++ uses FlexNet Publisher from revenera for license management. Licenses are managed on a license server (floating licenses) or provided in a license file located at the client side (e.g., evaluation licenses).License HandlingLicense HandlingTestwell CTC++ uses FlexNet Publisher from revenera for license management. Licenses are managed on a license server (floating licenses) or provided in a license file located at the client side (e.g., evaluation licenses).The license server used to manage floating licenses for Testwell CTC++ is an independent software package available for Windows, Linux and macOS. An existing license server from revenera / FlexNet Publisher can be used, or the full package can be downloaded from the customer area of Verifysoft. Putting the License Server into ServicePutting the License Server into ServiceThe license server used to manage floating licenses for Testwell CTC++ is an independent software package available for Windows, Linux and macOS. An existing license server from revenera / FlexNet Publisher can be used, or the full package can be downloaded from the customer area of Verifysoft. Every Testwell CTC++ instance needs a license to run. This license can be obtained by a local license file, often named testwell.lic, or from a license server managing floating licenses for Testwell CTC++.License Supply for the ClientLicense Supply for the ClientEvery Testwell CTC++ instance needs a license to run. This license can be obtained by a local license file, often named testwell.lic, or from a license server managing floating licenses for Testwell CTC++.