Verifysoft Technology GmbH Testwell CTC++ Version 9.1.4.1 11 April 2022 This file describes the changes in successive versions of Testwell CTC++. The latest version is described first. Version 9.1.4.1 (11 April 2022) ------------------------------- This version fixes an issue in ctc introduced with the bug fix in version 9.1.4 regarding line numbers of include directives. When included files are instrumented, some constellations of include and other preprocessor directives could lead to the following issue: In the symbol file (MON.sym), ctc did not record the return from instrumented headers to the including file. In consequence, the functions from this file were treated as if they belonged to the header. If this issue has occured for a project instrumented with v9.1.4, a full rebuild with v9.1.4.1 is necessary to ensure that the symbol file is rewritten. Version 9.1.4 (18 March 2022) ----------------------------- Features and Changes: - With version 9.1.4, macOS (Big Sur, Monterey) is supported for x86_64 and arm64 (M1) architectures. Bug Fixes: - A crash of ctc due to an access violation could happen with version 9.1.3. - During instrumentation, ctc could get into an endless loop caused by cyclic type definitions like: typedef unsigned char uint8_t; typedef uint8_t uint8; typedef uint8 uint8_t; These kind of type definitions have been encountered in library code from Visual Studio, versions 15 and 17, and from clang. - While processing template parameters, additional nested templates could hinder ctc from recognizing the end of the first parameter list. This could lead to a syntax error of ctc for the end of the source file, or to uninstrumented code. - In the definition of a namespace, using attributes as in namespace [[deprecated("Reason")]] example { ... } hindered ctc from recognizing the namespace correctly. This led to misnamed functions in the reports of Testwell CTC++ but could also cause subsequent faults in processing templated types, for example. - With version 9.1.3, a new handling of line numbers associated with include directives was introduced. With some compilers (for example, DesignWare ARC MetaWare), this line number was not always derived correctly, and this could lead to rare problems like a crash of ctcxmlmerge. Version 9.1.3 (03 December 2021) -------------------------------- Features and Changes: - With the new environment variable CTC_DATA_NAME, the name of the data file written out during execution of an instrumented program can be set. This name substitutes the name set by default (“MON.dat”) or at instrumentation time with the parameter DATAFILE. With that possibility, it is easier to distinguish different test runs with the same executable. A differing path for the data file is still determined by the environment variable CTC_DATA_PATH. Example (Windows-oriented): Unit tests and integration tests share an instrumented library set CTC_DATA_NAME=UnitTests.dat … execute unit tests … set CTC_DATA_NAME=IntegrationTests.dat … execute integration tests … - On Linux, HOTA / BITCOV is now also part of the standard installation procedure (like already on Windows). The documentation of BITCOV is moved to the Testwell CTC++ Help. For Linux, the improved error message of dmp2txt (see release 9.1.2) is deployed with this update. - With the installation on Windows, files for Cygwin integration are copied to the Testwell CTC++ directory in folder \Cygwin_integ. There is a makefile to perform the actual installation of this integration. This package has been improved to work better regarding installation, configuration, and parallel usage with gcc compiler from MinGW. Bug Fixes - User-defined types introduced with using namespace ns_name; using ns_name::name; were not always recognized correctly by ctc during instrumentation. This led to different issues: - Declarations inside if-statements were not recognized. - Static casts like static_cast were not handled correctly. In both cases, the instrumentation resulted in uncompilable code. - A static cast with many nested angular brackets could lead to a syntax error of ctc for the end of the source file to be instrumented. - Paths with more than 260 characters could cause a crash of ctc. This is fixed by cutting the beginning of the path. - A special situation could lead to a wrong presentation in the source code view of the HTML report: - several instrumented header files are included in a source file, - one of them is reported as a part of its including source file by ctcpost because there was a different variant of the header file extracted before. To fix the presentation, the line numbers associated by ctc with the #include directives are changed and harmonized. Hence a full rebuild with a new symbol file MON.sym is necessary for this change to take effect for a project. - For C++ code, non-constant expressions in if-statements like if (A & 1); were wrongly recognized as constant, if they started with one non-constant operand followed by certain operators and several constant operands. Version 9.1.2 (05 August 2021) ------------------------------ Change: - During execution of instrumented programs, the standard Testwell CTC++ runtime libraries will not perform a license check any longer. This harmonizes the behavior between standard and HOTA libraries and leads to the following advantages: - If the test is executed on a different computer, no ctc.ini is needed there. - On Linux, the misleading warning no .eh_frame_hdr table will be created is no longer issued during linking. - On Windows, certain instrumented functions called by the entry function when loading a DLL (mainly C++ constructors of objects with static lifetime) could cause a hanging process. This is fixed by this change. Bug Fixes: - When the header extraction was switched off with option -nhe, the file exclusion with option -nf did not work in ctcpost. - A C++ function defined with a combination of noexcept and override as specifiers was not instrumented. - Less-than signs inside template <…> brackets were not always recognized correctly. This led to not instrumented source code. - Source files that are compiled both directly and included in other files (in tests for example) are reported twice in Testwell CTC++: In the text report, they appear as "Monitored Source File" and as "Monitored Header File". In the HTML report, one of these variants was missing. Now, both variants occur (with separated counter/ coverage), the second one from the text report is displayed like "foo.cpp (other)". - Source code comments starting on a function end line and reaching over several lines led to missing highlighting for the following code. - When a language element like if, for, do, while is spread over more than one line, Testwell CTC++ generally assigns the corresponding probe to the last line. However, if the character terminating the decision itself was on a new line (closing bracket or semicolon in case of "for"), the line before was assigned. That led to line highlighting issues in the HTML report. - After a combination of strings and line continuation characters inside and outside the string like printf("x is \ %d\n",\ 2); single source code lines were not shown in the HTML report. - The license information given with "ctc -V" was not correct if the port number of the license server was not set in ctc.ini or in the environment variables TESTWELLD_LICENSE_FILE or LM_LICENSE_FILE. The actual license check out was not affected by this issue. - When the build tool called with ctclaunch was not available, the error message was misleading. Additionally, if the build tool finishes with an exit code greater than zero, this message is now clearly distinguished from ctclaunch error codes. - When dmp2txt was used with a wrong or corrupted MON.aux file, the error message was misleading. Version 9.1.1 (22 October 2020) ------------------------------- Changes: - The 32 and 64-bit run-time libraries for Linux and MinGW are no longer distinguished by their name, but put to different folders. A switch between 32-bit and 64-bit builds is now possible without switching LIBRARY entries in ctc.ini. - The build launcher ctclaunch for Windows works also on Windows 7 now. - To use ctclaunch with the IAR build system/ IAR Embedded Workbench, the option "-iar" is available. - ctclaunch also works for build processes using batch files (.bat or .cmd) now. - The license information given with ctc -V additionally shows licenses granted by an INCREMENT. - On Linux, the ctc runtime is no longer built with the switch "-no-pie". Hence this switch does not have to be used any longer for linking the instrumented code with the runtime. In consequence, also instrumented shared objects can be built now on Linux. Bug Fixes: - In a code line like vector v2 = static_cast&&>(v); the rvalue reference declarator "&&" was mistaken for a logical AND and ctc produced uncompilable code. - When the ctc runtime layer or ctc2dat could not open the datafile for writing, a crash occurred. This error is caught now, and ctc2dat ends with an error message. - Assembly blocks containing a "#" sometimes led to uncompilable instrumented code. - ctcwrap also handles the ctc option "-dump-config" now. - An improvement in v9.1.0 caused performance issues for ctc on Windows for source code with many header files included. - ctcdiff did not recognize Java lambdas in the text reports to be compared. - With the MinGW compiler command "g++", ctclaunch did not work properly. - For GNU compilers, the misplaced option -Xopenmp-target is removed from PARAMS_SEPARATE in the ctc.ini delivered with installation. Version 9.1 (03 April 2020) --------------------------- Starting with this version 9.1 of Testwell CTC++, all components are synchronized in their version number. Features and Changes: - New Help System: Testwell CTC++ is delivered with a new help based on HTML instead of the PDF-based User Guide. In the \Doc folder of the installation, a file called "Testwell CTC++ Help" can be found to start this Help. - New mechanism for build integration on Windows: A new tool called ctclaunch is released that can be used to integrate ctc easily in build systems like CMake or MSBuild. The purpose of ctclaunch is in particular the usage on build machines when ctcwrap cannot be used directly, and ctcwrap with option "-hard" is not acceptable in its consequences. Please see the Testwell CTC++ Help for a detailed description. - Instrumentation of "missing defaults" in switch-case-statements: To obtain full coverage, it is now also required to test the default case even if there is no explicit default statement given, comparable to a "missing else" branch in an if statement. Note that this may decrease coverage for existing projects after an upgrade. - Configuration dump for ctc: During instrumentation with ctc, the values of configuration parameters are set based on different configuration files (ctc.ini) and command line options. With the new ctc option "-dump-config", all these settings are written to a log file in a temp folder: Windows: %TEMP%\ctclog (using the environment variable %TEMP%) Unixes: /tmp/ctclog The filename starts with "ctclog" and contains a date and a process ID. This dump is useful to analyze cases with unexpected behavior. - Extended diagnosis for ctc with option "-V": For the search of configuration files, every search position is given now independently from the concrete INI file, if available. If a license is available, detailed information is given about the location of the license, the owner of the license, and the expiration date. If no license is available, the same license errors are shown as before. - Rounding behavior for coverage ratios (TER): Previously, different rounding rules were used to calculate the Test Effectiveness Ratio as a percentage, depending on the range of the TER. Now every ratio is rounded down to an integer. 100% coverage is still reached only if all counting points have been tested. - HTML report improvements: For the code view in HTML report (Execution Profile), the table header sticks now at the top when the page is scrolled down. In overview tables, the wrapping behavior is improved. The fonts used in default CSS are slightly modernized: Verdana is not longer used and Consolas is used for source code instead of Courier New. - Improved text report comparator ctcdiff: Functions are now compared even if their source file changed, as long as the function itself did not change. - Warning for misplaced line-ending characters: If source code files contain untypical line-end characters, for example the carriage return character in a Linux environment, a warning is now issued by ctc2html for this file. The HTML report is nevertheless generated. In some cases, it may show a corrupted source code view for these files, but all coverage figures are yet valid. - Starting of a browser with ctc2html: If ctc2html has to start the browser, it does not longer wait until the browser is closed. - Attribute __declspec for armcc: This compiler specific attribute is now also accepted for armcc. - Namespace support: Functions defined in namespaces are now identified by Testwell CTC++ with the name of the namespace included. Example: A (member) function foo in a class A in a namespace n was identified as A::foo before in the symbolfile and in all reports. Now it is identified as n::A::foo. Note: If namespaces are used, the symbolfile MON.sym of a project shall be renewed after an upgrade to version 9.1, making a full rebuild of the instrumented program necessary. - Withdrawal of JSON report: The generation of a coverage report in JSON format by ctcpost is not longer provided. Customers using this format are kindly asked to contact Verifysoft before an upgrade. - Withdrawal of ctc2excel: The converter ctc2excel producing CSV files from the Execution Profile Listing is not longer part of the Testwell CTC++ package. Customers using this format are kindly asked to contact Verifysoft before an upgrade. - License and tool information in ctc.ini: The configuration file of Testwell CTC++ does by default not longer contain a restricted block with information about the tool version and the way of licensing. Bug Fixes: - Process hung of ctc: For C++ code having identically named types referring differently in different namespaces, it was possible that ctc got in an endless loop. - Rounding on directory level in HTML report: Test Effectiveness Ratios on directory level are now always rounded down, consistently to other reports. An issue showing 100% on directory level even if the real TER was below 100% is therefore fixed. - Fixes around SOURCE_IDENTIFICATION = basename: This setting caused crashes of ctc before. - Casts to user-defined types: Casting a decision like x = (uint8)(a && b); could prevent the instrumentation of this decision. - Uncompilable Code: Instrumenting certain while-loops directly followed by a ternary-? operator, ctc caused compilation errors. - Fixed inconsistency for decision coverage: Assignments containing a multicondition like x = a && b; were instrumented and taken into account if the instrumentation mode was multicondition and the report was reduced to decision coverage. This is now synchronized with the behavior if the instrumentation mode is decision coverage, giving the following result: This kind of assignment is not taken into account for decision coverage, but for multicondition coverage, MC/DC and condition coverage. - Source code comments in HTML report: After comments containing strings and multiple lines, the line coverage highlighting was corrupted. - The decltype specifier in a typedef are now processed correctly by ctc. - When generating an XML report with ctcpost for MC/DC coverage, an error could occur for decisions with too many conditions. This is fixed. - Warning for too complex multiconditions: If a multicondition would lead to more than 500 evaluation alternatives, ctc issues a warning during instrumentation and instruments the multicondition as a decision only. This warning was previously shown only as an information (appearing only if WARNING_LEVEL was set to "info"). Now it is shown also for the default setting WARNING_LEVEL = warn. Version 9.0.3 (29 May 2019) --------------------------- This version 9.0.3 of CTC++ has the following components: Preprocessor 9.0.3 (updated from 9.0.2) Run-time libraries 9.0.3 (updated from 9.0) Postprocessor 9.0.3 (updated from 9.0) Header file ctc.h 9.0 Configuration file ctc.ini 9.0 CTC++ Coverage Comparator 1.0 CTC++ to HTML Converter 6.0 CTC++ to Excel Converter 3.3 CTC++ XML Merger utility 4.0 ctc2dat receiver utility 6.0 (corrected) and the following Windows platform specific components: Visual Studio Integration 4.4 (corrected) CTC++ Wrapper for Windows 3.5 (corrected) and the following Unix/Linux platform specific components: CTC++ Wrapper for Unix 1.14 (corrected) Enhancements and Bug Fixes: - Enhancement: Harmonized behaviour for file lists used by ctcpost. With the option -f, a list of source files can be specified for which the coverage report is built. With option -nf, a list of source files to be excluded from the report can be defined. For the convenient use of wildcards in these lists, the behaviour has been unified among operating systems: The argument of -f and -nf should now be included in "...". Example: ctcpost -nf "*\my_project\test*.c" -p profile.txt Remark: As before, the file names refer to the identifiers in the symbolfile (MON.sym), not to real files at the file system available at report time. The way source files are identified in the symbolfile is determined by the configuration parameter SOURCE_IDENTIFICATION, and this identification has to be used. - Bug fix: Handling of nested template parameters When instrumenting templates with nested template parameters it could happen that ctc did not parse the closing angle brackets correctly. The resulting error depended on the context after the closing bracket. - Enhancement: The locking mechanism used for datafiles has changed. Instead of lock files, operating system functions are used. If the environment variable CTC_LOCK_MAX_WAIT is still set, the old behaviour is in effect. - Bug fix: A rare ctcpost crash with generated code has been fixed. Version 9.0.2 (18 February 2019) --------------------------------- This version 9.0.2 of CTC++ has the following components: Preprocessor 9.0.2 (was: 9.0.1; seen by -h option) Run-time libraries 9.0 (unchanged; seen by 'ident' command applied on the library in some environments) Postprocessor 9.0 (unchanged; seen by -h option and in the listings) Header file ctc.h 9.0 (unchanged; seen in the file) Configuration file ctc.ini 9.0 (unchanged; seen in the file) CTC++ Coverage Comparator 1.0 (unchanged; seen by -h option) CTC++ to HTML Converter 6.0 (unchanged; seen by -h option) CTC++ to Excel Converter 3.3 (unchanged: seen by -h option) CTC++ XML Merger utility 4.0 (unchanged; seen by -h option) ctc2dat receiver utility 4.0 (unchanged; seen by -h option) and the following Windows platform specific components: Visual Studio IDE Integration 4.3 (unchanged; seen by clicking the Tw-icon in the dialog program and selecting "About...") CTC++ Wrapper for Windows 4.0 (unchanged: seen by -h option) and the following Unix/Linux platform specific components: CTC++ Wrapper for Unix 2.0 (unchanged; seen by -h option) In the CTC++ preprocessor (ctc): - Enhancement: The locking mechanism used for symbol files has changed. Instead of lock files, operating system functions are used. If the environment variable CTC_LOCK_MAX_WAIT is still set, the old behaviour is in effect. - Bug fix: In for loops, function calls without parameters like for(;B();) could cause a crash during instrumentation. This has been fixed. - Bug fix: In some constellations inside multiconditions, casts to user defined types were not interpreted correctly and the multicondition was not fully instrumented. This has been fixed. - Bug fix: Two closing angle brackets in a cast to a templated type, as in static_cast> could me misinterpreted as right shift operator. This has been fixed. - Bug fix: In C++, range-based for loops with a lambda expression as direct body caused a syntax error. This has been fixed. - Bug fix: In C++, using a lambda [=] capture caused uncompilable code. This has been fixed. Version 9.0.1 (12 December 2018) --------------------------------- This version 9.0.1 of CTC++ has the following components: Preprocessor 9.0.1 (was: 9.0; seen by -h option) Run-time libraries 9.0 (unchanged; seen by 'ident' command applied on the library in some environments) Postprocessor 9.0 (unchanged; seen by -h option and in the listings) Header file ctc.h 9.0 (unchanged; seen in the file) Configuration file ctc.ini 9.0 (unchanged; seen in the file) CTC++ Coverage Comparator 1.0 (unchanged; seen by -h option) CTC++ to HTML Converter 6.0 (unchanged; seen by -h option) CTC++ to Excel Converter 3.3 (unchanged: seen by -h option) CTC++ XML Merger utility 4.0 (unchanged; seen by -h option) ctc2dat receiver utility 4.0 (unchanged; seen by -h option) and the following Windows platform specific components: Visual Studio IDE Integration 4.3 (unchanged; seen by clicking the Tw-icon in the dialog program and selecting "About...") CTC++ Wrapper for Windows 4.0 (unchanged: seen by -h option) and the following Unix/Linux platform specific components: CTC++ Wrapper for Unix 2.0 (unchanged; seen by -h option) In the CTC++ preprocessor (ctc): - Bug fix/regression: In an un-braced for-each loop body containing a C++11 initialiser gave un-compilable code. E.g: while (int i=2) f({s}); - Bug fix: A crash happened with a name clash on variable template with class template. This is fixed. - Bug fix: "\x00" and other longer escape sequences get now never split from CTC++ tokeniser. Previous that gave either un-compilable code or a change of string content. Version 9.0 (07 November 2018) ------------------------------ This version 9.0 of CTC++ has the following components: Preprocessor 9.0 (was: 8.2.2; seen by -h option) Run-time libraries 9.0 (was: 8.2.2; seen by 'ident' command applied on the library in some environments) Postprocessor 9.0 (was: 8.2.2; seen by -h option and in the listings) Header file ctc.h 9.0 (was: 8.2-2; seen in the file) Configuration file ctc.ini 9.0 (was: 8.2.2; seen in the file) CTC++ Coverage Comparator 1.0 (unchanged; seen by -h option) CTC++ to HTML Converter 6.0 (was: 5.5; seen by -h option) CTC++ to Excel Converter 3.3 (unchanged: seen by -h option) CTC++ XML Merger utility 4.0 (was: 3.4; seen by -h option) ctc2dat receiver utility 4.0 (was: 3.7; seen by -h option) and the following Windows platform specific components: Visual Studio IDE Integration 4.3 (unchanged; seen by clicking the Tw-icon in the dialog program and selecting "About...") CTC++ Wrapper for Windows 4.0 (was 3.5: seen by -h option) and the following Unix/Linux platform specific components: CTC++ Wrapper for Unix 2.0 (was 1.4; seen by -h option) General Enhancement in the Testwell CTC++ toolchain: The way how different versions of instrumented source files are identified is fundamentally changed: Instead of the instrumentation timestamp, a fingerprint is used. In consequence, counter data can be reused more often. See User Guide for details. Please note that with this change in version 9.0, it is not possible to use symbolfiles or datafiles from older versions. In the CTC++ preprocessor (ctc): - Enhancement: Objective-C code is now instrumented by ctc. To identify source files with Objective-C code, there is a new parameter for the corresponding name extensions: EXT_OBJC = m, mm, M This setting is delivered in ctc.ini for compilers supporting Objective-C. - Enhancement: The recognition of compile-time constant expressions, e.g. if ( 0 ) is highly improved. More complex expressions like if ( 0 + ( (int) 42)) are recognized. Consequently, one evaluation (true OR false) during test is sufficient for full structural coverage. - Enhancement: Variable templates are now supported (C++14). - Enhancement: For compilers already supporting functionality of C++20 draft, support is added for constrained function templates. The constraints are not executed and thus not instrumented. - Change: For Java, methods inside enums are now instrumented. - Bug fix: With SKIP_PARAMETER_LIST, some specific functions can be defined whose parameters shall not be instrumented. This setting is not longer ignored when instrumentation mode is multicondition and ternary-? expressions appear in a parameter. - Bug fix: In a switch-case structure, the attributes [[clang::fallthrough]] and [[fallthrough]] do not longer prevent instrumentation. - Bug fix: "Constexpr if"- statements (C++17), for example if constexpr(1<2) are now supported. Previously they caused a syntax error. - Bug fix: In some situations, variables in brackets like (my_variable) were misinterpreted by ctc as casting. This is now fixed. - Enhancement: GNU C extension allows to use labels as values with the unary && operator, like in label: void *ptr = &&label; This does not longer cause a syntax error. - Enhancement: For the dialect Dynamic-C, costate blocks are now instrumented. - Bug fix: Universal character names like \u00c0 are now supported. - Enhancement: When a certain compiler is configured for ctc, i.e. for the parameter COMMAND in ctc.ini like COMMAND = ccarm the version with '-orig' attached is implicitely taken into account. Hence it is no longer necessary to list it explicitely like COMMAND = ccarm, ccarm-orig The '-orig' versions are needed for the ctcwrap-hard setup, for example. This behaviour already existed on Windows and is new on Unix/Linux. - Bug fix: Functions with the attribute __attribute((naked)) are not longer instrumented, as this resulted in uncompilable code. - Bug fix: An EOF character (255) as character literal does not longer break instrumentation. - Bug fix: When an element of an enumeration inside a typedef was used in an expression, a syntax error could occur. This is fixed. - Enhancement: Added instrumentation support for statement lambdas in Java and C#. - Enhancement: The lock file handling is improved to avoid timeouts and deadlocks. In the CTC++ postprocessor (ctcpost): - Enhancement: For lamdas, language dependent descriptions are used in the reports: C++: lambda-[]() Java: lambda-()-> C#: lambda-()=> Obj-C: lambda-^() - Change: On Windows platform, the runtime Dll files are renamed to - ctcwin32.dll - ctcwin64.dll with common ordinals unchanged. - Bug fix: In some special cases, a header file included in several source code files was not correctly identified by ctcpost when the extracted report for this file was created. This is fixed. In the CTC++ XML Merger utility: - Bug fix: The handling of many ternary conditional operators at a single source line is improved. Previously it allocated too much memory. In the CTC++ HTML converter (ctc2html): - Enhancement: In source code, also CR line breaks are accepted. - Enhancement: When the ecjctc-package is used for Java/ C# development, the generation of the HTML report works now directly, without taking back the instrumentation manually. ---------- Previous versions are not listed here.