Test strategies for emC sources (C/C++-Test)

Tools (a few jar files) are able to load from internet, see first vishia/SwEng/html/srcFileTree.html: chapter Libs and tools on the source tree.

The git archive Test_emC/.git does not contain the tools files, instead they should be gotten from internet. But it contains all information to load this files:

Wget as known linux cmd is not available unfortunately in a standard MinGW installation, neither it is anyway a standard on any Linux System. Hence it is provided with the minisys_vishia.jar for all systems where Java runs. But minisys_vishia.jar does more.

The GetWebfile works with a bom, a bill of material, see articel in german: Jeff Luszcz "Risiken bei Open-Source-Software: Warum eine Bill-of-Materials sinnvoll ist"

The tools.bom contains the re-check of the vishiaMinisys.jar, and check and download of vishiaBase.jar and vishiaGui.jar. The bom contains MD5 checksums. With it the already existing vishiaMinisys.jar is checked whether the checksum is okay. It it is not so, a warning is output. The other files are loaded and checked (whether the download is correct). If there are existing (on repeated call), the MD5 checksum is build and compared. The MD5 checksum is noted in this archive. Hence it is not possible (with the safety of MD5) to violate the files all on server, download process and on the own PC.

The next importance is: It is documented which files are used from where. Other systems loads some downloaded stuff in a home directory (C:\Users... on Windows), not simple obviously which and from where. And the third importance is: The sources of this jar files are stored beside the jar file at the server. The jar files can be build reproducible (see https://www.vishia.org/Java/html5/source+build/reproducibleJar.html).

The build directory is only temporary. It is recommended to redirect it to a RAM disk, which may be the location of the $TMP directory in windows or the /tmp/ in Linux. Advantage of a RAM-disk: It is faster, your hard disk (SSD) is spared. for that the

for Windows and Linux can be used to create a linked directory ./build redirected to the tmp folder.

can be used to clean the situation with the build, for a newly build process or to finish the work. It cleans also the directory in the TMP / tmp area, not only removes the build link (important for carefully using the disk space).

The working tree should be free of some temporary or resulting files. It should contain only sources. That gives the possibility to build a 'file copy' in form of a zip file for example, with compressed content. This is another proper possibility to save a safety version than git or another possibility for share sources.

On the other hand, all stuff should be done in the only one working tree without complex external file path settings. With the 'gradle' file tree concept the building results are stored in the build directory. Now it is possible to really store the content inside the tmp directory (on linux per default /tmp) using a link for the build sub directory.

This ./results directory contains the last reports and generation results for comparison and study. It does not contain libraries. Libraries depends on the platform, used compiler etc. They are not provided here.

The ./IDE directory contains IDE = Integrated Development Environments in sub directories for some test environments, for example Visusal Studio, Eclipse CDT but also for some embedded platforms.

An elab0rately explaination is given in vishia/SwEng/html/srcFileTree.html chapter Directories for the IDE beside src. Whether the IDE configuration files are part of the git for Test-EmC or specific gits or mirrored in ./src/IDE*, it is specifically regulated.

They are two git repositories present:

The git archives are hosted under github/JzHartmut.

See chapter Get sources, build and test how to get and use.

Git does not store the time stamps of the files. The reason for that may be that a make system needs new time stamps to make. This topic is discussed conflicting in internet. They are better make systems than the classic C/Unix maker with only check newer time stamps to decide whether to build or not. A better make system saves and re-uses a hash of the files to detect whether they are changed.

The time stamps may be a point of interesting to find out when a file was changed. This can be essentially in developing. Especially if you compare file trees, it is very helpfully to see the original time stamps. If you have a specified version, all original files should have a time stamp oder than the version’s date. If they have a time stamp of the copying or clone date, you may worse decide which files are changed by yourself (with newer time stamp) and which were the originals, because the date of clone or copying may be in the same time range as your change. For example you work on your project, and in the mean time you update a repository.

The time stamp in Unix (before Linux) and also in Linux have another approach as usual in Windows. DOS and Windows have always preserv the time stamp on copying, which is better to see whats happen. In contrast, on Unix the principle of make was always present (Unix tools were often delivered only as sources for different machines). The standard Unix maker uses the time stamp, more exact the comparison of the newer time stamp of a source against the older time stamp of the compilation result. Thats why in Unix the copy or creation of a file has had created anyway a new time stamp.

Because the emC test uses a better maker than the standard on Linux/Unix the files does not need a newer time stamp to decide for compilation. Instead a comparison with a list is done. That’s why it is possible, and should be done, to adjust the time stamps of the files after cloning or getting a zip from github.

For that reason a file

is always part of the files in the repository. This file contains the time stamps, and also the length and a CRC check of all files. The list is/should be updated with the current time stamp before commit. This is automatically done using the vishiaGitGui.html. A manual update can be invoked by calling the +buildTimestamps.sh found on the root of the file tree:

The +buildTimestamps.sh script looks whether it finds the .\tools (proper in the working tree), and if not it calls ~/cmds/javavishia.sh to execute Java with the vishia jar files.

The option -m: (mask) selects dedicated files which are member of the list. An elaborately description of this mask argument can be found in org.vishia.util.FilepathFilterM (used inside org.vishia.util.FileList). Here it means that the directories [result/**|src/...] are only candidats to add to the list, but also all files on root level with ~_*. It means all files but not files starting with underliner. In the src directory only src/[IDE*|buildScripts|cmds|docs|load_tools|test] are regarded. The other directories inside src have their own .fileList and git repository. Inside this src/…​ directories /** means that all sub directories are listed, and for the file level [~#*|~*#]. That exclude files starting or ending with #.

A more simple mask is given for the pure source archive:

This more simple mask means for directories [**/[~#*|~*#] all deep levels, files without starting and ending # and for the file level the same as above: ~_*. The ~ is a negate (exclude) operator. * is the known wildcard symbol in file path masks. A choice of files is written with [..|..] on start sequence (not after a *), also possible nested and with wild cards. That`s all.

Refer to org.vishia.util.FileList for detail documentation of this tool.

The mask should be follow the content of the git archive. That is not identically in any case. git uses the .gitignore file for an adequate approach. Files which are not included in the .filelist but in the git archive does not get the proper time stamp, not a problem if it is a few files. Files which are included in the .filelist but are not existing while touching are reported on console. May be they are missing in a git archive, detected while restoring (checkout, update, clone). That is also not a meaningful problem. But the mask and the git content should be tuned as well as possible.

As you see in the script line here the approach described in chapter Linux and Window: jar files and some scripts in /home/usr/cmds or ~/cmds and ~/vishia/tools are used here. This command should be able to invoke in all situations, just also on the mirror files with .git archive, independent of ./tools in the working tree (chapter tools).

To restore the timestamps execute +adjustTimestamps.sh:

This script looks whether it finds the \tools (proper in the working tree), and if not it calls ~/cmds/javavishia.sh to execute Java with the vishia jar files.

On restoring a time stamp it is tested whether the file is unchanged. Only then the time stamp is restored, of course. A changed state of a file is detected primary by its length (fast operation). If the length is identically, the CRC code is built with the content. Only if the CRC is the same, the time stamp is applied. There is a very small gap that a changed file has the same length and the same CRC.

Refer to org.vishia.util.FileList for detail documentation of this tool.

Both systems should be able to use. On Windows MS Visual Studio is supported. For Windows and Linux Eclipse-CDT is used. For test gcc is used on both systems.

Important: Case sensitivity filenames

Filenames on Windows are used non-case sensitive. It means errors in writing upper/lower case are not detected. But on Linux file names are case sensitive. Windows supports and saves the case of letters, but doesn’t evaluate it.

Rule: All file names should be used as case sensitiv with the proper case. The case of the filename in the git repository is determining. All #inlcude <…​> lines should be proper for case sensitivity.

It means a test under Linux is necessary to detect errors in faulty case of filenames.

It was seen that a faulty case writing of files in the MS Visual Studio project files forces a faulty case while editing and saving the file. Other tools respects the case on the file system. Hence also the case in the MS VS project files should be proper.

Important: Line end \n

All files in emC should have the line ending \n as in Unix/Linux. This is possible also in Windows. All Windows programs accept a simple \n (not only \r\n) as line ending, except the standard-Windows Notepad. But that program is usual not necessary to use.

The git rule for line ending should be 'store as is, get as is'. It means the line ending \n is stored also in the emC git repositories.

An conversion to \r\n only because 'we have here Windows' as usual in the mainstream is contra productive and unnecessary.

If some files have \r\n als line ending, it is a mistake! It is not seen in some editors.

MS-Visual Studio supports especially working with \n as line ending. This is done by the file

See comments there. This file is definitely detected by MS-Visual Studio and used.

Important: Using slash in include paths and all other paths

All [cp} compilers also in Windows accepts the slash / in the include paths. Only specific Windows programs (MS-Visual Studio internally) uses the backslash \ internally.

Actually some Windows programs accepts the slash meanwhile.

Because the scripts to compile etc are shell scripts, the slash is used there too. JZtxtmcd scripts uses the slash. All Java file paths accepts the slash also in Windows.

Don’t using Tabs

Tabs in texts are a bad decision because some editors uses different tab widhts and the text is picked apart. It is not really necessary to decide the 'own' hability for indentations. The emC sources uses an identation of two characters per level. It is enough to be obviously, and the indentation can be typed simple manually if necessary. If 4 characters are used for indentation (often), it is necessary to count while typing - an unnecessary effort.

Tabs inside a text are often not visible, and it is confusing if the cursor moves to unexpected positions while typing the arrow keys.

Automatically remove trailing white spaces is a good decision.

On Windows MinGW, Cygwin or adequate should be installed to support sh.exe for unix shell scripts and to offer the Gnu Compiler suite (gcc). It should be familiar for persons which uses gcc etc.

I have had originally oriented to MinGW, it has run with gcc, but it seems to haven’t support for pthread. All exclusively my src_emC/emC_srcOSALspec/os_LinuxGcc/os_thread.c has worked. Then I have changed to Cygwin, used it, also test under origninal Linux, all runs well. I have not time yet for MinGW. The content regarding MINGW is shifted to the last chapter, maybe helpfull or not.

Git should be access-able via command line

It is available often, elsewhere it should be installed from git-scm. The git should be installed under MS-Windows to

Elsewhere it does not run probably. Git contains a git/mingw64 part too. Both bin directories should added to the system PATH (may be correctly done by installation). The mingw inside git does not contain any compiler, but usual the same linux commands inclusively sh.exe.

Any git graphical environment can be installed, for example tortoise-git, but it is not presumed for working with emC sources. It is presumed for git actions.

On a Windows PC I have installed an ordinary git.

The system PATH was enhanced by this installation and contains:

It means git and also the mingw32 stuff delivered with git is in the PATH. But mingw32 does not contain the gcc compiler suite.

Hence I have install Cygwin on PC, cygwin.org. select proper tools, and gotten:

It is not installed to the Windows standard folder but in an own directory tree. It is not included in the system’s PATH.

To simple execute unix (linux) shell scripts with the extension .sh you can write a batch file unix-script.bat , able to found in the PATH. Templates of this files are stored in src/load_tools/windows_shellscriptBatches_template/ You should copy this files in a directory included in the system’s PATH (on my computer in C:\Programs\Batch\* and associate the extension .sh to unix-script.bat

Here you can simple change by comment lines, whether one of the following scripts is executed.

The major difference between git-script.bat and cygwin-script.bat is: The last one sets the PATH to the compiler, which are not contained in the git shell installation. See next chapter.

Some details:

converts the backslash (given on double click in calling argument) to the necessary slash.

The HOMEPATH and HOMEDRIVE variables sets the home directory which is known in Unix/Linux. So you can execute Unix/linux shell scripts nearly usual as in the originals. aption of the operation system access to Windows).

This script sets also the PATH to the desired Java version. The operation system can use per default also a maybe other Java version, Java16 or such, but the Java parts are yet tested with Java8 only (2021-04). See next chapters.

Important:

Because git comes with some stuff of mingw32, it may be confusing with the used Cygwin. Hence the PATH refers to Cygwin should be firstly seen in the PATH variable. It may be possible to remove the c:\Program Files\git\mingw32\bin entry from git, but then some git stuff may nor run outside of this test environment. If the PATH contains Cygwin firstly, all executable available for Cygwin should be find firstly and used. If you add Cygwin at last in the PATH, some executables from git mingw are used in concurrence to Cygwin, and that gives a version mismatch.

If you want to use pure git independent of Cygwin (for other approaches) you may have another batch to include only git in the PATH (I have an adequate git-script.bat file) or you may inserted the PATH to git in the system installation.

tools

The jar files which are necessary for some operations to get the sources etc. are the same as described in sub chapter tools in the first chapter (directory tree). It may be recommended to store this jar files on a central position either in the own home directory in linux or in a central directory in /usr/vishia/tools/ for all user on a linux system. For windows it may be stored in C:\Programs\vishia\tools\. Then you can make symbolic links to this location, and you have anyway the same jar files with its sources present ready for use independent of your concretely work.

You can use the

from the Test_emC zip file or clone from git to get this jar files.

For the following explanation it is presumed that this central directory for the tools is

whereby ~/ is the home directory in Linux, and also in Windows on a proper location for execution of shell-scripts.

After downloading the tools (or copying from a zip) you find there:

Yous see here in this snapshot on a SSH shell to a Linux computer that also the sources as jar are presented here. The name of the jar files are equalized to canonical names. The real names with version are contained in the tools.bom, as snapshot:

cmds

To simply call java if different versions are available on the PC, or to call JZtxtcmd scripts without necessity of change the PATH in a Linux environmentt you should have a cmds directory in your home location.

/home/user/cmds/ or more short ~cmds/

This is also valid for a shell script execution environment, see chapter before: For MS-Windows: Asscociate extension .sh to unix-script.bat, see its settings of HOMEPATH and HOMEDRIVE

In this directory some script should be present and adapted to call JZtxtcmd scripts and Java programs. Note: JZtxtcmd is frequently used for generation in emC. This scripts are contained as template in ./src/load_tools/cmds_template to copy and adapt.

This script calls purely only the class org.vishia.jztxtcmd.JZtxtcmd with the script to execute.

In the last script there are some things to adapt:

The content of the script can be copied from here, and adapted if necessary.

You can enhance the PATH locally, how it is done with this unix-script.bat start batch. The enhancement of a script variable is also valid inside the called script (for Windows inside the whole console process). That approach is known by all experts.

The other possibility is: On installation process on a special tool the installer enhances the systems settings. Then the tool runs without any scripting. This is the common way for ordinary installations.

Setting a special path into the PATH variable in a script has the advantage for more experience. You will see what is really necessary. You can choose between different tools and versions which uses the same command names (sh.exe, gcc.exe etc.)

Java should be a standard on any PC system. For some build- and translation tools JRE8 is used. This should be checked with console command:

On Linux the tools are tested with OpenJDK Runtime 11.0.6, it runs.

If another JRE Version (Higher than 8) is used as default and it does not run, you can adapt the PATH for a JRE8, see chapter For MS-Windows: Asscociate extension .sh to unix-script.bat

This action defines a local enhanced PATH without change the environment of the system.

For Linux the gcc package (GNU) should be installed:

It should be familiar for Linux users which uses C/C++-Compilation.

On MS-Windows gcc is contained in Cygwin, see above.

Firstly the algorithm of the C-sources should be tested. It should be independent of the used compiler and there options. Hence any compiler can be used for test of the sources, for example a Visual Studio compiler, gcc or other.

Secondly, it is possible that an algorithm works proper with the selected compiler, but fails in practice on an embedded hardware. What is faulty? It is possible for example that the target compiler has better optimization, and a property keyword such as volatile is missing in the source. It is a real failure in the source, but it was not detected by the test run with lower optimization. Some other adequate mistakes may be existing in a software.

In conclusion of that, the compiler and its optimization level should be carefully set. The test should be done with more as one compiler and with different optimization levels. For nightly tests the night may be long enough.

The next question is: "Test in the real target hardware". An important answer is: "The test should not only be done in the special hardware environment, the sources should be tested in different environment situations". For example, an algorithm works properly in the special hardware environment because some data are shortened, but the algorithm is really faulty. Ergo, test it in different situations.

But the test in the real target environment, with the target compiler, running inside the real hardware platform may be the last of all tests. It can be done as integration test of course, but the modules can be tested in this manner too.

It means, the test should compile for the target platform, load the result into the target hardware, run there, get error messages for example via a serial output, but run it as module test. Because not all modules may be able to load in one binary (it would be too large), the build and test process should divide the all modules in proper portions and test one after another, or test parallel in more as one hardware board.

There are some IDE ("Integrated Development Environment") project files:

Offering special test projects for various topics has not proven successful, because the maintenance of some more projects is a too high effort. Instead, there is exactly one project for any platform (it means at least two, one for Visual Studio and one for Eclipse CDT). To test a special topic there is a main routine which’s calling statements are commented, only the interesting call is used, for single step in debug. This is simple to make.

This is a snapshot of the current situation. This main routine is used for both IDE.

The include path is IDE- and configuration-specific in the IDE. For both IDEs different path are used for the

This file(s) should be changed for several Variants for emC compilation. Of course any commit contains the last used situation, not a developer progress in any case.

The applstdef are located in

It is for Visual Studio. The same set of files, but other files are existing for Eclipse-CDT, see project.

This may be seen as preparation for the nightly 'test all' (c) but can also be seen as an intermediate test while development.

For this approach the Test Selection GUI can be used (chapter #testSelection). With the GUI the test cases and test conditions are set. On button [ gen selection ] this gui produces the necessary make..sh files and one file on src/test/cpp/emC_TestAll/fDefSelection.h which contains settings (#define) for compilation. This file is used both by the script controlled test and by the test in the GUI, for example Visual Studio. Both projects may/should include this file to have the same settings. Hence an testing error shown as test result can be explore by testing step by step in the GUI.

In the emC test usual the familiar make approaches are not used. Why?

Standard make files with complex settings are not simple to read, write and understand. Hence a more obvious system named Zmake was established for some years (starting in the 1990th). It uses a [JZtxtcmd]-script to generate shell scripts which invokes the compilation. Such script files are the sources to determine what and how to make.

The output from a Zmake invocation is a shell.sh script which contains the compiler invocation as command line with all obvious options. The script contains the immediately real compiler invocation. It is not a make script which builds the compiler invocation internally using some dependencies, settings etc. The advantage of immediately real compiler invocation is: It is immediately documented what is happen.

To generate this compiler invocation scripts the JZtxtcmd approach is used. This is done by the Test Selection GUI (chapter #testSelection) as also by the specified test files build.sh and inside the src/test/testScripts directory.

All this scripts are short, consisting of two parts:

Above is the shell-script part, invoking JZtxtcmd (the main class of the jar) with the script file itself. After them the generated script is executed to compile and confirm the test.

It follows the JZtxtcmd script part:

It includes the common generation JZtxtcmd script and invokes in the main() the common genTestCases(…​) routine (from the included script) with given specific arguments. This arguments comes from the Test Selection GUI (chapter #testSelection).

How does the common genTestCases(…​) routine works? It generates texts, selects files from some Fileset etc. See the comments in this script src/test/ZmakeGcc/test_Selection.jztsh.

The problem on testing the core emC sources is the variety of variants (yet 124 combinations) for ObjectJc, Exceptionhandling etc. Writing a lot of scripts, and adjusting the compile switches in applstdef_emC.h is a too high effort. Hence a 'Select Simulation' tool is used, which cames originally from Simulink stimuli selections written by me in the past. It is written in Java and contained in libs/vishiaGui.jar. This tool works with tables.

The same tables as for the manual operating graphic tool are used to arrange the conditions for the test cases. For example the table for Selecting the kind of reflection generation looks like (File src/test/ZmakeGcc/test_Selection.jztsh):

It is a data list in ../../../JZtxtcmd/html/JZtxtcmd.html. You see the magic character in the list and in the 'Select Simulation'. The table contains immediately the necessary compiler switches for each of the four test variants.

With the adequate information about the selected lines the sub routine

is invoked. It gets the selected line in each table. line1 is from the table above. With the information in the line the compiler switches in the test script can be arragenged in a simple way. The texts are contained in the line.

For execution of some test cases the same information is used.

contains in the select String the information, which lines are used, for example in the snippet of src/test/testScripts/testBasics_Simple.jzTc.sh:

The syntax of the select String is described in ../../../smlk/html/SmlkTimeSignals/SmlkTimeSignals.html#truegenerating-manual-planned-test-cases which uses the same tool for another approach. The combination is done with the called Java routine (part of vishiaBase.jar). See ../../../Java/docuSrcJava_vishiaBase/org/vishia/testutil/TestConditionCombi.html.

The 6. table in the 'Select Simulation' contains, which is to test. (The other tables contains, 'under which condition is to test'). It looks like (shortend):

The srcSet is the name of a file set, defined in the script src/test/ZmakeGcc/filesets.jzTc. It determines which files should be used, whereby a reference to further filesets are contained too:

A Fileset is a core capability from ../../../JZtxtcmd/html/JZtxtcmd.html. It names some files and sub Filesets.

In a Fileset some files are named for some application goals. This information can be used to select which emC files are need as part of a maybe simple application:

The fileset for the core files:

are also the core sources for test. Maybe not all, but from this selection may be necessary to use as core sources for an application, which uses emC. It documents the necessities and indirectly also the dependencies.

The distinction between C and C++ compilation can be done using either gcc for *.c-Files or g++ which always compiles as C++. This is the content of the special build_…​ routine. Some more build_…​ routines are existing for different used files and for decision between C and C++ compilation.

The distinction between conditional compilation (variants, see ../Base/Variants_emC.html are done with the different content of the cc_def variable. It contains '-D …​' arguments for the compilation. The other variant may be selecting different <applstdef_emC.h> files which is recommended for user applications. Then the include path should be varied. It needs some applstdef_emC.h files. This can be done too, the part of the include path to <applstdef_emC.h> is contained in the cc_def variable.

A file should be compiled:

The first two conditions are checked only with the 'is newer' aspect from an ordinary make file. For the third condition (indirect newly) the dependencies between the files should be known. For a classic make files this dependencies can be given - if they are known. In practice the dependencies depends on the include situation, it is not simple. Hence the real dependencies can only detect for a concretely version of the file, and the make script should be corrected any time. IDEs use their native dependency check usual proper.

Because this cannot be done easily, often there is a 'build all' mentality.

For repeated compilation the 'build all' mentality needs to much time.

For this approach a Java package org.vishia.checkDeps_C is used. See

../../../JZtxtcmd/html/CheckDeps_C.html.

This tool uses a comprehensive file deps.txt which contains the dependency situation of each file and the timestamp and content situation (via CRC checksum). The tool checks the time stamp and the content of all depending files from the list. If one file is changed, it is parsed by content, find out include statements and build newly the dependencies from this level. Ones of course the object should be recompiled, because another content may be changed. Secondly the dependencies for the test later are corrected..

Because the dependency file contains the time stamp of any source file, it is detected whether an older file is given. The comparison of time stamps is not the comparison between source and object, it is the comparison between the last used source and the current source time stamp. The newly compilation is done also if the file is older, not only newer than the object file. This is an expectable situation, if a file is changed by checkout from a file repositiory with its originally time stamp (the older one). Because git and some other Unix/linux tools stores an older file with the current timestamp this problem is not present on linux, but Windows restores or preserves the time stamp of a copied file, which may be the better and here supported approach.

If the dependency file is not existing, it means, the dependencies should be detected, build all is necessary and the dependency file is built. This is the situation on first invocation after clean.

The dependency file is stored inside the object directory:

It is a snapshot from the root of the object dir tree. The deps.txt has about 260 kByte, it is not too long. The Java algorithm to check the dependencies of all files reading this file needs only milliseconds, because like known, Java is very fast. It runs of course also in Linux.

You can view this file to explore the individual dependencies of each file, which may be informative.

The dependency check is part of each make..sh shell script (generated):

( The java invocation is a long line).

The check of the unchanged situation does only need reading the time stamps of all depending files, it is very fast because the file system is usual cached. If dependencies should be evaluate newly all source files are parsed. Of course already parsed included files are not proceeded twice. The parsing, and checking for # include statement, does only need a short time because Java is fast. The gcc compiler itself supports a dependency check too, but that is very slower (not because C++ is slow, but because it may be more complex). The checkDeps dependency check is more simple, for example it does not regard conditional compilation (a conditional include). It means, it detects a dependency to a included file which is not active in the compiling situation. But that is not a disadvantage, because the dependency can be exist, and the unnecessary compilation because of one conditional include does not need more time than the elaborately dependency check.

If the object file should be recompiled, the checkDeps algorithm deletes it and forces a recompilation because existence check of the object file before compilation. It is a simple liaison between this independent tools.

Inside an operation the begin and the end of the test is marked with:

The macros STACKTRC_…​ are necessary for exception messaging with stacktrace-report, see ../Base/ThCxtExc_emC.html.

The TEST_TRY(…​) macro declares a bTESTok variable and notice the start. For PC usage this macro is defined as

The compiler intrinsic FILE and LINE writes the file and line information in the test output, important for tracking of test results in the sources.

The _TEST_TRY_END checks the bTESTok variable and writes an proper message or stores any data information for the test result. It contains also the exception handling for unexpected exceptions while test. For PC usage this macro is defined as

The same can done also manually, with more writing overhead, but maybe better obviously what’s happen:

The adequate macros are defined as:

That is all for the test frame.

Of course selecting some test cases with a specific tool (gradle tests or such) is not possible. But there is a normal programming for test cases. Selection of specific test cases can be done with ordinary conditional compiling. It doesn’t need special knowledge.

For exceptions see ../Base/ThCxtExc_emC.html.

The test should be done under Exception control. On PC platform this should not be an problem. Either the C++ try-catch can be used, or the other possibilities. But it is the same as elsewhere setted in the <applstdef_emC.h>. Hence: If complicated algorithm are tested, the exception handling should be activated. Only on test of the exception handling itself, it would be set to DEF_NO_Exception_emC or DEF_NO_THCXT_STACKTRC_EXC_emC which disables the exceptions.

This TRY-CATCH frame does only except if an exception is not catched inside the tested algorithm. For a catched exception it is part of the algorithm itself.

If an unexpected exception occurs during the test, then the msgEndFileLine_testAssert_emC(…​) routine is not reached, instead the exceptionFileLine_testAssert_emC(…​) routine is invoked. It writes the exception with file and line and if possible (depending on setting of DEF_ThreadContext_STACKTRC) it writes the stack trace to evaluate where and why the exception was occurring.

The tests itself checks and logs results. For example the test in emc_Test_ObjectJc/test_ObjectJc.c:

Any boolean test result condition is built. Then the CHECK_TRUE with the test result is invoked, with a text which describes the positive expected result. The CHECK_TRUE produces only an output if the check is false:

in one line. Instead using TEST_TRUE outputs also the positive test:

This documents which tests are done. The output is valid for PC test or for test on an embedded platform with text output capability.

The test can also output values, in a printf manner. For example in emC_Test_Math/Test_Math_emC.c the results of mathematics are checked:

It outputs for example:

and informs about numeric test result, here for a fix point multiplication. The test is important because the fix point multiplication can fail on an embedded platform if register widths etc. are other as expected. Maybe using CHECK_TRUE may be more proper, information only on faulty results. The calculated faulty value can be used to detect the reason of the fault.

Furthermore the text can be prepared and reused for more test outputs. See the example testing a smoothing control block (T1) in test1_T1_Ctrl_emC() (src/test/cpp/emC_Test_Ctrl/test_T1i_Ctrl.c) the output of the control block for a step response is compared with the known mathematic result. The maximum of the abbreviation (error) is built in a variable dysh1_eMax. A small abbreviation is expectable because the T1 FBlock has only integer accuracy with a defined step width. It means in comparison to the exact mathematic behaviour not in the step time a small difference is expected and admissible. Hence the test is written as

The abbreviation is compared with a given value. Because some other tests are done too, the text is stored in an array which is used for more tests. Example:

You see, the functionality what and which to test is complex due to the technical approach. But the building of the test result is very simple. The CHECK_TRUE(COND, TEXT,…​) macro expects only a false or true as first argument, TEXT as second arguemnt which describes the expected behavior, and maybe some more variable arguments.

This test output macros are defined as:

The difference between TEST…​ and CHECK…​ is only the conditional call of the inner operation expectMsgFileLine_testAssert_emC on CHECK…​, hence nothing is outputted on true.

The known C macro `VA_ARGS`is used to forward the variable arguments to the macro expansion. The operation for test output itself is defined as:

As you see, not too complicated, using the known variable arguments in C programming together with the vsnprintf(…​) as save variant of the traditional known sprintf(…​).

For test on PC or test on a text supporting embedded platform a text output is created.

With distinguish TEST_TRUE and CHECK_TRUE one can add more information about executed tests. A test output with some executed tests looks like:

You see the test case starts with Test: …​ left aligned, and ok after the test is also written left aligned as the finish line. Between them some messages ` ok: …​` which documents which tests are executed, or ` ERROR: …​` if a test fails:

The simple form looks like:

If during processing the test algorithm an exception is thrown, then (using the TEST_TRY(…​) macros) this test is aborted with an error message:

This exception in the chapter above had occurred because the macro for INIZsuper_ClassJc(…​) was faulty, not all elements are initialized. The type is tested inside a C++ constructor outside of the test itself, and that causes the exception. For that case the test is finished with the EXCEPTION …​ line, the other tests of this block are not executed. That is fatal.

In addition, the stack trace is output. With that the source of the exception was able to found without elaborately debug tests: In line 21 an ASSERT_emC(…​) was found, which checks the base type. Setting a breakpoint there (Visual Studio) shows, the information about the ClassJc…​superClass was missing, which was caused by the faulty macro for the initialization.

It is also possible to write `TRY { …​. }_TRY CATCH { …​. } ` statements inside the test, to catch an exception in the test algorithm. Then the CATCH block should contain:

This logs the not expected exception for the test output. But if the exception behaviour is tested itself as test case, it can be written (see emC_Test_Stacktrc_Exc/TestException.cpp):

In the CATCH clause a test can also confirm, but to check whether the CATCH and also here FINALLY has entered, a boolean variable is set and test after the TRY block.

The evaluation can be done with a not sophisticated text analyse, see ../TestOrg/testStrategie_en.html#trueview-of-test-results

Each test for emC has two stages of test execution:

It means in this case three files where outputted:

The testcase.out is produces as empty file (1 space) if the test does not run. Additonally

is produces.

The sources are hosted on github. You can either get a zip with the sources from a given version, and unpack it manually, or get the whole repository as clone.

Firstly the Test_emC git archive should be cloned using

This clones the repository and unpacks the sources for the master branch. Refer to git description. Without any more options it writes the repository and the files into the sub directory Test_emC

After cloning you can immediately start

Alternatively in Windows you can click build.bat. Note that nevertheless the shell script interpreter should be given, internally unix-script.bat is called, which should use the Cygwin sh.exe and environment. See chapter For MS-Windows: Asscociate extension .sh to unix-script.bat.

This statements starts all other necessities and starts a build of 4 test cases which are fundamental. You see on the output whether all is ok. Then you can also visit the directory tree as described in chapter Directory-tree for sources, test sources and test environment.

What is done (look in the build.sh)

Then you get a fast feedback, look on sources and result.

Especially if you want to work with a mirror location for git (recommended, see vishia.org/Git/html/GitUsage.html) also to work with dedicated sources of emC for specific embedded projects, you should do the following:

Clone for a mirror location of the repositories

This both statements clones the repository and a file tree (not bar) to the both locations:

It means you have two locations, sensible side by side, for this two parts of emC: Test and core sources.

Now you can manually adjust the time stamps of all files going into this locations and execute for both:

For internals of the +adjustTimestamps.sh script see next chapter The time stamps of all files

Because the ./tools are not able to found in this locations, you must first set up the tools and cmds as described in chapter Linux and Window: jar files and some scripts in /home/usr/cmds or ~/cmds and ~/vishia/tools. It presumes that the extension .sh on Windows is related to the shell script executer, see chapter For MS-Windows: Asscociate extension .sh to unix-script.bat, and that also the infrastructure on your PC for tools is clarified, especially the directory ~/cmds and ~/vishia/toools as described in chapter Linux and Window: jar files and some scripts in /home/usr/cmds or ~/cmds and ~/vishia/tools.

Some scripts to deal with the archive

You find a set of files hidden in Test_emC/src/docs/gitMirrorArchiveScripts/* which should be copied on the root location of the both git archives (path/to/gitArchive/Test_emC and …​/src_emC), all this scripts starts with _…​. It helps manage the archives. look into. The most important of them is _CleanResyncRepository.jz.sh. This script removes all files and gets it newly from the local .git archive, inclusively adjusting the time stamps of the files. This is necessary to update the files after commits from the remote location, see next details. Do not copy this file into your working tree, then it may remove all your (changed and not commit) sources. Thats why this script is not part of the git archive on the correct working position, should be setup manually only for the archive position.

Some scripts help to push, pop, work with tags. You can contribute to emC, if you get access rights.

Copy from cloned mirror location

If you have the files on the git archive directory with the proper time stamp, you can copy all files manually to your working location. Do not copy the .git.

To work with the git repository also in the working tree with mirror location for the .git archive you should use a simple text file with the name .git. This file should contain the following content:

With this replacement instead the real .git repository tree you can do all action with git, also in command line, also with for example tortoise git, and also with the vishiaGitGui.html but with the referenced repository and just from more as one location.

It is also possible to get the sources from a zip archive, also from the github:

To get your working files without a local git, you should simple copy the content of a zip also from a specific version from github, see right image, or just from the mirror location.

Then you should get the both zips, also from src_emC in the correct proper version. It is recommended to use the ./src-back in the working tree for this zips. Then you should unzip the content to the correct positions. The directory names should be proper, especially src_emC inside src should be used, not the name with date stamp from the zip form github. You should be sure yourself which version you are working with.

Then you can afterwards get the git archives, setup the link from your working tree to the mirrored repository, set git to the correct version and commit. - Or alternatively compare with other given versions, save your work in a zip, update, try, get experience, …​

General it is recommended use both, backups (in zip) and version control:

Compare the source tree with the mirror location

If you want to only get some sources from emC, for example for simple embedded projects, you can use known tools to compare file trees, given for example with the www.ghisler.com: Total Commander or with vishia/Fcmd: The.file.Commander or some other comparison tools. This is very convenient when you have changed only a few files and you want to compare with the originals, maybe update and contribute for the original files. Then you should run all tests after updating in the test working tree, immediately before doing the effort to commit any changes.

For MS-Windows, you should firstly create the build directory recommended with a link to the TMP location on your hard disk by invocation of

After them you have an empty build directory as link (Junction) to a TMP location, containing only one file readme.txt for check.

The same should be done on linus with

Distinguish here between Linux and windows, because the statements to create a linked folder are different.

On Linux you may change the execution flag of the files with

Then you can invoke the first

The build.sh contains the invocation of src/load_tools/+loadTools.sh if the ./tools directory does not contain a ./tools/vishiaBase.jar. See chapter tools.

The ./build.sh executes the src/test/testScripts/testBasics_Simple.jzTc.sh.

This executes 4 tests, 3 basically, one of them with more sophisticated sources, with fast feedback that the test system runs.

If somewhat does not work, you should have a look inside build.sh, set stop points (pause in windows, an extra cmd call or some echo outputs) to see what’s happen. Refer the chapters before whether the environment is ok. Especially gcc should work and sh.exe should invoke the shell script correctly.

The results can be seen in the ./build/result as text outputs respectively in ./buld/objZmake

You can run more tests or a nightly test with scripts inside

You can select special tests via start a stimuli GUI in

You can start Visual Studio or Eclipse CDT, or Texas Instruments Code Composer Studio to run special tests with debugging.

The scripts contain two things in one Script: The shell commands and the JZtxtcmd statements. The shell script runs till exit 0, the JZtxtcmd statements starts with the following label.

In the JZtxtcmd script part it is determined which is tested. It includes ../ZmakeGcc/test_Selection.jztsh, which does the work. The subroutine genTestcases(…​) produces the ready to run compilation shell script with the proper selection arguments, see testStrategie_en.html, chapter How does it works, genTestCases(…​).

The problem on emC is the diversity of compiler switches which determine:

This supports poor (small) processors so far as well powerful capabilities. The combination of this decision is a matrix. If some fails, usual the reason is simple, but it should be detect.

It is possible to execute specific tests manually as also tests for continues integration. A GUI helps to select the tests.

See testStrategie_en..html.

The check of newly of files is done in comparison to their previous time stamp and hash, not against built files (object, exe). It is done with a compare list. It is the vishia.org/…​./CheckDeps_C approach. For example files can be replaced by other versions, maybe older ones, the original time stamp is preserved, and CheckDeps_C detects that they are newly stored. Another example are generated files, with new time stamp, but with unchanged content. CheckDeps_C can compare the content in comparison with the previous generated files with ignoring comments (often contains generated meta information). If only comments are changed, the files need not be newly compiled.

The given *.jzTc.sh script can select different compilers with control statements, if it should be used for PC simulation and similar for a special embedded target. But it is possible to use included files, for example for the fileset, to prevent similar sources (do not repeat yourself). This fact and all other conditions can be changed immediately in the script.

The gcc/g++ compiler on Linux and Windows with MinGW is fortunately identical.

It is possible to use an IDE (Integrated Development Environment) either immediately for the embedded cross compilation, and additional for PC compilation and test. On the other hand it is possible to use only an PC IDE (it may be Visual Studio) to write sources, test it with PC-Debugging with a PC simulated environment, and build the target system only with such a JZtxtcmd script. Of course compilation errors are visible only immediately as compiler output messages, but if the sources are tested on PC with any IDE, the failure rate is less.

For this test system not the debugging is superficial, but the build of the text-executable with simple check of its outputs, ok or non ok.

If the compiler and linker process has no errors (expected case), the

is generated and runs.

If this file is absent, view the

The built emCBase.test.exe writes some information to stdout (simple information per test case) and writes errors of tests to stderr. If stderr is empty, the test is ok. The stdout shows, which tests are executed. This test system is simple. Internally there are checks

Both outputs are written to

After running the test the output is additional shown on command window.

For Linux the gcc package (GNU) should be installed:

It should be familiar for Linux users which uses C/C++-Compilation.

On MS-Windows gcc can be supported for example using mingw.org. C/++ Developer on MS-Windows should know it. MinGW is not so far minimal.

Another Possibility is cygwin.org.

If one of both is installed, the system’s PATH should be refer to the necessary …​/bin directories (more as one) for the executables. The other possibility is (example):

see -setEnv.bat, to start an environment to execute the build and test for emC. Using this local-path-strategy it is possible to use different gcc tools or abbreviated tools from the system settings (for other usages).

For this situation, the file -setEnv.bat is existing, it can be adapted. This file is called on start of build-bat for MS-Windows.