1. Requirements for PC Systems

The emC Software should be tested on PC, either with Windows (Win-10 recommended) or with Linux. For Compilation of the test files gcc (GNU Compiler Suite) is used, which is available under Windows with MinGW or Cygwin.

Especially for Windows usage, some Visual Studio Projects are part of

src/test/VS15/*

This projects can be used individually, opening the adequate *.sln file.

The emC software is for embedded, but the concept includes test of algorithm in originally C / C++ on PC platform.

1.1. Java JRE8 available

Java should be a standard on any PC system. For some build- and translation tools JRE8 is used. This should be checked with console command:

java --version

On Linux the tools are tested with OpenJDK Runtime 11.0.6, it runs.

If another JRE Version (Higher than 8) is used as default and it does not run, you can start all emC build actions from a cmd window, which is started with in an cmd script for example in windows with

set PATH=path/to/JRE8/bin;%PATH%   ..enhance the PATH, firstly find JRE8
start cmd.exe

This action defines a local enhanced PATH without change the environment of the system.

For this situation, the file -setEnv.bat is existing, it can be adapted. This file is called on start of build-bat for MS-Windows.

1.2. gcc available

For Linux the gcc package (GNU) should be installed:

apt-get install gcc
apt-get install g++

It should be familiar for Linux users which uses C/C++-Compilation.

On MS-Windows gcc can be supported for example using mingw.org. C/++ Developer on MS-Windows should know it. MinGW is not so far minimal.

Another Possibility is cygwin.org.

If one of both is installed, the system’s PATH should be refer to the necessary …​/bin directories (more as one) for the executables. The other possibility is (example):

set PATH=c:\Programs\MinGW\bin;c:\Programs\MinGW\msys\1.0\bin\;%PATH%

to start an environment to execute the build and test for emC. Using this local-path-strategy it is possible to use different gcc tools or abbreviated tools from the system settings (for other usages).

For this situation, the file -setEnv.bat is existing, it can be adapted. This file is called on start of build-bat for MS-Windows.

1.3. shell available

Some scripts are Unix-shell scripts. If MinGW or Cygwin is installed on Windows, there is a sh.exe Command available. This is used to start shell scripts using

sh -c path/to/myscript.sh

This command runs as Linux command too, hence calling scripts via sh -c …​ are the same using Windows or Linux. That is the approach for compatibility.

Inside a sh.exe all Windows commands can be used too, inclusively java …​ start. The scripts are tested under Linux. Hence it is assured that no Windows-only command is used.

1.4. Git available

Git should be access-able via command line

git ....

It is available often, elsewhere it should be installed from git-scm. The git should be installed under MS-Windows to

C:\Program Files\git

Elsewhere it does not run probably. Git contains a git/mingw64 part too. Both bin directories should added to the system PATH (may be correctly done by installation). The mingw inside git does not contain any compiler, but usual the same linux commands inclusively sh.exe.

Any git graphical environment can be installed, for example tortoise-git, but it is not presumed for working with emC sources. It is presumed for git actions.

2. Getting git archives

The git archives are hosted under github/JzHartmut.

  • The archive Test_emC includes the test environment and docu in asciidoc.

  • The archive src_emC is a sub archive inside the Test_emC. It is download via script if the build is started.

Firstly the Test_emC git archive should be cloned using

git clone https://github.com/JzHartmut/Test_emC.git

After cloning, the, on Linux:

cd Test_emC
chmod 777 build.sh
build.sh          ... Linux

should be called. In Windows:

build.bat
calls internally `sh.exe -c build.sh`, but before that `-setEnv.bat`
and `-mkLinkBuildGradle.bat` are called, because of special handling in Windows.

The build.sh calls inside:

./+resolveDeps.sh

This sub-shell-script (can be invoked manually too) loads one jar file from internet and clones the src_emC git Archiv from github, see next. This action is only done one time respectively if the named destinations do not exist.

2.1. Depending parts loaded from internet

The git archive Test_emC/.git contains only files, which are used exclusively for the emC-Test. There are two necessary add-ons which should be gotten from internet:

./+resolveDeps.sh

contains statements to load this components from internet with the dedicated URL. To download it a small libs/minisys_vishia.jar is used as part in the git archive as only one common. It contains the necessary Wget class. Wget as known linux cmd is not available unfortunately in a standard MinGW installation, neither it is anyway a standard on any Linux System. Hence it is provided with the minisys_vishia.jar for all systems where Java runs.

  • The downloaded/jars/vishiaBase.jar is a Java executable (class files) with about 1.2 MByte, which contains especially the JZtxtcmd script interpreter. That is used to generate the test scripts and for Reflection generation (further usage of sources). It is a necessary component. This file is downloaded from a given URL in internet. If necessary you can find the sources to this jar file beside the jar file in the same remote directory. With the sources you can step debugging the tools for example using eclipse IDE.

  • The src/main/cpp/src_emC/…​ is the source tree of emC. It is provided as an extra git archive, because only usage of the sources for example for an embedded project does not need the Test_emC sources. This is true for a test environment on PC for a special C/++ project too. Hence the src_emC/.gits can be seen as git submodule. But the git submodule - capability of git is not used, it is too sophisticated. It is some more simple to get the src_emC\.git as an own part but inside the Test_emC file tree, under Test_emC/src/main/cpp/src_emC. For that

    src/main/cpp/+gitclone_src_emC.sh
  • …​ is part of the git-archive files in Test_emC/.git. It installs the correct requested version of src_emC per default, but the version handling (commit, checkout etc) is independent after that.

This distribution does not store any content from internet in any non obvious locations on a hard disk, exclusively gradle does it, used for Asciidoc text generation. Gradle is not necessary to build and run the tests of the emC sources.

2.2. Time stamps of files

Git does not store the time stamps of the files. The reason for that may be that a make system needs new time stamps to make. This topic is discussed conflicting in internet. They are better make systems than the classic C/Unix maker with only check newer time stamps to decide whether to build or not. A better make system saves and re-uses a hash of the files to detect whether they are changed.

The time stamps may be a point of interesting to find out when was a file changed. This can be essential in developing. Hence the time stamps are stored in a file .filelist. The time stamp of each file is applied to the files via invocation of the Java class org.vishia.util.FileList as part of vishiaBase.jar. This is done as one line inside

+resolveDeps.sh

firstly after clone of the files from git archive. It is done via invocation of

java -cp downloaded/jars/vishiaBase.jar org.vishia.util.FileList T -l:.filelist -d:.

The .filelist contains a hash, it applies the timestamp only if the content matches. On a commit with the special vishia GitGui or with invocation of the org.vishia.util.FileList to create a file list the current time stamps are stored before the commit. Hence the commit has current time stamps.

3. Built files, temporary content in the build directory

The working tree should be free of some temporary or resulting files. It should contain only sources. That gives the possibility to build a 'file copy' in form of a zip file for example, with compressed content. This is another proper possibility to save a safety version than git or another possibility for share sources.

On the other hand, all stuff should be done in the only one working tree without complex external file path settings. With the 'gradle' file tree concept the building results are stored in the build directory. Now it is possible to really store the content inside the tmp directory (on linux per default /tmp) using a link for the build sub directory. The possibility of symbolic linked directories is given under Unix since 1970 with

ln -s path/to/dst build

For Windows it is also possible, since "Windows Vista" but not so far public. The adequate command mklink /D …​ needs unfortunately administrator rights, it is really not able to handle. But the soft form

mklink /J build path\to\dst

runs easy. It is a really symbolic link. It is not obvious why both mklink /D with administrator rights and the soft form mklink /J are differentiated. Unfortunately the Java build-in variant

java.nio.files.Files.createSymbolicLink(link, target);

invokes the administrator safeguarded variant inside the MS-Windows operation system API call, hence it is not proper to use.

Because of that the creation of directory links are programmed twice, inside -mkLinkBuildGradle.bat for MS-Windows and as part of +mkLinkBuildGradle.sh for Linux.

3.1. Linked directories

The following linked directories are created:

Working_tree
 +- build --> $TMP/Test_emC/build
 |   +- src --> Working_tree/src
 |
 +-.gradle --> $TMP/Test_emC/.gradle

The src link inside build back to the working tree is necessary because the generated build shell script runs in the build directory. A ../ would not point back to the position before build, it is inside the linked file tree. Hence the sources should be gotten via the src link.

The scripts to create the links checks whether build and .gradle exist (independent) and cleans and creates the temporary directories $TMP/…​. It means,

if build or create as link are removed manually or via call of +clean.bat or +clean.sh then the destination content is cleaned too. Only the symbolic links need manually be removed to force a newly build.

Pay attention, do not call rmdir /S/Q build\src', unfortunately it removes all sources in the Working tree. But call of `rmdir build does not work recursively, and it removes only the link if it is a link. If it is not a link, it removes nothing, returns with error. The same does rm build for Linux.

3.2. Using a RAM disk

A RAM disk has the benefit that the access is faster, and especially a SSD hard disk will be spared. The content on the build and .gradle is only temporary necessary. Results of build should be anyway copied to a distribution. So the RAM disk is the ideal solution to store built files. The content of the RAM disk should not be kept after should down of the PC.

All temporaries can be stored on this non permanent medium, inclusively some windows stuff. Hence the TMP environment variable of the MS-Windows System can be redirected to the RAM disk. (Using System control, Enhanced system settings). The linked destinations uses $TMP, hence the RAM disk if TMP refer it, or any other temporary directory.

4. Build and test

On MS-Windows you can start

build.bat

immediately after clone. It loads the further content from internet (see chapter Dependencies to parts from internet), creates Links for temporary data (see linkedDirs) and calls then sh.exe -c build.sh

On Linux you can firstly call

chmod 777 build.sh
build.sh

build.sh is the only one shell script where executable rights may be set manually. All other shell scripts (all with file-extension .sh) will be set to executable inside the build.sh if it is invoked.

build.sh calls

./+resolveDeps.sh
./+mkLinkBuildGradle.sh

to download the depending content from internet and create links to temporary (see chapter Dependencies to internet) and then it call

##The jzTc.cmd does the task of build and test:
java -jar downloaded/jars/vishiaBase.jar jzTc.cmd

This JZtxtcmd script (see www.vishia.org/JZtxtcmd/…​) calls the test_emC() sub routine in the sub script

src/test/ZmakeGcc/All_Test/ZmakeGcc.jzTc.sh

4.1. Build with Zmake and JZtxtcmd

There are some reason to use JZtxtcmd. The result of running the script ZmakeGcc.jzTc.sh is a generated shell script in

build/make_test_emC.sh

which contains the invocation of the g++ compiler and linker and some more stuff for build. This resulting shell script can be stored (outside temporary build), studied and repeated called for some reason of compilation. The compilation invocation can be seen immediately. But this script is temporary. It is no worth to invest work into it. It’s only usefully to study how the compiler is obviously invoked.

The make_test_emC.sh script contains

  • all files one after another to compile and link. Advantage: Selection of less files for special projects is possible. But given a wildcard mask, all files in one directory is possible too.

  • all options for compilation, able to control via conditional statements.

  • all include paths.

  • the text generation rules for the compile and link cmd call.

  • All it is simple, easy and obviously, better than in a classic maker.

  • The check of newly of files is done in comparison to their previous time stamp and hash, not against built files (object, exe). It is done with a compare list. It is the vishia.org/…​./CheckDeps_C approach. For example files can be replaced by other versions, maybe older ones, the original time stamp is preserved, and CheckDeps_C detects that they are newly stored. Another example are generated files, with new time stamp, but with unchanged content. CheckDeps_C can compare the content in comparison with the previous generated files with ignoring comments (often contains generated meta information). If only comments are changed, the files need not be newly compiled.

The given *.jzTc.sh script can select different compilers with control statements, if it should be used for PC simulation and similar for a special embedded target. But it is possible to use included files, for example for the fileset, to prevent similar sources (do not repeat yourself). This fact and all other conditions can be changed immediately in the script.

The gcc/g++ compiler on Linux and Windows with MinGW is fortunately identical.

It is possible to use an IDE (Integrated Development Environment) either immediately for the embedded cross compilation, and additional for PC compilation and test. On the other hand it is possible to use only an PC IDE (it may be Visual Studio) to write sources, test it with PC-Debugging with a PC simulated environment, and build the target system only with such a JZtxtcmd script. Of course compilation errors are visible only immediately as compiler output messages, but if the sources are tested on PC with any IDE, the failure rate is less.

For this test system not the debugging is superficial, but the build of the text-executable with simple check of its outputs, ok or non ok.

4.2. Compiling and linking, errors and warnings

If the compiler and linker process has no errors (expected case), the

build/emCBase.test.exe

is generated and runs.

If this file is absent, view the

gcc_out.txt      ... stdout of compiling
gcc_err.txt      ... errors, warnings of compiling
ld_out.txt       ... stdout of liking
ld_err.txt       ... errors, warnings of linking

4.3. Check of test results

The built emCBase.test.exe writes some information to stdout (simple information per test case) and writes errors of tests to stderr. If stderr is empty, the test is ok. The stdout shows, which tests are executed. This test system is simple. Internally there are checks

Both outputs are written to

build/test.out
build/test.err

After running the test the output is additional shown on command window.