From 7c963bf8e4f21723397bd74dabb777867cd08684 Mon Sep 17 00:00:00 2001 From: saschbe Date: Sat, 3 Oct 2026 14:06:05 +0200 Subject: [PATCH] docs: document building and running the test suites --- CONTRIBUTING.md | 151 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9a84d8a80..cf5f0f6a0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,6 +12,157 @@ Here are some important resources: ## Testing +### Build the tests + +Start with the dependencies and platform-specific setup in +[INSTALL.md](INSTALL.md). Tests additionally require the Qt6 Test module. +The commands below use CMake/CTest 3.20 or newer (`--test-dir` needs 3.20), +a C++17 compiler, and the Qt6 development packages, including GuiPrivate, +Svg and LinguistTools. Configuration may download dependencies with +FetchContent, including Catch2, so an initial build needs network access. + +Run the following commands from the repository root. Keep build products in +a separate directory and use Debug for the regression tests: + +```sh +git submodule update --init --recursive +cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DPACKAGE_TESTS=ON +cmake --build build --config Debug --parallel +``` + +`PACKAGE_TESTS` defaults to `ON`; specify it explicitly when reusing a build +directory that previously disabled tests. `BUILD_WITH_KF` defaults to `ON`. +If KDE Frameworks are unavailable, add `-DBUILD_WITH_KF=OFF` to the configure +command, as described in INSTALL.md. Use a separate build directory when +changing compiler, Qt kit, or generator. + +### Run the tests with CTest + +After a successful build, list the registered tests, run them, or select a +test by name with `-R`: + +```sh +ctest --test-dir build -C Debug -N +ctest --test-dir build -C Debug --output-on-failure +ctest --test-dir build -C Debug -R '^tst_qetstrings$' --output-on-failure +``` + +`-N` only lists tests; it does not execute them. `-C Debug` selects the +configuration for multi-configuration generators such as Visual Studio. +For single-configuration generators such as Ninja and Unix Makefiles, +`CMAKE_BUILD_TYPE=Debug` selects it at configure time. CTest runs the tests +registered by CMake; it does not build their executables. + +Some tests create Qt widgets and need a graphical environment. On a Linux +machine without a display, use Xvfb as the Linux CI does: + +```sh +xvfb-run -a ctest --test-dir build -C Debug --output-on-failure +``` + +Tests that depend on optional features may not be registered. In particular, +several scripting tests require the Qt6 Qml module at configure time. +Review the test list and any skipped tests before reporting what was covered. + +### Windows + +Follow INSTALL.md for SQLite3 and the MSVC or MinGW toolchain. The compiler +and Qt kit must match. For MSVC, use a Visual Studio developer PowerShell; +the example below assumes Visual Studio 2022 and a 64-bit MSVC Qt kit. +Replace the example Qt path with your installation, and supply any SQLite3 +paths or toolchain file required by your setup: + +```powershell +$qtPrefix = 'C:/Qt/6.x.x/msvc2022_64' +cmake -S . -B build-msvc -G 'Visual Studio 17 2022' -A x64 ` + "-DCMAKE_PREFIX_PATH=$qtPrefix" -DBUILD_WITH_KF=OFF -DPACKAGE_TESTS=ON +cmake --build build-msvc --config Debug --parallel +$env:Path = "$qtPrefix/bin;$env:Path" +ctest --test-dir build-msvc -C Debug --output-on-failure +``` + +Putting the matching Qt `bin` directory on `PATH` lets test executables find +Qt DLLs. Other shared dependencies must also be available on `PATH`. +For MinGW with Ninja, use the matching MinGW Qt kit and compiler environment, +omit `-A x64`, select `-G Ninja`, and set `-DCMAKE_BUILD_TYPE=Debug`. +The Linux shell regression scripts below do not run natively on Windows. + +### Tests outside the CTest suite + +The Catch2 executable `C_unittests` is built with `PACKAGE_TESTS=ON`, but is +currently not registered with CTest. Run it separately from the repository +root after building: + +```sh +./build/tests/catch/C_unittests +``` + +This executable creates a Qt GUI application too. On headless Linux, use: + +```sh +xvfb-run -a ./build/tests/catch/C_unittests +``` + +For the Visual Studio example above, use this PowerShell command after +setting up `PATH`: + +```powershell +& ./build-msvc/tests/catch/Debug/C_unittests.exe +``` + +The Python MCP suite is also separate from CTest. Its unit and protocol +tests can be run without a QElectroTech binary. Some fixtures launch POSIX +shell scripts, so use a Unix-like environment for this suite: + +```sh +python3 misc/qet-mcp/test_qet_mcp.py +``` + +Use `python` instead of `python3` if that is your interpreter's command. +Integration tests need a built QElectroTech and the environment variables +documented in [the MCP testing guide](misc/qet-mcp/README.md#testing), +including `QET_ENABLE_SCRIPTING=1` for script-driven tests. Missing integration +prerequisites cause tests to be skipped, so a successful unit run alone does +not establish integration coverage. + +### Linux regression prerequisites + +In addition to the build dependencies in INSTALL.md, headless widget tests +need `xvfb-run`, Xvfb and `xauth`. The existing +[Linux workflow](.github/workflows/linux-build.yml) lists the packages used +by CI, including the QtSvg development package and SQLite driver for QtSql. +Package names vary by distribution. + +The IPC open-forwarding regression requires Bash, Xvfb, Openbox and xdotool. +Use a Qt6 Debug build; the documented reproduction used Qt 6.10.2. It uses +`examples/industrial.qet` by default and starts its own display, so run it +directly rather than wrapping it in `xvfb-run`: + +```sh +tests/ipc-regression/run.sh --binary build/qelectrotech +``` + +This test is separate from CTest. Exit code `0` means the scenario survived, +`1` means a crash, and `2` means inconclusive or unusable input. An inconclusive +run is not a pass. See [the IPC guide](tests/ipc-regression/README.md). + +The quit-during-modal regression needs gdb with Python support and a binary +with the symbols used by the test (use an unstripped Debug build). It uses +Qt's offscreen platform and needs no X server. It is registered with CTest +on Linux and can be selected with: + +```sh +ctest --test-dir build -C Debug -R '^modal_quit_regression$' --output-on-failure +``` + +Missing gdb, missing Python support in gdb, or a stripped binary can result +in exit code `77`, which CTest treats as skipped. See +[the modal-quit guide](tests/modal-quit-regression/README.md). + +When submitting changes, state the build configuration, commands actually +run, results, and any skipped or unavailable tests. If commands were only +checked against the build files or documentation, say so explicitly. + ## Submitting changes Always write a clear log message for your commits.