Commit Graph

406 Commits

Author SHA1 Message Date
ispyisail b4bbd17933 Fix a .qetz that cannot be opened after only a folio's title changed
With a .qetz and lazy folios (both the default), changing only the
title, author, date or revision of a folio holding a picture, while that
folio was not displayed, then saving, gave a .qetz that refused to open:
'picture "images/<sha>.png" is missing'.

Such a folio is saved from its items as read, with its new header
(Diagram::deferredXmlWithHeader()). Read from a .qetz, its <image> names
its picture's file (qetz-file) instead of holding the picture, and
QetContainer::split() takes out only pictures held inline: the file was
never written, while the folio still named it. The folio now gets its
pictures written in, from the bytes the project keeps
(QETProject::containerPicture()), as a built folio writes them.

On master, a picture on folio 2 of grafcet, iso_sfc_example, 741,
tremie_vibrante or Projet_vierge is lost this way; found on three forum
projects by comparing lazy saves with saves of every folio built. Saving
as .qet was not affected.

New tst_qetzheaderpictures (grafcet.qet: picture on folio 2, saved as
.qetz; its title changed with folios built when needed; reopened with
its picture and title): fails without this change ("picture ... is
missing"). ctest 107/107.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:33:51 +13:00
ispyisail d97434cec4 Merge pull request #1504 from ispyisail/feat/qetz-recover
Add recovering what can be read of a damaged .qetz project (--recover)
2026-10-16 11:26:55 +13:00
ispyisail 8301856ee7 Merge pull request #1502 from ispyisail/fix/report-text-after-info-edit
Fix a folio report pointing at its partner's old place after its properties are edited
2026-10-16 11:26:16 +13:00
ispyisail 7e82d290f9 Merge pull request #1501 from ispyisail/feat/keep-previous-versions
Add keeping the previous versions of a project each time it is saved
2026-10-16 11:25:52 +13:00
ispyisail 2bd19e04be Add recovering what can be read of a damaged .qetz (--recover)
A .qetz that does not open as it is -- a picture's file missing (as
the title-change fault wrote them), a folio file missing, a row deleted
from project.sqlite by hand (issue #1458), a damaged database -- was
refused whole, and nothing in QElectroTech could get anything back.

QetContainer::recover() reads what it can and says what it left out:
an entry that cannot be unpacked, a missing folio file or picture, a
symbol or wire that does not match the database, and, when the
database cannot be read at all, the information it held (the wires go
too, as it holds their ends). join(), restore() and QetZip::fromBytes()
take an optional list for this; given none they refuse as before.
`qelectrotech --recover damaged.qetz out.qet` writes what was read to a
.qet and lists what was left out.

Measured on industrial (50 folios, 618 symbols, 671 wires): a wire row
deleted -> 670 wires, the rest whole; a folio file missing -> 49
folios; a database page overwritten -> every folio and symbol drawn, no
wires and no symbol information. A zip cut short (no central directory)
still cannot be read.

Tests: recoverLeavesOutOnlyTheDamage (nothing, folio file, wire row,
database page), recoverMissingPicture, recoverNotAZipRefused; they fail
with the database side kept strict. The strict cases pass unchanged.
ctest 106/106.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 10:15:37 +13:00
ispyisail e8201abb8a Keep the previous versions of a project when saving it
Saving replaced the project file with nothing kept of what it was: one
bad save -- a fault in QElectroTech, a full disk, a wrong edit saved by
mistake -- and the last good version was gone. AutoCAD keeps a .bak on
every save, KiCad several backups.

Before a project is saved over, the file as it was is now copied to a
folder beside it, "<name>-backups", named by the time that version was
saved ("industrial 2026-10-16 08-30-12.qetz"); only the newest are kept,
5 by default (Settings > General > Projects, 0 for none). A copy is
never a move, so a save that fails leaves the file in place; a copy that
fails only warns, the save goes on. The same bytes as the newest version
kept are not kept twice.

The prompt on opening ("Create a backup copy?"), which writes one dated
copy beside the project and never removes any, is left as it is.

New tst_previousversions (newest kept, same bytes once, same second,
nothing for a new file or a count of 0, a failed copy said); it fails
with the oldest never removed. Checked with the program: two scripted
saves of a .qetz leave the original and the first save in
industrial-backups, the project holds the second. ctest 107/107.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:00:07 +13:00
ispyisail fcc797342e Fix a folio report pointing back at its partner's old place after its information changes
A folio report's label shows where its partner is (updateReportText()).
Changing any other information of the report -- a comment, a function --
ran elementInfoChanged(), which set the text from the label stored in
the report's information instead: the partner's position when the file
was last saved. Move the partner from column 4 to column 9, give the
report a comment, and it showed (and saved) 1-C4 again.

With lazy folios the partner is often built after the change, so the
report was recomputed and the two modes saved different files; that is
how this was found (FINDINGS F116, tools/lazy-folios/edit-diff.py).

New tst_reporttextafterinfo: move the partner, comment the report, save,
check the report says 1-C9, with folios built when needed and with every
folio built. Both rows fail without this change ("1-C4").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:54:27 +13:00
ispyisail e3fa4a049c .qetz: read the project database the way SQLite advises for untrusted files
A .qetz's project.sqlite comes from a file, which may be damaged, made
by another tool, edited by hand (issue #1458) or made to harm. SQLite's
advice for such files (https://www.sqlite.org/security.html) was only
partly followed: trusted_schema was off and every name had to be a plain
table. readAsUntrusted() now also turns on the defensive mode, turns off
triggers and views, checks each cell's size as its page is read, limits
a value to 256 MiB, and runs quick_check before anything is read, for
the database read on opening and for the one a save copies kept folios
from.

A damaged database is now refused as "the project database is damaged",
where it was "cannot be read: database disk image is malformed" or a
missing row. Damage on disk was already caught before SQLite: the zip
checks each entry (one flipped bit in project.sqlite: refused, "unexpected
decompressed size"). quick_check does not see a changed value inside a
row (49 of 400 random small changes read silently, quick_check "ok"):
only the zip's checksum does.

New test damagedDatabaseRefused (a page overwritten at three places):
fails without this change on the message. Opening nuri (136 folios):
3.56-3.66 s with and without. ctest: the 7 container, save and lazy
folio suites pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 08:41:27 +13:00
ispyisail d50f27dc8b .qetz: bring an older database to the current format one step at a time
The reader of project.sqlite knew two formats at once: every query asked
the database which column names to use (v2() ? "folio_id" : "folio_uuid"
and so on), and each new format would have added another branch to every
query. Asked in discussion #1440.

An older database is now brought up a format at a time when a .qetz opens
(readyToRead()): format 2 to 3 renames its tables' columns, adds the show
columns and puts the symbols' and wires' uuids and the symbols' type back
in the folio files; 3 to 4 has nothing to change for reading. The reader
then knows only the current format. The next format adds one step.

Checked: the format 2 test file (tests/qttest/fixtures/qetz_format2.qetz)
resaved as .qet and as .qetz gives the same files as master. format2Read
now also checks that every symbol and wire gets its uuid and type back:
it failed with the type left out of the step, which the test did not see
before. ctest 106/106.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 06:59:46 +13:00
ispyisail b085dcd5dd Pair terminals moved by one shift, and terminals sharing a place
The first pass refused a pair whenever a saved terminal had two live
candidates in reach. That happens when a whole drawing moves by half the
spacing of its terminals (pins 10 apart moved 5), and when two terminals
share a place. When one offset fits every unmatched terminal, as when a
whole drawing moved, that offset now settles all pairs; terminals sharing
a place pair in order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 21:42:43 +13:00
ispyisail 786f8574d9 Reattach wires to a terminal that moved a little since the project was saved
A project saves, for every placed symbol, where each terminal docked, its
orientation and its number. When the symbol's definition is replaced by one
whose terminal sits a few units elsewhere, wires were matched by exact
position and so were dropped on the next open: legacy wires by Terminal::
fromXml(), uuid wires by the uuid derived from the terminal's new place.

Element::fromXml() now pairs each saved terminal that nothing matched
exactly with the live terminal of the same orientation whose docking point
is within 5 units in x and y, only when each is the other's only candidate.
A pair registers the saved number, and the uuid derived from the old place,
which Diagram::findTerminals() checks after the derived uuid of the current
place. Wire ends reattached this way are counted and logged once per folio.

tst_movedterminals covers a move of 3 and 5 units (number and derived
uuid), 6 units, two candidates, and an unmoved terminal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 20:26:30 +13:00
ispyisail fba67c3b2a Merge pull request #1483 from ispyisail/fix/qetz-1472-followups
Fix saving to a read-only folder, and the #1472 review follow-ups
2026-10-15 13:31:03 +13:00
ispyisail 41baea32a8 Fix tst_qetcontainer on a Windows checkout (#1482)
specMatchesSchema() finds the schema in QETZ-FORMAT.md by looking for
"```sql\n". A Windows checkout gives the file CRLF line endings, so the
marker is never found and the test fails with "no schema in
QETZ-FORMAT.md". Drop the carriage returns after reading the file.

Checked by converting QETZ-FORMAT.md to CRLF: the test failed with the
CI message before this change and passes after it; with LF it passes
both ways. All 51 cases of tst_qetcontainer pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:11:29 +13:00
ispyisail cd3a726e26 Name the .qetz MIME type application/x-qet-project+zip
Every other QElectroTech file type is application/x-qet-*; the zipped
project was application/x-qelectrotech-project+zip. No release has
shipped it yet, so it is renamed now.

Files written with the old name still open: the reader accepts both, and
the shared-mime-info file keeps the old name as an alias and matches
either on content.

Review of #1472 (scorpio810, point 2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 06:20:57 +13:00
Laurent Trinques dbecfae35b Merge pull request #1481 from ispyisail/fix/pdf-orientation-per-folio
Linux build and tests / Build and test (Qt 6, Debug) (push) Failing after 1m39s
Add per-folio page orientation to the PDF export (#1480)
2026-10-14 17:27:15 +02:00
Laurent Trinques 2ce508e373 Merge pull request #1472 from ispyisail/feat/qetz-db-truth-v2
Save as .qetz by default, with the project database as the source of truth
2026-10-14 17:22:44 +02:00
ispyisail f42958d2c2 PDF export: turn each page to its folio's orientation (#1480)
The print window exported every page in one orientation, landscape
unless changed, so a portrait folio was shrunk onto a landscape page.
With "Fit in page" on, a PDF export now turns each page to portrait or
landscape as its folio is drawn. The preview keeps one orientation
(QPrintPreviewWidget lays out every page at one paper size), and a
printer keeps it too (only Qt's PDF engine accepts a new page layout
once printing has started). The orientation chosen in the window is
put back afterwards, so it is still the one saved.

The cross-reference links took their target from painter->viewport(),
which QPainter sets once when it opens, so on a turned page they missed
their text. They now use the device's size, as render() does.

tst_printpdforientation exports through the window: an A3 landscape and
an A3 portrait folio get one page each way; tiled pages keep the chosen
orientation; and the links on examples/industrial.qet's turned pages
land where they do when the export starts portrait.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 23:38:00 +13:00
ispyisail 33a731b56f Merge pull request #1477 from ispyisail/perf/parallel-picture-decode
Linux build and tests / Build and test (Qt 6, Debug) (push) Failing after 2m6s
Speed up opening folios with photos: decode pictures in parallel
2026-10-14 19:10:59 +13:00
ispyisail c0a5084bc4 Merge master into feat/qetz-db-truth-v2
The .qetz files take this branch's side: it already holds #1473, and
adds the folio parts to join().

qetscriptapi.cpp: #1456 gave setConductorProperty() the voltage that
crosses a function separated terminal element; here that function hands
its work to applyConductorProperties(), so the change goes there, with
the properties captured before the pushes. The database path for folios
not built already gives up at such a terminal, so it builds the folio and
takes the same route.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:57:29 +13:00
ispyisail 373a705bbb Merge master into perf/parallel-picture-decode
tests/qttest/CMakeLists.txt: this branch's tst_folioimageload and
master's tst_svgsmallpictures (#1478) were added at the same place;
both are kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:36:36 +13:00
ispyisail af142aba90 Merge pull request #1473 from ispyisail/fix/qetz-format-ids-types-version
Fix .qetz database: rows keyed by position, untyped, unversioned
2026-10-14 18:29:57 +13:00
ispyisail 8cda53f4ce Merge pull request #1456 from Kellermorph/terminal-potential
Terminal: rename potential separation to function separation
2026-10-14 18:26:00 +13:00
ispyisail 1768518b44 Merge master into fix/qetz-format-ids-types-version
join() and readDocument() gained a parameter on both sides: master's
Pictures map (#1470) and this branch's format. Both are kept, pictures
first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:56:32 +13:00
ispyisail da6eb83035 Add an SVG export option to store pictures smaller
An SVG keeps every picture at full resolution, whatever size it is
drawn at: a folio with 22 phone photos shown at a tenth of their size
gave a 270 MB SVG and took 65 s to write, most of it compressing the
photos again.

The export dialog gets "SVG: smaller pictures" (off by default, kept
with the other export settings), and --export-svg gets
--svg-small-pictures. With it, a picture is stored at four times the
size it is drawn at when that is smaller than the picture; smaller
pictures stay whole. Four times is about 375 dpi on a printed folio.

The exporter sets Diagram::smallPictures() for its own drawing only,
so screen, print, PDF and PNG are unchanged. It is not part of
applyProperties(), which every format uses.

On that project: --export-svg 84.9 s -> 34.5 s (opening is 20 s of
both), 270 MB -> 57 MB, peak memory 2.45 -> 1.86 GB. Rendered at twice
the folio size, the two SVGs differ by more than 32/255 on 0.23 % of
pixels, along photo detail.

tst_svgsmallpictures: an 800x600 picture drawn at 0.1 is stored at
320x240 with the option, through --export-svg and through the
dialog's QPicture route, and at 800x600 without it; a 40x30 picture
stays whole. It fails when the factor is changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:12:38 +13:00
ispyisail 6751fc6f19 Speed up opening folios with photos: decode their pictures in parallel
Each picture of a folio was decoded in turn while its item was built,
a quarter of a second for a 12 megapixel photo. Diagram::fromXml() now
reads every picture of the folio first, decodes them all at once on a
pool of its own (one thread per core, at most one per picture), then
builds the items in file order in the GUI thread as before.

The decoding (base64 text, then the picture file) moves out of
DiagramImageItem::fromXml() into decodePicture(), which touches only
its argument. QImage::fromData() then QPixmap::fromImage() is what
QPixmap::loadFromData() does, so the pixels are the same.

On a 263 MB project with 22 photos on one folio: --info 21.3 s ->
12.5 s, the folio's content 10.6 s -> 3.2 s, opening its .qetz 15.6 s
-> 9.3 s. Peak memory +220 MB (1.81 -> 2.03 GB), the pictures' text
being converted by several threads at once. A resave is byte-identical
to master's, and so is a PNG export.

tst_folioimageload: six pictures come back with their own pixels and
positions, and an <image> without a picture does not stop the others.
It fails when the decoded pictures are handed to the wrong items.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:09:09 +13:00
ispyisail bf465f9245 Merge #1473's format document; describe format 4
QETZ-FORMAT.md now describes format 4: the computed text left out of the
folio files, QElectroTech's own database built on opening rather than
stored, the tables written without keys, and format 3 among those read.

Tables written without keys keep their declared column types (INTEGER
was written as INT); tst_qetcontainer checks the columns and types
against the document, and that the file has no keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:34:52 +13:00
ispyisail f65b24250c Document the .qetz format
QETZ-FORMAT.md describes format 3: the archive and its files, the
manifest and the version rules, every table and column of project.sqlite
and what it comes from, how rows are keyed, the markers a moved block
leaves, and what reading checks. Asked for in #1440: the format had no
written specification.

tst_qetcontainer checks that the schema the document gives is the one
written: the same tables and columns, in the same order, with the same
types and keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:14:12 +13:00
ispyisail a9ed99293e Merge #1473: a property's show flag gets a column
A folio is copied as it was only from a database whose folio_property and
element_info have the columns this writes too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:54:09 +13:00
ispyisail 590bd6e532 .qetz: a property's show flag gets a column
The JSON column of a property row held only its show flag, in every
project QElectroTech saves: the 25 examples and a 136-folio project used
it for nothing else. Folio and project properties now store it as a
number, as symbol information does, so a saved project's database holds
no JSON; JSON is left for attributes QElectroTech does not write (an
older or hand-edited file).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:36:19 +13:00
ispyisail 286779a2e4 Merge #1473: no position columns, a later format opened read-only
A symbol changed on a folio kept as it was needs the place its
information goes back to: its part is checked for the marker
(QetContainer::split()), where the position column said it before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:17:21 +13:00
ispyisail 4c3d87e3f7 .qetz: no columns for where a block sat; open a later format read-only
The database held where each moved block sat among its parent's
children (props_at, info_at, links_at), though reading only asked
whether there was one: the marker the folio file keeps says that, and
where it goes back. The columns are gone; format 2 files leave the same
markers and still open.

A .qetz of a later format whose min-reader allows this version was
still refused by the database's own format check. It opens now,
read-only, with a message: saving it here could lose what this version
does not know. Save As writes a copy in this version's format.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:47:08 +13:00
ispyisail 56c704be1f Merge #1473: a symbol's type stays in its folio, mismatched databases refused
The checks run once every folio is read. A .qetz written by this branch
has no keys in its tables, so a row twice is caught by the counts too.
A folio is copied as it was only from a database with the columns this
writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:00:59 +13:00
ispyisail 5975b75766 .qetz: keep a symbol's type in its folio; refuse a database that does not match
A folio file in a .qetz did not say which symbol each item is: the type
(embed://...) was moved to project.sqlite, so a folio read alone could
not be drawn or diffed. It stays in the folio file now; format 2 files,
which moved it, are still read.

Nothing checked that project.sqlite and the folio files agree: a row no
item has, an item's row on another folio, or information rows of no
symbol were read as far as they went. Reading now refuses such a file and
says what does not match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:57:25 +13:00
ispyisail c259cae429 Keep each fact once in a .qetz: drop the stored project database
A .qetz held the engineering data twice: the document's tables, and
beside them QElectroTech's own project database (stage D2/D3), which
QElectroTech also builds from the document. Two copies of one fact can
disagree, and the copy cost a third of the file. Now project.sqlite holds
the document's tables only; QElectroTech builds its own project database
from them when the project opens, as it does for a .qet.

Measured (this build, lazy folios):
  industrial (50 folios): 620 -> 374 KB, open 0.37 -> 0.65 s
  f2099/nuri (136 folios): 3.27 -> 2.20 MB, open 2.2 -> 3.5 s
The same projects as .qet open in 3.3 s and 23 s.

Saving still copies the file of every folio not built, with its rows
(stage D4): the project keeps the part each folio was read from. nuri
opened and saved again: 6.3 s, no folio built, the same bytes.

A symbol's terminal list is left out of the folio file when every
folio, symbol and wire has a uuid of its own, now checked on the
document (QetContainer::ownUuids()) rather than by the stored database
being written.

Checked: 342 corpus projects open, .qetz -> .qet equals the plain
resave, a .qetz saved twice gives the same bytes; tst_lazyfolios,
tst_qetcontainer, tst_resaveunchanged (which now checks project.sqlite
holds only the document's tables).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:35:34 +13:00
Kellermorph 408ef5114d Terminal: pass the voltage across a separation only when it was edited
The three code paths that copied the voltage / protocol to the far
side of a function separated terminal -- the conductor properties
dialog, search/replace and the script API -- compared it against the
far wire's own properties instead of against the edit itself. The two
sides differ on that field just as often as not, so the comparison
held no matter what was edited and every change overwrote the far
side's voltage: setting a colour on a 24 V side dragged the 0 V one
along, where master left it alone. Compare the edit -- what the
dialog was filled with against what the user made of it -- so that a
change of the voltage / protocol is what crosses, which is what the
selection panel multi-edit already did. The values are read before
the undo commands are pushed, since QPropertyUndoCommand applies on
push.

An explicitly selected wire is never the far side of anything:
picking one wire on each side of a separation and editing them edits
them both, in full, however the selection happens to be walked. Nor
is a wire that belongs to a potential on any path, so reaching it
once normally and once across a separation gives it the full edit.
2026-10-13 20:13:18 +02:00
ispyisail 39855e1bb5 Merge #1473 into feat/qetz-db-truth-v2: format 4
#1473 keys the .qetz database's rows by the items' uuids, stores
true/false as 1/0 and writes the format to PRAGMA user_version, as
format 3. This branch's own changes (computed text left out of the
folios, tables without keys, folios kept as they were) become format 4;
formats 2 and 3 are still read.

Folios kept as they were copy their rows by folio uuid, and the keys of
those rows are kept from the other items' keys. The document's tables
are named doc_<name> only beside the project database's own tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:12:57 +13:00
ispyisail f48bd6717d Merge master into feat/qetz-db-truth-v2
#1470 keeps a .qetz's pictures out of the document; the index path still
reads them in, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:00:37 +13:00
ispyisail 8d118faf36 .qetz: key database rows by uuid, type true/false, version the schema
Answers the points arummler raised in discussion #1440 about the
project.sqlite that .qetz files carry (format 2, PR #1462):

- Folio rows were keyed "f1", "f2"... by position, and every symbol and
  wire carried a separate key beside its uuid. A row's key is now the
  item's own uuid, which stays in the folio file; no key attribute is
  added to the folio files of a project as QElectroTech saves it. An item
  without a usable uuid (none, or one another item shares, F100) gets a
  uuid made from its own content, the same on every save, never from
  where it is.
- freezeLabel is stored as 0/1 (INTEGER) and an element information's
  "show" in its own column; a value QElectroTech would not write stays in
  the folio file verbatim. The tag is kept in the JSON only when it is
  not the usual one.
- The database's PRAGMA user_version holds the format.

Format 3. Format 2 files, written by development builds since #1462,
still open: tst_qetcontainer reads one (fixtures/qetz_format2.qetz,
grafcet.qet saved by master aa503aec8) and saves it again.

Checked: tst_qetcontainer 38/38, and 343/343 readable projects of the
legacy corpus round-trip exactly. Every example saved as .qetz then .qet
equals its plain resave, and every format 2 file read by this build
equals what master writes from it. Each new test goes red with the
behaviour it checks broken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:34:35 +13:00
Laurent Trinques 8b680af938 Merge pull request #1471 from ispyisail/fix/lazy-refused-opens-eager
Linux build and tests / Build and test (Qt 6, Debug) (push) Failing after 1m41s
Fix lazy folios opening older projects slower than without them
2026-10-13 12:47:40 +02:00
Laurent Trinques 7c2a09f2bc Merge pull request #1470 from ispyisail/perf/qetz-picture-load
Speed up opening .qetz projects that hold large pictures
2026-10-13 12:42:15 +02:00
Laurent Trinques 132eefb8b8 Merge pull request #1467 from ispyisail/fix/qetz-zip-entry-date
Fix the dates of the files inside a .qetz project
2026-10-13 12:41:42 +02:00
ispyisail bfa9912c5f Make modal_quit_regression deterministic
The test interrupted QElectroTech at a random moment inside the dialog's
loop and called quitQET() there. Landing inside malloc(), the call waited
on the allocator's lock for ever: 1 run in 20 timed out at 120 s, on
master too. It now lets the program run on to the start of the main
thread's next QEventLoop::processEvents(), between two events, and calls
from there; the event loop watchdog wakes the loop every 50 ms.

A run could also stop before any dialog: SingleApplication keys a System V
shared memory block from a file in the temporary folder, 16 bits of the
key varying, and each run killed by gdb leaves its block; once a new
run's key fell on one of them (976 here), it refused to start. The run
now removes the blocks it leaves, attached to no process, and is tried
again (a new binary path, a new key, up to three times) when
SingleApplication stops it that way: the scenario never ran. The "dialog
entered" marker no longer prints when the program had exited.

60 runs under load, 976 old blocks present: 60 passed, one try repeated.
With refuseCloseWhileModal() disabled: 3 of 3 FAIL (signal 6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:51 +13:00
ispyisail 798010cc83 Change a wire's properties by uuid without building its folios (stage D5)
Scripts get conductorPropertyByUuid() and setConductorPropertyByUuid(),
the wire named by its uuid (conductorUuids()) rather than by a symbol's
terminal. A change goes to every wire at the wire's potential, as
setConductorProperty() does. When those wires are all on folios not
built, none is numbered by a formula, and what changes in each is what a
.qetz database holds in its own columns (number, function, voltage,
colour and section of the wire, cable, bus), it goes through the project
database: the store and the conductor rows, undoable by uuid. The
potential is followed in the database (projectDataBase::potentialWires():
through wire ends, folio reports and terminal blocks, as
Conductor::relatedPotentialConductors() follows it; a terminal block that
isolates its terminals answers nothing). Building a wire takes its change
(Conductor::fromXml()); a .qetz save writes it in the database, the folio
file copied; a folio written from its XML gets it there; a .qet builds.
Anything else builds the folio, as before.

tst_lazyfolios: every wire's potential from the database = as followed,
1 342 of 1 342 (industrial, eager and from a .qetz); a cable set by uuid
on a 14-wire potential builds no folio, reaches every wire, saves as when
built, undoes, shows once built (fails when building ignores the change,
and when the .qetz writer does). The test's run limit is raised (500 s ->
1 500 s, ctest 1 800 s): under load it ran past 500 s. ctest 96/96 both
ways.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:51 +13:00
ispyisail a6d81a21a8 Build the folios before reading the drawn items' view (stage D6)
A query of drawing_item_view, which is made of the shape, text and
picture tables, built no folio, and with folios built only when needed
(the default since D6) it found nothing: the drawn items are written to
the database by built folios. A query naming the view builds them, as
one naming the tables does.

Found by the qet MCP server's suite (four drawn-item tests failed lazily,
passed with every folio built). tst_lazyfolios: the view's rows from a
.qetz as when built (5 = 5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:51 +13:00
ispyisail 60489c9893 Fix two faults the end-state gates found (stage D7)
A .qetz re-saved after opening from its index saved wrong folio numbers
in its database: reading each folio's header wrote its row with a
number counted from the folios read so far ("1/1", "2/2" for "1/3",
"2/3"), and an index-opened project keeps the rows it was saved with,
so nothing wrote them again. The folio table's rows are no longer
written while the project is being read (projectDataBase::
diagramInfoChanged()). Folio list tables showed it.

A .qetz whose project.sqlite is not a database crashed QElectroTech: the
first statement could not be prepared and SQLite was handed a null
statement. A statement that cannot be prepared now binds nothing and
steps to no row, and the file is refused.

Gates: 342 corpus projects,
written by QElectroTech 0.3 to 0.200, all open; .qet -> .qetz -> .qet
equals the plain resave with every folio built; a .qetz re-saved twice
holds the same bytes entry by entry but the save's own name, path and
time (the first fault made the first re-save differ); hostile .qetz
(truncated, garbage database, view for a table, trigger, escaping and
repeated names, later format, missing folio, 2 GiB zip bomb) neither
crash nor hang and none opens; an unknown extra entry still opens.
tst_lazyfolios: folio numbers with totals from the index as when built
(fails without the fix); tst_qetcontainer: garbage database refused
(segfaults without it). ctest 96/96 both ways; save and export gates
same on 342.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:51 +13:00
ispyisail 5c4fa8fadf Build folios only when needed, by default (stage D6)
Every project now opens without building its folios; a folio is built
when it is shown or something needs its items (QETProject::lazyFolios()).
QET_LAZY_FOLIOS is gone; QET_EAGER_FOLIOS=1 builds every folio on
opening, as before, for comparisons and troubleshooting. Comments that
named the old variable now say "lazy folios"; the tests switch modes
with the new one.

The first save of a .qet as .qetz still builds every folio: a .qet does
not say which program wrote it, so its folios are written again rather
than copied. A .qetz opens and saves without building.

ctest 96/96 by default and with QET_EAGER_FOLIOS=1; 342 corpus projects
save the same in all four arms and export the same. In the GUI
(qet-gui-dialog): a 136-folio .qetz opens with no folio built, shows
folios 1 to 4, saves building none, the save = the original but the time
spent in it; a .qet opens lazily, saves as a .qetz beside it after the
notice, the .qet unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:51 +13:00
ispyisail b151f52428 Save a new title, author or revision without building the folio (stage D5)
A folio not built whose own data changed was built to save, with the
folios its links reach. When what changed is only title block fields
nothing on a folio is computed from -- its title, author, date,
revision index, version, file name, template; a formula reads the
folio's label, plant, location, extra fields and grid -- its items as
read are what building it would write: it is saved from them with its
own data as it is now (Diagram::deferredXmlWithHeader()), .qet and
.qetz alike, with the information of its symbols changed through the
project database (or built, when one of those needs it). Any other
change still builds it.

136-folio .qetz, revision index and author set on every folio by a
script: 0.8 s, saved in 6.8 s, no folio built; with every folio built,
the same file. tst_lazyfolios: a new revision on every folio of a .qet
and of a .qetz saved as when built, building only the folios holding a
table (fails, 47 built, when every change counts); a folio whose label
changed is built (fails when none counts). ctest 96/96 eager and lazy;
342 corpus projects save the same in all four arms and export the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:44 +13:00
ispyisail 244de05421 Change a symbol's information without building its folio (stage D5)
A script changing the information of a symbol on a folio not built, when
nothing else shows that information (no symbol built holds it, no
partner is linked to it, no label formula), changes it through the
project database: the store and the symbol's element info row, undoable
by uuid (ChangeStoredElementInformationCommand). Reading it back reads
the store. Building the symbol later takes the change
(projectDataBase::takeUnbuiltEdit(), in Element::fromXml()), the label
and texts following as for any change. Saving a .qetz writes the
changed information in the database in place of the rows the folio was
read with, its file copied, the folio still not built; a symbol whose
file holds no information block, or a .qet, builds the folio to save.

136-folio .qetz, "comment" set on the 3 513 symbols the database can
change: 0.9 s, no folio built, saved in 6.1 s, none built (27 s and 116
folios when saving built them). The same edits on the 4 039 symbols with
no partner and no formula, lazily and with every folio built: the two
saved projects are the same file. tst_lazyfolios: a script changes a
symbol not built, building none; saved as when built (fails when
building ignores the change, and when the .qetz writer does); undone;
the folio built afterwards shows it. ctest 96/96 eager and lazy; 342
corpus projects save the same in all four arms and export the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:44 +13:00
ispyisail 9078e4e321 Answer scripts from the folio asked for, without building every folio (stage D5)
The scripting API (scripts and the assistant's live mode) reached a folio
through diagrams(), which builds every folio, for every call. A folio's
own data -- count, title, uuid, title block, border, wire defaults --
now comes from folios(), built or not; a call reading or changing a
folio's items builds that folio, as showing it does (built()); selecting
a symbol by uuid builds the folio the project database places it on.
Search and replace and the continuity check still build every folio.

DiagramContent worked out one wire of each potential when made, which
follows potentials through folio reports and built the folios at the
other end, for every caller; it is now worked out when first asked
(Search and replace asks; a merge asks both sides, as before).

tst_lazyfolios: on a .qetz, a script reads every folio's title, uuid and
border building none (fails with diagrams()), and the symbols and wires
of the folio with the most links to other folios as when built, building
that folio only (27 before potentials were lazy). ctest 96/96 but
modal_quit_regression in both modes: a race in that test (gdb interrupts
the main thread inside malloc while the folio view repaints during the
dialog, master too), 2 of 23 runs here, 0 of 10 on master, passing alone.
342 corpus projects save the same in all four arms, and export the same
wiring list and wire numbers lazily.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:44 +13:00
ispyisail 65b10b01d9 Export the wiring list without building every folio (stage D5)
The wiring list export read the project as QETProject::toXml() writes
it, which builds every folio. It now reads a document made without
building: a built folio written as saving writes it
(QETProject::folioXml(), which keeps its symbols' links to partners not
built), a folio not built copied from its XML, and what QElectroTech
computes for a wire's ends -- its symbols' labels, their last partner's
label, the terminals' names -- filled in from the symbols built and from
the project database. A symbol still waiting for a partner not built
takes its partners from the database, saved order.

136-folio .qetz, wiring list from the command line: 30.1 s with every
folio built, 4.5 s lazily, same file. tst_lazyfolios: the wiring list of
a .qet and of a .qetz as when built, building no folio. New corpus gate
(it fails on
all 6 projects tried with terminal names broken): --export-cables and
--export-wires lazily from the .qet and the .qetz = eager on 342
projects; one (forum f3386) differed until waiting symbols took their
partners from the database. ctest 96/96 eager and lazy (one earlier
run had modal_quit_regression and tst_resaveunchanged fail once each,
both passing on rerun in both modes); 342 corpus projects save the same
in all four arms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 23:03:44 +13:00