join() and readDocument() gained a parameter on both sides: master's
Pictures map (#1470) and this branch's format. Both are kept, pictures
first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An SVG keeps every picture at full resolution, whatever size it is
drawn at: a folio with 22 phone photos shown at a tenth of their size
gave a 270 MB SVG and took 65 s to write, most of it compressing the
photos again.
The export dialog gets "SVG: smaller pictures" (off by default, kept
with the other export settings), and --export-svg gets
--svg-small-pictures. With it, a picture is stored at four times the
size it is drawn at when that is smaller than the picture; smaller
pictures stay whole. Four times is about 375 dpi on a printed folio.
The exporter sets Diagram::smallPictures() for its own drawing only,
so screen, print, PDF and PNG are unchanged. It is not part of
applyProperties(), which every format uses.
On that project: --export-svg 84.9 s -> 34.5 s (opening is 20 s of
both), 270 MB -> 57 MB, peak memory 2.45 -> 1.86 GB. Rendered at twice
the folio size, the two SVGs differ by more than 32/255 on 0.23 % of
pixels, along photo detail.
tst_svgsmallpictures: an 800x600 picture drawn at 0.1 is stored at
320x240 with the option, through --export-svg and through the
dialog's QPicture route, and at 800x600 without it; a 40x30 picture
stays whole. It fails when the factor is changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QETZ-FORMAT.md describes format 3: the archive and its files, the
manifest and the version rules, every table and column of project.sqlite
and what it comes from, how rows are keyed, the markers a moved block
leaves, and what reading checks. Asked for in #1440: the format had no
written specification.
tst_qetcontainer checks that the schema the document gives is the one
written: the same tables and columns, in the same order, with the same
types and keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The JSON column of a property row held only its show flag, in every
project QElectroTech saves: the 25 examples and a 136-folio project used
it for nothing else. Folio and project properties now store it as a
number, as symbol information does, so a saved project's database holds
no JSON; JSON is left for attributes QElectroTech does not write (an
older or hand-edited file).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The database held where each moved block sat among its parent's
children (props_at, info_at, links_at), though reading only asked
whether there was one: the marker the folio file keeps says that, and
where it goes back. The columns are gone; format 2 files leave the same
markers and still open.
A .qetz of a later format whose min-reader allows this version was
still refused by the database's own format check. It opens now,
read-only, with a message: saving it here could lose what this version
does not know. Save As writes a copy in this version's format.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A folio file in a .qetz did not say which symbol each item is: the type
(embed://...) was moved to project.sqlite, so a folio read alone could
not be drawn or diffed. It stays in the folio file now; format 2 files,
which moved it, are still read.
Nothing checked that project.sqlite and the folio files agree: a row no
item has, an item's row on another folio, or information rows of no
symbol were read as far as they went. Reading now refuses such a file and
says what does not match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three code paths that copied the voltage / protocol to the far
side of a function separated terminal -- the conductor properties
dialog, search/replace and the script API -- compared it against the
far wire's own properties instead of against the edit itself. The two
sides differ on that field just as often as not, so the comparison
held no matter what was edited and every change overwrote the far
side's voltage: setting a colour on a 24 V side dragged the 0 V one
along, where master left it alone. Compare the edit -- what the
dialog was filled with against what the user made of it -- so that a
change of the voltage / protocol is what crosses, which is what the
selection panel multi-edit already did. The values are read before
the undo commands are pushed, since QPropertyUndoCommand applies on
push.
An explicitly selected wire is never the far side of anything:
picking one wire on each side of a separation and editing them edits
them both, in full, however the selection happens to be walked. Nor
is a wire that belongs to a potential on any path, so reaching it
once normally and once across a separation gives it the full edit.
Answers the points arummler raised in discussion #1440 about the
project.sqlite that .qetz files carry (format 2, PR #1462):
- Folio rows were keyed "f1", "f2"... by position, and every symbol and
wire carried a separate key beside its uuid. A row's key is now the
item's own uuid, which stays in the folio file; no key attribute is
added to the folio files of a project as QElectroTech saves it. An item
without a usable uuid (none, or one another item shares, F100) gets a
uuid made from its own content, the same on every save, never from
where it is.
- freezeLabel is stored as 0/1 (INTEGER) and an element information's
"show" in its own column; a value QElectroTech would not write stays in
the folio file verbatim. The tag is kept in the JSON only when it is
not the usual one.
- The database's PRAGMA user_version holds the format.
Format 3. Format 2 files, written by development builds since #1462,
still open: tst_qetcontainer reads one (fixtures/qetz_format2.qetz,
grafcet.qet saved by master aa503aec8) and saves it again.
Checked: tst_qetcontainer 38/38, and 343/343 readable projects of the
legacy corpus round-trip exactly. Every example saved as .qetz then .qet
equals its plain resave, and every format 2 file read by this build
equals what master writes from it. Each new test goes red with the
behaviour it checks broken.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comments added with the lazy folios cited planning notes kept outside the
QElectroTech repository, which a reader of the source cannot open. The
sentences now stand on their own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With QET_LAZY_FOLIOS=1, a project whose database cannot be filled from
the file (folios or symbols with no saved uuid, two symbols sharing one,
a folio number using %autonum, wires naming their ends the older way)
had its folios left unbuilt, then built all at once while the database
was filled from them. That is slower than building them as they are
read: a 17 MB project reported on the pull request took 24.8 s to open
instead of 19.2 s, and most files saved by an older version have no
folio uuids.
The file is now checked for those reasons before the folios are read;
when one applies the project opens as without QET_LAZY_FOLIOS. The rarer
reasons, which need the definitions or every folio's links, still fall
back as before.
tst_lazyfolios opens a copy of industrial.qet with one symbol's uuid
given to another, and checks no folio is built after reading.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A .qetz keeps each folio picture as a PNG file. On opening, join() put
every picture back into the document as base64 text, and the folio then
decoded that text again, twice. On a project with 20 pictures of 7 MB
(140 MB of PNG, the shape of the project on PR #1462) that cost about
8 of the 12 seconds the open took:
QDomDocument::createTextNode() on the base64 3.9 s
DiagramImageItem::fromXml(), base64 decoded 3.9 s (2 x 1.9 s)
the PNG decoding itself 0.3 s
join() now takes an optional map. Given one, each picture stays a file:
the <image> keeps its qetz-file attribute and the bytes go in the map.
QETProject keeps the map for as long as the project is open, and
Diagram::fromXml() hands a picture's bytes to the new
DiagramImageItem::fromXml(element, picture). Kept, not cleared after
the load, so a folio built later finds its pictures too; the bytes are
shared with the PNG cache of the pictures built from them, so this costs
almost no memory. readDocument() without a map, as the automatic
numbering import uses it, still gives the whole document.
fromXml() of a .qet picture also decodes its base64 once instead of
twice: the first text node, which the PNG cache already used. e.text()
also collects <image_base>'s text, but loadFromData() stops at the end
of the first picture, so the picture shown is the same.
Measured, same file, 3 runs each, offscreen:
.qetz 11.8 s -> 3.8 s, peak memory 1034 MB -> 655 MB
.qet 15.7 s -> 13.6 s
Resaving either file with either binary gives the same bytes, a cropped
and colour-keyed picture included, and every folio exports to the same
PNG pixels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
miniz is built with MINIZ_NO_TIME so that saving the same project twice
gives the same bytes. Without a clock it writes 0 as each entry's DOS
date: day 0 of month 0, 1980, which is not a date. Unzip tools show it
as 30 Nov 1979, others fail the conversion and show 13 Dec 1901
(reported on PR #1462).
Write 1 Jan 1980 00:00, the first valid DOS date, into the local and
central header of every entry once the archive is built. The bytes stay
reproducible, and miniz stays unmodified. Using the save time instead
was rejected: two saves would no longer give the same file, and miniz
converts through the local time zone, so even a fixed time_t would
differ between machines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The checkbox of a terminal element was an all-or-nothing switch: while
it was unticked every conductor property (function, wire colour,
section, number...) crossed to the other side of the terminal, and
once it was ticked nothing did at all.
Rename it to "Function separation" and let one property still cross:
the voltage / protocol, which describes the electrical node itself
rather than one side of it. Conductor::functionSeparatedConductors()
returns the conductors reachable only by crossing such a terminal --
the potential computed with the separations ignored, minus the one
that respects them -- and
ConductorProperties::applyTensionProtocolChange() gives them that one
property, in the conductor properties dialog, the selection panel
multi-edit, the script API and search/replace.
The stored element information stays "potential_isolating", so
existing symbols and projects keep their setting, and the existing
sub-case of terminals sharing a potential-group identifier is
untouched. The German translation of the new label is deliberately
left out of this commit and belongs with the other .ts updates.
The project's terminal strips -- strip information, the terminals of
each physical terminal by level, bridges and what they bridge -- move to
four tables of project.sqlite, when rebuilding them from the rows gives
the same XML exactly (as QElectroTech writes them). Anything else, or
strips that carry text a save would not write, stays in project.xml.
Test: tst_qetcontainer::terminalStripsInDatabase (two strips with a
bridge and a two-level terminal on industrial.qet's own terminals: out
of project.xml, back exactly; stays in the XML when the rebuild is
broken). Outside ctest: a strip fixture saved as .qetz then .qet by
QElectroTech equals its straight resave, and the strips are rows any
SQLite tool reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
Fourth stage of the zipped project file (#1440), format 2: what an
engineer would put in a list -- folio title-block fields and
properties, project properties, a symbol's uuid, type, prefix,
information and links, a wire's ends, number, function, colour,
section, cable and bus -- moves out of the XML into project.sqlite in
the .qetz, and lives only there. The folio files keep where things are
drawn. On opening it goes back into the document before the usual
reader runs, so nothing else in QElectroTech changes yet (the program
reading the database directly is the next step of the roadmap).
Each moved item keeps a key in the XML (qetz-key: its uuid where
unique); each moved block leaves a marker where it was, which keeps the
whitespace around it apart. A block of a shape this does not model
stays in the XML. A file is untrusted: the database is read with
trusted_schema off, query_only, and only from names that are plain
tables. Format 2 readers still open format 1; a format 1 reader
refuses format 2.
Tests (tst_qetcontainer): every example back exactly, whitespace
included; databaseHoldsTheData (industrial.qet: the folio files hold no
information, links, wire numbers or properties; the row counts match).
Outside ctest: 341 of 341 projects QET can open give the same .qet
through a format 2 .qetz.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
Third stage of the zipped project file (#1440). A project is saved as a
.qetz when its file name ends in .qetz -- Save As offers it as a second
file type, and --resave, --set-titleblock, the script API's save() and
the live-mode snapshot follow the name too. Saving a .qet project still
writes a .qet; nothing changes unless the user picks .qetz.
Opening recognises a .qetz by its content (a zip), not its name, and
reads it as bytes: text mode would alter it on Windows. The project
document is joined back and read by the same code as a .qet. Crash
backups stay plain XML. Drag and drop, the open dialog and the
numbering import accept .qetz; the database export names its file from
the project's name without assuming ".qet".
Parts are now written without added indentation (one line each): the
symbol definitions are kept as read, whitespace included, and parts
that gained indentation made the next save differ (five forum projects).
Tests: tst_resaveunchanged::containerSavesTheSameProject (four examples
saved as .qetz, then as .qet, equal the straight .qet; pictures checked
to fail without the fix), tst_qetcontainer::noWhitespaceAdded. Outside
ctest: 341 of 341 projects QET can open (examples, bugtracker, GitHub,
forum) give the same .qet through a .qetz; 181.6 MB of .qet became
36.9 MB of .qetz.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
Second stage of the zipped project file (#1440): QetContainer::split()
turns a project document into the entries of a .qetz -- mimetype first
and stored, a manifest with the format and the oldest format allowed to
read it, project.xml with each part replaced by <qetz-part file=.../>,
one file per folio, per symbol definition and per title block template,
and folio pictures as PNG files named by their SHA-256, stored once
each. join() puts it back and refuses a file of a later format, a
missing part or anything that is not a .qetz.
A picture leaves its folio only if its base64 comes back exactly, so
nothing here can change a project. Parts are written as QET writes a
.qet and read as it reads one. Still nothing opens or saves a .qetz.
Test: tst_qetcontainer, every example split, zipped, unzipped and
joined back to the same document (whitespace between elements aside);
pictures out of the folios once each; later format and missing part
refused. With QET_CONTAINER_CORPUS it also runs on a folder of projects:
341 of 341 readable projects collected from the bugtracker, GitHub and
the forum come back the same.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
First stage of the zipped project file discussed in #1440 (and #865):
the zip itself, nothing reads or writes a project with it yet.
miniz 3.1.2 (MIT, one .c and one .h, no zlib) is bundled unmodified in
sources/container/miniz, as the LZMA SDK is for .edz files, so builds
need nothing new. QetZip wraps it: plain ZIP with Deflate, entries stored
where asked (pictures, the mimetype entry), written through QSaveFile so
a failed save keeps the previous file, without timestamps so that the
same project gives the same bytes. Reading refuses unsafe or repeated
entry names, encrypted entries and archives over 2 GB once unpacked.
Test: tst_qetzip (round trip, same bytes twice, names refused both ways,
truncated and non-zip input). A written archive also passes unzip -t and
Python's zipfile.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
English has been the source language of the program for a while now
(upstream: "Make English the source language ..."), the cable feature
came in with French ones: about 200 translatable strings over 30 files,
the two settings pages built from .ui files, four comments and one test
which asserted the old wording (blocking review comment by ispyisail).
The wording follows what the code base already uses where it exists:
Plant (=) and Location (+) as qetinformation.cpp names those element
information, Designation for the field the cable form calls its BMK,
Length, Sheet for folio, Automatic numbering, Cross-reference, and Right
aligned / Centred / Left aligned the way cabletextformatdialog.cpp
already spells the three alignment entries.
Two things are more than a word-for-word swap:
- xrefpropertieswidget.cpp:291 wrote the label of the master field on
every type change with a hardcoded tr("Maitre"), so the English
"Master" coming from the .ui was overwritten for every type, cable or
not: a real bug next to being French. The wording now comes back from
the .ui and only a cable renames the field to "Text:", so the two
cannot drift apart again.
- tst_cabletypelist kept asserting that a new catalogue is *labelled*
in French. That label line is the translated header and reads English
now. The test still reads files with French headers on purpose:
headerAliasMap() keeps the five old French labels as aliases next to
the English ones, so catalogues written by earlier builds, including
the one the reviewer reads her own types from, keep loading.
The .ts files are deliberately not regenerated here. Upstream
regenerates them on master, as it did for the language change itself,
and a translation is keyed on the source string: while the sources were
French a German user saw French in the cable dialogs, and an English
source string with no translation yet shows as it is.
Build clean, ctest 92/93, the one failure being the pre-existing
headless tst_menubarkeyboard.
CI builds without Qt Qml, so --run is not compiled in and the test's
QElectroTech opened the project and waited until the 180 s timeout.
The test now skips when QET_HAS_SCRIPTING is not defined, as the other
--run tests are left out of such builds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
The catalogue is written with QSaveFile, which cannot write into a
folder that is not there. A fresh profile has no documents folder, and a
path typed into the settings may name a folder which was never created,
so the first write failed with "No such file or directory" and no way
for the user to see that the folder was the problem rather than the file
(non-blocking review comment by ispyisail).
writeFile() makes the folder first, which covers every writer: creating
the template, appending a type and editing one all go through it.
Covered by tst_cabletypelist::aMissingFolderIsCreated, writing into
<tmp>/not/there/yet/qet_cable_types.csv. Verified in both directions:
without the fix the test fails with 'createFile(path, &error) returned
FALSE. (No such file or directory)', with it the file is created and
reads back.
Deleting a cable and pressing Ctrl+Z twice leaves the cable out of the
project while AddCableCommand still stands and still thinks it has to
clean that cable up. The next edit of any kind makes the stack throw the
undone commands away: ~RemoveCableCommand frees the cable, and
~AddCableCommand frees the same memory again. A plain build hides it,
AddressSanitizer reports it as a heap-use-after-free in
addcablecommand.cpp (blocking review comment by ispyisail).
AddCableCommand now holds the cable as a QPointer, which is what
RemoveCableCommand already did: whoever of the two runs first frees it,
the other one sees a null pointer and has nothing left to do, in either
order of destruction. The destructor also stopped dereferencing
project() without checking it, which it did on that very line.
Covered by tst_cableundointegration, a probe linked against the
application's objects the way the other integration probes are. It draws
a cable, deletes it, undoes twice and pushes a further edit -- the
sequence which used to crash. Verified in both directions: with the raw
pointer the probe dies with SIGSEGV inside ~AddCableCommand called from
QUndoStack::push, with the QPointer it prints its PASS line.
A folio report arrow's text looks for a wire to show: on the arrow
itself, else on the arrow it is linked to, through that arrow's first
terminal. Some report symbols have no terminal -- ref_voyant_2_h.elmt in
the collection, and older collections' arrows -- so a project linking an
arrow with no wire to one of them crashed QElectroTech as it opened.
The other arrow's terminals are now checked first, as the arrow's own
already were.
Found on a project attached to the forum (saved by 0.70), which crashed
on opening.
Test: tst_resaveunchanged::reportArrowWithoutTerminalOpens builds the
case from Projet_vierge.qet's own symbols; it crashes without the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
The project database is kept up to date edit by edit, and a script's
qet.query() and the folio tables read it without a full rebuild. Three
edits left it out of date until the next rebuild:
- adding or removing a folio: the folios after it kept their old
position and folio number, and a removed folio's diagram_info row
stayed;
- moving a symbol: its folio cell (element.pos) stayed where the symbol
was added;
- deleting a wire: its two ends stayed in the terminal table, which
lists the terminals a wire ends on.
Positions and folio numbers are now written again after a folio is
added, removed or moved (one function for the three), the removed
folio's diagram_info row goes, a moved symbol's cell is written when
the database is next read (as drawing items are), and a deleted wire's
ends go unless another wire ends there.
Found by comparing the tables after random scripted edits with those
a fresh open of the saved project gives. Left alone: labels and wire
numbers made from a folio formula, which the drawing itself keeps until
the project is reopened.
Tests (tst_databasefromdocument): folioAddedAndRemoved,
movedElementCell, deletedWireTerminals; each fails without its part of
the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
A summary or parts-list table split over several folios saves its data
with its first part only; each later part names the part before it.
Deleting a folio keeps its tables in memory for undo, still in the
chain, so saving then named the deleted part as the previous table of
the next one. On reopening, every part after the gap had no data, and
closing the project crashed: the first orphaned part, destroyed, handed
its missing data on to the next part through a null model.
toXml() now names the nearest previous part still in the project, or
saves the data if there is none. setPreviousTable() no longer copies a
null model, so files already saved this way open and close.
Tests (tst_resaveunchanged): missingPreviousTableOpens,
deletedFolioKeepsTableChain; both fail without the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
One row per element and element it is linked to -- a coil and its
contacts, a pair of folio reports -- from each side, with the contact
group the element saved for it. Nothing reads it yet: cross-references
are the main thing one folio needs from another, and this is what lets
them be answered without building every folio.
Filled like the other tables: from the document when it can be read
exactly (a link only one side lists, between kinds that cannot link, or
giving a contact a second coil is left to the folios), from the folios
otherwise. A link made or undone afterwards rewrites that element's rows
when the database is next read, as drawing items are, since the contact
group is set after the link is made.
Test: tst_databasefromdocument compares the link table both ways on
every example; linksAreListed checks a coil and contact (with a group)
in m_000.qet, oneSidedLinkFallsBack the refusal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
On opening a project, a coil and its contacts are linked in the order
the folios' items are visited. When contacts are on an earlier folio
than the coil they link first, so the coil's list followed the folio
order instead of the file's, and contacts on one folio swapped places
from run to run. Saving the same project twice could then give two
different files.
After linking, Element::initLink() now puts the linked elements back in
the order the file listed them. Elements linked but not in that list
keep their place after it.
Test: tst_resaveunchanged::savedLinkOrderKept reverses a coil's two
contacts in m_000.qet and checks a resave keeps that order; it fails
without the fix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Wga9B92BNurDEtxxtCDxf
Upstream moved 56 commits ahead of the last merge. Six files conflicted,
all of them files where upstream's French -> English tr() translation wave
touched lines we had also touched with the cable feature. Resolution rule
applied everywhere: take upstream's English source strings, keep our cable
additions.
Resolved:
- sources/ui/xrefpropertieswidget.ui : kept our structure (m_labels_gb
pulled out of the display group, m_slave_row, the m_font_pb "Police..."
button of the cable cross-reference) and mechanically applied upstream's
20 English string replacements on top of it.
- sources/ui/xrefpropertieswidget.cpp : our version plus upstream's 7
translated combo entries; the tr("Cable") item we add for the cable
cross-reference type stays.
- sources/autoNum/ui/numparteditorw.cpp : upstream's English part names
("number format 1", "Sheet no.", "Plant", "Location", ...) for the
number-format list, with our comment explaining why type 3 (cable) takes
the conductor kinds of variable.
- sources/ui/configpage/configpages.cpp : upstream's English tab titles
(Conductors / Elements / Sheets) plus our Cables tab.
- sources/ui/configpage/projectconfigpages.cpp : upstream's "Sheets" tab
title plus our Cables tab block (single-rule SelectAutonumW(3)).
- sources/qetdiagrameditor.cpp : union of both sides - upstream's new
"Add a generic device" action (icon, status tip, data id) and its
English menu names ("File"/"Edit"/"Project"/"Display"), together with our
cable action, the Numbering and Lists menus, the cable list actions and
the CSV export in the Project menu.
Kept French: our own cable tr() source strings stay French for now. That
is the smaller change and the translation pass can follow separately.
Verified: full build 1065/1065, ctest 91/92 (only the pre-existing
headless tst_menubarkeyboard fails: no window manager, so Alt+F and F10
cannot open a menu bar), smoke start exit 124, no conflict marker left in
the tree.
ispyisail