A `%` column's cell was read as given when absolute, or joined to the
CSV's folder or `base`, so a CSV could make dataset.import open a file
on another computer (a UNC path or a `file:` URL with a host).
Each cell now goes through fileio::document_path, as an image
variable's path does when a data set is selected
(crates/engine/src/cmd/links.rs:1169-1173), with the CSV's folder or
`base` as the document's folder. Paths on another computer or on
another network share, devices and URLs are refused before any file is
read, with the line, the column and the reason. Absolute local paths,
`file:` URLs without a host and relative paths are still read, as
image paths are.
The params text and docs/mcp.md state the rule, and that relative `%`
paths are taken from the CSV's folder at import while relative `@`
paths are taken from the document's folder when a data set is
selected.
dataset.import refused every header cell that starts with % unless it was
a variable's exact name, so the five-column example on the data source
set-up page did not import. A %Name header now defines the graph variable
Name, or names it when the document has it, and each cell of the column is
the path of a graph data file. The file is read as the Graph Data window's
Import Data reads one, and the data becomes the data set's graph value,
which applies as graph.setData applies it.
A relative path is taken from the new base parameter, else from the folder
of the CSV file at path. A file that is missing, can't be read or is larger
than 1 MB, a file that gives a graph value longer than 256 KB, and %
columns that hold more than 256 MB in all refuse the import with the line
and the column, and the document is left as it was. Each file is read once
per import.
Import Data's text decoding, cell splitting and sheet-to-data steps move
from the Graph Data dialog to the engine (graph/data.rs), so both imports
read a file the same way. dataset.export's warning now lists a % header
over values as refused and a %-prefixed variable of another kind as read
back with another kind.
The Graph Data window's Import Data read UTF-16 only after a byte-order
mark, so a UTF-16 file without one filled the cells with NUL characters.
It now reads such a file in the byte order Import Data Sets' rule (#7348)
gives it: at least half of the two-byte units hold a NUL byte, and the
byte order is the one that reads the line breaks. The text is then read
as after a byte-order mark: an odd last byte is dropped and an unpaired
surrogate reads as U+FFFD. A file the rule refuses is read as UTF-8, as
before.
The rule now also gives a byte order to a one-line file, which reads a
line break in neither order: big-endian when more of its units start
with a NUL byte than end with one, else little-endian as before. Import
Data Sets still refuses a one-line file, which has no data rows; read in
the new order, such a file can be refused with a different error first.
A CSV file that holds NUL bytes and has no byte-order mark was refused.
It is now read as UTF-16 when at least half of its two-byte units hold
a NUL byte, in the byte order that reads a line break. A file with fewer
NUL bytes (UTF-8 or Windows-1252 text with stray NUL bytes, or UTF-16
text made mostly of characters above U+00FF) is still refused, and so is
a file that reads a line break in both byte orders, has an odd length,
isn't valid UTF-16 in that order or holds a NUL character. Refused files
get the same message as before.
dataset.import and variable.loadLibrary added an undo step, cleared Redo
and marked a saved document modified even when the variables and data
sets came out as they were, such as the same CSV or library loaded
again. Both now build the new variables on a copy and record a step only
when it differs. variable.loadLibrary's params text and docs/mcp.md say
so.
dataset.import read two header cells that named one variable (City
and City, or Shown and #Shown) into the same column, so each row kept
only the last non-blank cell and the other value was dropped without a
word. The import is now refused with the two column numbers, and the
message has a row in every complete-message catalog.
dataset.import replaced the reason for a refused cell with "value for
`X` must match its variable kind", so a visibility value such as
maybe and a graph value over 262,144 bytes gave the same message. The
error now keeps the line and gives the reason, and both messages have
a row in every complete-message catalog.
dataset.export checked only the 1 MiB size, so a document with more than
1000 data sets or 256 variables exported a CSV that dataset.import then
refused. Export now refuses such a document with a message naming the
limit, and the messages have rows in every complete-message catalog.
Found while testing crates/engine/src/cmd/variablecmds/csv.rs:146 (csv_escape) and :158-163 (cell_to_value)
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>