Reading the values a workbook already carries, and how big it claims to be.
Every formula cell of a saved workbook holds two things: the formula, and the
result the spreadsheet application last computed for it. This module reads the
second — the value the user actually saw on screen — which is what the report
compares its own recalculation against, and the only source for a cell no
engine can compute.
Two readers, on purpose. python-calamine is the fast one and returns native
Python types, so a date is a date rather than a number wearing a format
string. openpyxl is the lazy one, and it takes over for a file calamine cannot
open or must not be handed: it builds a sheet as a dense rows × columns array
before returning anything, and a workbook declaring A1:XFD1048576 would ask
the allocator for 512 GiB. That is why :func:declared_cells exists, and why
it is consulted before anything reads a cell.
CachedValues
Values the spreadsheet application cached in the file.
They are what the user last saw on screen. Workbooks written by openpyxl
carry no cache for formula cells, workbooks saved by Excel or LibreOffice
do; either way constants and their number formats are always readable.
Source code in src/linexcel/loader.py
| class CachedValues:
"""Values the spreadsheet application cached in the file.
They are what the user last saw on screen. Workbooks written by openpyxl
carry no cache for formula cells, workbooks saved by Excel or LibreOffice
do; either way constants and their number formats are always readable.
"""
def __init__(
self,
values: dict[tuple[str, int, int], Any],
date_cells: set[tuple[str, int, int]],
epoch_1904: bool,
truncated_sheets: set[str] | None = None,
):
self._values = values
self._date_cells = date_cells
self.epoch_1904 = epoch_1904
self.truncated_sheets = truncated_sheets or set()
def get(self, sheet: str, row: int, col: int) -> Any:
return self._values.get((sheet, row, col))
def is_date(self, sheet: str, row: int, col: int) -> bool:
return (sheet, row, col) in self._date_cells
def __len__(self) -> int:
return len(self._values)
|
declared_cells
declared_cells(data: bytes) -> int
The largest rectangle any sheet of the package declares, in cells.
Not what it holds — what it says it uses. The two differ wildly: one stray
cell at XFD1048576, and a sheet with three numbers in it declares 17
billion. Read from the <dimension> element rather than from the cells,
because the whole point is to know the size before reading anything.
0 when no sheet declares one, which is also what a package this cannot
parse returns: the callers treat it as "no reason to worry", since a writer
that omits <dimension> is not the one that writes a stray corner cell.
Source code in src/linexcel/loader.py
| def declared_cells(data: bytes) -> int:
"""The largest rectangle any sheet of the package *declares*, in cells.
Not what it holds — what it says it uses. The two differ wildly: one stray
cell at XFD1048576, and a sheet with three numbers in it declares 17
billion. Read from the ``<dimension>`` element rather than from the cells,
because the whole point is to know the size before reading anything.
``0`` when no sheet declares one, which is also what a package this cannot
parse returns: the callers treat it as "no reason to worry", since a writer
that omits ``<dimension>`` is not the one that writes a stray corner cell.
"""
largest = 0
try:
with zipfile.ZipFile(io.BytesIO(data)) as zf:
for part in zf.namelist():
if not re.fullmatch(r"xl/worksheets/sheet\d+\.xml", part):
continue
with zf.open(part) as handle:
match = _DIMENSION_RE.search(handle.read(_DIMENSION_WINDOW))
if match is None:
continue
columns = col_to_num(match.group(1).decode("ascii"))
largest = max(largest, columns * int(match.group(2)))
except Exception:
return 0
return largest
|
load_cached_values
load_cached_values(
data: bytes,
warnings: list[str] | None = None,
reporter: Reporter | None = None,
*,
max_cells_per_sheet: int | None = None,
max_dense_cells: int | None = None,
) -> CachedValues
Read the file's cached values once, keyed by (sheet, row, col).
python-calamine (the Rust engine) is the hot path: it returns native Python
types directly, so dates are detected by type instead of by number_format
string, and it is roughly an order of magnitude faster than openpyxl on
large files. openpyxl remains the fallback for the rare file calamine cannot
open; there its number_format-based date detection keeps the edge case (a
number formatted as a date but stored as a float) covered.
A sheet that declares more than :data:MAX_DENSE_CELLS never reaches
calamine at all. It builds a sheet as a dense rows × columns array before
returning anything to Python, so A1:XFD1048576 asks the allocator for
512 GiB — and an allocation failure in Rust aborts the process. That is
not an exception, and no try around this call would see it. openpyxl's
read-only reader is lazy and already bounded, so it takes the file instead.
Source code in src/linexcel/loader.py
| def load_cached_values(
data: bytes,
warnings: list[str] | None = None,
reporter: Reporter | None = None,
*,
max_cells_per_sheet: int | None = None,
max_dense_cells: int | None = None,
) -> CachedValues:
"""Read the file's cached values once, keyed by (sheet, row, col).
python-calamine (the Rust engine) is the hot path: it returns native Python
types directly, so dates are detected by type instead of by number_format
string, and it is roughly an order of magnitude faster than openpyxl on
large files. openpyxl remains the fallback for the rare file calamine cannot
open; there its number_format-based date detection keeps the edge case (a
number formatted as a date but stored as a float) covered.
A sheet that declares more than :data:`MAX_DENSE_CELLS` never reaches
calamine at all. It builds a sheet as a dense rows × columns array before
returning anything to Python, so ``A1:XFD1048576`` asks the allocator for
512 GiB — and an allocation failure in Rust *aborts the process*. That is
not an exception, and no ``try`` around this call would see it. openpyxl's
read-only reader is lazy and already bounded, so it takes the file instead.
"""
cell_limit = limit_or_default(
"max_cells_per_sheet", max_cells_per_sheet, MAX_CELLS_PER_SHEET
)
dense_limit = limit_or_default("max_dense_cells", max_dense_cells, MAX_DENSE_CELLS)
reader_kwargs = (
{} if max_cells_per_sheet is None else {"max_cells_per_sheet": cell_limit}
)
def finish(cached: CachedValues, *, report_truncation: bool = True) -> CachedValues:
if warnings is not None and report_truncation:
for sheet in sorted(cached.truncated_sheets):
warnings.append(
f"Cached values on sheet '{sheet}' were limited to "
f"the first {cell_limit:,} cells in row-major order; "
f"omitted values are unavailable"
)
return cached
declared = declared_cells(data)
if declared > dense_limit:
if warnings is not None:
warnings.append(
f"A sheet declares a used range of {declared:,} cells. Values "
f"were read the slow way, and only the first "
f"{cell_limit:,} cells of each sheet were kept, so "
f"some may be missing from the report. If the sheet does not "
f"really hold that much: {STRAY_CORNER_ADVICE}"
)
return finish(
_load_cached_values_openpyxl(data, reporter, **reader_kwargs),
report_truncation=False,
)
try:
return finish(_load_cached_values_calamine(data, reporter, **reader_kwargs))
except Exception as exc:
# Not silent: the slow reader detects dates from the number format
# rather than from the type, which is a different answer on an edge
# case, and someone comparing two runs deserves to know which read it.
if warnings is not None:
warnings.append(
f"Values were read with openpyxl rather than the fast reader "
f"({type(exc).__name__}: {exc}). The lineage is unaffected; a "
f"cell whose date is stored as a plain number may read "
f"differently."
)
return finish(_load_cached_values_openpyxl(data, reporter, **reader_kwargs))
|