Product evidence metrics
These metrics turn the package's safety and predictability goals into reviewable acceptance evidence. They help users decide whether the package fits an integration and help maintainers detect a weakened contract.
Purpose and claim boundary
The package is not positioned as universally faster, smaller, or higher quality than every image library. Native codecs, devices, source images, thermals, and filesystem state make that claim indefensible from one runner capture.
The primary question is narrower: does a supported request behave according to its declared byte-budget, failure-safety, and runtime-capability contracts? Current figures are a v0.4.0 evidence snapshot, not production adoption or incident-rate statistics.
Known v0.4.0 defect: the retained iOS capture has a vertically inverted output. It remains public so the provenance record is not silently rewritten, but it does not count as successful visual-integrity evidence. The 0.4.1 source candidate fixes the renderer and gates future captures with auto-oriented SSIM plus a vertical-flip control.
Primary metrics
Byte-budget attainment
Definition: supported cases whose completed output is at or below output.maxBytes, divided by supported cases that declare an attainable target in a versioned fixture plan.
byte-budget attainment = cases with byteSize <= maxBytes / supported budget casesThe release target is 100% for declared attainable fixtures. An intentionally unreachable target is measured separately: the result must be the smallest generated candidate and its byteSize must make the miss observable. Output dimensions, decode-back validity, and metadata policy are guardrails; reducing bytes by returning the wrong geometry or an unreadable file does not count.
The current native walkthrough snapshot contains two supported cases and both meet an 8,000-byte target. The retained v0.4.0 iOS result is vertically inverted, however, so only 1 / 2 captures currently passes the combined byte and visual-integrity guardrail. Neither count estimates a general success rate.
Failure-safe completion
Definition: failed or cancelled calls that leave a newly published output or temporary file. The target is exactly zero.
residual-output rate = calls with a new residual output / failed or cancelled callsThe contract is exercised at JavaScript preflight, queued, and running abort boundaries, Android queued/running and target-search cleanup, and iOS target-search and post-write cleanup. A completed call remains a separate guardrail: it must return a decodeable file whose measured bytes and dimensions match the result object.
This is deterministic regression evidence, not a production crash-rate or field-reliability estimate.
Runtime capability agreement
Definition: exercised capability claims whose corresponding operation behaves as reported, divided by all capability claims exercised on the same runtime.
capability agreement = matching exercised claims / exercised capability claimsThe target is 100% within each recorded runtime. Unsupported output must remain an explicit rejection rather than a soft success, and supported output must decode back with the declared format behavior.
The current public walkthrough records Android and iOS capabilities before the compression call. Unit, native, and instrumentation tests cover format routing, but two recorded runtimes are not broad enough to publish a cross-device agreement percentage. Expanding the device and OS matrix is required before this becomes a representative rate.
Driver and guardrail metrics
Planned working-pixel reduction
For a resize request, this reports how much smaller the planned decode is than the source pixel count.
planned reduction = 1 - planned decoded pixels / source pixelsThe deterministic Android policy fixture maps an 8,000 × 6,000 source (48,000,000 pixels) to a 1,600 × 1,200 decode plan (1,920,000 pixels), a 96% reduction in planned decoded pixels. The iOS native large-image suite executes the same 48 MP to 1.92 MP request through ImageIO thumbnail downsampling.
This is not a peak-memory measurement. Resident memory must be measured separately on representative physical devices before making an RSS or OOM-rate claim.
Sensitive metadata retention
The privacy guardrail counts named sensitive fixture fields that remain after a safe or strip operation.
The Android safe fixture retains 0 of 7 checked sensitive fields: GPS latitude, GPS longitude, camera owner, body serial, lens serial, user comment, and image unique ID. On iOS, safe and strip build destination properties without source metadata. preserve is measured separately because retaining source metadata is its declared behavior.
Packed-consumer build coverage
The v0.4.0 release matrix contains four release-required consumer configurations—React Native 0.73 Legacy, React Native 0.86 Legacy, React Native 0.86 New Architecture, and Expo 57 / React Native 0.86 New Architecture—on Android and iOS. All 8 / 8 platform targets passed with the packed candidate.
This confirms installation and native build compatibility only. It does not claim that every intermediate React Native version, OS version, device codec, or application dependency graph was exercised.
Evidence charts
Separate source fixtures; utilization is not a quality comparison.
Pixel plan only; this is not a peak-memory measurement.
Current evidence snapshot
| Signal | v0.4.0 observation | Source |
|---|---|---|
| Native byte + visual-integrity cases | 1 / 2 valid; both are under 8,000 B, but v0.4.0 iOS is vertically inverted | Native walkthrough |
| Android output | 2,264 B, 28.3% budget utilization, 16.7% of source bytes | Demo manifest |
| iOS output | 2,353 B and under budget, but visual integrity failed because the pixel layout is vertically inverted | Demo manifest |
| 48 MP resize plan | 1.92 MP planned decode, 96% planned-pixel reduction | Android resource-policy test |
| Cancellation cleanup | Zero residual output is the asserted invariant; JavaScript, Android, and iOS cleanup tests pass | Verification architecture |
| Sensitive metadata fixture | Android safe: 0 of 7 named sensitive fields retained; iOS safe/strip: no source metadata copied | Android metadata test |
| Packed-consumer targets | 8 / 8 native builds passed | Compatibility matrix |
Android and iOS demo compression ratios must not be compared: the source fixtures have different byte sizes. Budget utilization shows distance from the requested ceiling for that exact fixture; lower utilization does not by itself mean better visual quality.
Interpretation limits
- Two native demo cases do not establish a general byte-budget success rate.
- Planned pixels are not peak resident memory.
- Deterministic cleanup tests are not production incident telemetry.
- Compatibility builds are not a device-codec census.
- Byte size without a perceptual metric such as SSIM or DSSIM does not establish image-quality superiority.
- The retained v0.4.0 iOS capture is defect evidence, not a valid quality or byte-budget success once visual integrity is included.
- The exact-plan timing benchmark is an environment-bound observation and is documented separately from these product-contract metrics.
Large-photo source-tree capture contract
The Native Demo Evidence workflow also captures a separate, kit-only 12 MP JPEG scenario from the checked-out source tree. It uses one project-generated, non-personal 4,000 × 3,000 fixture, a 1,600 × 1,200 contain request, JPEG quality 90, a 500,000-byte ceiling, a strip metadata policy, two warmups, and ten measured calls. Iteration 10 is the representative output.
This source-tree bundle is intentionally separate from the current v0.4.0 site snapshot. A result is accepted only when source/output bytes and hashes, decoded geometry, the exact environment and capabilities, SSIM ≥ 0.90, an upright-over-vertical-flip margin ≥ 0.02, source retention, and cleanup of all 12 package-owned outputs agree. The source-to-output byte difference is signed; it is not an avoided-transfer, storage-savings, or cost claim because the source remains and no matched baseline is supplied.
See the benchmark methodology for the complete timing boundary, artifact verifier, and interpretation limits. The economic resilience archive documents the append-only source-tree preservation contract and shows its claim boundaries. It renders a methodology preview while the index is empty, then renders only archive-derived bytes, visual scores, lifecycle facts, retained assets, and run provenance after an exact-master capture is present.
Reproduce the evidence
pnpm verify:demo-evidence
pnpm test:coverage
pnpm verify
pnpm fixtures:compatibility:check
pnpm example:android-unit-test
pnpm example:ios:large-image-test
pnpm example:ios:metadata-testSee the benchmark methodology for raw timing samples, comparator boundaries, and the prohibition on universal performance claims.