OpenJOC production architecture¶

This is the canonical description of the current production data flow. It describes implemented boundaries, not a promise that every historical design goal is complete.

OpenJOC implements its spatial rendering DSP directly rather than delegating object rendering to platform-specific spatial audio engines. Operating-system audio APIs may be used for integration and I/O, but they do not define OpenJOC's spatial rendering result. One renderer. Same spatial semantics across platforms.

The CLI, GStreamer plugin, and external FFmpeg bridge are transport frontends over the same OpenJocSession; they do not contain separate render pipelines. GStreamer owns its buffer/caps/segment lifecycle. The FFmpeg bridge uses libavformat for demux and public libavutil AVFrame allocation. In both cases OpenJOC retains E-AC-3/JOC decode, scene construction, DRC/dialnorm, speaker rendering, binaural HRTF rendering, latency, and drain state.

Complete access-unit classification¶

openjoc-api::admission owns only stateless inspection of one complete AU: indexing, grouping, audio-frame syntax, JOC carrier parsing and decoder topology. The adapters select an explicit metadata policy:

FFmpeg uses StrictOrObservedVendor, trying ETSI strict validation before the existing observed-vendor compatibility profile
GStreamer uses FirstJocPayload, preserving its direct first-payload check without introducing EMDF profile validation into its classifier

These policies intentionally remain different. For example, a syntactically complete carrier with mismatched payload group IDs or only a JOC payload is rejected by FFmpeg's profile classifier but can be classified as JOC by GStreamer's payload classifier. Classification is not permission to bypass the subsequent OpenJocSession admission rules. Profile rejection remains separate from parser errors, and the FFmpeg wrapper retains original error text and frame-count facts.

parse_access_unit_bounds continues to own AU boundaries in openjoc-eac3. FFmpeg staging/EOF, GStreamer parsing/caps, browser chunks/CMAF and LAV negotiation remain adapter-owned. The shared classifier does not manage any of their state. The 149-case policy contract freezes both adapters' complete pre-extraction classification transcripts.

Data flow¶

The explicit render-scene workflow is implemented in openjoc-render-scene. It depends on openjoc-render, openjoc-sofa, and openjoc-wave; it is deliberately separate from openjoc-scene, so decoder metadata and ReconstructionBasis rows cannot enter the caller-bound source contract.

raw EC-3 / seekable ISO BMFF
          │
          ▼
input/container ownership and access-unit delivery
          │
          ├── E-AC-3 base decode ──► channel-labelled PCM / RcLfe
          │
          └── EMDF payloads
                  ├── OAMD ──► metadata objects and timed state
                  └── JOC  ──► reconstruction-basis rows
                                      ├──► codec-domain JOC bridge
                                      │       (T(t) remains unresolved)
                                      │
                                      └──► exact clean binding gate
                                               │
                                               ▼
                                      bound decoded JOC Objects
                                               │
                                               ▼
                                      reconstructed dynamic ADM

The binding branch is deliberately narrower than the decoder and renderer branches. OAMD supplies decoded dynamic metadata; JOC supplies decoded object audio rows. Only the exact admitted carrier profile pairs them. Other scene consumers still receive metadata and decoder-coordinate rows without an automatic authored-object interpretation.

The reconstructed ADM branch establishes decoded-scene and ADM structural correctness within that profile. It does not recover the authored Atmos master or prove perceptual equivalence with a native JOC final renderer; native JOC playback remains the reference for renderer-identical localization.

The render-joc workflow adds an explicit experimental speaker branch after the decoded component boundary:

raw EC-3 / seekable ISO BMFF
          ↓
bounded AU delivery → E-AC-3 Base + RcLfe + JOC/OAMD decode
          ↓
decoded Base/RB codec-coordinate bundle + automatic bridge-control assembly
          ↓
persistent JocSpatialBridge → active N-channel speaker planes
  ├── Base LFE/RcLfe → LFE plane only
  └── active planes + LFE → shared final linked speaker gain
                              ↓
                      incremental semantic WAV/CAF output

The CLI presets are data-only registrations over the generic SpatialLayout and JocSpatialBridge projection path. The public library can consume a caller-defined channel registry, geometry, and output order without a preset-specific projection algorithm; the CLI accepts the same versioned geometry through --layout-file.

The CLI keeps its speaker/binaural rendering and F64 intermediate processing in joc_render, while its private joc_output module owns output staging, container selection, channel descriptions and incremental WAV/CAF writers. The existing publication order, overwrite rules, failure cleanup and error text are covered by the complete-file output contract. The paired ADM publication transaction remains separately owned by main.rs. This file-output boundary does not alter session precision, queues or FIR/LFE tail handling.

The 22.2 preset is ITU-R BS.2051-3 Sound System H (9+10+3): a four-layer bottom/middle/upper/top topology with 22 spatial speakers and two semantic LFE destinations. It uses the same N-layer point projector and semantic operation boundaries as the established layouts; LFE channels are never projection vertices.

Automatic assembly derives codec-coordinate control from validated OAMD and decoded Base/RB state. A complete sidecar is optional and takes precedence as an explicit override/test source; automatic and explicit sources are not implicitly merged. Topology/count, coordinate dimensions, metadata updates, and Base topology changes are validated at the integration boundary; no guessed row/object renderer is constructed. The public semantic PCM order is selected by the explicit preset; exact speaker WAV masks remain backend specific. 5.1 remains FL, FR, FC, LFE, Ls, Rs.

The parser reads what is present in the carrier. Validation then applies an explicit profile. The decoder consumes an accepted representation; it does not hide vendor compatibility decisions.

Layer boundaries¶

Input and container¶

Raw E-AC-3 uses the in-process incremental reader. Ordinary seekable ISO BMFF uses a sample cursor with container ownership kept separate from the AU consumer. Non-seekable and fragmented MP4 are outside the current contract.

E-AC-3 base¶

The base decoder keeps frame, audio-block, coupling, SPX, AHT, rematrix, substream and TDAC state explicit. Channel labels and RcLfe are retained as base-carried information; RcLfe is not a dynamic reconstruction row.

OAMD and profiles¶

OAMD metadata is parsed into typed state and timed updates. ETSI_STRICT enforces the published validation rules. OBSERVED_VENDOR_COMPAT is explicit and partial: it preserves original metadata and records deviations, but does not assign meaning to unresolved vendor continuation.

User-facing decode and decode-payload commands expose a separate AUTO selection policy. It parses once, evaluates strict validation first, and uses the existing compatibility policy only when every blocking deviation is already whitelisted. Malformed, unsafe, unknown, and non-whitelisted failures remain failures. Selection diagnostics include the requested and selected profiles, strict status, deviation set, and reason. AUTO is not a parser or renderer profile; explicit ETSI_STRICT never falls back, while normative inspection remains strict by purpose.

The observed OAMD warp_mode value raw=3 remains reserved under ETSI strict parsing. The exact observed compatibility profile preserves raw3 as opaque and may use the decoded OAMD spatial metadata directly for the tested scene scope; no production alias, transform, offset, or trim guess is present.

Scene and binding¶

ObjectScene retains metadata objects, timed positions, and decoder-coordinate rows as separate domains. For the exact admitted decoded-JOC/OAMD profile, SemanticBindingState::ResolvedWithinCarrier records a carrier-local pairing for the reconstructed ADM path. For all other profiles it remains Unresolved; there is no implicit row == authored object, slot identity, or dominant-row fallback.

JOC ReconstructionBasis¶

JOC reconstruction produces rows with structural indices and deterministic numerical behavior. The rows remain decoder coordinates and are not authored- object PCM. In the exact admitted carrier profile, a row also has a scoped decoded JOC Object interpretation for the dynamic ADM path; that does not turn it into an authored stem. Diagnostic WAV export uses diagnostics/reconstruction_rows/row_NNN.wav.

The stable component boundary uses ReconstructionBasisRowIndex as a local decoder-coordinate identity. DecodedComponentLayout and the CLI diagnostics/components.json manifest distinguish Base full-band channels, separate Base LFE, indexed RB rows, and SemanticBindingState::Unresolved or its admitted carrier-local counterpart without retaining another PCM copy. Operations requiring authored-object audio identity fail explicitly even when decoded-object binding is admitted; component-domain decode and streaming remain available.

JOC spatial reconstruction bridge¶

openjoc-scene exposes JocSpatialFrameBridge and the versioned openjoc.joc-spatial-reconstruction.v1 codec-domain contract. A borrowed CodecBasisBlock carries explicitly labelled Base full-band PCM, indexed ReconstructionBasis rows, and separate RcLfe; JocSpatialMetadataFrame carries the current OAMD payload and structural programme dimensions; and SampleRange gives each committed decoder frame an absolute half-open sample interval. The bridge is streaming and retains no duration-proportional PCM.

The semantic operation is modelled as o(t) = T(t)c(t) followed by the independent renderer operator. T(t) is not known: JocSpatialOperatorState therefore remains Unresolved, and require_resolved_operator() is a hard gate. There is no automatic conversion from decoded components to ExplicitSpatialScene. The scoped decoded-object binding resolves only the carrier-local association between decoded JOC row j and decoded OAMD dynamic ordinal j for reconstructed ADM; it does not resolve T(t), a speaker-rendering operator, or an authored source identity. There is no fixed RB-row/authored-object mapping and no implicit matrix or permutation. The readiness census remains a repository artifact and is available as docs/joc_reconstruction_readiness.json.

The explicitly activated JocSpatialBridge is a downstream spatial function with experimental maturity. It consumes a losslessly retained topology/ coordinate snapshot, projects into a caller-supplied public layout, applies the Q32 gain scheduler, and accumulates linearly into caller-owned buffers. It does not change profile validation, assign authored-object identity, or resolve SemanticBindingState for authored or renderer semantics; its raw warp-3 field is retained as opaque data and excluded from projection arithmetic. The supported ordinary domain and activation surface remain an experimental implementation boundary. The E-AC-3 JOC overview shows how this branch relates to reconstructed ADM export.

After Base and ReconstructionBasis contributions have been accumulated into the final semantic speaker planes, the shared renderer applies a causal, common FinalLinkedGain stage for the admitted 48-kHz 32-sample adapter blocks. It includes active LFE in the linked channel set, adds one block of speaker output history, and is reset with the stream/timeline lifecycle. The stage is not applied to the SOFA binaural path; Base downmix overload protection and pre-gain contribution linearity remain separate contracts.

Base full-band scene-composition gate¶

Decoded non-LFE Base/full-band channels are known JOC reconstruction inputs. That fact is separate from whether an independent Base contribution survives into the final JOC delivery scene. The experimental bridge may accumulate Base coordinates and ReconstructionBasis rows together, but that codec-basis composition is not proof of the final-scene equation while T(t) remains unresolved. In particular, strong energy in decoded Base C is not evidence that Base C should be exported a second time, and the original authored Bed is not used to explain or authorize that export. Base-plus-Objects must pass an explicit double-counting proof before ADM scene composition changes.

Explicit spatial renderer¶

The openjoc-render crate is a separate Layer-A/Layer-B foundation. It accepts only caller-supplied ExplicitSpatialSource blocks with an opaque source ID, mono PCM, explicit Cartesian position, and explicit linear gain. Its initial renderer maps the front horizontal hemisphere to FL, FR with a public equal-power law and mixes borrowed blocks into caller-owned floating-point buffers. It has no dependency on openjoc-scene, DecodedJocComponents, or ReconstructionBasis, so unresolved decoder rows cannot become authored spatial sources through an implicit conversion.

The initial StereoRenderer rejects rear-hemisphere and undefined horizontal directions, ignores elevation for stereo, performs no distance/room/occlusion processing, and does not clip by default. SpeakerLayout2d and LayoutRenderer2d add arbitrary validated horizontal layouts with deterministic adjacent-pair, checked 2x2 VBAP-style gains. The caller's speaker order is the public planar output order; unsupported angular gaps fail explicitly. The 2D renderer ignores elevation and has no LFE/bass-management path. The separate binaural renderer and experimental JOC spatial bridge do not change this 2D contract or provide automatic JOC semantic binding.

SpatialState2d, TrajectorySegment2d, and SourceTrajectory2d add an explicit, piecewise-linear automation contract. Segment endpoints are inclusive absolute sample indices; azimuth follows an explicit shortest/increasing/ decreasing path policy and source gain is interpolated linearly in the linear domain. StereoRenderer::render_trajectory_block and LayoutRenderer2d::render_trajectory_block evaluate that state per sample, so one block, irregular blocks, and one-sample blocks have the same result for the same absolute timeline. Trajectory blocks borrow PCM and caller-owned output planes, perform bounded preflight validation, and allocate neither per sample nor for the full timeline. The trajectory is directional only: radius, z, distance, Doppler, room effects, elevation, and HRTF are not rendered.

Speaker3d, SpeakerTriplet3d, and SpeakerLayout3d add an explicit three-dimensional topology contract. The caller supplies the public speaker order and every admissible VBAP triplet; LayoutRenderer3d never performs Delaunay, hull, coverage, or “best triplet” inference. Each declared triplet is solved as the public 3×3 system S g = p, with finite/non-singular checks, non-negative gain checks, and unit-energy normalization. Exact speaker hits are deterministic one-hot gains. If multiple declared triplets cover a direction, their complete public-order gain vectors must agree or rendering fails with an ambiguity error. Partial layouts fail explicitly for unsupported directions, and LFE/bass management remains outside this renderer contract. The 3D renderer accepts only explicit sources and caller-owned planar f64 outputs. SpatialState3d, TrajectorySegment3d, SourceTrajectory3d, and TrajectorySourceBlock3d add an additive sample-accurate dynamic path over that same immutable topology. Each segment uses shortest great-circle SLERP between canonical unit directions, a stable small-angle branch, linear gain interpolation, and explicit rejection of antipodal ambiguity. Callers supply intermediate keyframes for longer routes; no path inference is performed. LayoutRenderer3d::render_trajectory_block evaluates absolute sample indices and preserves static output equivalence, endpoint/keyframe continuity, and byte-identical block-partition invariance. It preflights every sample before clearing caller-owned planar f64 outputs and performs no per-sample heap allocation. Distance, Doppler, listener orientation, room effects, LFE, HRTF/binaural rendering, JOC, ObjectScene, and authored-object bridges remain outside the contract.

HrirPair, HrirEntry, and HrirBank provide a compact caller-supplied exact-direction HRIR contract, with optional explicit per-ear delay metadata retained for construction-time interpolation. StaticBinauralSource binds a fixed explicit source ID, canonical direction, linear gain, and HRIR entry; it does not infer authored-object identity. BinauralRenderer uses the fixed listener convention (+Y forward, +X right, +Z up), emits LEFT_EAR then RIGHT_EAR, and performs direct causal time-domain FIR convolution. It preserves leading HRIR delay, keeps bounded per-source history across caller-owned input blocks, and exposes explicit tail draining and reset semantics. openjoc-sofa resolves exact directions first and performs bounded delay-aligned spherical interpolation before registering static sources. PartitionedBinauralRenderer is an additive, caller-selected uniform overlap-add backend: its fixed power-of-two partition P uses a 2P FFT, reports one-partition scheduling latency, accepts exactly P-sample input partitions plus one explicit final partial operation, and drains exactly the largest registered M-1 causal tail. It precomputes HRIR spectra and keeps only bounded filter-length frequency/time state; no duration-proportional PCM history or adaptive backend selection is used. Direct FIR remains the numerical oracle, so cross-backend validation is numerical rather than a promise of bit identity.

The separate openjoc-sofa crate is a construction-time, read-only adapter from a deliberately narrow SimpleFreeFieldHRIR SOFA contract into HrirBank. It depends on openjoc-render; the renderer remains independent of file parsing, NetCDF/HDF5 libraries, and OS-specific APIs. The current portable reader accepts the project-tested NetCDF classic CDF-1 and NetCDF-4/HDF5 SOFA subsets, fixed listener pose, spherical degree/degree/metre source positions, exactly two receivers, common sample rate, and integer sample delays. Receiver geometry, not array order, determines left/right ear mapping. Exact lookup is preferred; non-exact requests use a deterministic local spherical segment/triangle with shared ear weights, while sparse/outside coverage fails closed. After construction no SOFA file handle is retained and neither renderer performs file I/O per audio block. The SOFA adapter parses CDF-1 or HDF5 while it builds the bank. Custom HRIRs are resampled to the renderer rate once at setup; banks already at that rate retain their original taps. Moving sources, SOFA writing, and JOC semantic bridging remain outside this boundary.

Experimental device-independent listener orientation¶

openjoc-api adds an explicit opt-in control path above the same virtual- speaker binaural renderer. A cloned ListenerOrientationPreparer owns shared, immutable HRTF/layout resources and resolves all non-LFE HRIRs away from the render call. It interprets a normalized (x, y, z, w) active listener-local- to-scene quaternion by inverse-rotating each fixed world-space speaker direction, then uses the orientation-specific bounded exact/interpolation resolver. This follows the public quaternion rotation identity and the existing SOFA spherical interpolation contract; the ordinary static resolver is unchanged.

The prepared update contains the complete ordered speaker kernel set, a resource/layout fingerprint, stream epoch, and monotonic sequence. Admission checks these fields atomically. An active shared 240-sample linear crossfade (5 ms at 48 kHz) preserves each source's causal input history; one latest target may wait during the fade. The dynamic path retains full HRIR taps up to its resource-derived limit, capped at 8,192 taps, and rejects the whole update if any direction is uncovered or any pair exceeds it. It does not shorten a filter, resample an already-48-kHz built-in bank, reduce the virtual speaker set, or alter the speaker projection or gain policy. Identity-orientation PCM is regression- checked against the static path.

An additional opt-in pull surface queues at most one 1,536-sample projected virtual-speaker AU and binauralizes at most 1–256 samples per receive. This lets a host submit pose targets between chunks; an active 240-sample crossfade can defer the next target. The compressed bridge retains its lazy positive-JOC admission. It does not read a sensor, timestamp poses, own an audio device, guarantee callback timing, or reduce the independent HRIR-preparation time. The default API, CLI, WASM and DirectShow/LAV output remain static. See the Rust API, C ABI, and known limitations.

Capture and streaming¶

Capture mode may retain metadata and diagnostic artifacts. Streaming mode uses bounded AU/frame state and incremental output finalization; it does not silently capture an unbounded ObjectScene or reconstruction-row vector. Malformed or truncated raw E-AC-3 and consumed ISO BMFF structures are rejected with bounded diagnostics; streaming output is staged and promoted only after a complete successful decode, so failure cannot publish a canonical partial.

Session lifecycle invariants¶

The private drained flag records that drain has started, or that the session has failed closed. It does not by itself mean that output has been exhausted. Pending output comprises three distinct stores: owned PCM frames, deferred virtual-speaker blocks, and the partially rendered current pull block. The deferred sample counter covers only the unrendered samples in the latter two stores. A pull-session FIR/LFE tail is separate from all three stores and is eligible for receiving only after drain starts.

State or operation	Result and preserved transition
Fresh, or successfully flushed/reset	Drain not started; no queued output; not drain-complete
Open session with any pending output	Push and drain return `OutputPending`; a requested discontinuity does not reset state
Ordinary receive	Pops one owned PCM frame; pull sessions use the separate bounded receive method
Bounded pull receive	Consumes deferred/current input first and decrements its remaining sample count; after drain starts it can consume the FIR/LFE tail
Drain with no pending output	Marks drain started, then emits reconstruction and binaural tails; an empty session reaches `EndOfStream`
Drain started, pending output remains	Push returns `AlreadyDrained`; another drain returns `OutputPending`; receiving continues
Drain started, only a pull tail remains	Another drain returns `FrameAvailable`; completion stays false until the tail is exhausted
Drain started, all output exhausted	Completion is true; repeated drain returns `EndOfStream`; push still returns `AlreadyDrained`
Failed closed	Clears all three output stores and the deferred sample count, marks drain started, and records the terminal error; completion is true even if internal FIR history remains
Successful flush/reset	Clears queued output before resetting decoder/render/timeline state, advances an orientation epoch, then clears the drain and terminal flags
Accepted discontinuity on an open, output-free session	Resets stream state before timestamp validation and decoding; pending, drained, and terminal guards take precedence

Guard priority is part of the compatibility contract: push checks terminal error, drain-started, pending output, empty/size checks, then discontinuity and PTS; drain checks terminal error, pending output, then drain-started; bounded pull checks terminal error before its mode requirement. Reset failure remains fail-closed. The original typed terminal error is retained privately while repeat calls preserve their historical public Render wrapper and full text. Queue clearing does not itself reset FIR history, timestamps, epochs, or the terminal flag; the calling lifecycle operation owns those ordered steps.

Scoped decoded-JOC/OAMD binding¶

The payload boundary has one explicit clean-room structural admission gate for the ordinary strict profile and the exact observed raw3-compatible profile: 15 decoded JOC objects, no OAMD bed, one Base LFE at total index 0, no ISF, 15 dynamic OAMD objects, and 16 total OAMD entries. Only after this gate passes does the canonical typed mapping produce joc_ordinal = dynamic_ordinal and oamd_total_index = joc_ordinal + 1. The +1 is centralized in DecodedJocBindingProfile; it is not an element-ID lookup, audio-content match, or PCM heuristic.

SemanticBindingState::ResolvedWithinCarrier means only that decoded JOC PCM and decoded OAMD metadata are paired inside this admitted carrier profile. It does not recover original authored ADM identity. Bed-bearing, ISF-bearing, alternative-LFE, count/order-mismatched, unknown-compatibility-deviation, and incomplete-Base-LFE cases remain unresolved; inactive transitions are not admitted by the dynamic ADM metadata exporter.

The dynamic ADM path reuses the existing absolute scene sample domain. It retains metadata events for the export plan but never duplicates programme PCM: each generated OpenJOC Reconstructed JOC Object NN track consumes its bound reconstruction row during the existing bounded streaming write, while its ADM blocks are derived from the corresponding OAMD event boundaries. Reset, discontinuity, flush, and stream reopen create a fresh decoder/scene epoch; stale metadata and PCM are not combined.

Error and evidence model¶

Malformed input, unsupported container shapes, strict profile violations and output failures are classified separately. A diagnostic or empirical result cannot upgrade semantic binding. Current claim boundaries are summarized in the capability matrix and known limitations.