Skip to content

LanguageModel Architecture Compatibility and Artifact Contract

  • Status: Implemented adoption contract (production rollout and legacy migration remain separate work)
  • Last updated: 2026-08-03
  • Scope: EvoSpikeNet-Core and applications, such as Mineral Exploration, that train, select, and infer LanguageModels through the SDK

Purpose

Allow applications to create LanguageModels with different d_model values from a UI without confusing models or loading the wrong architecture, tokenizer, or checkpoint for training, continued training, or inference.

d_model alone does not identify a model. Compatibility is determined by an immutable model specification that includes architecture, layer configuration, SNN parameters, tokenizer, vocabulary, and checkpoint artifacts.

Scope and non-goals

  • In scope: artifact registration, continued training, and inference selection for dense SpikingEvoTextLM, SparseEventMemoryLM, and future MemoriedSpikeNetLM models.
  • In scope: the contract between a LanguageMode UI, API, SDK adapter, and Core training artifacts.
  • Out of scope: weight conversion across different d_model values, changing layer counts during continuation, or partial vocabulary-matrix transfer. These require an explicit model-surgery or migration design.

SparseEventMemoryLM is a separate architecture from dense SpikingEvoTextLM; their checkpoints cannot be loaded into each other. See the Sparse Event-Memory LM specification.

Verified current state (2026-08-03)

Core

The Core training script saves weights, config.json, and a tokenizer in an artifact bundle. When continued training receives a usable base_model_path and a colocated config.json, Core reads the saved configuration, reconstructs the model, and performs a strict load_state_dict().

Therefore, models with different d_model values can coexist when each has an independent, complete artifact bundle and is loaded with its own configuration. A strict-load shape mismatch is a failure signal, not compatibility.

ChronoSpikeAttention requires:

\[d_{model} \bmod n_{heads} = 0\]

The approximate parameter count grows as:

\[P \approx 2Vd + 12Bd^2\]

where \(V\) is vocabulary size, \(d\) is d_model, and \(B\) is the number of transformer blocks. The block term is quadratic in \(d_{model}\); doubling d_model materially increases parameter, gradient, and optimizer-state requirements.

Mineral Exploration

LanguageMode UI/API registers and lists d_model, n_heads, num_blocks, time_steps, neuron_type, and architecture as an immutable model_spec. Core saves a checkpoint, config.json, tokenizer, and SHA-256-backed artifact_manifest.json; the SDK returns the resulting durable URIs.

The model registry record at storage/models/language/{model_id}/model.json persists the model_spec, manifest, parent model, and training job ID. Existing records without this evidence are shown as legacy_unverified and cannot be used for continued training. A temporary session://... URI remains display compatibility data only.

Continued-training and inference preflight validate checkpoint/config/tokenizer presence, SHA-256 digests, config/manifest model_spec equality, and the selected architecture. Docker-restart deployment and administrator migration of legacy artifacts remain separate work.

Required model_spec contract

Every registered model must persist the following immutable model_spec. Altering a value creates a new model through a new training run and a new model_id.

{
  "model_spec_version": 1,
  "model_family": "spiking_text_lm",
  "architecture": "dense_chronospike",
  "d_model": 128,
  "n_heads": 4,
  "num_transformer_blocks": 2,
  "time_steps": 20,
  "neuron_type": "EvoLIF",
  "vocab_size": 32000,
  "tokenizer": {
    "name": "cl-tohoku/bert-base-japanese-v3",
    "revision": "optional-immutable-revision",
    "files_sha256": "optional-tokenizer-bundle-digest"
  },
  "core": {
    "artifact_format_version": 1,
    "core_version": "recorded-build-or-commit"
  }
}

The registry record must also contain at least:

{
  "checkpoint_uri": "durable-uri-to-model.pt",
  "config_uri": "durable-uri-to-config.json",
  "tokenizer_uri": "durable-uri-to-tokenizer-bundle",
  "artifact_sha256": "bundle-or-manifest-digest",
  "parent_model_id": null,
  "training_job_id": "lmtrain-..."
}
  • model_uri can remain as a backward-compatible display alias, but must not substitute for checkpoint_uri.
  • config.json must agree with model_spec; registration verifies both.
  • URIs must resolve after a process restart. A session identifier alone does not meet this requirement.
  • Digests detect swapped or mismatched weights, configuration, and tokenizer bundles.

UI and API adoption contract

New training

A new LanguageMode training request explicitly contains the user-selectable fields from model_spec. At minimum, add:

  • architecture
  • d_model
  • n_heads
  • num_blocks
  • time_steps
  • neuron_type
  • seq_len
  • learning_rate

Before sending work to Core, the API validates:

  1. Every numeric value is positive and within service limits.
  2. \(d_{model} \bmod n_{heads} = 0\).
  3. Only parameters applicable to the selected architecture are present.
  4. The device-memory estimate and job limits are satisfied. For dense SpikingEvoTextLM, Mineral GPU capacity validation checks the requested configuration against free VRAM on the GPU currently visible to Core.
  5. Vocabulary and tokenizer information can be resolved.

The UI provides approved Small / Base / Large presets rather than accepting arbitrary large values by default. If detailed values are permitted, it presents the same constraints before submission.

Capacity validation does not automatically change batch_size, seq_len, learning rate, or any other model configuration. The UI displays insufficient capacity and disables launch; the launch API returns HTTP 422 without changing the configuration. Core/SDK execute a valid submitted configuration unchanged. See the LanguageModel GPU Capacity Validation Policy for GPU selection, the estimate, H100 portability, and runtime-OOM limitations.

Continued training

Continued training is not architecture migration. Once a base model is selected, the UI displays its model_spec and tokenizer read-only, and resolves checkpoint_uri to a Core-readable base_model_path.

The following must match exactly:

Category Required matching fields
Structure model_family, architecture, d_model, n_heads, num_transformer_blocks, time_steps, neuron_type
Input/output vocab_size, tokenizer name/revision/digest
Artifact checkpoint, configuration, and tokenizer bundle are all reachable and their digests match

A mismatch, missing artifact, or legacy model with unknown specification is rejected as an HTTP 422 compatibility error before Core attempts to load weights. Use training_type: new to train a new architecture.

Inference

Inference builds only from the selected model_id and its registered artifact manifest. It must not reuse current form values, environment defaults, or another model's d_model.

Before inference, the API verifies:

  1. model_spec, checkpoint, config.json, and tokenizer bundle exist.
  2. config.json agrees with model_spec.
  3. Artifact digests match.
  4. Model quality permits generation.
  5. The SDK/inference server can load the requested artifact URI.

On failure, return an explicit model, artifact, or compatibility error. Do not return garbled output or silently infer with a different model.

Compatibility matrix

Operation Condition Result
New training with a different d_model Register an independent complete artifact bundle and model_spec Allowed
Separate inference for model A/B Load the selected model's own configuration, tokenizer, and weights Allowed
Continued training with identical specification Structure, tokenizer, and artifact validation all succeed Allowed
Continued training with different d_model State-dict tensor shapes differ Rejected
Continued training with equal d_model but different blocks/heads/time steps/neuron Runtime semantics or weight shapes differ Rejected
Continued training with equal structure but different vocabulary/tokenizer Embedding/output rows or token semantics differ Rejected
Dense/sparse cross-load Checkpoint format and model implementation differ Rejected
Continued training from an unknown legacy specification Reproducibility information is incomplete Rejected until an administrator migrates the manifest

Migration and backward compatibility

Existing model.json records without model_spec are shown as legacy_unverified.

  • Permit inference only when the artifact manifest can be reconstructed and verified.
  • Prohibit continued training until an administrator records checkpoint, configuration, tokenizer bundle, and a final model_spec.
  • A value guessed from logs is not evidence; do not use guessed d_model or tokenizer values for formal compatibility decisions.
  • Do not immediately remove the legacy model_uri API field. Treat it as a read-only compatibility field until migration to checkpoint_uri is complete.

Acceptance criteria

Implementation is complete only after automated tests prove at least the following:

  1. Register d_model=8 and d_model=16 artifacts and distinguish their specifications in the list response.
  2. Inference for each model selects its own configuration, tokenizer, and checkpoint without cross-contamination.
  3. Continued-training requests with mismatched d_model, head count, block count, tokenizer, or architecture return an understandable 422 before job creation.
  4. Continued training with an identical specification actually loads the checkpoint and records parent/child lineage.
  5. Missing checkpoint/configuration/tokenizer or digest mismatches reject inference and continued training.
  6. The UI model list and selector show architecture and primary dimensions (d_model, heads, blocks, tokenizer).
  7. Introducing this contract does not mutate the request or artifacts of a currently running training job.