Distributed brain simulation - node configuration and model mapping

[!NOTE] For the latest implementation status, please refer to Functional Implementation Status (Remaining Functionality).

Implementation notes (artifacts): See docs/implementation/ARTIFACT_MANIFESTS.md for the artifact_manifest.json output by the training script and recommended CLI flags.

Copyright: 2026 Moonlight Technologies Inc. All Rights Reserved.

Author: Masahiro Aoki

Last Updated: January 8, 2026

Purpose and use of this document

Purpose: To quickly reference the node configuration, model mapping, and hub configuration of the distributed brain.
Target audience: Implementation/operation members in charge of node deployment and model assignment.
First reading order: Overview → New architecture → Simulation type/each node configuration.
Related links: Execution script is examples/run_zenoh_distributed_brain.py, PFC/Zenoh/Executive details are implementation/PFC_ZENOH_EXECUTIVE.md.

2026-04-20 Added: Distributed ASR/Whisper integration execution settings

For video/audio analysis on distributed nodes, switch the ASR route using node environment variables.

VIDEO_ANALYSIS_ASR_BACKEND: asr_fallback (compatible default) or whisper_real
VIDEO_ANALYSIS_WHISPER_MODEL: example tiny, base
VIDEO_ANALYSIS_WHISPER_DEVICE: example cpu, cuda
VIDEO_ANALYSIS_ASR_PREPROCESS: Preprocessing switching (1 / 0)

In docker-compose.distributed.yml, the above environment variables can be set for each brain-node-*. If you want to enable Whisper, please specify VIDEO_ANALYSIS_ASR_BACKEND=whisper_real at startup.

EvoSpikeNet's distributed brain simulation employs a hierarchical architecture in which multiple nodes work together. Each node simulates a specific brain region and exchanges spike signals via the Zenoh communication protocol.

overview

EvoSpikeNet's distributed brain simulation employs a hierarchical architecture in which multiple nodes work together. Each node simulates a specific brain region and exchanges spike signals via the Zenoh communication protocol.

New architecture: Separation of Sensor Hub and Motor Hub

Updated on December 12, 2025: Separated classifications that manage motor cortex and sensor information and introduced a more efficient architecture.

Sensor Hub

Integrated management of all sensor inputs (visual, auditory, tactile)
Responsible for preprocessing and integration of sensor data
Provides sensor data integrated into PFC

Motor Hub

Integrated management of all motor outputs (trajectory control, cerebellar coordination, PWM control)
Convert commands from PFC into actual motion control
Manage coordination of multiple motor subsystems

Advantages

Parallel processing: Sensor inputs can be processed simultaneously
Specialization: Each hub is responsible for specialized functions
Extensibility: Easy to add new sensors/motion types
Efficiency: Separation of sensor and motion enables processing optimization

Data flow

Overview

EvoSpikeNet's distributed brain simulation employs a hierarchical architecture in which multiple nodes work together. Each node simulates a specific brain region and exchanges spike signals via the Zenoh communication protocol. Sensor Hub → PFC → Motor Hub ↓ ↓ ↓ Visual Compute Motor-Traj Auditory Lang-Main Motor-Cereb Speech Motor-PWM```

Simulation type

分散脳シミュレーションは以下の5つのタイプをサポートしています：

Language Focus (6プロセス) - 言語処理に特化
Image Focus (7プロセス) - 視覚処理に特化（Sensor Hub使用）
Audio Focus (10プロセス) - 聴覚・発声処理に特化（Sensor Hub使用）
Motor Focus (7プロセス) - 運動制御に特化（Motor Hub使用）
Full Brain (24プロセス) - 全機能統合（Sensor Hub + Motor Hub）

Node hierarchy configuration

1. Language Focus (10 processes)

言語処理を中心とした構成。埋め込みベクトル生成、タスク分解、RAG検索などの高度な言語機能を含みます。

Rank 0: PFC (Execution Control) ├─ Rank 1: Visual (visual input) ├─ Rank 2: Motor (motor output) ├─ Rank 3: Compute ├─ Rank 4: Lang-Main (Main language processing) ├─ Rank 5: Auditory (auditory input) ├─ Rank 6: Speech (voice output) ├─ Rank 7: Lang-Embed (Embedded vector generation) ├─ Rank 8: Lang-TAS (task decomposition) └─ Rank 9: Lang-RAG (RAG search)

使用モデル: - PFC: SimpleLIFNode - Lang-Main, Lang-Embed, Lang-TAS, Lang-RAG: SpikingEvoTextLM - Visual: SpikingEvoVisionEncoder - Auditory, Speech: SpikingEvoAudioEncoder - Motor: SimpleLIFNode (フォールバック) - Compute: SpikingEvoTextLM

2. Image Focus (7 processes)

視覚処理階層を重点的にカバー。Sensor Hubの下にVisualモジュールを配置し、エッジ検出、形状認識、物体認識の3段階処理。

Rank 0: PFC (Execution Control) ├─ Rank 1: Compute └─ Rank 2: Sensor-Hub (Sensor integrated management) └─ Rank 3: Visual (main visual processing) ├─ Rank 4: Vis-Edge (edge detection) ├─ Rank 5: Vis-Shape (shape recognition) └─ Rank 6: Vis-Object (object recognition)

使用モデル: - Visual, Vis-Edge, Vis-Shape, Vis-Object: SpikingEvoVisionEncoder - Sensor-Hub: SimpleLIFNode (統合管理) - PFC, Compute: SpikingEvoTextLM

3. Audio Focus (10 processes)

聴覚・発声処理階層を重点的にカバー。Sensor Hubの下にAuditoryモジュールを配置し、MFCC特徴抽出、音素認識、意味理解の3段階処理。Speechモジュールは独立して配置。

Rank 0: PFC (Execution Control) ├─ Rank 1: Compute ├─ Rank 2: Sensor-Hub (Sensor integrated management) │ └─ Rank 3: Auditory (main auditory processing) │ ├─ Rank 4: Aud-MFCC (MFCC feature extraction) │ ├─ Rank 5: Aud-Phoneme (phoneme recognition) │ └─ Rank 6: Aud-Semantic (semantic understanding) └─ Rank 7: Speech (main speech processing) ├─ Rank 8: Speech-Phoneme (phoneme generation) └─ Rank 9: Speech-Wave (waveform synthesis)

使用モデル: - Auditory, Aud-MFCC, Aud-Phoneme, Aud-Semantic: SpikingEvoAudioEncoder - Speech, Speech-Phoneme, Speech-Wave: SpikingEvoAudioEncoder - Sensor-Hub: SimpleLIFNode (統合管理) - PFC, Compute: SpikingEvoTextLM

4. Motor Focus (7 processes)

運動制御階層を重点的にカバー。Motor Hubの下にMotorモジュールを配置し、軌道制御、小脳協調、PWM制御の3段階処理。

Rank 0: PFC (Execution Control) ├─ Rank 1: Compute └─ Rank 2: Motor-Hub (Motor integrated management) └─ Rank 3: Motor (main motion control) ├─ Rank 4: Motor-Traj (orbit control) ├─ Rank 5: Motor-Cereb (cerebellar coordination) └─ Rank 6: Motor-PWM (PWM control)

使用モデル: - Motor, Motor-Traj, Motor-Cereb, Motor-PWM: SimpleLIFNode (運動制御用) - Motor-Hub: SimpleLIFNode (統合管理) - PFC, Compute: SpikingEvoTextLM

5. Full Brain (24 processes)

全ての階層処理を統合した完全な脳シミュレーション。Sensor HubとMotor Hubを使用した新しいアーキテクチャを採用。

Rank 0: PFC (Execution Control) ├─ Rank 1: Sensor-Hub (Sensor integrated management) │ ├─ Rank 2: Visual (visual main) │ └─ Rank 3: Auditory (mainly auditory) ├─ Rank 4: Motor-Hub (Motor integrated management) │ └─ Rank 5: Motor (mainly exercise) ├─ Rank 6: Compute ├─ Rank 7: Lang-Main (Language Main) ├─ Rank 8: Speech (voice main) ├─ Rank 9: Vis-Edge (edge detection) ├─ Rank 10: Vis-Shape (shape recognition) ├─ Rank 11: Vis-Object (object recognition) ├─ Rank 12: Motor-Traj (orbit control) ├─ Rank 13: Motor-Cereb (cerebellar coordination) ├─ Rank 14: Motor-PWM (PWM control) ├─ Rank 15: Aud-MFCC (MFCC features) ├─ Rank 16: Aud-Phoneme (phoneme recognition) ├─ Rank 17: Aud-Semantic (semantic understanding) ├─ Rank 18: Speech-Phoneme (phoneme generation) ├─ Rank 19: Speech-Wave (waveform synthesis) ├─ Rank 20: Lang-Embed ├─ Rank 21: Lang-TAS (task decomposition) └─ Rank 22: Extra-1 (spare)

model mapping

Node type normalization

サブノードは_get_base_module_type()メソッドによってベースタイプに正規化され、対応するSpikingモデルが割り当てられます。

# Language subnode → lang-main
"lang-embed", "lang-rag", "lang-tas", "compute"
→ SpikingEvoTextLM

# visual system subnode → visual
"vis-edge", "vis-shape", "vis-object"
→ SpikingEvoVisionEncoder

# Auditory subnode → audio
"aud-mfcc", "aud-phoneme", "aud-semantic"
→ SpikingEvoAudioEncoder

# Vocal subnode → audio
"speech-phoneme", "speech-wave"
→ SpikingEvoAudioEncoder

# Motor system subnode → motor
"motor-traj", "motor-cereb", "motor-pwm"
→ SimpleLIFNode (Fallback, plan to use MotorModule in the future)```

### Model details

#### SpikingEvoTextLM (language model)
- **vocab_size**: 30522 (BERT互換)
- **d_model**: 128
- **n_heads**: 4
- **num_transformer_blocks**: 2
- **time_steps**: 10
- **Model Type**: "text"
- **Category**: "LangText"

**使用ノード:**
- Lang-Main, Lang-Embed, Lang-TAS, Lang-RAG, Compute

#### SpikingEvoVisionEncoder (visual model)
- **input_channels**: 1 (グレースケール)
- **output_dim**: 128
- **image_size**: (28, 28) (MNIST互換)
- **time_steps**: 10
- **アーキテクチャ**: 2層CNN + LIF
- **Model Type**: "vision"
- **Category**: "Vision"

**使用ノード:**
- Visual, Vis-Edge, Vis-Shape, Vis-Object

#### SpikingEvoAudioEncoder (audio model)
- **input_features**: 13 (MFCC係数)
- **output_neurons**: 128
- **time_steps**: 10
- **Model Type**: "audio"
- **Category**: "Audio"

**使用ノード:**
- Auditory, Aud-MFCC, Aud-Phoneme, Aud-Semantic
- Speech, Speech-Phoneme, Speech-Wave

#### SpikingEvoMultiModalLM (Multimodal model)
- **text_vocab_size**: 30522
- **vision_input_channels**: 1
- **audio_input_features**: 13
- **d_model**: 128
- **n_heads**: 4
- **time_steps**: 10
- **Model Type**: "multimodal"
- **Category**: "MultiModal"

**使用ノード:**
- 統合タスク用 (将来的拡張)

#### SimpleLIFNode (General Purpose Spiking Neuron)
- **size**: 128
- **基本的なLIF (Leaky Integrate-and-Fire) ニューロンレイヤー**
- **Model Type**: N/A (汎用)
- **Category**: N/A

**使用ノード:**
- PFC
- Motor系 (Motor, Motor-Traj, Motor-Cereb, Motor-PWM)

---

## How to start a node

### Basic commands

```bash
python examples/run_zenoh_distributed_brain.py \
  --node-id <node-id> \
  --module-type <module-type> \
  [--model-artifact-id <artifact-id>]

Startup example

1. PFC node startup

python examples/run_zenoh_distributed_brain.py \
  --node-id pfc-0 \
  --module-type pfc

2. Start language main node (using trained model)

python examples/run_zenoh_distributed_brain.py \
  --node-id lang-main-4 \
  --module-type lang-main \
  --model-artifact-id 236d1796-547b-4b74-9f67-02e506c1a706

3. Visual subnode launch

python examples/run_zenoh_distributed_brain.py \
  --node-id vis-object-9 \
  --module-type vis-object

4. Auditory subnode activation

python examples/run_zenoh_distributed_brain.py \
  --node-id aud-mfcc-13 \
  --module-type aud-mfcc

Model loading operation

1. When specifying an explicit artifact ID

1. Search for artifact specified with --model-artifact-id 2. Resolve session ID 3. Download config.json and weights.pth from API 4. Initialize the model and load weights 5. For language models, also load tokenizer

2. When artifact ID is not specified

1. Get the latest model session from API 2. Download latest session artifacts 3. Initialize the model and load weights

3. When API connection fails

1. Initialize the model with default parameters 2. Operates with random weights (no learning) 3. For language models, use default BERT tokenizer

Automatic node generation (Auto Node Mapper)

目的：公開コネクトームデータから分散脳ノード構成（rank→node）を自動生成し、structural_mask NPZ と node_manifest.yaml を出力する。

主要要件：

入力：connectome JSON/NPZ（neurons[], synapses[], metadata）
出力：per-rank_id NPZ (row_indices,col_indices,weights,delays,ei_mask) と node_manifest.yaml（node_id, ranks, mask_url, version_uuid, etag）
削減手法：Policy F（F-1 層別サンプリング / F-2 スペクトル削減 / F-3 クラスタ代表）を実装、並列評価して最良案を選択
検証：E/I 差分 <5%、degrees-distribution KS p-value >0.05、メモリ制限内（rank_profiles）
運用：HCP 等の契約データは manual approval 必須、CAVE はバックオフ + NPZ キャッシュ

推定フロー：

プリプロセス（metadata 抽出、cell_type/layer/ROI 集計）
削減候補生成（F-1/F-2/F-3）
RANK 割当（ルールベース + スコア最適化）
NPZ 出力 + node_manifest.yaml 生成
デプロイ：connectome/metadata/{node_id} へ manifest を publish、ノードは apply_mask API を呼び出す

推奨実装成果物：

scripts/auto_node_mapper.py（PoC：C. elegans → 29-node manifest）
evospikenet/connectome/node_mapping.py（マッピングルール・Policy F 実装）
config/connectome_config.yaml の rank_profiles 拡張
tests/test_auto_node_mapper.py, tests/test_connectome_integration_mapper.py

この設計は docs-dev/connectome_schema.md と互換であり、evospikenet/connectome/connectome_loader.py の apply/validate/rollback ワークフローと連携します。

FPGA safety controller integration

全てのノードはFPGAセーフティコントローラーと統合され、2秒ごとにハートビートを送信します。

Heartbeat transmission timing

セッション解決中
モデルロード前
config.json ダウンロード後
モデル初期化後
weights.pth ダウンロード後
重みロード後

これにより、長時間のモデルロード中にもウォッチドッグタイムアウトが発生しません。

node validation

Test script

全ノードタイプのモデル初期化を検証：

python tests/test_node_types.py

検証項目: - Lang系: lang-main, lang-embed, lang-rag, lang-tas, compute - Visual系: visual, vision, vis-main - Audio系: audio, auditory, aud-main, speech - Multimodal: multimodal

Verification results

✅ lang-main: SpikingEvoTextLM
✅ lang-embed: SpikingEvoTextLM
✅ lang-rag: SpikingEvoTextLM
✅ lang-tas: SpikingEvoTextLM
✅ compute: SpikingEvoTextLM
✅ visual: SpikingEvoVisionEncoder
✅ vision: SpikingEvoVisionEncoder
✅ vis-main: SpikingEvoVisionEncoder
✅ audio: SpikingEvoAudioEncoder
✅ auditory: SpikingEvoAudioEncoder
✅ aud-main: SpikingEvoAudioEncoder
✅ speech: SpikingEvoAudioEncoder
✅ multimodal: SpikingEvoMultiModalLM

Passed: 13/13

troubleshooting

1. FPGA Watchdog Timeout

症状: 50秒以上のモデルロード中にタイムアウト

解決策: - AutoModelSelector.get_model()にsafety_controllerを渡す - モデルロード中の6箇所でハートビートを送信

2. Model initialization error (TypeError)

症状: name 'd_model' is not defined

解決策: - _load_from_api()でTypeErrorをキャッチ - デフォルトパラメータでフィルタリング - フィルタ済みパラメータで再初期化

3. API connection error

症状: Failed to resolve 'api'

解決策: - Docker環境: apiホスト名を使用 - ローカル環境: localhostを使用 - 環境変数API_HOSTで制御可能

4. Tokenizer load failure

症状: 言語モデルでトークナイザーが見つからない

解決策: - _load_tokenizer_from_session()失敗時に自動フォールバック - デフォルトBERTトークナイザー(bert-base-uncased)を使用

Future development

Priority: High

MotorModule統合
MotorModuleをAutoModelSelectorに統合
Motor系サブノードで専用モデルを使用
モデル保存・読込の強化
トークナイザーの自動保存
config.jsonの完全性チェック

Priority: Medium

階層的モデル選択
サブノードごとに異なるモデルを使用可能に
視覚階層: Edge→Shape→Objectで異なるモデル
動的ノード発見
ノード発見サービスの強化
動的なトポロジー変更対応

Priority: Low

分散学習サポート
複数ノード間での協調学習
勾配同期機構

Structure of config.json

各モデルタイプで保存されるconfig.jsonの詳細な構造です。

SpikingEvoTextLM (language model)

{
    "vocab_size": 30522,
    "d_model": 128,
    "n_heads": 4,
    "num_transformer_blocks": 2,
    "time_steps": 16,
    "neuron_type": "lif"
}

パラメータ説明: - vocab_size: 語彙サイズ（BERT互換は30522） - d_model: Transformerの隠れ層次元 - n_heads: マルチヘッドアテンションのヘッド数 - num_transformer_blocks: Transformerブロックの層数 - time_steps: スパイキングニューロンの時間ステップ数 - neuron_type: ニューロンタイプ（"lif" または "alif"）

保存スクリプト:

python examples/train_spiking_evospikenet_lm.py \
  --run-name lang_20251209_132845 \
  --upload-to-db

アーティファクト: - config.json - モデル設定 - spiking_lm.pth - モデル重み - spiking_lm_tokenizer.zip - BERTトークナイザー

SpikingEvoVisionEncoder (visual model)

{
    "input_channels": 1,
    "output_dim": 128,
    "time_steps": 20,
    "num_classes": 10,
    "dataset": "mnist",
    "image_size": [28, 28]
}

パラメータ説明: - input_channels: 入力画像のチャンネル数（1=グレースケール、3=RGB） - output_dim: 出力特徴ベクトルの次元 - time_steps: スパイキングニューロンの時間ステップ数 - num_classes: 分類クラス数（分類タスクの場合） - dataset: 学習に使用したデータセット名 - image_size: 入力画像サイズ [高さ, 幅]

保存スクリプト:

python examples/train_vision_encoder.py \
  --run-name vision_20251208_164304 \
  --dataset mnist \
  --upload-to-db

アーティファクト: - config.json - モデル設定 - vision_encoder.pth - エンコーダー重み - vision_classifier.pth - 分類器重み（オプション）

SpikingEvoAudioEncoder (audio model)

{
    "input_features": 13,
    "output_neurons": 128,
    "time_steps": 10
}

パラメータ説明: - input_features: 入力特徴量の次元（通常はMFCC係数数） - output_neurons: 出力ニューロン数 - time_steps: スパイキングニューロンの時間ステップ数

保存スクリプト:

python examples/train_audio_modal_lm.py \
  --run-name audio_20251208_164638 \
  --upload-to-db

アーティファクト: - config.json - モデル設定 - audio_encoder.pth - モデル重み

Usage flow of config.json

1. When saving the model

# inside the learning script
config_to_save = {
    'vocab_size': tokenizer.vocab_size,
    'd_model': args.d_model,
    'n_heads': args.n_heads,
    'num_transformer_blocks': args.num_blocks,
    'time_steps': args.time_steps,
    'neuron_type': args.neuron_type,
}

with open('config.json', 'w') as f:
    json.dump(config_to_save, f, indent=4)

```python

Download config.json

with open(config_path, 'r') as f: config = json.load(f)

add device

config['device'] = 'cpu'

model initialization

try: model = SpikingEvoTextLM(config) except TypeError as e: # Filter when parameters do not match default_params = _get_default_params(task_type) filtered_config = {k: v for k, v in config.items() if k in default_params or k == 'device'} model = SpikingEvoTextLM(filtered_config)```

3. Error handling

config.jsonに余分なパラメータが含まれている場合：

```python

Example: config.json has parameters such as "aeg_module" that the model does not accept

{ "vocab_size": 30522, "d_model": 128, "aeg_module": {...} # SpikingEvoTextLM is not accepted }

TypeError occurs → filtering with default parameters

filtered_config = { "vocab_size": 30522, "d_model": 128, "time_steps": 10, # complete with default value "device": "cpu" } ````

Reference materials

Last updated: December 10, 2025

Version: v1.1

Distributed brain simulation - node configuration and model mapping

Copyright: 2026 Moonlight Technologies Inc. All Rights Reserved.

Author: Masahiro Aoki

Purpose and use of this document

2026-04-20 Added: Distributed ASR/Whisper integration execution settings

overview

New architecture: Separation of Sensor Hub and Motor Hub

Sensor Hub

Motor Hub

Advantages

Data flow

Overview

Simulation type

Node hierarchy configuration

1. Language Focus (10 processes)

2. Image Focus (7 processes)

3. Audio Focus (10 processes)

4. Motor Focus (7 processes)

5. Full Brain (24 processes)

model mapping

Node type normalization

Startup example

1. PFC node startup

2. Start language main node (using trained model)

3. Visual subnode launch

4. Auditory subnode activation

Model loading operation

1. When specifying an explicit artifact ID

2. When artifact ID is not specified

3. When API connection fails

Automatic node generation (Auto Node Mapper)

FPGA safety controller integration

Heartbeat transmission timing

node validation

Test script

Verification results

troubleshooting

1. FPGA Watchdog Timeout

2. Model initialization error (TypeError)

3. API connection error

4. Tokenizer load failure

Future development

Priority: High

Priority: Medium

Priority: Low

Structure of config.json

SpikingEvoTextLM (language model)

SpikingEvoVisionEncoder (visual model)

SpikingEvoAudioEncoder (audio model)

Usage flow of config.json

1. When saving the model

2. When loading the model (AutoModelSelector)

Download config.json

add device

model initialization

3. Error handling

Example: config.json has parameters such as "aeg_module" that the model does not accept

TypeError occurs → filtering with default parameters

Reference materials