DIRU：受树突启发的循环单元，用于学习混沌动力学

1 Introduction

Recurrent neural networks (RNNs) have been widely employed for sequential data modeling. Traditional architectures such as Long Short-Term Memory (LSTM) networks [1] capture temporal dependencies using gated units, which are learned mechanisms that regulate how much information from previous time steps is retained, updated, or discarded as the sequence is processed. This allows the network to maintain a memory of past inputs over many time steps. However, modeling chaotic and highly nonlinear systems remains challenging due to single-compartment processing and vanishing/exploding gradient issues, despite useful extensions having been proposed [2, 3].

In contrast, biological neurons utilize elaborate dendritic trees with multiple nonlinear compartments to perform hierarchical computations within a single neuron [4]. These dendritic branches are not merely passive cables, but active processing units capable of local nonlinear integration, coincidence detection, and input-specific filtering [5]. This compartmentalized architecture enables neurons to implement computationally powerful operations, such as direction selectivity, logical operations, and gain modulation at the subcellular level [6, 7].

In this perspective, Brenner and colleagues proposed a "tractable" framework for dendritic RNN [8]. In their model, dendrites are structured but fundamentally passive nonlinear subunits designed for mathematically analyzable system reconstruction and interpretability, prioritizing tractability, stability, and identifiability. Other studies, more focused on biophysical realism [9], went even further by demonstrating that the complex input–output transformation of a single cortical neuron can be accurately approximated by a deep artificial neural network, suggesting that dendritic trees effectively implement multilayer nonlinear computation at the cellular level, although this was not specific for time series or chaotic systems.

Inspired by these principles, we propose the Dendrite-Inspired Recurrent Unit (DIRU), which integrates multiple computational compartments within each recurrent cell. Unlike conventional RNN units that function as monolithic integrators, DIRU’s compartmentalized design allows for parallel nonlinear processing, hierarchical feature extraction, and improved gradient flow during training, by mirroring the functional advantages of biological dendritic computation. In contrast with the Brenner’s tractable framework [8], DIRUs model dendrites are considered as active computational subunits rather than passive signal integrators. Each dendritic branch applies a learnable gating mechanism to its inputs, allowing synaptic signals to be selectively amplified or attenuated before contributing to the unit output. In addition, a global modulatory signal adjusts the overall gain of the neuron through a multiplicative scaling term, enabling the network to adapt its sensitivity to inputs depending on the global context or hidden state. The dendritic branches further implement nonlinear transformations that are combined hierarchically before reaching the somatic output, effectively introducing multi-stage feature processing within a single recurrent unit. This structure is inspired by models of dendritic computation showing that individual neurons can approximate multilayer nonlinear processing [4, 10]. Finally, the recurrent update incorporates residual connections so that previous hidden states are additively combined with the current update. Residual recurrence improves gradient propagation and stabilizes the learning of long-range temporal dependencies [11]. Overall, this design transforms the recurrent unit into a structured nonlinear dynamical system capable of adaptive gating, context-dependent gain modulation, and hierarchical feature processing. This transforms the architecture from a standard recurrent unit, which performs a single global transformation of inputs and hidden states, into an adaptive dynamical system with multiple interacting compartments, capable of context-dependent computation, dynamic routing, regime switching, and richer nonlinear representations, trading simplicity for expressive power, biological realism, and learning capacity.

2 Background

The use of neural networks to model and predict the behavior of nonlinear dynamical systems has become an active area of research, spanning data-driven approaches for system identification, state prediction, and control in settings where explicit analytical models are difficult to derive [12]. Nevertheless, chaotic dynamical systems are fundamentally challenging for recurrent learning architectures due to their intrinsic properties of sensitivity to initial conditions, especially with non-periodic attractor structures. In such systems, small approximation errors are rapidly amplified over time, leading to trajectory divergence, attractor drift, and long-horizon prediction instability. Classical recurrent architectures, including LSTMs, encode dynamics in a single homogeneous state space, limiting their ability to capture multi-attractor landscapes, regime transitions, and hierarchical nonlinear interactions [13]. Consequently, chaotic dynamics are often approximated by smoothed surrogates rather than faithful internal models, reducing stability [14].

We propose to address those limitations, taking inspiration from the detailed computation of dendrites in biological neurons. Neurons are not single integrative points but complex nonlinear processors composed of dendritic subunits capable of local integration, coincidence detection, and hierarchical nonlinear transformation [4], as summarized in the simplistic drawing in Fig. 1. These compartmentalized structures enable functionally deep computation within individual neurons, allowing biological systems to perform parallel processing and context-sensitive modulation at the cellular level. Rather than serving as metaphorical inspiration, dendritic architectures represent a powerful structural inductive bias for learning complex dynamical systems [5]. Embedding such compartmentalization into artificial recurrent architectures introduces internal computational hierarchies that enable multiscale representation, which is particularly beneficial in chaotic systems with strong nonlinear interactions, extending the representational capacity beyond that of monolithic recurrent units [15].

These architectural principles are particularly relevant for biological time series, where dynamics are inherently nonlinear, multiscale, and regime-dependent. Very often, these points of view are studied mostly from cognitive neuroscience, predominantly on transitions between functional states [16]. Moreover, nonlinear and dynamical systems analyzes have been applied in clinical contexts to characterize pathological alterations in brain dynamics. For example, nonlinear electroencephalography (EEG) analyses have revealed altered complexity and dynamical properties in schizophrenia, including changes in measures such as correlation dimension and Lyapunov exponents relative to healthy controls [17, 18]. Other researchers have studied multistability in large-scale models of brain activity. For example, Golos and colleagues, even without focusing on a specific clinical disease, explored how attractor landscapes and multistable dynamics can emerge from connectome-based neural mass models [19].

Dendritic computation has inspired prior works, but many of these models treat dendrites as passive nonlinear subunits, designed primarily for system reconstruction rather than adaptive learning. In contrast, active dendrites perform local, nonlinear computations on incoming signals before contributing to the neuron’s output, enabling compartmentalized processing that goes beyond the monolithic integration performed by standard LSTM units [4, 10]. Biophysically detailed models [9] capture dendritic complexity but are not designed for sequence learning tasks, and are computationally expensive. Given these observations, what is missing is an architecture that: (a) embeds active dendritic computation into recurrent units through learnable gating, allowing each compartment to amplify or suppress inputs selectively; (b) preserves gradient flow through structured compartmentalization, where parallel dendritic compartments process inputs nonlinearly and contribute to a shared recurrent state, facilitating stable learning across long sequences; and (c) scales to real-world biological time series by enabling end-to-end optimization without handcrafted features. In practice, our goal is to fill this gap by introducing DIRUs, which combine learnable gating, parallel compartmental processing, and implicit coordination through the shared hidden state, effectively bridging neuroscience-inspired dendritic computation with modern deep sequence learning architectures such as RNNs and LSTMs.

Neural signals such as EEG exhibit nonstationarity, cross-frequency coupling, metastable states, and rapid transitions between dynamical regimes, reflecting the dynamics of the neural population underlying [20]. Traditional recurrent models often struggle to disentangle these interacting temporal scales and nonlinear interactions within a single latent state representation. Here, the term single latent state representation refers to the hidden state \(h_t\) of a standard RNN or LSTM, which compresses all past information into a single vector at each time step. This can limit the network’s ability to disentangle interacting temporal scales and nonlinear interactions. Multi-compartment recurrent architectures offer a natural framework for modeling such signals by enabling parallel processing of distinct temporal and spectral components, hierarchical integration across scales, and context-dependent modulation of internal dynamics. This positions dendrite-inspired recurrent units as a biologically grounded and computationally suitable architecture for modeling complex neural time series, including EEG, where learning meaningful dynamical structure is essential for both prediction and interpretability.

This work focuses on a specific but fundamental question: can dendritic-inspired compartmentalization improve recurrent network performance in chaotic dynamics, such as the Lorenz system, where hierarchical structure may better capture nonlinear interactions and stabilize long-term predictions? Rather than attempting to model full brain-state evolution, we demonstrate that even simple compartmentalized architectures provide measurable benefits for both synthetic chaotic systems and real neural time series. In the following sections, we describe the architecture in detail, and in the experiments, we compare the proposed method with the widely accepted LSTM and the method proposed by Brenner and colleagues, which is called, for the rest of the paper, "Tractable" RNN.

Epilepsy is among the most common serious neurological disorders, characterized by a predisposition to recurrent, unprovoked seizures arising from abnormal neuronal discharges in the brain [21]. Unlike epilepsy later in life, neonatal seizures are typically acute symptomatic events secondary to an underlying insult, such as hypoxic-ischemic encephalopathy, intracranial hemorrhage, infection, or metabolic derangement, and their timely identification matters clinically, since unrecognized or prolonged seizures are linked to worse neurodevelopmental outcomes [22]. This combination of clinical urgency and difficulty of visual interpretation has motivated substantial interest in automated seizure detection, especially considering the recent significant evidence that EEG dynamics during epileptic seizures can be chaotic [23, 24], which poses a challenge for representation-learning methods that implicitly or explicitly assume smoother or more stationary dynamics.

3 Methods

3.1 DIRU architecture

Each DIRU cell consists of K nonlinear compartments. Let \(\textbf{x}_t \in \mathbb {R}^{d_{x}}\) denote the input at time t, \(\textbf{h}_t \in \mathbb {R}^{d_h}\) the hidden state, and \(\textbf{c}_t^{(i)}\) a generic compartment state i. Given K parallel compartments, we first define the hidden dimension \(d_h\) and the number of compartments K.

For each compartment \(i \in \{1,\dots ,K\}\), we compute a candidate state \(\tilde{\textbf{c}}_t^{(i)} \in \mathbb {R}^{d_h}\) through an independent nonlinear transformation:

$$\begin{aligned} \tilde{\textbf{c}}_t^{(i)} = \tanh \!\Big ( W_{\text {in}}^{(i)} \textbf{x}_t + W_{\text {rec}}^{(i)} \textbf{h}_{t-1} \Big ) \end{aligned}$$

(1)

where \(W_{\text {in}}^{(i)} \in \mathbb {R}^{d_h \times d_x}\) and \(W_{\text {rec}}^{(i)} \in \mathbb {R}^{d_h \times d_h}\) are the input and recurrent weight matrices for compartment i. Note that \(W_{\text {rec}}^{(i)}\) corresponds to the i-th \(d_h\)-sized block of a shared matrix \(W_{\text {rec}} \in \mathbb {R}^{(K \cdot d_h) \times d_h}\), chunked along the output dimension.

Unlike previous partitioned-compartment models, such as tractable dendritic RNNs, DIRU introduces context-dependent modulation via a softmax attention mechanism over the stacked compartment outputs:

$$\begin{aligned} \textbf{w}_t = \textrm{softmax}\!\Big ( W_g \big [\tilde{\textbf{c}}_t^{(1)} ; \cdots ; \tilde{\textbf{c}}_t^{(K)}\big ] \Big ), \quad \mathbf {W_g} \in \mathbb {R}^{K \times (K \cdot d_h)} \end{aligned}$$

(2)

where \(\textbf{w}_t \in \mathbb {R}^K\) and each scalar \(w_t^{(i)}\) is the attention weight assigned to compartment i, with \({\sum} _{i=1}^K w_t^{(i)} = 1\), and \(\textbf{Wg}\) being a time-invariant gate matrix, shared across time steps.

The final hidden state is computed as the attention-weighted sum of all compartment outputs:

$$\begin{aligned} \textbf{h}_t = \sum _{i=1}^{K} w_t^{(i)}\, \tilde{\textbf{c}}_t^{(i)} = \sum _{i=1}^{K} w_t^{(i)}\, \tanh \!\Big ( W_{\text {in}}^{(i)}\textbf{x}_t + W_{\text {rec}}^{(i)}\textbf{h}_{t-1} \Big ) \end{aligned}$$

(3)

where \(\textbf{w}_t\) is defined by Eq. 2, yielding \(\textbf{h}_t \in \mathbb {R}^{d_h}\). In this manner, K compartments operate in parallel and their outputs are aggregated via learned attention weights to produce the next hidden state. The gating mechanism uses the softmax function, ensuring that the compartment contributions are normalized and interpretable as a soft selection over parallel processing pathways.

Structurally, DIRU is organized as a set of parallel computational compartments. Interaction between compartments does not occur through direct lateral synapses; instead, coordination is mediated implicitly through the shared recurrent state and the attention-based aggregation in Eq. 2. This design ensures stable information integration while preserving scalability and avoiding the instability associated with dense cross-compartment coupling.

For a DIRU architecture with K compartments, the total number of trainable parameters is given by:

$$\begin{aligned} \#\theta _{\text {DIRU}} = \underbrace{K \cdot d_h \cdot d_x}_{\text {input weights } W_{\text {in}}^{(i)}} + \underbrace{K \cdot d_h^2}_{\text {shared recurrent } W_{\text {rec}}} + \underbrace{K^2 \cdot d_h}_{\text {attention gate } W_g} + \underbrace{2K \cdot d_h}_{\text {biases } b_{\text {in}}^{(i)},\, b_{\text {rec}}}. \end{aligned}$$

(4)

All baseline models were configured with the same hidden dimension \(d_h\) to ensure controlled comparisons. In particular, the LSTM contains \(4(d_x d_h + d_h^2 + d_h)\) parameters (including biases), while the Tractable dendritic RNN, which uses partitioned compartments of dimension \(d_c = d_h / K\) with the constraint \(d_h \equiv 0 \pmod {K}\), contains \(K(d_c d_x + d_c d_h) + d_h^2\) parameters. The DIRU model contains 5,888 parameters, the Tractable dendritic RNN contains 3,457 parameters, and the LSTM baseline contains 5,908 parameters. The Tractable model is smaller due to its partitioned compartment design, yet it provides a meaningful baseline to assess the impact of active attention-based gating in DIRU.

DIRU improves gradient propagation through its compartmentalized architecture. The Jacobian of the hidden state with respect to the previous hidden state is:

$$\begin{aligned} \frac{\partial \textbf{h}_t}{\partial \textbf{h}_{t-1}} = \sum _{i=1}^{K} \left[ w_t^{(i)} \cdot \textrm{diag}\!\big (1 - \tanh ^2(\cdot )\big ) \cdot W_{\text {rec}}^{(i)} + \frac{\partial w_t^{(i)}}{\partial \textbf{h}_{t-1}} \cdot \tilde{\textbf{c}}_t^{(i)\top } \right] , \end{aligned}$$

(5)

where the implicit argument of \(\tanh ^2(\cdot )\) is \(W_{\text {in}}^{(i)}\textbf{x}_t + W_{\text {rec}}^{(i)}\textbf{h}_{t-1}\), and \(\frac{\partial w_t^{(i)}}{\partial \textbf{h}_{t-1}}\) reflects the gradient through the softmax attention gate.

This structure provides three advantages for gradient flow. First, errors propagate through K parallel pathways with independent weight matrices \(W_{\text {rec}}^{(i)}\), mitigating the accumulation of repeated nonlinear transformations that typically lead to vanishing gradients in monolithic architectures. Second, softmax attention weights \(w_t^{(i)}\) dynamically concentrate gradient flow through the most active compartments while attenuating it through saturated ones, preventing gradient vanishing during long sequences. Third, parallel compartmentalization distributes computation across independent subspaces, mitigating the compounding nonlinearities that cause vanishing gradients in monolithic architectures. Gradient norms were clipped to \(\Vert \nabla \theta \Vert _2 \le 1.0\) during training for numerical stability. The code related to this implementation and experiments is publicly available at https://github.com/alecrimi/DIRU.

These properties structurally distinguish DIRU from classical gated recurrent models such as LSTMs, which rely exclusively on temporal gating, and from tractable dendritic RNNs, which employ passive nonlinear compartments without adaptive gating.

3.2 Synthetic chaotic training data

To evaluate the capacity of our model, Brenner’s tractable dendritic RNNs, and LSTM on nonlinear and chaotic dynamics, we conducted controlled experiments on canonical low-dimensional chaotic systems. These systems provide well-defined ground-truth dynamics, known attractor structures, and strong sensitivity to initial conditions, making them suitable benchmarks for assessing the representational power, stability, and long-horizon predictive performance of recurrent architectures.

We primarily focused on two classical chaotic systems: the Lorenz attractor and the Mackey–Glass time-delay system. Those are explained in the rest of the section and are depicted in Fig. 2 (a) and (b), respectively.

Lorenz System

The Lorenz system is defined by coupled nonlinear differential equations

$$\begin{aligned} \frac{dx}{dt}&= \sigma (y - x), \end{aligned}$$

(6)

$$\begin{aligned} \frac{dy}{dt}&= x(\rho - z) - y, \end{aligned}$$

(7)

$$\begin{aligned} \frac{dz}{dt}&= xy - \beta z, \end{aligned}$$

(8)

where \(\sigma\), \(\rho\), and \(\beta\) are system parameters controlling the chaotic regime. We used the standard chaotic configuration \(\sigma = 10\), \(\rho = 28\), and \(\beta = 8/3\). The trajectory was generated through explicit numerical integration using a fixed time step \(\Delta t = 0.01\), starting from randomized initial conditions. This produced three-dimensional state trajectories evolving on the classical Lorenz strange attractor, characterized by multistability, sensitive dependence on initial conditions, and nonlinear coupling between state variables.

Mackey–Glass System

The Mackey–Glass equation is a nonlinear time-delay differential equation given by

$$\begin{aligned} \frac{dx(t)}{dt} = \frac{\beta x(t-\tau )}{1 + x(t-\tau )^n} - \gamma x(t), \end{aligned}$$

(9)

where \(\tau\) is the delay parameter and \(\beta\), \(\gamma\), and n control the nonlinearity and feedback strength. It is considered an attractor system because its dynamics evolve toward a stable set of states in phase space, based on delays that introduce memory and nonlinearity, which are crucial for generating rich dynamics. For these reasons, it is used as a benchmark for RNN and to model the concentration of blood cells, for example, in chronic granulocytic leukemia [25]. We employed the standard chaotic configuration with \(\tau = 17\), \(n = 10\), \(\beta = 0.2\), and \(\gamma = 0.1\). The system was simulated using discrete-time numerical integration, with an initial transient period discarded to ensure convergence to the chaotic attractor. This produces a one-dimensional time series that exhibits long-range temporal dependencies, delayed feedback, and complex nonlinear dynamics.

Dataset Construction

From each simulated trajectory, the overlapping input-target sequences were extracted using a sliding window approach. The input sequences consisted of contiguous segments of length \(L=100\) time steps, while the targets corresponded to future trajectories on a prediction horizon of \(H=10\) steps. This formulation defines a multi-step forecasting task in which the model must learn the underlying system dynamics rather than short-term autoregressive correlations. For this purpose, we also investigated longer horizons on the same generated datasets, such as with 25, 50 and 100 steps.

The Lorenz system was treated as a multivariate prediction problem in \(\mathbb {R}^3\), while the Mackey–Glass system was a univariate delayed dynamical prediction task. All trajectories were normalized to zero mean and unit variance to ensure stable optimization and comparable scaling across systems.

Training Protocol Chaotic Time Series

Models were trained to minimize the mean squared error (MSE) between predicted and true future states over the prediction horizon:

$$\begin{aligned} \mathcal {L} = \frac{1}{H} \sum _{h=1}^{H} \Vert \hat{\textbf{x}}_{t+h} - \textbf{x}_{t+h} \Vert ^2, \end{aligned}$$

(10)

where \(\hat{\textbf{x}}_{t+h}\) denotes the predicted state at horizon step h and \(\textbf{x}_{t+h}\) the ground-truth state. In this way, the loss function itself represents the MSE of the prediction. Optimization was performed using the Adam optimizer for 80 epochs.

This training setup evaluates the models’ ability to learn intrinsic system dynamics, maintain stable internal representations over long temporal horizons, and generate structurally consistent trajectories in chaotic regimes. In contrast to short-horizon prediction tasks, this formulation explicitly probes long-term stability, horizon-dependent error accumulation, and regime preservation, making it particularly suitable for assessing the benefits of multi-compartment recurrent architectures such as DIRU.

Moreover, we assessed the statistical significance of the best validation loss across methods using paired t-tests (Sect. 3).

3.3 Helsinki neonatal EEG data

There is significant evidence that the EEG time series during epileptic seizures can be chaotic [23, 24]. Therefore, for experiments with real biological signals, we used the challenging Helsinki Neonatal EEG dataset, a publicly available collection of multichannel EEG recordings of term neonates admitted to the neonatal intensive care unit (NICU) of the Helsinki University Hospital [26]. The dataset comprises EEG recordings from 79 human neonates, with a median recording duration of 74 minutes. Of these 79 neonates, 39 were recorded to have seizures, 22 were classified as seizure-free, and the remaining subjects exhibited various annotations between experts [26]. It is also important to bear in mind that all subjects were atypical, and not controls, even if marked as seizure-free. Most neonates had hypoxia, encephalopathy, metabolic disturbances, structural lesions, or medication effects. In summary, the data set is of a pathological population with few seizure events. Moreover, despite the length of the complete recording, the seizures marked by the annotators were about 1 minute or less, and often appeared only once in the recordings. Of the 79 files, 3 did not pass quality control and were excluded (sample #4, #29, and #50). Nevertheless, the dataset thus provides a heterogeneous sample of neonatal EEG dynamics useful for evaluating representation learning in real nonlinear time series.

Pre-processing

The remainder of the files were preprocessed following standard pipelines. Briefly, the EEG recordings were first band-pass filtered between 0.5 and 40 Hz to remove slow drifts and high-frequency noise. The signals were re-referenced using a bipolar montage, where each channel represents the voltage difference between two adjacent electrodes, and then normalized per-channel by subtracting the mean and dividing by the standard deviation [27]. With this representation, an example of the dataset showing a labeled seizure (ictal phase) and the period before the seizure (pre-ictal) can be shown in Fig. 3. Moreover, we use this representation as an approximate source-level inference [28] where sensor-based location is converted to source-based using sensors as seeds.

Data representation

Importantly, following the comments in [29], we used a subset of recorded electrodes, as the full research configuration does not reflect the realistic clinical setting in the NICU. This reduction is consistent with clinical recommendations for neonatal EEG, where reduced-channel montages have been validated for seizure detection [30, 31]. This is a decision that comes in handy, especially considering the small subset available and the potentially high dimension of the feature vectors. As in [29], we therefore moved from 19 electrodes to 9 as shown in Fig. 4. We also neglected the signal from the CZ, further reducing the subset to 8 electrodes. The CZ electrode was excluded from the reduced-channel configuration due to its limited spatial specificity and high correlation with multiple channels in neonatal EEG recordings. As a midline electrode, CZ often captures global signal components rather than region-specific activity, providing a limited discriminative value for seizure classification. Its removal reduces redundancy in the input space while preserving physiologically meaningful spatial information. However, it is recognized that these considerations need to be re-evaluated in the case of source-localization or functional connectivity [32].

Then those 8 time series for each subject/recording were subdivided into windows/epochs of 3 seconds and with a 75% overlap, without optimizing those parameters. Windows were assigned contiguously to folds such that no overlapping windows span the train/test boundary. This is in line with similar work in the literature using epochs between 2 and 6 seconds [29, 33, 34].

Despite the highly imbalanced dataset, we did not remove or subsample non-seizure windows. We introduced a ratio of 0.5 as a threshold for labeling the window positive: if \(\ge 50\%\) of the 3 s window contains a seizure, the label is 1; otherwise 0. Although this threshold may appear arbitrary in isolation, it is motivated by intrinsic physiological and signal-processing considerations of EEG-based seizure detection. For completeness, we also report results obtained when varying this threshold to more permissive values (less than 50%) in the Results section. From a neurophysiological perspective, seizure dynamics are characterized by rapid transitions and nonstationary temporal evolution. Clinically relevant ictal activity may emerge locally and transiently within a time window before becoming spatially or temporally dominant. A strict majority-based threshold (e.g., much more than \(50\%\)) would systematically exclude short but clinically significant ictal onsets, effectively biasing the labeling process toward sustained seizures only and introducing systematic false negatives at the limit of seizures. A 10% threshold would be too permissive and prone to labeling windows with minimal ictal activity as positive. The at least \(50\%\) criterion instead reflects a detection-oriented formulation, in which the presence of any non-negligible ictal content within a window is treated as diagnostically relevant.

From a signal-processing standpoint, short ictal fragments can significantly alter the spectral, nonlinear, and dynamical features of EEG signals, even when they occupy a minority fraction of the temporal window. Consequently, windows containing brief seizure activity are not statistically equivalent to purely interictal windows, and treating them as negative samples would introduce label noise and feature contamination.

More formally, let \(s_c(t)\) denote the EEG signal from channel c, with \(c = 1, \dots , 8\).

concatenating all 8 channels, the input vector at time t is:

$$\begin{aligned} \textbf{x}_t = [s_1(t), s_2(t), \dots , s_8(t)]^\top \in \mathbb {R}^{8}. \end{aligned}$$

(11)

The RNN receives the temporal sequence:

$$\begin{aligned} \textbf{X} = (\textbf{x}_1, \textbf{x}_2, \dots , \textbf{x}_T), \end{aligned}$$

(12)

and models recurrent dynamics:

$$\begin{aligned} \textbf{h}_t&= \Phi (\textbf{x}_t, \textbf{h}_{t-1}) \end{aligned}$$

(13)

$$\begin{aligned} \hat{y}&= g(\textbf{h}_T) \end{aligned}$$

(14)

where \(\Phi\) is the recurrent update function (e.g., LSTM/Tractable/DIRU cell) and g is the output mapping.

Most prior work on classifying neonatal seizures using EEG has employed optimized features or architectures, or applied strong data augmentation. O’Shea and colleagues [35] developed a fully convolutional neural network to detect seizures from raw multi-channel neonatal EEG, reaching high accuracy with the raw EEG signal, but training the model first on a much larger dataset with 1,389 annotated seizure events [36], which is unfortunately not fully publicly available. Indeed, advanced architectures based, for example, on CNN and transformers could reach high-level accuracy even with raw EEG, assuming a very large dataset is available. Given this, on small dataset as the one used in this paper, Nemati et al. [33] used short-time Fourier transform (STFT) spectrograms and discrete wavelet transform (DWT) sub-band decomposition with Inception-ResNetV2 and XGBoost. Micheli et al. [29] used spikiness features and time-frequency characteristics. Mazurek and colleagues investigated the influence of Katz fractal dimensionality and band power energies [37, 38]. The risk of looking for the ideal combination of features and architecture with such a small dataset is that we may find features that increase the classification score for this dataset, but we cannot assure that this will generalize to other datasets. Therefore, we limit ourselves to raw EEG split into small windows, as explained above. Although tuning window size, overlap, and threshold could further increase classification metrics on this dataset, we avoid such optimization to reduce the aforementioned risk of overfitting to a single benchmark.

Training Protocol EEG

Models were trained to minimize a weighted binary cross-entropy (BCE) loss between predicted seizure probabilities and ground-truth window labels:

$$\begin{aligned} \mathcal {L} = -\left[ w^{+} \, y \log (\hat{y}) + w^{-} (1-y)\log (1-\hat{y}) \right] , \quad w^{+} = \frac{N_{\text {neg}}}{N_{\text {pos}}}, \; w^{-} = 1 \end{aligned}$$

(15)

where \(\hat{y}\) denotes the predicted probability of seizure for a given window, \(y \in \{0,1\}\) is the ground-truth label, and \(w^{+} = \frac{N_{\text {neg}}}{N_{\text {pos}}}\) is a class-balancing weight compensating for the strong imbalance between non-seizure and seizure windows. Indeed, this is cost-sensitive learning applied to unbalanced binary classification by using a weighted binary cross-entropy loss [39]. This weighting strategy increases the contribution of rare seizure events without subsampling non-seizure data.

Optimization was performed using the AdamW optimizer with weight decay, gradient clipping, learning-rate warm-up, and adaptive learning-rate scheduling, for a maximum of 40 epochs per fold. Training and evaluation were conducted using leave-one-recording-out cross-validation (LOO-CV) at the patient level, ensuring strict subject-level separation between training and testing data and preventing data leakage.

This training formulation evaluates the models’ ability to learn discriminative temporal representations of transient pathological dynamics, capture rapid regime transitions, and maintain stable internal representations in highly imbalanced and noisy biological signals. Rather than short-horizon signal reconstruction, the objective explicitly probes regime discrimination, nonlinear temporal integration, and robustness to sparse event structure, making it particularly suitable for assessing the advantages of multi-compartment recurrent architectures such as DIRU in real-world biological time series. The performance of the model was evaluated using the area under the receiver operating characteristic curve (AUC). The AUC was selected as the primary metric as it measures discriminative performance across all possible classification thresholds simultaneously. Given the LOO-CV we also report the confidence intervals to show statistical differences. For threshold-dependent metrics (sensitivity, specificity, F1, and accuracy), we report those results, using the automatically defined Youden’s J threshold [40], often used in similar studies on seizures detection from EEG [41, 42].

Nonetheless, we computed the classification with raw EEG, and using K-fold subdivision 60% Training, 20% testing, and 20% validation to investigate those settings, and report those results. Ablation settings are argued in the Discussion section, and mostly comprise a reduction of compartments.

4 Results

DIRU consistently outperformed LSTM across all benchmarks. DIRU achieves lower validation loss and faster convergence as shown in Figs. 5, 6 and Tables 3, 2 and 5. Table 6 summarizes instead the median convergence epoch for all methods across the datasets, defined when training early stopped with a patience of 8 training epochs. The validation loss (BCE) was monitored for early stopping, using 8 as patience. For the synthetic datasets the same criterion was used to detect where the convergence was happening, though we let the training continue.

4.1 Synthetic chaotic data

Although the validation loss curves with both synthetic data show some overlap between DIRU and the Tractable Dendritic RNN across epochs, this is expected and does not contradict the statistical results. The plotted curves represent the mean ± standard deviation across seeds for each epoch, capturing the variability in learning dynamics. In contrast, the paired t-test was performed on the best validation loss per seed, focusing on the peak performance achieved by each model. Even if the curves overlap at certain epochs, DIRU consistently attains lower minima across seeds, which is reflected in the significant p-values. Thus, the overlap in the curves is consistent with the observed statistical advantage of DIRU. Tables 3 and 2 summarize the best validation loss improvements for all datasets in the final epochs.

4.2 Neonatal EEG data

Tables 5 and Fig. 7 report the classification metrics for the seizure classification task using the three methods for raw EEG. Particularly, we report the AUC values with a relative confidence interval CI for statistical significance. The results of varying the threshold defining how much seizure is included in a window are reported in Table 7.

Reduced performances were observed with the currently selected features and the splitting of the data set into several k-fold training, test, and validation subsets. Indeed, this produced \(AUC \approx 0.70-0.75\) for all three methods. This discrepancy should be interpreted cautiously, since the LOO-CV AUC was computed by pooling held-out predictions across all leave-one-recording-out folds, whereas the K-fold estimate was obtained from a substantially smaller held-out partition. Therefore, the two values should not be interpreted as directly comparable estimates of generalization performance, particularly given the limited size and clinical heterogeneity of the Helsinki cohort.

4.3 Ablation studies

Tractable dendritic RNN can already represent a structured ablation study that simultaneously removes DIRU’s two distinguishing inductive biases: full-width compartment projections and soft attention-weighted. A further ablation that has been carried out is related to the influence of the number of compartments, which showed the following results for the prediction experiments in Tables 8 and 9.

5 Discussion

This work introduces a dendrite-inspired recurrent neural unit as a fundamentally different class of recurrent architecture, shifting from monolithic state-space modeling toward internally structured recurrent computation, with compartments already in the individual neuronal units. Unlike conventional RNNs and prior dendritic-inspired models that emphasize mathematical tractability and system reconstruction, DIRUs are designed as adaptive dynamical systems in which multiple interacting compartments actively shape information flow, memory routing, and nonlinear transformation.

Existing dendritic RNN formulations, such as tractable dendritic models, interpret dendrites primarily as structured nonlinear transformations designed for identifiability, stability, and analytical interpretability. While this yields elegant system-identification frameworks, it constrains expressive capacity and limits adaptability in highly nonlinear regimes. In contrast, DIRUs explicitly trade theoretical tractability for representational power and adaptive computation. By embedding compartment-specific gating, residual recurrence, and global modulatory control into recurrent dynamics, DIRUs operate closer to adaptive dynamical systems than to structured regression models. This positions DIRU as a bridge between neuroscience-inspired computation and modern deep sequence learning architectures, integrating biological plausibility with high-capacity function approximation.

The results on the synthetic chaotic series show a better reconstruction error defined within the loss function, but also a faster convergence, even in the Mackey-Glass series, which were relatively noisier in the process. Fewer epochs to convergence suggest that DIRU not only provides improved learning capacity, but also reduces training time. Those performances also reflect qualitative differences in the learned dynamics. Beyond DIRU’s performance, tractable dendritic RNN significantly outperformed LSTM on the Lorenz attractor (77.46% improvement, \(p < 0.0001\)), while LSTM showed a non-significant advantage on the Mackey-Glass system (−79.94%, p = 0.1715). Results are consistent when using horizon longer than 10 steps with DIRU still outpeforming the other methods.

The parameter asymmetry between DIRU and the Tractable model is not a confounding but a structural consequence of their respective designs. The Tractable model’s passive compartment formulation does not include per-compartment gating, global modulation, or residual recurrence, making these components architecturally absent rather than merely undersized. Inflating the Tractable model to match DIRU’s parameter count would require adding precisely the architectural elements whose absence defines it, producing a different model entirely. Moreover, in the Lorenz attractor, the Tractable model outperforms LSTM by 77% despite having roughly half of the parameters, demonstrating that structural inductive bias is the primary driver of performance, not capacity. DIRU’s further improvement over Tractable, therefore, reflects the benefit of active gating and residual dynamics, not a capacity advantage. This divergence likely reflects the distinct dynamical properties of each system: the Lorenz attractor’s multi-variable coupling may favor structured compartmentalization, whereas the Mackey-Glass system’s delayed feedback may align better with LSTM’s gating architecture.

DIRU is explicitly designed as an extension of the tractable dendritic RNN framework. The two models occupy endpoints of a compact design space: tractable RNN uses partitioned projections with linear integration, while DIRU employs full-width projections with attention-based aggregation. This design choice allows direct assessment of these two inductive biases without requiring exhaustive ablation, as the performance gap between them isolates the contribution of active gating and attention mechanisms. Replacing DIRU’s full-width projections with partitioned ones and substituting its softmax attention gate with a linear integration layer recovers the tractable dendritic RNN exactly [8]. The two models, therefore, occupy the endpoints of a compact two-axis design space. Rather than treating Tractable dendritic RNN as an arbitrary baseline, it is more precisely interpreted as a structured ablation study that simultaneously removes DIRU’s two distinguishing inductive biases, full-width compartment projections and soft attention-weighted aggregation, allowing their joint contribution to be assessed directly from the reported performance gap. There is a slight reduction in performance when reducing the number of compartments; however, the most pronounced change occurs when we move from 1 to 2 compartments.

Classical recurrent architectures encode system evolution within a single homogeneous latent space, thereby compressing all nonlinear interactions, temporal dependencies, and regime transitions into a single state manifold. This representation is inherently limited for chaotic systems, where multistability, attractor coexistence, delayed feedback, and sensitivity to initial conditions dominate the dynamics. As a result, traditional models tend to learn smoothed surrogates of chaotic trajectories rather than stable internal representations of the underlying attractor structure. DIRUs mitigate this limitation by distributing computation across multiple interacting compartments, enabling internal decomposition of nonlinear dynamics. From a dynamical systems perspective, DIRUs can be interpreted as learning internal computational hierarchies that approximate complex nonlinear operators through structured composition rather than flat recurrence. This aligns with the theoretical views of biological neurons as functionally deep computational units rather than pointwise integrators. In this sense, DIRUs draw conceptual inspiration from the principles of compartmentalized neural computation, where local nonlinear processing, coincidence detection, and hierarchical integration occur at the cellular level.

These architectural properties are also relevant for biological time series. Neural signals such as EEG exhibit nonstationarity, cross-frequency coupling, metastability, delayed feedback, and rapid transitions between dynamical regimes. Traditional recurrent models struggle to disentangle these interacting temporal scales within a single latent representation [43]. Multi-compartment recurrent architectures provide a natural inductive bias for such signals by enabling parallel processing of distinct spectral and temporal components, hierarchical integration across scales, and context-dependent modulation of internal dynamics. This makes dendrite-inspired recurrent units especially suitable for modeling neural population dynamics, where the meaningful structure lies not only in short-term correlations but also in regime transitions, attractor structure, and long-range temporal organization.

The EEG classification results tell a more nuanced story than the chaotic benchmarks. Rather than demonstrating a decisive architectural advantage, DIRU maintains competitive performance relative to LSTM and the Tractable dendritic RNN, with AUC differences remaining within a narrow band across all models. The results suggest that the recurrent architectures are capable of extracting informative temporal patterns directly from raw EEG signals, without relying on handcrafted spectral or subband feature representations. This is consistent with the nature of the task: because the models operate directly on short raw EEG windows, the discriminative signal may be dominated by local ictal waveform patterns and by the severe imbalance of the dataset, partially reducing the apparent marginal benefit of more structured recurrent computation. When the input representation itself performs much of the dynamical abstraction, the marginal benefit of compartmentalized recurrent computation is reduced. Nonetheless, DIRU does not degrade on this challenging task. It remains competitive under extreme class imbalance, small sample size, and strict leave-one-out evaluation, without data augmentation or architectural specialization. Regarding the metrics, despite the relatively low F1-score, this result is consistent with the extreme class imbalance of the neonatal seizure dataset. In such settings, even when sensitivity and specificity are reasonably high, the very low prevalence of the positive class leads to reduced precision, which in turn markedly penalizes the F1-score. The strong AUC achieved by DIRU indicates good class separability overall, while the low F1 mainly reflects the difficulty of maintaining high precision when seizure events are rare and transient in raw EEG recordings.

A further methodological point concerns the relationship between our LOO-CV and K-fold estimates on the EEG classification task (AUC = 0.884 vs. AUC \(\approx 0.70-0.75\), respectively). These two estimators are not directly comparable: the LOO-CV AUC is computed by pooling predictions across all leave-one-out folds into a single ROC curve, aggregating information across the full cohort (76 held-out recordings, one per fold), whereas the K-fold estimate reflects performance on a single, considerably smaller held-out partition (\(\sim\)20% of the cohort). The latter is therefore inherently more sensitive to the particular composition of that one test split, especially given the small sample size and clinical heterogeneity of the cohort. Given the modest size and heterogeneity of the Helsinki cohort, we caution against treating either estimate in isolation as a definitive measure of generalization performance, and we consider validation on larger, independent neonatal EEG cohorts an important direction for future work. A further limitation of the study is that its evaluation is based on window-level classification metrics; we did not compute an event-level false-alarm rate per hour [35], a standard metric in the seizure-detection literature that requires converting window-level predictions into continuous detected events via minimum-duration and merging criteria, and we identify this as an important direction for future validation of DIRU’s clinical utility.

This work establishes dendritic-inspired recurrent architectures as a promising computational substrate for learning complex nonlinear systems. Future work can build on this foundation by exploring multi-scale hierarchies, adaptive connectivity, and integration with graph-based architectures. DIRU’s compartmentalized design naturally extends to hierarchical architectures, which we leave for future investigation.

6 Conclusion

DIRU demonstrates superior capability in modeling nonlinear and chaotic systems compared to LSTM and the Tractable RNN model. Its dendrite-inspired compartmental architecture allows for hierarchical nonlinear processing, faster convergence, and better prediction accuracy. These findings motivate further investigation of compartmentalized recurrent architectures in clinical time-series analysis. In the neonatal EEG seizure detection task, DIRU outperformed the compared recurrent baselines in terms of AUC, although validation on additional cohorts will be necessary to assess clinical generalizability.

References

1. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

   Article  Google Scholar 

2. Shen Y, Tan S, Sordoni A, Courville A (2018) Ordered neurons: Integrating tree structures into recurrent neural networks. International Conference on Learning Representations (ICLR). p 2019 arXiv:1810.09536

3. Chung J, Ahn S, Bengio Y (2016) Hierarchical multiscale recurrent neural networks. In: International Conference on Learning Representations (ICLR) 2017. arXiv:1609.01704

4. Poirazi P, Brannon T, Mel BW (2003) Pyramidal neuron as two-layer neural network. Neuron 37(6):989–999

   Article  Google Scholar 

5. Spruston N (2008) Pyramidal neurons: dendritic structure and synaptic integration. Nat Rev Neurosci 9(3):206–221

   Article  MathSciNet  Google Scholar 

6. Poirazi P, Brannon T, Mel BW (2003) Arithmetic of subthreshold synaptic summation in a model CA1 pyramidal cell. Neuron 37(6):977–987

   Article  Google Scholar 

7. London M, Häusser M (2005) Dendritic computation. Annu Rev Neurosci 28(1):503–532

   Article  Google Scholar 

8. Brenner M, Hess F, Mikhaeil JM, Bereska LF, Monfared Z, Kuo P-C, Durstewitz D (2022) Tractable dendritic RNNs for reconstructing nonlinear dynamical systems. In: International conference on machine learning, pp. 2292–2320. PMLR

9. Beniaguev D, Segev I, London M (2021) Single cortical neurons as deep artificial neural networks. Neuron 109(17):2727–2739

   Article  Google Scholar 

10. Mel BW (1992) Nmda-based pattern discrimination in a modeled cortical neuron. Neural Comput 4(4):502–517

    Article  Google Scholar 

11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

12. Zhang L, Sun Y, Wang A, Zhang J (2023) Neural network modeling and dynamic behavior prediction of nonlinear dynamic systems. Nonlinear Dyn 111(12):11335–11356

    Article  Google Scholar 

13. Sussillo D, Barak O (2013) Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput 25(3):626–649

    Article  MathSciNet  Google Scholar 

14. Ott E (2002) Chaos in dynamical systems. Cambridge University Press, Cambridge

    Book  Google Scholar 

15. Pagkalos M, Chavlis S, Poirazi P (2023) Introducing the dendrify framework for incorporating dendrites to spiking neural networks. Nat Commun 14(1):131

    Article  Google Scholar 

16. Gao J, Hu J, Tung W-W (2011) Complexity measures of brain wave dynamics. Cogn Neurodyn 5(2):171–182

    Article  Google Scholar 

17. Röschke J, Fell J, Beckmann P (1995) Nonlinear analysis of sleep EEG data in schizophrenia: calculation of the principal lyapunov exponent. Psychiatry Res 56(3):257–269

    Article  Google Scholar 

18. Lee YJ, Zhu YS, Xu YH, Shen MF, Tong SB, Thakor NV (2001) The nonlinear dynamical analysis of the EEG in schizophrenia with temporal and spatial embedding dimension. J Med Eng Tech 25(2):79–83

    Article  Google Scholar 

19. Golos M, Jirsa VK, Daucé E (2013) Multistability in large scale models of brain activity. BMC Neurosci 14(Suppl 1):84

    Article  Google Scholar 

20. Breakspear M (2017) Dynamic models of large-scale brain activity. Nat Neurosci 20(3):340–352

    Article  Google Scholar 

21. Fisher RS, Acevedo C, Arzimanoglou A et al (2014) Ilae official report: a practical clinical definition of epilepsy. Epilepsia 55(4):475–482. https://doi.org/10.1111/epi.12550

    Article  Google Scholar 

22. Pressler RM, Cilio MR et al (2021) The ilae classification of seizures and the epilepsies: modification for seizures in the neonate. Epilepsia 62(3):615–628. https://doi.org/10.1111/epi.16815

    Article  Google Scholar 

23. Kwessi EA, Edwards LJ (2021) Analysis of EEG data using complex geometric structurization. Neural Comput 33(7):1942–1969

    Article  MathSciNet  Google Scholar 

24. Zhao Y, Grayden DB, Boley M, Liu Y, Karoly PJ, Cook MJ, Kuhlmann L (2025) Cortical stability and chaos during focal seizures: insights from inference-based modeling. J Neural Eng 22(3):036021

    Article  Google Scholar 

25. Glass L (2001) Synchronization and rhythmic processes in physiology. Nature 410(6825):277–284

    Article  Google Scholar 

26. Stevenson NJ, Tapani K, Lauronen L, Vanhatalo S (2019) A dataset of neonatal EEG recordings with seizure annotations. Scientific Data 6:190039

    Article  Google Scholar 

27. Tapani KT, Nevalainen P, Vanhatalo S, Stevenson NJ (2022) Validating an SVM-based neonatal seizure detection algorithm for generalizability, non-inferiority and clinical efficacy. Comput Biol Med 145:105399. https://doi.org/10.1016/j.compbiomed.2022.105399

    Article  Google Scholar 

28. Rojas GM, Alvarez C, Montoya CE, Iglesia-Vaya M, Cisternas JE, Gálvez M (2018) Study of resting-state functional connectivity networks using EEG electrodes position as seed. Front Neurosci 12:235

    Article  Google Scholar 

29. Micheli C, Thelen A, De Vos M, Dereymaeker A, Haueisen J, Fiedler P (2025) Validation of an automated seizure detection procedure for multi-channel neonatal EEG. Appl Sci 16(1):52

    Article  Google Scholar 

30. Tekgul H, Bourgeois BF, Gauvreau K, Bergin AM (2005) Electroencephalography in neonatal seizures: comparison of a reduced and a full 10/20 montage. Pediatr Neurol 32(3):155–161

    Article  Google Scholar 

31. Stevenson NJ, Lauronen L, Vanhatalo S (2018) The effect of reducing EEG electrode number on the visual interpretation of the human expert for neonatal seizure detection. Clin Neurophysiol 129(1):265–270

    Article  Google Scholar 

32. Zhang L, Wang P, Zhang R, Chen M, Shi L, Gao J, Hu Y (2020) The influence of different EEG references on scalp EEG functional network analysis during hand movement tasks. Front Hum Neurosci 14:367

    Article  Google Scholar 

33. Nemati N, Meshgini S, Rezaii TY, Afrouzian R (2025) Neonatal seizure detection from EEG using inception resnetv2 feature extraction and xgboost optimized with particle swarm optimization. Sci Rep 15(1):41493

    Article  Google Scholar 

34. Zhou W, Zheng W, Feng Y, Li X (2024) LMA-EEGNet: a lightweight multi-attention network for neonatal seizure detection using EEG signals. Electronics (Basel) 13(12):2354

    Google Scholar 

35. O’Shea A, Lightbody G, Boylan G, Temko A (2020) Neonatal seizure detection from raw multi-channel EEG using a fully convolutional architecture. Neural Netw 123:12–25

    Article  Google Scholar 

36. Temko A, McEvoy R, Dwyer D, Faul S, Lightbody G, Marnane WP Cork Neonatal EEG Seizure Dataset. EEG recordings from 18 full-term neonates with hypoxic-ischemic encephalopathy (HIE) and annotated seizure events collected at Cork University Maternity Hospital. https://cora.ucc.ie

37. Mazurek S, Blanco R, Falcó-Roget J, Crimi A (2024) Explainable graph neural networks for EEG classification and seizure detection in epileptic patients. 2024 IEEE Int Symposium Biomedical Imaging (ISBI). IEEE, pp 1–5

38. Mazurek S, Moore S, Crimi A (2025) Graph attention networks for detecting epilepsy from EEG signals using accessible hardware in low-resource settings. IEEE Open J Eng Med Biol. https://doi.org/10.1109/OJEMB.2025.3642070

    Article  Google Scholar 

39. Buda M, Maki A, Mazurowski MA (2018) A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw 106:249–259

    Article  Google Scholar 

40. Fluss R, Faraggi D, Reiser B (2005) Estimation of the Youden index and its associated cutoff point. Biom J 47(4):458–472

    Article  MathSciNet  Google Scholar 

41. Daşdemir A, Örnek HK (2025) Epileptic seizure prediction with deep learning-based fusion methods. Eng Sci Tech Int J 72:102212

    Google Scholar 

42. Bangash AH, Bercu MM, Byrne RW, Pavuluri S, Salehi A (2025) Application of machine learning approaches to predict seizure-onset zones in patients with drug-resistant epilepsy: a systematic review. Front Neurol 16:1687144

    Article  Google Scholar 

43. Kaplan AY, Fingelkurts AA, Fingelkurts AA, Borisov SV, Darkhovsky BS (2005) Nonstationary nature of the brain activity as revealed by eeg/meg: methodological, practical and conceptual challenges. Signal Process 85(11):2190–2212

    Article  Google Scholar 

Download references