Mobi Tandem Neural Network Pruning Lab

Forecast horizon is the number of minutes ahead the model tries to predict CGM glucose. A 60 minute horizon asks the network to map the current state into glucose roughly one hour later. Longer horizons are harder because meals, exercise, infusion-site behavior, and sensor artifacts can enter after the current feature window.

Hidden layer 1 receives the engineered feature vector. In this page the feature vector includes current and lagged glucose, short-term slopes, direct entered carb grams, recent bolus and carbohydrate sums, basal delivery summaries, meter BG context, time of day, and day of week. Layer 1 neurons are early pattern detectors; a unit may become sensitive to combinations such as rising glucose plus recent carbohydrates or falling glucose plus insulin activity.

Hidden layer 2 combines the layer 1 detectors into higher-level states. These units can represent broader situations such as post-meal rise, overnight stability, sustained high glucose, or correction-driven decline. Because layer 2 is closer to the output, pruning layer 2 often has a more direct effect on the final forecast.

Pruning removes whole hidden neurons by masking them to zero. This is structured pruning, not just setting a few individual weights to zero. If a layer 1 neuron is pruned, its output is removed before it reaches layer 2. If a layer 2 neuron is pruned, its contribution to the glucose forecast is removed before the final output.

Activation x outgoing weight is the default pruning score. Mean activation estimates how much a neuron is used on validation data. Outgoing weight magnitude estimates how strongly later layers depend on it. Multiplying the two is a practical salience score: neurons that rarely activate and have weak downstream weights are pruned first.

Fine-tuning trains the remaining unpruned neurons for a short period after masks are applied. Pruning can abruptly remove capacity, so fine-tuning lets the surviving weights compensate. If fine-tuning improves test error, the original network probably had redundant hidden units. If fine-tuning cannot recover accuracy, the pruning percentage was probably too aggressive or the removed units were important.

MAE is mean absolute error in mg/dL, which is usually the easiest forecast error to interpret. RMSE penalizes larger misses more strongly. R2 compares the model against predicting the average target value; negative R2 means the held-out forecast is worse than that simple average baseline.

• High pruning with small MAE change means the dense model had redundant hidden capacity for this export.
• Large MAE increase after pruning means the removed neurons were carrying useful information or the model did not have enough data to learn redundant representations.
• Layer 1 pruning tests whether many low-level feature detectors are unnecessary.
• Layer 2 pruning tests whether the final hidden representation has redundant state combinations.