Prognostic models

Predictive models are algorithms that estimate future values or events based on historical data. They underlie product analytics, risk management, marketing, anti-fraud and operational optimization. Below is a practical guide: from choosing the wording of the task to operation and quality control.

1) Task formulation and signals

Regression: continuous value forecast (LTV, deposit size, check size).
Classification: probability of an event (customer outflow of 30 days, fraud/non-fraud).
Ranking: ordering objects by relevance (recommendations).
Time series: calendar prediction (traffic, load, revenue).
Survival/time to event: estimated time to outflow/repay.
Uplift modeling: the increase in the effect of the impact (which stock really changes behavior).
Multi-class/multi-label productions: multiple states/events at the same time.
Bayesian and probabilistic models: prediction with apparent uncertainty.

2) Data and preparation

Grain and observation window: Define the forecast unit (user/session/day) and an honest "window" for collecting characteristics up to the moment of forecast.
Target variable: clear definition, horizons (7/30/90 days), elimination of future leaks.
Features: aggregates by windows (7/30/90), behavioral frequencies, sequences, categorical encodings, interactions, segment statistics.
Data quality: omissions, outliers, duplicates, circuit changes, timestamp consistency.
Class balance: stratification, weighting, oversampling/undersampling - careful not to introduce bias.

3) Choice of algorithms

Basic linear: linear/logistic regression, regularization (L1/L2/Elastic Net) - fast, interpretable.
Trees and ensembles: Random Forest, Gradient Boosting (XGBoost/LightGBM/CatBoost) - strong baselines.
Neural networks: MLP/Seq2Seq/Transformer for complex patterns (texts, sequences, time series).
Time series: ARIMA/ETS, Prophet, Gradient Boosting with lags, DeepAR/N-BEATS/Temporal Fusion Transformer.
Uplift: two-model approach, T-Learner/S-Learner/X-Learner, causal effect trees.
Survival: Cox, AFT, Random Survival Forests.
Bayesian: GLM/hierarchical models with prior distributions.

Practice: Start with a simple and reproducible baseline, add difficulty only with tangible gains in metrics and stability.

4) Separation and validation

Temporal CV: for tasks with time - sections by dates (rolling/expanding windows).
Stratification: for unbalanced classes.
Group validation: prevent "leaks" between groups (user_id, campaign_id).
Out-of-time test: final check on the "future" period.
Baselines: naive (last value, average), frequency, simple logreg - to measure real value.

5) Quality metrics

Classification: ROC-AUC, PR-AUC (important in rare events), logloss, Brier score, F1, precision/recall @ k, lift/NDCG.
Regression: RMSE/MAE/MAPE, sMAPE, R ², quantile loss (for percentiles).
Time series: MAPE/sMAPE/MASE, Pinball loss (quantile prediction).
Survival: Concordance index, Brier for time-to-event.
Uplift: Qini, uplift@k, AUUC.
Business metrics: incremental profit, retained users, fraud reduction, accuracy SLAs.

Recommendation: always store two levels of metrics - "ML-quality" and "business effect."

6) Calibration and thresholds

Probability calibration: Platt, isotonic, temperature scaling.
Selection of the threshold: by business goals (maximum profit/limitation of false positive), by costs/fines.
Threshold stability: monitoring taking into account changes in distributions.

7) Interpretability and explainability

Global: importance of characteristics (gain, permutation), PDP/ICE, SHAP summaries.
Locally: SHAP/LIME to explain the object solution.

Caveat: Interpretability is a tool for hypothesis validation and trust, not an absolute "truth."

8) Combat deployment

Serving: online (REST/gRPC, low latency) and batch (hourly/daily).
Versioning: models, data, schemes and features (registry).
Fichestores: online/offline consistency, point-in-time correctness.
Delay control: feature extraction budget, inference, post-processes.
Fail-safe: default rules, baseline degradation, timeouts and retrays.
A/B and gradual rollout: canary releases, shadow/inference mirroring.

9) Monitoring and drift

Quality: metrics on sale (ROC-AUC/PR-AUC by deferred labels, business KPIs).
Distributions: PSI/JS-divergence/KL, feature drift and target.
Data Health: share of gaps, cardinality of categories, spikes, pipeline delays.
Alerting and SLO: thresholds, runibooks (what to do when degraded).
Retraining: by schedule, by drift triggers, by seasonality; quality regression control.

10) Safety, compliance and ethics

Privacy: data minimization, pseudonymization, access control.
Fairness: checking bias by segments (false positive/negative rate), adjusting samples/thresholds.
GDPR/local norms: processing objectives, shelf life, right to explain.
Auditing and reproducibility: version tracing, reproducible assemblies, solution log.

11) Templates for typical cases

Churn: binary classification; metrics - PR-AUC, uplift test for retention campaigns; conclusion - prioritization of offers.
Antifraud: high-balance classification; metrics - PR-AUC, recall @ low-FPR; strict latency restrictions.
LTV/ARPPU: regression or quantile regression; regular recalibration.
Load/revenue forecast: time series; ensemble of statistical and ML-models; seasonality and holidays as features.
Personal recommendations: ranking/matrixing/seq-models; online updates and exploitation (mult-armed bandits).

12) Process (ML Lifecycle)

1. Problem and KPI → 2. Data and features → 3. Baseline → 4. Models and → selection

2. Calibration and thresholds → 6. Validation and A/B → 7. Serving and monitoring → 8. Retraining/evolution.

Each stage is documented: data diagrams, training configs, versions of artifacts, experimental results.

13) Feature Engineering (Brief)

Aggregates: sums/averages/quantiles by window.
Counters and frequencies: conversions, rarity of categories.
Time lags and trends: deltas, exponential smoothes.
Encoding categories: target/stats encoding (with leaks carefully), WoE, one-hot/Hashing.
Texts and events: TF-IDF, embeddings, sequence features.
Business logic: domain characteristics (for example, "number of days with last payment").

14) Experiment management

Tracking: metrics, parameters, artifacts, datasets.
Hyperparameters: Bayesian claim, random/grid search, early stop.
Feature control: data catalog, circuit contract, compatibility tests.

15) Pre-sale checklist

No future leaks; correct temporal validation
Metrics are shear resistant; have an OOT test
Calibration verified; business threshold selected
Data Feature and Contract Documentation
Degradation and alert functions are configured
Retraining plan and stopping criteria
Legal and ethics checks passed

Mini Glossary

Distribution drift: change in data statistics/target over time.
Calibration: reduction of probabilities to the real frequency of events.

Uplift: prediction of effect difference between "treated" and "control."

OOT test: model verification on a period completely invisible during training/validation.

Total

A predictive model is not only an algorithm, but a process: correct statement of the problem, data discipline, strict validation, reproducible depla pipelines, monitoring and ethics. Following the described cycle reduces the risk of "beautiful offline metrics" and increases the real value in the product.

Prognostic models