Optimisation skills map¶
What senior and staff optimisation roles ask for, where this book teaches it, and what is still to come. The book's rule is one problem, many techniques: the same flagship energy problems come back with new formulations, algorithms and solvers, so you learn to compare methods on problems you already understand, not on toy examples.
Status: taught · introduced · planned
Optimisation fundamentals¶
| Skill | Where | Status |
|---|---|---|
| Decision variables, objectives, constraints, feasible regions | Ch 1 | |
| Existence and uniqueness of optima: infeasible, unbounded, degenerate and tied problems | T2 | |
| Ill-posedness and sensitivity: when a small change in data flips the answer; regularisation | T2; Ch 6 (the cliff), Ch 7 | |
| Local vs global optima; convexity | Toolbox T4; Ch 7 (marginal value overshoots) | |
| Gradient descent, momentum, step sizes, convergence | T3 (power-curve fitting: 2/L rule, Nesterov, heavy-ball transient, Adam) | |
| Least squares: normal equations, QR, conditioning, robust (LAD/Huber) regression | T3 (LAD and monotone fits as LPs; lost-energy bias) | |
| Maximum likelihood with censoring | Ch 3 (Weibull) | |
| State estimation: Kalman filter as recursive least squares; forecast intervals vs model error | Ch 10 (battery state of health) | |
| Non-convex optimisation: multistart, piecewise-linear relaxations, global solvers | Toolbox T4 (wake steering, Weibull likelihood); introduced in T3 (logistic curve symmetry) |
Constrained optimisation and operations research¶
| Skill | Where | Status |
|---|---|---|
| Simplex method by hand: standard form, slack/surplus/artificial variables, big-M and two-phase, ratio test, degeneracy and Bland's rule | T0 (Case A tableaux; checked against HiGHS) | |
| Choosing a method from the structure of a problem | Choosing a technique; a "Why these techniques?" section in every chapter | |
| Linear programming, geometry, duals and shadow prices | Ch 1, Ch 2, Ch 8, Ch 10 (shadow price of a throughput budget = degradation cost), Ch 12 (hourly shadow price of a shared line; parametric LP sizes the line) | |
| Mixed-integer programming: binaries, time-indexed scheduling, inventory, big-M | Ch 2, Ch 4, Ch 5, Ch 8, Ch 11 (fixed-cost campaigns, indicator for a capacity contract, concave revenue by epigraph; duals with integers fixed) | |
| Explaining a MILP: what-if re-solves, optimality gaps | Ch 5 | |
| Formulation strength, branch and bound, cuts, symmetry | Toolbox T6 (Case C) | |
| Chance constraints and expectations under uncertainty | Ch 4, Ch 6, Ch 7 | |
| Convex optimisation: QP, conic, CVaR | Ch 13 (CVaR as an LP, its dual as a tail measure); Ch 14 (mean–variance QP vs mean–CVaR LP vs lots MILP; duals as stress probabilities); Toolbox T5 (conic, BESS degradation) | |
| Risk measures: VaR, CVaR/expected shortfall, coherence, Euler contributions, VaR backtests (Kupiec) | Ch 13 (portfolio revenue risk, P50/P90/P99, driver Shapley) | |
| Constraint programming (CP-SAT) | Toolbox T6 (crew rostering) | |
| Dynamic programming and model predictive control | Ch 9 (rolling-horizon MPC, bids from the SOC dual); Ch 11 (DP policy for battery augmentation, validated against the MILP; rolling re-planning) | |
| Cooperative game theory: Shapley value, the core, side payments | Ch 12 (line-sharing agreement between separately owned farms) | |
| Stochastic and robust optimisation | Ch 11 (Monte Carlo backtest of life policies; value of re-planning); Ch 14 (scenario-based hedging, optimiser's curse, out-of-sample evaluation); Ch 15 (two-stage SP, VSS/EVPI, SAA bounds, realistic recourse); Ch 16 (robust LP with Bertsimas–Sim budgets by duality and by cutting planes, price of robustness, risk-based vs rule-based security, distributionally robust CVaR with bounded failure rates and unknown dependence, promised vs delivered); Toolbox T8 (two-stage bidding, robust windows) | |
| Power-flow modelling for optimisation: DC power flow, PTDFs, N−1 contingency rows | Ch 16 (a virtual transmission line) | |
| Decomposition at scale (Benders, Lagrangian) | Ch 10 (Lagrangian relaxation of a throughput budget, solved day by day); Ch 15 (L-shaped / Benders: 12 s vs 74 s for the extensive form); Toolbox T9 (fleet, Milestone 8) | |
| Metaheuristics, and when exact methods are better | Toolbox T10 |
Solvers and modelling frameworks¶
| Tool | Where | Status |
|---|---|---|
HiGHS through SciPy (linprog, milp) |
Chapters 1–8 | |
| Pyomo (algebraic modelling, solver-independent) | T1: Case A in Pyomo with HiGHS | |
| OR-Tools (GLOP, PDLP, CLP, CBC, SCIP; CP-SAT later) | T1; CP-SAT in T6 | |
| CBC, SCIP (MIP), PDLP (first-order LP) | T1 | |
| Gurobi, CPLEX (optional, licensed) | T1: same Pyomo model, detected and skipped when absent | |
| Solver operations (status, gaps, tolerances, duals cross-checks, version pinning) | T1; SolveReport throughout |
Data science foundations¶
| Skill | Where | Status |
|---|---|---|
| Statistical modelling: Poisson rates and intervals, Weibull, lognormal | Ch 3; Ch 16 (Clopper–Pearson bound on a failure rate, as the edge of an ambiguity set) | |
| Calibrating synthetic models to real aggregates | Ch 3, Ch 6 | |
| Probabilistic forecasts and their decision value | Ch 4, Ch 6 | |
| Backtesting without look-ahead | Ch 9 | |
| Monte Carlo scenario generation, common random numbers, bootstrap intervals | Ch 13 |
Software engineering¶
| Skill | Where | Status |
|---|---|---|
| Typed, tested Python package; tests that assert mathematics | Lab 1, tests/ |
|
| CI: lint, types, tests, notebooks executed, assets checked | .github/workflows/ |
|
| Version control and code review practice | Contributing guide | |
| Backtesting without look-ahead; benchmarks; counterfactual and Shapley attribution | Ch 9 | |
| Algorithm design: rainflow cycle counting (ASTM E1049), bisection on a monotone dual | Ch 10 | |
| Data engineering: ingestion with retries and fallback, idempotent upserts on natural keys, medallion layers (DuckDB, Parquet), lineage, freshness checks in SQL | Lab 3 | |
| APIs, containers, CD, scheduling, observability | Engineering labs 2, 4–6, Milestone 10 |
The toolbox track¶
Each toolbox chapter takes a flagship problem the reader already knows and solves it several ways:
| Chapter | Problem | Techniques compared | |
|---|---|---|---|
| T1 ✓ | One model, many solvers | Case A, shared connection | SciPy/HiGHS · Pyomo + HiGHS / CBC · OR-Tools GLOP · optional Gurobi/CPLEX |
| T2 ✓ | Does an optimum exist, and is it unique? | Case A | Infeasibility certificates · unboundedness · degeneracy and ties · ill-conditioned data · regularisation |
| T3 ✓ | Least squares and gradient descent | Power curve from SCADA | Normal equations · QR · gradient descent · momentum/Adam · LAD as an LP |
| T4 | Local, global and non-convex | Wake steering; Weibull likelihood | Multistart · convexity checks · piecewise-linear MILP · global solver |
| T5 | Convex and quadratic | BESS degradation; portfolio (Case L) | QP · conic · CVaR |
| T6 | Inside a MILP | Case C campaign; crew rostering | Formulation strength · branch and bound · cuts · CP-SAT |
| T7 | Dynamic programming and MPC | BESS state of charge | DP vs LP · rolling horizon |
| T8 | Stochastic and robust | Bidding; maintenance windows | Two-stage · scenario trees · robust counterparts |
| T9 | Decomposition | Fleet (Milestone 8) | Benders · Lagrangian relaxation |
| T10 | Heuristics, honestly | Case C at scale | Greedy · local search · when to stop |