Constrained Flow Optimization via
Sequential Fine-Tuning for Molecular Design

1Department of Computer Science, ETH Zurich
2Department of Chemistry and Applied Biosciences, ETH Zurich
3ETH AI Center
4NCCR Catalysis
*Equal contribution
Accepted at ICML 2026
Constrained generative optimization via fine-tuning.

CFO fine-tunes a pre-trained flow model to maximize a reward while provably satisfying arbitrary expected-value constraints — without manual Lagrange-multiplier tuning.

Abstract

Adapting generative foundation models, in particular diffusion and flow models, to optimize given reward functions (e.g. binding affinity) while satisfying constraints (e.g. molecular synthesizability) is fundamental for their adoption in real-world scientific discovery applications such as molecular design or protein engineering. While recent works have introduced scalable methods for reward-guided fine-tuning of such models via reinforcement learning and control schemes, it remains an open problem how to algorithmically trade-off reward maximization and constraint satisfaction in a reliable and predictable manner. Motivated by this challenge, we first present a rigorous framework for Constrained Generative Optimization, which brings an optimization viewpoint to the introduced adaptation problem and retrieves the relevant task of constrained generation as a sub-case. Then, we introduce Constrained Flow Optimization (CFO), an algorithm that automatically and provably balances reward maximization and constraint satisfaction by reducing the original problem to sequential fine-tuning via established, scalable methods. We provide convergence guarantees for constrained generative optimization and constrained generation via CFO. Ultimately, we present an experimental evaluation of CFO on both synthetic, yet illustrative, settings, and a molecular design task. Across these evaluations, CFO achieves consistent increases in reward while ensuring high constraint satisfaction, showcasing its practical utility for constrained generative optimization.

Constrained Generative Optimization

Given a pre-trained flow (or diffusion) model with terminal distribution $\ppre_1$ over a space $\Xcal$, a scalar reward $r : \Xcal \to \mathbb{R}$, and a scalar constraint $c : \Xcal \to \mathbb{R}$ with bound $B \in \mathbb{R}$, we look for a fine-tuned policy $\pi$ whose induced distribution $p^\pi_1$ solves:

Constrained Generative Optimization via Flow Fine-Tuning
$$ \begin{aligned} \operatorname*{arg\,max}_\pi \;\; & \Esymb_{x \sim p^\pi_1}[r(x)] \;-\; \alpha\,\DKL\!\big(p^\pi_1 \,\|\, \ppre_1\big) \\ \text{s.t.} \;\; & \Esymb_{x \sim p^\pi_1}[c(x)] \;\le\; B \end{aligned} $$

The formulation is fully agnostic: no differentiability of $r$ or $c$ is required at the problem level. Setting $r \equiv 0$ recovers the classical task of constrained generation; for non-trivial $r$ it becomes a constrained optimization problem on top of a pre-trained generative model.

A natural first attempt is the fixed-weight Lagrangian $\Lcal_\mu(\pi) = \Esymb[r(x)] - \alpha\,\DKL(p^\pi_1\|\ppre_1) - \mu\,\big(\Esymb[c(x)] - B\big)$ with $\mu \ge 0$. This fails in practice for three reasons:

  • No feasibility certificate. The optimum is feasible only above an unknown, problem-dependent threshold on $\mu$.
  • Non-monotone in $\mu$. There is no guaranteed map from $\mu$ to the resulting constraint violation, so trial-and-error is needed.
  • Reward hacking. When $r$ is unbounded or approximate (a learned proxy), the optimum can move mass to high-reward, infeasible regions.

CFO removes these failure modes by adaptively updating both a Lagrange multiplier and a penalty.

The CFO Algorithm

CFO solves the constrained problem above as a sequence of unconstrained, entropy-regularized fine-tuning subproblems via an augmented Lagrangian scheme. Intuitively, CFO embeds the constraint into an augmented reward via two adaptive dual variables — the penalty $\rho_k$ and the Lagrange multiplier $\lambda_k$ — and at every outer iteration steps a standard fine-tuning solver (e.g. Adjoint Matching, DiffusionNFT, Flow-GRPO) toward feasibility while improving reward.

Algorithm 1 — Constrained Flow Optimization (CFO)
  1. Input: pre-trained model $\ppre$; iterations $K$; safeguard $\lambda_{\min} < 0$; initial penalty $\rho_\text{init} > 0$; growth rate $\eta \ge 1$; contraction value $0 < \tau < 1$.
  2. Initialize: $\lambda_1 = 0$, $\rho_1 = \rho_\text{init}$.
  3. for $k = 1,\,2,\,\dots,\,K$ do
    Step 1. Form the augmented objective $$f_k(x) \;=\; r(x) \;-\; \tfrac{\rho_k}{2}\left[\max\!\Big(0,\; c(x) - B - \tfrac{\lambda_k}{\rho_k}\Big)\right]^{\!2}.$$
    Step 2. Fine-tune via an inner solver $$\pi_k \;\gets\; \Solver(f_k,\,\ppre).$$
    Step 3. Estimate gap & contraction $$G_k = \Esymb_{x\sim p^{\pi_k}_1}[c(x)] - B,\qquad V_k = \min\{G_k,\,-\lambda_k/\rho_k\}.$$
    Step 4. Update the Lagrange multiplier $$\lambda_{k+1} \;\gets\; \max\!\Big\{\lambda_{\min},\; \min\{0,\, \lambda_k - \rho_k G_k\}\Big\}.$$
    Step 5. Update the penalty $$\rho_{k+1} \;=\; \begin{cases} \rho_k, & \text{if } k = 1 \text{ or } V_k \le \tau\,V_{k-1}, \\ \eta\,\rho_k, & \text{otherwise.} \end{cases}$$
  4. end for
  5. Return $\pi_K$.

Step by Step

Step 1 — Augmented reward

We form $f_k = r - \tfrac{\rho_k}{2}[\max(0,\, c - B - \lambda_k/\rho_k)]^{2}$: the task reward minus a shifted, quadratic constraint penalty. The non-negative shift $-\lambda_k/\rho_k$ moves the penalty threshold toward the current expected constraint, so moderate penalties suffice and inner subproblems stay well-conditioned.

Step 2 — Inner fine-tuning

Any off-the-shelf entropy-regularized fine-tuning oracle maximizes the augmented reward: $\pi_k \in \arg\max_\pi\, \Esymb_{x\sim p^\pi_1}[f_k(x)] - \alpha\,\DKL(p^\pi_1\|\ppre_1)$. In our experiments we instantiate the oracle with Adjoint Matching, and demonstrate that CFO is solver-agnostic by also plugging in the gradient-free DiffusionNFT.

Step 3 — Empirical gap and progress

With samples from $p^{\pi_k}_1$ we estimate the constraint gap $G_k = \Esymb[c(x)] - B$ and a contraction statistic $V_k = \min\{G_k,\, -\lambda_k/\rho_k\}$ that measures whether the iterate is genuinely tightening on feasibility (rather than artificially relaxing via a large shift).

Step 4 — Projected dual update

The Lagrange multiplier is updated by a projected step $\lambda_{k+1} = \Pi_{[\lambda_{\min},\,0]}(\lambda_k - \rho_k G_k)$. If the constraint is violated ($G_k > 0$), $\lambda_{k+1}$ becomes more negative and the penalty sharpens. If the constraint is satisfied ($G_k < 0$), $\lambda_{k+1}$ relaxes toward $0$ — reward gets more weight.

Step 5 — Penalty schedule

We grow $\rho$ only when needed: if $V_k$ fails to contract by a factor $\tau$, the penalty is scaled by $\eta > 1$; otherwise it is held fixed. This avoids unnecessary stiffness while guaranteeing the penalty is eventually large enough to enforce feasibility.

Theoretical Guarantees

Under a mild approximate-solver assumption on the inner oracle, the iterates produced by CFO inherit classical augmented-Lagrangian guarantees:

  • Feasibility (Thm. 1). Any limit point $\bar{\pi}$ of $\{\pi_k\}$ minimizes infeasibility: $\max\{0, G(\bar{\pi})\} \le \max\{0, G(\pi)\}$ for all $\pi$. Hence if a feasible policy exists, $\bar{\pi}$ is feasible (Cor. 1).
  • Optimality (Thm. 2). If additionally the solver error vanishes ($\epsilon_k \to 0$), then any limit point is a global maximizer of the constrained problem.

Critically, the formulation requires no differentiability of the reward or constraint at the problem level — any such requirements are inherited only from the chosen inner solver. This means CFO directly composes with gradient-free fine-tuners such as DiffusionNFT or Flow-GRPO.

Results

2D illustration — constrained optimization on a mixture of Gaussians

On a 2-D mixture of Gaussians with a reward (distance to the white target) and a hard constraint (samples must lie inside the red triangles), CFO with either inner solver — gradient-based Adjoint Matching (AM) or gradient-free DiffusionNFT — concentrates mass in the feasible regions while pushing reward up. The unconstrained baselines alone improve reward but violate the constraint.

Pre-trained samples

Pre-trained

CFO+AM samples

CFOAM (ours)

Adjoint Matching samples

Adjoint Matching

CFO+NFT samples

CFONFT (ours)

$\Esymb[r(x)]\ \uparrow$ $\Esymb[c(x)]\ \downarrow$
Pre-trained $-7.62 \pm 0.03$ $0.58 \pm 0.07$
CFOAM (ours) $-4.75 \pm 0.04$ $0.12 \pm 0.06$
Adjoint Matching $-2.93 \pm 0.03$ $2.47 \pm 0.11$
CFONFT (ours) $-5.28 \pm 0.14$ $0.06 \pm 0.01$
DiffusionNFT $-3.59 \pm 0.10$ $1.76 \pm 0.05$

Constrained generation — dialling the bound $B$

Setting $r \equiv \text{const}$ recovers classical constrained generation. By varying the constraint bound $B$, CFO smoothly controls how strictly the resulting flow model respects the constraint region (red triangle) — a single dial for the feasibility/coverage trade-off.

Pre-trained Gaussian

Pre-trained

CFO B=0

CFO with $B = 0$

CFO B=1

CFO with $B = 1$

Molecular design — energy-constrained dipole maximization

Applied to FlowMol pre-trained on GEOM-Drugs, CFO maximizes the molecular dipole moment (reward) while keeping the total xTB energy inside an $-80$ Ha bound (constraint). With either Adjoint Matching or gradient-free DiffusionNFT as inner solver, CFO drives the expected energy inside the feasible region — a boundary that neither baseline crosses on its own — while attaining comparable dipole moment.

CFOAM — Adjoint Matching as inner solver (gradient-based)

CFO+AM dipole trajectory

Reward (dipole, D) ↑

CFO+AM energy trajectory

Constraint (energy, Ha) — bound at $-80$

CFONFT — DiffusionNFT as inner solver (gradient-free)

CFO+NFT dipole trajectory

Reward (dipole, D) ↑

CFO+NFT energy trajectory

Constraint (energy, Ha) — bound at $-80$

Pre-trained CFOAM AM CFONFT DiffusionNFT
$\Esymb[r(x)]\ \uparrow$ $6.55_{\pm 0.07}$ $8.39_{\pm 0.10}$ $8.37_{\pm 0.07}$ $8.27_{\pm 0.19}$ $8.30_{\pm 0.07}$
$\Esymb[c(x)]\ (\le -80)$ $-77.86_{\pm 0.22}$ $-82.28_{\pm 0.41}$ $-78.31_{\pm 0.38}$ $-80.72_{\pm 0.93}$ $-78.67_{\pm 0.62}$

Numeric evaluation (95% CI). Green satisfies the $-80$ Ha energy bound; red violates it. Both CFO variants satisfy the constraint, neither baseline does.

Generated molecules

Drug-like samples generated by the CFO-fine-tuned model, annotated with ground-truth dipole and energy.

Generated molecule a

$15.7$ D / $-86.9$ Ha

Generated molecule b

$9.1$ D / $-83.4$ Ha

Generated molecule c

$12.5$ D / $-93.8$ Ha

Pareto — CFO vs. manually tuned fixed $\mu$

Compared against 18 fixed-$\mu$ Adjoint-Matching baselines (sweeping $\mu \in [10^{-6}, 10^{6}]$), only two baselines simultaneously reach CFO-level dipole and satisfy the energy constraint. Manual tuning is both expensive and unreliable — CFO finds the right operating point automatically.

Pareto plot CFO vs fixed mu
Dipole distribution

BibTeX

@inproceedings{gutjahr2026constrained,
  title  = {Constrained Flow Optimization via Sequential Fine-Tuning for Molecular Design},
  author = {Gutjahr, Sven and De Santi, Riccardo and Schaufelberger, Luca and Jorner, Kjell and Krause, Andreas},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year   = {2026}
}