Optimal sequencing of heterogeneous, non-instantaneous interventions

We consider a stochastically deteriorating system that generates reward at a rate that depends on its condition. There are three heterogeneous interventions/actions that can be performed. Two of these are one-time interventions, each of which is either ineffective or effective with some known probability after a specific delay period. The third intervention is palliative in nature (i.e., does not affect the deterioration process), and has no effectiveness delay. To maximize the total expected reward generated by the system, we examine the problem of optimally sequencing these interventions to simultaneously balance three inherent trade-offs: the time until (potential) effectiveness is revealed, probability of effectiveness, and cost. This problem is motivated by decision-making in the treatment of chronic diseases where providers must determine the order in which to prescribe treatment options (e.g., medications) with uncertain outcomes. We provide both theoretical conditions and numerical examples that indicate when it is optimal to reserve the palliative intervention as a last resort and in what order to implement the other two interventions.