Age-optimal Packet Scheduling with Resource Constraint and Feedback Delay

This paper considers the design of age-optimal packet scheduling policies in networks with delayed feedback, long-term resource constraint and various packet arrival models. The problem is formulated as a constrained partial observable Markov decision process (CPOMDP) in general. We first derive the achievable age of information (AoI) by the random and determined transmission policies, which works in the absence of feedback. Then, we propose a greedy policy that traces the expected receiver AoI based on the delayed feedback, and selects the action that minimizes the expected immediate Lagrange cost defined in the formulated CPOMDP. The proposed policy outperforms the existing policies for a wide range of feedback delay, even in the absence of feedback, yet the implementation complexity is still low since the decision threshold is derived in the closed-form, as a function of the network parameters.