O lovely boy, you hold Time's glass and paradoxically grow lovelier even as you wane; yet Nature, though she retains you now, must eventually render you back to Time.
The first four couplets establish a stunning paradox: the beloved holds 'Time's fickle glass his fickle hour,' yet 'hast by waning grown.' He becomes more lovely precisely as he ages. His lovers wither while he grows. This suggests that the beloved is somehow outside normal temporal flow—he alone escapes Time's usual erosion. Nature ('sovereign mistress over wrack') has made him the exception to mortality's rule, keeping him 'to this purpose, that her skill / May time disgrace, and wretched minutes kill.' The beloved becomes Nature's weapon against Time.
Yet the final couplets deliver the envoy's harsh truth: 'Yet fear her O thou minion of her pleasure, / She may detain, but not still keep her treasure!' Nature's preservation is temporary. Her 'audit (though delayed) answered must be,' and 'her quietus is to render thee.' Quietus is the final accounting, the literal death-sentence. The beloved must eventually be rendered back to Time. The Fair Youth sequence closes not with triumph but with the knowledge that even the beloved, however preserved by Nature, will eventually be claimed by mortality. This is the formal farewell: you cannot escape time, and after you, the sequence turns to darker, more troubled devotion.
Someone stays beautiful and seems frozen in time while everyone around them ages. But eventually everyone pays the same price to time—there's no exception, no perpetual youth. Nature can delay the reckoning, but it can't cancel it.