The speaker forgives the beloved for being tempted by a woman, since the beloved's beauty and youth naturally attract desire; yet the beloved should have resisted for the speaker's sake.
Lines 1–8 construct an argument in which the beloved is almost not responsible for infidelity. His beauty is a natural magnet ('temptation follows where thou art'), and his gentleness makes him susceptible to seduction. More provocatively, when a woman 'woos,' no 'woman's son' (no man) will resist 'till he have prevailed'—the line suggests that masculine conquest is virtually automatic, not a choice. Lines 5–6 use a military vocabulary ('to be won,' 'to be assailed') that presents the beloved as a besieged fortress, not an autonomous agent. The beloved's body—his beauty and youth—is his doom. He is almost innocent because he is almost not-himself; he is merely an instrument of his own desirability.
Yet lines 9–14 reverse this logic abruptly. Line 9's 'Ay me, but yet' signals protest: the speaker acknowledges the beloved's circumstances but then demands resistance anyway. The beloved 'mightst my seat forbear'—could have kept the speaker's place in his heart sacred. Lines 10–12 escalate this: the beloved should have 'chide[d]' his own beauty and youth, fought against his own nature, refused to be led into 'riot' by his youth. The final couplet consolidates this: the beloved broke a 'twofold truth'—he wronged both the other woman (by seducing her with false beauty) and the speaker (by being faithless). The sonnet's conclusion is that while circumstances explain the beloved's infidelity, they do not excuse it. Love demands that we resist temptation, even when resistance is hardest.
Someone beautiful and young gets hit on constantly. You know it's not their fault that people want them. But you still wished they'd said no, if only for your sake. You understand why they fell; you just wish their love for you had been stronger than the temptation. It's not quite blame, but it's not quite forgiveness either.