The beloved's heart contains the ghosts of all the speaker's past loves; the beloved has become a living tomb and treasury of everyone the speaker has ever cherished.
The image of the beloved as a 'grave where buried love doth live' collapses the boundary between death and life, tomb and temple. This is not necrophilia but a metaphysical claim: past loves are not erased but preserved alive within the beloved's being. The beloved becomes a monument ('hung with trophies'), a reliquary, a space where all the speaker's previous emotional attachments are simultaneously honored and transcended. The paradox is profound: the beloved is simultaneously a grave (deadness) and a womb (renewal). All prior loves exist within the beloved not as ghosts but as active, constituent presences.
Line 12's 'That due of many, now is thine alone' marks the complete transfer of the speaker's affective estate: what was once scattered among many people now concentrates entirely in a single beloved. The final line—'And thou, all they, hast all the all of me'—multiplies pronouns in a dizzying way: you (beloved) contain them (all past loves), and therefore own me (the speaker) completely. The speaker's identity dissolves into the beloved's identity. There is no self left separate from the beloved; the 'I' has been entirely absorbed into the 'thou.' This is devotion taken to its logical—and unsettling—extreme.
You've loved several people—friends who moved away, relationships that ended, relatives who passed. Then you meet someone who reminds you of all their best qualities, and you realize you can love all those people again through loving this one person. They become the container for every attachment you've ever made, and somehow that makes the past connections feel not dead but continuously alive.