Your question is:

How do those who support the perpetual virginity of Mary doctrine harmonize the Protevangelium of James with the Gospel of Matthew?

It feels like there's an implicit assumption that those who support the Perpetual Virginity of Mary must follow the concept from the Protoevangelium of James that Joseph was an older widow with children from a previous marriage.

There's actually a different belief that many supporters of the Perpetual Virginity of Mary hold, which is that the "brothers" of Christ were cousins (rather than Joseph's children from a previous marriage).

This belief shows up in the Fragments of Papias section 10 (~100 AD) , and is first clearly articulated as an argument in Jerome's The Perpetual Virginity of Blessed Mary: Against Helvidius. section 15 (383 AD).

In brief, the belief goes like this:

• John 19:25 tells us that Mary mother of Jesus had a sister named Mary of Clopas (who was Jesus' aunt)
• Mark 15:40 (and Matthew 27:55-56) tells us that Mary of Clopas was mother of James the less and Joseph.
• Jude 1 speaks of "Jude, the brother of James". This is likely the Jude who is son of Mary of Clopas, brother of James.
• Matthew 13:55 identifies James and Joseph and Simon and Judas as 'adelphos' of Christ. We see James and Joseph are his cousins through Mary of Clopas, so it follows that Simon and Judas are likely also children of Mary of Clopas.
• This video lays out the full biblical case graphically, supplementing it with historical records

So to answer your question, many of those who support the perpetual virginity of Mary don't bother attempting to harmonize the Protoevangelium of James with the Gospel of Matthew. We see the Protoevangelium simply as an uninspired, though respected, ancient church document that we don't need to harmonize with the inspired Gospel of Matthew.

Your question is:

How do those who support the perpetual virginity of Mary doctrine harmonize the Protevangelium of James with the Gospel of Matthew?

It feels like there's an implicit assumption that those who support the Perpetual Virginity of Mary must follow the concept from the Protoevangelium of James that Joseph was an older widow with children from a previous marriage.

There's actually a different belief that many supporters of the Perpetual Virginity of Mary hold, which is that the "brothers" of Christ were cousins (rather than Joseph's children from a previous marriage).

This belief shows up in the Fragments of Papias section 10 (~100 AD) , and is first clearly articulated as an argument in Jerome's The Perpetual Virginity of Blessed Mary: Against Helvidius. section 15 (383 AD).

In brief, the belief goes like this:

• John 19:25 tells us that Mary mother of Jesus had a sister named Mary of Clopas (who was Jesus' aunt)
• Mark 15:40 (and Matthew 27:55-56) tells us that Mary of Clopas was mother of James the less and Joseph.
• Jude 1 speaks of "Jude, the brother of James". This is likely the Jude who is son of Mary of Clopas, brother of James.
• Matthew 13:55 identifies James and Joseph and Simon and Judas as 'adelphos' of Christ. We see James and Joseph are his cousins through Mary of Clopas, so it follows that Simon and Judas are likely also children of Mary of Clopas.
• This wiki lays out the full biblical case, supplementing it with historical records.

So to answer your question, many of those who support the perpetual virginity of Mary don't bother attempting to harmonize the Protoevangelium of James with the Gospel of Matthew. We see the Protoevangelium simply as an uninspired, though respected, ancient church document that we don't need to harmonize with the inspired Gospel of Matthew.