It seems more likely that Paul was referring to James the Just (Christ's familial "brother" or possibly a "brother" in name only), but we have no definitive way of knowing this.
- There is no agreement on whether or not Christ had a familial brother named James. Below is an excerpt from a commentary, "The Brothers and Sisters of Jesus: Anything New?" from Francois Rossier, at the University of Dayton, in Ohio (pdf).
"...it cannot be said that the New Testament identifies them without doubt as blood brothers and sisters and hence as children of Mary... It remains true that the word "brother," in Hebrew, also means "blood sibling." Since it is the most obvious--but by far not the exclusive--meaning, it cannot be simply dismissed. The use of the word adelphos remains, therefore, a challenge for those who uphold Mary's virginity post partum. Yet, reading this word as referring only to actual siblings may also raise some difficulties."
As your passage indicates, he appeared to "five hundred brothers" but Christ didn't have 500 familial brothers. This became even more apparent in our lifetimes when the James Ossuary was allegedly discovered and it created quite a bit of controversy. Below is the famous inscription found on the ossuary. According to a 2012 article in Biblical Archaeology Review (pdf), it reads: "James, son of Joseph, brother of Jesus."
- There were two apostles named James, not one: James, the son of Zebedee and James, the son of Alphaeus (often referred to as James the Less or James the Younger).
"Now the names of the twelve apostles are these; The first, Simon, who is called Peter, and Andrew his brother; James the son of Zebedee, and John his brother; Philip, and Bartholomew; Thomas, and Matthew the publican; James the son of Alphaeus, and Lebbaeus, whose surname was Thaddaeus; Simon the Canaanite, and Judas Iscariot, who also betrayed him." (Matthew 10:2-4, KJV)
Typically, James the Just is known as the familial "brother" of Christ but we don't have proof of this and these terms are debatable.
If Christ appeared to one of the apostles named James and not James -a familial brother of Christ- it's more likely that Paul would have specified which one since there were two.
Because of the way the passage reads, it's more likely that Paul meant a James that did have a familial relationship with Christ OR that he meant a James that was distinctive for another reason (there is none that we know of). I say this because he is setting apart James from "all of the apostles."
In 1 Corinthians 15:7, the Codex Sinaiticus, the oldest manuscript we have of Corinthians, shows their website translating this passage even more definitively.
"After that he appeared to James, after that to all the apostles."
- There is no record that I am aware of that links Paul to any James other than James the Just -head of the church at Jerusalem. If it was not James the Just (Christ's familial "brother"), this would have been a unique reference from Paul.
"And when we were come to Jerusalem, the brethren received us gladly. And the day following Paul went in with us unto James; and all the elders were present. And when he had saluted them, he declared particularly what things God had wrought among the Gentiles by his ministry." (Acts 21:17-19, KJV)
- The son of Alphaeus is typically known as James the Less (or James the Younger.)
"There were also women looking on afar off: among whom was Mary Magdalene, and Mary the mother of James the less and of Joses, and Salome." (Mark 15:40, KJV)
In 2006, Pope Benedict addressed an audience in the Vatican city that, according to Catholic Online, was "Contributed to Integrate the Original Jewish Dimension of Christianity." His entire address is about James the Less and it can be found here: (pdf)
From Pope Benedict's address:
"...another James appears in the gospel who is called James "the Less." He also forms part of the list of twelve apostles chosen personally by Jesus, and is always specified as "son of Alphaeus."