Our priest does such a great job at our church. I shook his hand after church and said "Bless you". I meant to say "Bless you for all you do", but was it offensive to bless a priest?
"Bless you!" is short for "[May God] bless you!" So, you are not blessing a priest, but asking God to.
I think what you did is rather good. Even pope Francis 1st, just elected, asked everybody to bless him:
And now I would like to give the blessing, but first - first I ask a favour of you: before the Bishop blesses his people, I ask you to pray to the Lord that he will bless me: the prayer of the people asking the blessing for their Bishop. Let us make, in silence, this prayer: your prayer over me.