I don't know Hebrew, so I can't offer a whole lot of guidance here, but it seems the apparent contradiction could be a result of different language paradigms.
It is interesting that the Hebrew texts for the verses seem to use the same word for "fool" and "not a fool".
The grammatical structure and conjugation is baffling to me, but according to the Blue Letter Bible:
Proverbs 26:4 reads as follows:
אַל־תַּעַן כְּסִיל כְּאִוַּלְתֹּו פֶּֽן־תִּשְׁוֶה־לֹּו גַם־אָֽתָּה׃
where:
- ענה - Answer
- כסיל - not a fool
- אולת - according to his folly
- שוה - lest thou also be like unto him
Proverbs 26:5 reads as follows:
עֲנֵה כְסִיל כְּאִוַּלְתֹּו פֶּן־יִהְיֶה חָכָם בְּעֵינָֽיו׃
where
- ענה - Answer
- כסיל - a fool
- אולת - according to his folly
- חכם - lest he be wise
- עין - to his own conceit
I think Mike's answer is a good one. The Proverbs provide wise topical advice using a pithy and often symmetrical or somewhat poetic format. The two verses together seem to be warning that it's not wise to stoop to a fool's level while answering his folly, but that it's also not wise to let a fool's folly go totally unchecked. It seems to utilize a bit of equivocation in what it means to answer him "according to his folly" (i.e. identify his bad assumptions or conclusions and try to correct them, but don't be baited into arguing foolishly yourself)