We raised Katahdin hair sheep. We tried Barbados (another hair breed) but they were too skittish. I made a point to handle all my lambs a lot, not hard to do as they're quite adorable and daily walked around my flock with treats in my hand, so my girls were very tame. We did a few Dorper/Katahdin crosses, and I have no complaints about them, but my real love was with the Katahdins. They're beautiful, nice disposition, excellent, excellent meat, hardy and just so easy to care for. I don't know if this is true or not, but my understanding is that hair sheep are less muttony tasting due to the lack of lanolin amounts the wool sheep have.
Personally, as with all meat, I think it has more to due with what they eat.
As for goats, we raise Nubians. We're just getting back into them, but raised them some years back. I love the breed, they are almost puppy like in attitude. Their milk is very good, very high in butterfat. If you want to up the meat amount in them, I understand that a Nubian/Boer cross makes both an excellent dairy and meat animal.
We keep ours for dairy production, and basically only butcher the young bucklings because I can't tolerate the chance of someone falling in love with how cute they are when little, buying one and later abusing it because they're a handful and stink.
I think if I had to make a choice between goats and sheep it would have to be based more on my lifestyle than which might be better. If you worked outside the home, and have a busy life, I'd go with sheep. If you or someone is on the farm all day working it, I'd probably choose goats.
On the other hand, if you keep your buck/ram separate from the girls, the goats and sheep can run together on pasture or in the woods just fine together, but you need to separate them at feeding time! The have somewhat different nutritional needs, goats need copper, copper is DEADLY to sheep.
Another plus on the sheep (at least Katahdins) is that they're naturally polled. I dearly hate disbudding goat kids, but fencing has made that a necessity with them, that or risk them getting their heads stuck in it and a dog/coyote coming along.
You might consider starting with 2 each does/ewes, spend some time getting to know them and make a choice from there.
Both can provide meat and milk. It really is a matter of preference.