How do you handle the milk? Do you sterilize your milking equipment?
I only milk into semless stainless steel or glass. All buckets and jars are scrubbed with dairy soap, get an acid wash 3x's a week, and a chlorine rinse. You could use dish soap, scrub with vinegar to remove any milk stone (rinse very well afterwards), then soak a bit in a 10 parts water, 1 part bleach to sterilize. Air dry.
At milking, wash udder well, dry, then empty first few squirts into a separate container to check for lumps/strings/blood. If all clear, milk into your sterile jar/bucket. Make sure your hands are clean prior to milking (I take a tub of Wetones antibacterial hand wips to use prior to milking....can also use them to clean the udder too).
Get your milk inside as soon as possible, filter & chill. I use a ss milk filter, but you can sterilize bits of cloth (have used cut up old tshirts before that I washed and boiled). Filter milk into a sterilized jar (I like the 1/2 gallon Ball jars with the plastic lids) & place in freezer for at least an hour. The cleaner you are when handling milk & the quicker you get it ice cold, the better the flavor & longer the shelf life. I usually put 1/2 gallon jars full in the freezer for 2 hours then store in the back, bottom of the fridge where it is coldest. Handle your milk gently, don't go shaking up the jars or anything.
If you havent, I'd trim the belly hair, udder & insides of back legs to get as much debri holding long hair away from your "milking area".
If not mastitis, goaty milk is often caused from poor handling. Good sanitation, filtering & rapid chilling can go a long way in helping with the flavor.
Don't forget to use a good teat dip/spray after milking.
As to possible pregnancy,
Home | BioTracking.com offers pregnancy testing for $6.50.