Fias Co is a good place to get some basics and start your reading, although some of the stuff there is a little off my preference scale as I continue to learn.
I would recommend looking for breeders in your area and joining a few mini-goat clubs, if you can. Try google! We have a great club up here called Alaska Mini Goat Cache and it is just great for beginners and old hands, alike. Most small clubs let non-members come to meetings and will mentor newbies exploring the idea of getting into goats, even if it turns out not to be your thing. Many members will invite you to their farm for a visit too! The good thing is that you can ask a lot of questions and make informed decisions before being disappointed or unprepared. Nothing worse than getting all excited about something that doesn't pan out the way you thought it would!
In what breeds are you interested? For what purpose do you want them (milk, pets, meat, etc.)? In the US, ADGA and AGS only officially recognize one type of mini DAIRY goat, the Nigerian Dwarf, and they must be purebred (no crossing) to be recognized at least by ADGA. You can get information on breeders in your area from ADGA and AGS and there is another registry for them too, called NDGA. In the US, one other common true mini is the pygmy and they are most commonly used as meat and pet goats. Some are used for milk but they are not recognized by either association as dairy goats. If none of that matters, there is a huge growing world of crossing these true mini breeds with other large breeds to produce minis (like mini Nubians, etc.) but they are NOT recognized by ADGA and I understand that it is actually against the rules, as an ADGA member, to do such crossing. If you don't care about all the association recognitions, you have a lot of choices. Just be sure you don't ever want to go down the road of showing, going on milk test, and appraising before you leap in that direction. It may happen one day but it would be an uphill battle to get the associations to change their minds.
Again, try using Google for breeders and certainly check websites for those of us here that have miniature goats (as well as links to the breeders where our goats originated).

Mine came from
Silveraurora. Be sure to visit the links at the bottom of her page where you can see her bucks, kids, senior does, first fresheners, kidding schedule, and goats for sale. She doesn't have all her goats on there but she has a good representation on the ND breed. My website doesn't yet have much on goats.