In the Elder Days, all Windows applications used MME for sound. The operating system would create a multimedia engine; the sound card would hook up to one side of it, and the host application to the other. The host app would be able to do basically nothing other than send a data stream to the engine, and the OS would send that information to the card. All this system overhead leads to rather high latencies, often well over 500 milliseconds.
Recently, some smart people have used faster machines and some heavy driver tweaking to get this much lower, but even under the most ideal circumstances and all else being equal, MME drivers have higher latencies than every other type of driver covered here. In addition, MME drivers use an OS level engine to deal with the audio streams, so by nature they do not allow a host application to control any additional features on a sound card (like a synth or a DSP chip) through the MME driver.
So, why do they even still exist? There are a few reasons. First, according to developers I talked with, they are quite simple to write. They take very little CPU overhead to work, and there is very wide support for them. So, MME is a quick and easy way to get audio in and out of a computer, it's just not an ideal way to be working when you are recording music or working with software synthesis. But, everything has its place. By the way, when the term "wave driver" is used, it usually means MME driver, though some people will on occasion use the term "wave driver" to mean any sound driver that is a Microsoft driver, including DirectSound drivers. This can lead to all manner of confusion, of course, but in general "wave driver" = MME driver.
Often called DirectX drivers, these are some of the most common drivers available today. DirectX offers a good deal to the home recordist. DirectX drivers for sound offer many of the same advantages of DirectX drivers for video cards. First, by allowing the vendor much more direct access to the hardware on the sound card, drivers can be optimized for much better performance. The second big advantage of DirectSound is that it has mixing capabilities built it. This means that multiple apps can easily use the same card, which can be extremely convenient. Another mark in favor or DirectSound is that nearly every audio recording app supports some level of DirectX, so nearly any app will be able to use a card with DirectSound.
Latency with DirectSound drivers varies wildly with the card and driver. It can be among the lowest or quite high, depending on the quality of the hardware used, and the time spent by the vendor working on DirectX performance. It's certainly worth gathering real world information about latency from current users with similar configurations to yours (both software and hardware) before deciding to use a card with DirectX drivers. The news can be very bad or very good.
Finally, by allowing the host application much more direct access to hardware than MME, DirectSound drivers can also control some amount of DSP on a sound card from the host application. Again, this mirrors the advantages of DirectX in video drivers that might be more familiar to you. DirectSound, like MME, makes no special accommodations for professional level synchronization between apps or machines, leaving the user at the mercy of MIDI Time Code. Not the best sync option in the world, but also eminently useable.