First and foremost, Jack Vance never quite followed the conventions of whichever genre he was writing in. I haven't read every single thing ever written by him (unfortunately, but I shall rectify this deficiency within a year) - but from what I have read, he never wrote anything that could be classified as "Swords & Sorcery." Jack Vance used wordplay, capricious and cruel personalities, surreal dialogue, exaggerated acts based on real events, and a number of other elements far more than he used the trappings of genres. It's almost as if it doesn't matter whether he is writing science-fiction or fantasy, as Vancian writing is like a genre unto itself. Both magic and technology alike were never the "purposes" of these stories, so much as convenient devices to move stories in certain directions and to cause certain interactions between characters.
"Vancian" magic itself is supposed to have been primarily inspired by the Dying Earth stories. In particular, The Excellent Prismatic Spray is taken directly from these stories - a spell that instantly obliterates foes with a spray of lights and sparks. Since you are all quite familiar with D&D's magic, I will focus on the portrayal of magic within the Dying Earth stories.
1. Magic was indeed memorized - but from whole books, if not multiple books per spell. The process of memorizing a spell was described as being "forced upon the mind." There was no explicit rule that a magic-user had to be well-rested before memorizing his spells, nor that they limited by day. Rather, the main limitation was the ability of the spellcaster to keep the spells memorized without error.
2. Magic was not structured in numbered levels. Indeed, there is very little indication that any particular spell could be twice as difficult to memorize as any other, for any reason other than sheer length. The vast majority of competent magicians possessed librams of The Excellent Prismatic Spray within their libraries, meaning that the equivalent of a "1st-Level Magic User" could instantly kill any one creature if given the opportunity. Suffice to say, Jack Vance did not feel that "balance" was necessary when it came to pitting ordinary men against magicians - vanity, greed, trickery and craftiness were the great equalizers between these two categories of men.
3. Magic could be practiced by anyone who was able to memorize it. Of course, if the spells were not memorized or performed correctly, they could fizzle or backfire. Even one of the lowliest of folk is able to attempt to memorize and cast a spell, and comes very close to executing it perfectly. It is for this reason that deadly man-eating creatures have not yet eaten the whole of humankind, for they greatly fear magic.
4. Arthur C. Clarke's Third Law was definitely involved with magic in the Dying Earth stories. Some magical creations are clearly the products of genetic engineering, some magical devices are actually flying cars, etc. Nevertheless, these things are indistinguishable from spells, demons, magical creatures, summoned helpers, etc. and all fall under the broad umbrella of "magic." The Excellent Prismatic Spray was just as mysterious to the common man as a flying car, even though the true workings of the former were never hinted at.
5. Magic was never as important as the people who used it. If one magician desired the secret knowledge of another, it was usually in order to satisfy some sort of vanity. For example, one magician desires to have the perfect genetically-engineered woman as his bride, and bears the malice he does towards another magician because of jealousy. A powerful magician spends more time arguing with his summoned servants than he does actually employing his powers.
6. Magic was based on an existing belief system, contrary to claims that "Vancian" magic was chosen for having no such roots. This existing belief system is named "I get what I want, when I want it." You may be intimately familiar with this belief system - it is the driving force behind the use of all magic in the Dying Earth stories.
7. All of the great inventors of spells were long since gone. By the time of the Dying Earth stories, any academic efforts were mostly focused on preserving existing lore or discovering lost lore, and the vast majority of magicians certainly were not researching anything new. The distinguishing powers of many magicians were a result of lost knowledge they had rediscovered.
I should probably also explain IOUN stones. These stones came from the remains of dead stars, and their properties were mainly to absorb and store magical energy. They did spin around the user's head, but they did not have inherent powers independent of their stored magical energy. They were able to release their stored energies in a controlled manner, making them highly valued by the great archmagicians.
No game is obliged to possess true "Vancian" magic, true IOUN stones, true elves or dwarves, etc. However, there is a great asymmetry of what the majority of D&D DMs and players think "Vancian" magic is versus what it actually was. In comparison, more than a few D&D DMs and players have done their due diligence on the Lord of the Rings novels, the Conan stories, etc. I have written this blog post in the hope that you would consider advancing Jack Vance's books on your reading lists, in order to see a Gary Gygax influence that was rather different from the others. I personally have found myself becoming an even stronger fan of the Vancian genre than the Sword & Sorcery genre.
EDIT: Just discovered http://jrients.blogspot.com/2008/06/just-so-you-know-this-is-vancian-magic.html which contains some direct quotes from the Dying Earth stories