Naturally this was/is an excellent question since reading about these wondrous & mystical realms is easy while writing one can be and typically is a great challenge. Yet, often enough, trying to craft your own unique system of magic that fits your story can seem like the hardest part of all. So here is a slightly expanded version of my response to Caitlin as to how I go about it. (Yes, just a slight expansion. When my friends ask for advice, I do not hold back.) She said she found it helpful, so hopefully you, dear reader, do as well.
The first thing you must decide is how ornate and prevalent you want you magic system to be in your story. Some, such as J.R.R. Tolkien, have a restrained use of magic: You do not see Gandalf casting a lot of spells or throwing fireballs in The Lord of the Rings; when danger comes, Gandalf draws Glamdring (his sword). Yet there is still a steady presence of magic in Middle-earth, subtle and mysterious yet bright, clear, and elegant. Then there are authors like Rowling, whose Harry Potter is full of flashing spells – but with carefully laid and complex rules behind and them, making the magic system work and adding realism and depth to it, albeit with lessened sense of mystery.
In general, the larger the role a magic system plays the more complex it is. Neither style is better, just different, as while the restrained kind is typically easier to craft, the often complex rules of the ornate system can help drive the story. The restrained kind often lends to a more epic feel, yet the ornate can be analyzed by the reader and bent in truly fascinating ways.
After you choose, the rest – i.e. the actual system of magic you are striving to create – depends entirely on the world and characters you have crafted. For example, you will have to decide whether your system will conform/relate to one of The Nine Magics, and how prevalent or otherwise you want arcane items to be in your tale (for this last, see my article on The Role and Proper Usage of Magic Thingamajigs).
Inspirational readings I can also offer: the first five being some of the best myth-based Fantasies around, and the bottom two being the best examples or the more ornate yet unique/excellent systems of magic (beware, the two categories often blur):
- The Sea of Trolls trilogy by Nancy Farmer (Celtic and Norse myths)
- The Secrets of the Immortal Nicholas Flamel series by Michael Scott (All myths, believe it or not)
- The Dark is Rising Sequence by Susan Copper (Arthurian legends)
- The Children of the Lamp series by P.B. Kerr (Arabian myths)
- In the Forests of Serre by Patricia A. McKillip (Slavic myths)
- The Abhorsen Series by Garth Nix
- The Vows and Honor trilogy and its companion By the Sword by Mercedes Lackey.
Beyond this there is little more I can offer save the earnest recommendation that one read the justly acclaimed Fantasy author Brandon Sanderson’s First, Second, and Third Law of Magic, for Sanderson is the genre's acknowledged Master of Magic Systems.
Advice to Young Magicians
Hence just as Tarma has some practical advice on adventuring, so too does the White Winds sorceress Lady Kethryvaris (called Kethry, Keth, and Greeneyes) of House Pheregrul wish to give some Advice to Young Magicians.
"The firebird knows your anger
And the firebird feels your fear;
For your passions will attract her
And your feelings draw her near.
But hate and fear and fury
Only make her flame and fly.
You must rule your heart, magician,
Or by her bright wings you die!
Now the cold-drake lives in silence
And he feeds on dark despair
Where the shadows fall the bleakest -
You will find the cold-drake there.
For he seeks to chill your spirit
And to lure you down to death.
Learn to rule your soul magician,
Ere you dare the cold-drakes breath!
The griffon is a proud beast,
He’s master of the sky;
And no one can forget the sight,
Who has seen the griffon fly
But his will is formed in magic
And not mortal flesh and bone -
And if you would rule the griffons will
You must first control your own!
The kyree is a creature
With a soul both old and wise.
You must never think to fool him
For he sees through all disguise.
If you seek to call a kyree
All your secrets he shall plumb.
So be certain you are worthy -
Or the kyree will not come!
Your own heart you must conquer,
If the firebird you would call,
You must know the dark within you,
Ere you seek the cold-drake’s hall.
Here is better read, magician,
Than those books upon your shelf.
If you seek to master others,
You must master first yourself!"