Recently traveled on the Disney Magic to the western Carib. Fantasy is about twice as large. Went on the Magic to your ports of call many years ago, so keep that in mind.
St. Maarten was very nice. We took a snorkeling excursion. Your kids sound a bit young for that, but that's up to you. We walked around the port area, wandered into a restaurant and casino and made a deposit. Walked the main beach area, which was nice. Since it's Dutch/French, there will be some topless sunbathing.
St. Thomas excursions were a bit expensive, so the wife and I spent the afternoon walking/eating/drinking in Charlotte Amalie. We took a cab over to the Marriott and swam. Probably would follow the earlier advice about taking off for St. John.
Castaway Cay is the highlight of most Disney cruises. Your 5 year old should have a blast. Hope it's warm enough. When we were there in December, it was chilly for swimming. One word of warning about CC and traveling on the Fantasy/Dream: With that amount of passengers, CC can get very busy, ESPECIALLY if both dock together. That seems unlikely, but it does occasionally happen. My point is that if you want a specific activity/excursion, you should book it as soon as you know. Even if the Fantasy is docking solo, you still have thousands of people competing for those slots. (This advice goes for your other ports of call as well.) Your 5 year old might like the stingray area/activity. Food there is so-so. If you are a runner and time isn't an issue with you day's activities, run in the 5k -- everyone gets a medal -- just be warned it's the same medal you can win repeatedly on the ship.
As far as the ship, I cannot speak to the Fantasy. Our family enjoyed some of the nightly shows, but not all. I would think in your situation, you might enjoy the earlier dinner seating. Also, get ready to unplug from the Internet, as access was cost prohibitive (IMO). If you need it, do it, but I didn't need it that badly. Some of the activities were quite fun, including an Amazing Race like scavenger hunt. We also all enjoyed bingo. Some of the jackpots can get quite high, so good luck on that account. They don't like to bring too much cash back to port, so if you play a few times you get a discount on the final jackpot game. You could win thousands. Most of the early, prior jackpots can run $50-a few hundred, depending on the # of winners.
Most of all, as stated above, just have fun. Your family will thank you repeatedly, I hope.