I never let any of my adventurers leave the castle WITH the princess, or the total contents of the treasury, and, for sure, no ballistas, catapults, or other siege weapons.
On the other hand, I can almost guarantee that every character I've ever played had been outfitted with a knife, a waterskin, and some rope or twine of some kind... and if they left the castle without them, they regretted it in short order.