Your purse holds the amount of gold pieces that you have, excluding any gold that you hid in a bag or other container. The amount of gold in your purse appears on the status line, or you can count it by pressing [$]. Your purse is available for making payments (to shopkeepers, to priests, to bribe demons, for example).
Note that the purse is not an actual item in NetHack. It is not visible in your inventory (though the gold in it is), you cannot drop it, and it cannot be taken from you. Nor does such an object appear anywhere else in the game. It is just a device to explain how the adventurer manages to keep their finances organized.
When a leprechaun steals your gold, the message is, "Your purse feels lighter."
When you are in situations where you might lose your gold (like fighting leprechauns, reading spellbooks, or sitting on thrones), then you can protect your gold by first removing it from your purse. If you have a bag, then you could hide your gold in there. You could also stash your gold into a chest or large box, or just drop or throw it on the floor (Note that if you drop gold, a leprechaun can still steal it by grabbing it from under your feet). An exception to normal purse behavior is when a vault guard demands that you drop all of the gold that you are carrying. The guard is able to sense if you have gold stored in a container and will attack if you do not remove the gold and drop it.