A palace is the official residence of a head of state, head of church or dignitary, especially monarchical or imperial system of government. A palace is not as secure as a castle as it is not fortified against attacks. It is designed for leisure.
A castle is a large building that is fortified and contains many defenses such as thick walls, massive towers, and so on. Castles are often built of bricks or stone for strength. A nobleman or a king often inhabit it. They are created to fight battles and to withstand the attacks of enemies.