There is really only speculation as to why there is no Nobel prize in math. One of the most widely known and believed reasons is that Alfred Nobel decided not to include a prize in math for personal and petty reasons. These reasons are unfounded but it has circulated that a woman he either proposed to, married, or had as a mistress either rejected him or cheated on him with a famous mathematician named.
It is often said that Gosta Mittag-Leffler is the guilty party in the situation. There is no actual factual evidence to back up these claims even though they are the most widely believed reasoning for the exclusion of a Nobel Prize in Mathematics.