Grendel casts a spell on their weapon which makes it useless to it against him. So to defeat him, they have to engage him in a hand in hand combat which unfortunately, none of them can withstand him except for Beowulf..
However, in the course of the final battle between Grendel and Beowulf, the battle concludes with Beowulf's ripping Grendel's arm off, fulfilling his promise to Hrothgar to kill the monster with his bare hands. Beowulf nails Grendel's severed arm to the mead hall rafters and, although Grendel runs away, he is later found to have bled to death in his underwater lair.