Yeah if it is just the wallet exploding, it would be more of a nuisance than a danger. Who is carrying their hardware wallet on their person all the time? 

Keep your keys separate and restore to another device.