The only metadata they can get from Signal is frequency of messages and the size of the message files. Everything else is encrypted. There's a feature with Signal called Sealed Sender, it makes it so anyone looking at the data being exchanged can't know who messages came from. This means that unless you and your contact are both active targets of the government, they really can't do much with that minimal metadata... Or, at least, nothing they can't do with other options. 

Now, if you need total anonymity due to being a whistleblower or something like that, I'd definitely recommend Session instead, as it's anonymous, has almost no visible metadata and is onion routed. It's not great for a daily option, however, thus why Signal is the best option for the average person.