Just a small post so I can remember this to the next time.
If you want to get rid of the warning when using published applications with Remote App on Windows Server 2012 R2 you will need to configure the following.
On the server hosting your RD stuff, start server manager and find the remote desktop section. Check that the broker publisher has been configured with the correct certificate.
Set the GPO for your users to trust the certificate used for signing the remote apps. Just start gpedit.msc and find the entry displayed below under User Configuration.