SamlSecurityTokenHandler.ValidateToken : ID4220

Hi ! We're trying to implement some security within a NServiceBus process. So the UI gets a token from thye ADFS first. Then, whenever a persistence action is launched, we get the token from the BootstrapContext and pass it thru to the NServiceBus Host (with IMessageMutators). On the receiving side, we try to read the token to get a ClaimsPrincipal with all claims. But when I do so, I get the infamous error 

System.IdentityModel.Tokens.SecurityTokenValidationException: ID4220: The SAML Assertion is either not signed or the signature's KeyIdentifier cannot be resolved to a SecurityToken. Ensure that the appropriate issuer tokens are present on the token resolver. To handle advanced token resolution requirements, extend SamlTokenSerializer and override ReadToken. at System.IdentityModel.Tokens.SamlSecurityTokenHandler.ValidateToken(SecurityToken token) at System.IdentityModel.Tokens.SecurityTokenHandlerCollection.ValidateToken(SecurityToken token

Here's the configuration in my App.config file : 

<system.identityModel><identityConfiguration name="adfsConfig"><audienceUris mode="Never"><add value="https://proacces-dev1.universitas.ca/"/></audienceUris><certificateValidation certificateValidationMode="None"/><issuerNameRegistry><trustedIssuers><add thumbprint="67f68f65a9cbb85dd21e3718ec1a0cd4e70b3cf9" name="http://stsdev.universitas.ca/adfs/services/trust"/></trustedIssuers></issuerNameRegistry></identityConfiguration></system.identityModel><system.identityModel.services><federationConfiguration name="adfsServiceConfig" identityConfigurationName="adfsConfig"><serviceCertificate><certificateReference x509FindType="FindByThumbprint" findValue="56755fef0b9a26689713821ef3aa7798944cfe29" storeLocation="LocalMachine" storeName="My"/></serviceCertificate></federationConfiguration></system.identityModel.services>

And here's the code I use to validate the token : 

        private void ValidateToken(string token)
        {
            var identityConfig = new IdentityConfiguration("adfsConfig");
            var federationConfig = new FederationConfiguration("adfsServiceConfig")
            {
                IdentityConfiguration = identityConfig
            };

            var handlers = federationConfig.IdentityConfiguration.SecurityTokenHandlers;


            var samlSecurityToken = handlers.ReadToken(new XmlTextReader(new StringReader(token))) as SamlSecurityToken;


            if(samlSecurityToken == null)
                throw new SecurityTokenException("Impossible de lire le jeton SAML");

            var identity = handlers.ValidateToken(samlSecurityToken).FirstOrDefault();

            if (identity == null)
                throw new SecurityTokenException("Impossible de valider le jeton SAML");


            var claimPrincipal = ClaimsPrincipal.Current;


            identity.Label = Common.Constants.Securite.NomIdentitePrincipale;


            claimPrincipal.AddIdentity(identity);

        }

But it seems to crash on the server at the "handlers.ReadToken" line. Note that when I try to replicate it on my machine, it works fine (hate to say that it "works on my machine" but it's the case).

Does anyone have an idea about what could cause this ? The signin certificate is installed on the server under trusted people