Should I normalize SOAP vectors before applying kernels? Also getting 'ValueError: Input X contains NaN.'

[alejandroarche]

Hello!

I’m using SOAP descriptors for building similarity and wondering if I should explicitly normalize each SOAP vector before computing the kernel. Some references suggest dividing by the vector norm (or normalizing the kernel), but it’s not obvious from the DScribe documentation whether this is done automatically. Your example does the following:

a_features = desc.create(a)
b_features = desc.create(b)
re = AverageKernel(metric="linear")
re_kernel = re.create([a_features, b_features])

Would it be better to normalize the SOAP vectors beforehand? Something like:

soap_list_norm = []
for a_feat, b_feat in zip(a_features, b_features):
    a_norm = np.linalg.norm(a_feat, axis=1, keepdims=True)
    b_norm = np.linalg.norm(b_feat, axis=1, keepdims=True)
    
    soap_list_norm.append(a_feat / a_norm)
    soap_list_norm.append(b_feat / b_norm)

I’m asking because I frequently encounter the error: ValueError: Input X contains NaN when using the REMatchKernel. This might indicate an issue with descriptor generation or a division by zero in the kernel computation. Do you have any guidelines on best practices for avoiding NaNs in DScribe?

Thanks for your help!

Best,
Alejandro

[lauri-codes]

Hi @alejandroarche!

AverageKernel has a parameter normalize_kernel which defaults to True. So normalization is done by default, but you can turn it off:

re = AverageKernel(metric="linear", normalize_kernel=False)

I think normalization is a sensible thing to do by default, but you can make the final decision yourself.

As for the NaNs: you will have to investigate yourself a bit how and where the NaNs end up being produced. One option is indeed that one of your vectors is completely empty. There is not really any best practices I could advise on this this: typically NaNs indicate a problem in your data that you need to react to.

[alejandroarche]

Hi @lauri-codes,

I've noticed that the problem is indeed not with the normalisation of feature vectors. I'm computing the kernel in two different ways:

option 1:

avg_kern = AverageKernel(metric="linear")
K = avg_kern.create(soap_list)

option 2:

avg_kern = REMatchKernel(metric="linear", alpha=2, threshold=1e-3)
K = avg_kern.create(soap_list)

For option 1, each C_ij = C_ij_dict[i, j] is computed just as the mean in get_global_similarity as K_ij = np.mean(localkernel), and that's why there are no problems. However, in option 2, get_global_similarity is not simply the mean and it is exactly in its second line K = np.exp(-(1 - localkernel) / self.alpha) where K becomes a vector of inf values. This is expected when localkernel is a big number because the exp(-(1-x)/alpha) explodes very quickly. I believe that in NumPy the maximum exponent you can safely compute is around 700. So, any vector component that is over 700, will trigger an inf and therefore a nan in the following line pity = np.multiply(np.multiply(K, u.reshape((-1, 1))), v).

This seems strange to me, as the values of a soap-generated feature vector can easily exceed these magnitudes, and the REMatch kernel will result in a matrix full of nan values. That's why one approach was to normalise the vectors, but I doubt that is mathematically accurate within the SOAP kernel framework.

Any thoughts overall?

Best,
Alejandro