Understanding Applicability Domain

Oct 13, 2025 by talynt.org 75 views

Introduction

Applicability domain (AD) is a crucial concept in cheminformatics and QSAR modeling, which checks whether a new molecule is within the densely sampled space near the training set. The farther out a new molecule is, the harder it is to predict it accurately. This is particularly important when internal evaluation splits like random or MaxMin are used. In this article, we will delve into the world of AD algorithms, explore their implementation, and discuss the importance of treating them similarly to filters.

Understanding Applicability Domain

Applicability domain is a measure of how well a model can predict a new molecule based on its similarity to the training set. The idea is that if a new molecule is far away from the training set, the model may not be able to predict it accurately. This is because the model has not seen molecules with similar properties before, and therefore, it may not have learned to generalize well to such cases.

Importance of Applicability Domain

Applicability domain is essential in QSAR modeling because it helps to prevent over-optimism and overfitting. When a model is trained on a dataset, it may perform well on the training set, but it may not generalize well to new, unseen data. By checking the applicability domain, we can ensure that the model is not being over-optimistic and that it is not making predictions that are far outside the range of the training data.

Algorithms for Applicability Domain

There are several algorithms available for checking the applicability domain, including:

Comparative Evaluation of Applicability Domain Definition Methods for Regression Models: This paper compares different methods for defining the applicability domain, including the use of molecular fingerprints, structural alerts, and physicochemical properties.
Applicability Domain: Towards a more formal definition: This paper proposes a formal definition of the applicability domain, which is based on the concept of a "region of interest" in the molecular space.
Applicability Domain: A Step Toward Confident Predictions and Decidability for QSAR Modeling: This paper discusses the importance of the applicability domain in QSAR modeling and proposes a method for checking the applicability domain based on the use of molecular fingerprints.

Implementation of Applicability Domain Algorithms

Implementing applicability domain algorithms is similar to implementing filters. We can either filter out molecules that are not in the applicability domain or flag them as 0/1 (0 - outside AD). The inputs to these algorithms are most often molecular fingerprints, which are a way of representing molecules as numerical vectors.

Treatment of Applicability Domain Algorithms

We should treat applicability domain algorithms similarly to filters. This means that we should either filter out molecules that are not in the applicability domain or flag them as 0/1 (0 - outside AD). We should not scale or preprocess the molecular fingerprints inside the applicability domain algorithms in most cases. Instead, the user should do this himself, but documentation should mention this.

Example Use Cases

Here are some example use cases for applicability domain algorithms:

Filtering out molecules that are not in the applicability domain: We can use an applicability domain algorithm to filter out molecules that are not in the applicability domain. This can help to prevent over-optimism and overfitting.
Flagging molecules that are outside the applicability domain: We can use an applicability domain algorithm to flag molecules that are outside the applicability domain. This can help to identify molecules that are unlikely to be predicted accurately by the model.
Checking the applicability domain of a new molecule: We can use an applicability domain algorithm to check the applicability domain of a new molecule. This can help to determine whether the model is likely to be able to predict the molecule accurately.

Conclusion

Applicability domain algorithms are an essential tool in cheminformatics and QSAR modeling. They help to prevent over-optimism and overfitting by checking whether a new molecule is within the densely sampled space near the training set. By implementing these algorithms, we can ensure that our models are making accurate predictions and that we are not over-optimistic about their performance.

Future Work

There are several areas of future work in applicability domain algorithms, including:

Development of new algorithms: There is a need for the development of new algorithms that can check the applicability domain more accurately and efficiently.
Improvement of existing algorithms: Existing algorithms can be improved by incorporating new features and by using more advanced machine learning techniques.
Evaluation of applicability domain algorithms: There is a need for the evaluation of applicability domain algorithms to determine their performance and to identify areas for improvement.

References

Comparative Evaluation of Applicability Domain Definition Methods for Regression Models: This paper compares different methods for defining the applicability domain, including the use of molecular fingerprints, structural alerts, and physicochemical properties.
Applicability Domain: Towards a more formal definition: This paper proposes a formal definition of the applicability domain, which is based on the concept of a "region of interest" in the molecular space.
Applicability Domain: A Step Toward Confident Predictions and Decidability for QSAR Modeling: This paper discusses the importance of the applicability domain in QSAR modeling and proposes a method for checking the applicability domain based on the use of molecular fingerprints.
MLChemAD library: This library provides a set of tools for checking the applicability domain, including algorithms for filtering out molecules that are not in the applicability domain and for flagging molecules that are outside the applicability domain.

Code

Here is an example of how to implement an applicability domain algorithm in Python:

import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Load the dataset
X = np.loadtxt('dataset.csv', delimiter=',')
y = np.loadtxt('target.csv', delimiter=',')

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale the molecular fingerprints
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Train a random forest classifier
clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X_train_scaled, y_train)

# Check the applicability domain of the testing set
ad = clf.predict_proba(X_test_scaled)[:, 1]

# Filter out molecules that are not in the applicability domain
filtered_X_test = X_test_scaled[ad > 0.5]

# Flag molecules that are outside the applicability domain
flagged_X_test = X_test_scaled[ad <= 0.5]

# Evaluate the performance of the model on the filtered and flagged testing sets
y_pred_filtered = clf.predict(filtered_X_test)
y_pred_flagged = clf.predict(flagged_X_test)
print('Accuracy on filtered testing set:', accuracy_score(y_test, y_pred_filtered))
print('Accuracy on flagged testing set:', accuracy_score(y_test, y_pred_flagged))

This code trains a random forest classifier on the training set and checks the applicability domain of the testing set using the predict_proba method. It then filters out molecules that are not in the applicability domain and flags molecules that are outside the applicability domain. Finally, it evaluates the performance of the model on the filtered and flagged testing sets using the accuracy_score function.

What is Applicability Domain?

Applicability domain (AD) is a measure of how well a model can predict a new molecule based on its similarity to the training set. It checks whether a new molecule is within the densely sampled space near the training set.

Why is Applicability Domain Important?

Applicability domain is essential in QSAR modeling because it helps to prevent over-optimism and overfitting. By checking the applicability domain, we can ensure that the model is not being over-optimistic and that it is not making predictions that are far outside the range of the training data.

What are the Different Types of Applicability Domain Algorithms?

There are several types of applicability domain algorithms, including:

Molecular fingerprint-based algorithms: These algorithms use molecular fingerprints to check the applicability domain.
Structural alert-based algorithms: These algorithms use structural alerts to check the applicability domain.
Physicochemical property-based algorithms: These algorithms use physicochemical properties to check the applicability domain.

How Do I Choose the Right Applicability Domain Algorithm for My Model?

Choosing the right applicability domain algorithm depends on the specific requirements of your model. You should consider the following factors:

Type of data: Different algorithms are suitable for different types of data.
Model complexity: More complex models require more sophisticated applicability domain algorithms.
Computational resources: Some algorithms are more computationally intensive than others.

Can I Use Applicability Domain Algorithms with Other Machine Learning Models?

Yes, you can use applicability domain algorithms with other machine learning models. In fact, applicability domain algorithms can be used with any machine learning model that requires predictions on new, unseen data.

How Do I Evaluate the Performance of an Applicability Domain Algorithm?

Evaluating the performance of an applicability domain algorithm involves checking its accuracy, precision, and recall. You can use metrics such as the area under the receiver operating characteristic curve (AUC-ROC) and the area under the precision-recall curve (AUC-PR) to evaluate the performance of the algorithm.

Can I Use Applicability Domain Algorithms to Filter Out Molecules That Are Not in the Applicability Domain?

Yes, you can use applicability domain algorithms to filter out molecules that are not in the applicability domain. This can help to prevent over-optimism and overfitting.

How Do I Use Applicability Domain Algorithms to Flag Molecules That Are Outside the Applicability Domain?

You can use applicability domain algorithms to flag molecules that are outside the applicability domain by setting a threshold value. Any molecule that has a predicted probability below the threshold value is flagged as being outside the applicability domain.

Can I Use Applicability Domain Algorithms to Check the Applicability Domain of a New Molecule?

Yes, you can use applicability domain algorithms to check the applicability domain of a new molecule. This can help to determine whether the model is likely to be able to predict the molecule accurately.

How Do I Implement Applicability Domain Algorithms in My Model?

Implementing applicability domain algorithms in your model involves the following steps:

Choose an applicability domain algorithm: Select an algorithm that is suitable for your model and data.
Train the algorithm: Train the algorithm on your training data.
Check the applicability domain: Use the trained algorithm to check the applicability domain of your new molecule.
Filter or flag molecules: Filter out or flag molecules that are not in the applicability domain.

What are the Common Challenges Associated with Applicability Domain Algorithms?

Some common challenges associated with applicability domain algorithms include:

Overfitting: Applicability domain algorithms can overfit to the training data, leading to poor performance on new, unseen data.
Underfitting: Applicability domain algorithms can underfit the training data, leading to poor performance on the training data.
Computational resources: Some applicability domain algorithms can be computationally intensive, requiring significant resources to train and evaluate.

How Can I Address the Challenges Associated with Applicability Domain Algorithms?

To address the challenges associated with applicability domain algorithms, you can try the following:

Use regularization techniques: Regularization techniques, such as L1 and L2 regularization, can help to prevent overfitting.
Use ensemble methods: Ensemble methods, such as bagging and boosting, can help to prevent underfitting.
Use parallel computing: Parallel computing can help to speed up the training and evaluation of applicability domain algorithms.

What are the Future Directions for Applicability Domain Algorithms?

Some future directions for applicability domain algorithms include:

Development of new algorithms: New algorithms that can check the applicability domain more accurately and efficiently.
Improvement of existing algorithms: Existing algorithms can be improved by incorporating new features and using more advanced machine learning techniques.
Evaluation of applicability domain algorithms: There is a need for the evaluation of applicability domain algorithms to determine their performance and to identify areas for improvement.