RDKit blog

GPU-Accelerated Clustering with nvMolKit

Fri, 27 Feb 2026 23:00:00 GMT

GPU-Accelerated Molecular Clustering with nvMolKit

Note: This is a guest post written by Kevin Boyd at NVIDIA.

nvMolKit is an open-source library that provides GPU-accelerated implementations of common RDKit cheminformatics operations. The APIs closely mirror RDKit’s, with batching as needed for GPU efficiency, making it easy to drop nvMolKit into existing workflows while achieving significant speedups on large datasets.

As of January 2026, nvMolKit v0.3 supports: - Batch Morgan fingerprint calculation - Many to many Tanimoto and Cosine similarity calculations - Butina clustering - Batch ETKDG conformer generation - Batch MMFF force field optimization

In this post, we’ll demonstrate nvMolKit’s capabilities through a molecular clustering workflow. We’ll compute Morgan fingerprints, calculate pairwise Tanimoto similarities, and perform Butina clustering, with both the RDKit and nvMolKit APIs.

Installation

nvMolKit is available on conda-forge alongside RDKit. nvMolKit requires an NVIDIA GPU with compute capability 7.0 (V100) or higher. Some nvMolKit results are on the GPU, a CUDA-compatible torch installation can be used to interpret results.

!conda install -c conda-forge rdkit nvmolkit -y -q

Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

import numpy as np
import torch
assert torch.cuda.is_available(), "CUDA is required for nvMolKit"

Configuration

We’ll process 20,000 molecules from Enamine REAL for this comparison.

# Number of molecules to process
n_mols = 20000

# CPU threads for multithreaded RDKit operations
n_cpu_threads = 16

# Morgan fingerprint parameters
fp_radius = 3
fp_nbits = 1024

# Butina clustering threshold
distance_threshold = 0.6

Loading Molecules

First, we’ll parse molecules from SMILES using RDKit. We’ll take the first molecules from the Enamine Real 10.4M sample

import pandas as pd
from rdkit.Chem import MolFromSmiles

smi_file = "enamine_real_10M.csxmiles"

smis = pd.read_csv(smi_file, nrows=n_mols, usecols=[0], sep='\t').iloc[:, 0].to_list()
mols = [MolFromSmiles(smi) for smi in smis]
mols = [mol for mol in mols if mol]  # Remove any parse failures
n_mols = len(mols)
print(f"Parsed {n_mols} molecules")

Parsed 20000 molecules

Workflow Description

This example workflow consists of 3 steps: 1. Fingerprinting: Generate Morgan fingerprints 2. Similarity: Compute pairwise Tanimoto distances 3. Clustering: Run Butina algorithm on the distance matrix

RDKit Workflow

import time
from rdkit.Chem import rdFingerprintGenerator
from rdkit.DataStructs import BulkTanimotoSimilarity
from rdkit.ML.Cluster.Butina import ClusterData

# Step 1: Fingerprinting
t_start = time.time()
generator = rdFingerprintGenerator.GetMorganGenerator(radius=fp_radius, fpSize=fp_nbits)
fps = generator.GetFingerprints(mols, numThreads=n_cpu_threads)
t_fp = time.time() - t_start
print(f"Fingerprinting: {t_fp:.2f}s")

# Step 2: Pairwise Tanimoto distances
t_start = time.time()
distances = []
for i in range(len(mols)):
    distances.extend(BulkTanimotoSimilarity(fps[i], fps[:i], returnDistance=True))
t_sim = time.time() - t_start
print(f"Similarity matrix: {t_sim:.2f}s")

# Step 3: Butina clustering
t_start = time.time()
rdkit_clusters = ClusterData(
    np.array(distances),
    n_mols,
    distance_threshold,
    isDistData=True,
    distFunc=None,
    reordering=True
)
t_clust = time.time() - t_start
print(f"Butina clustering: {t_clust:.2f}s")

rdkit_total_time = t_fp + t_sim + t_clust
print(f"\nTotal RDKit time: {rdkit_total_time:.2f}s")

Fingerprinting: 0.04s
Similarity matrix: 11.17s
Butina clustering: 7.65s

Total RDKit time: 18.85s

nvMolKit Workflow

Note that we add synchronizes to delimit individual timings, but the entire workflow can be run asynchronously with a single sync at the end

from nvmolkit.fingerprints import MorganFingerprintGenerator
from nvmolkit.similarity import crossTanimotoSimilarity
from nvmolkit.clustering import butina

# Step 1: Fingerprinting
t_start = time.time()
nvmolkit_fpgen = MorganFingerprintGenerator(radius=fp_radius, fpSize=fp_nbits)
nvmolkit_fps = nvmolkit_fpgen.GetFingerprints(mols, num_threads=n_cpu_threads)
torch.cuda.synchronize()
t_fp_nv = time.time() - t_start
print(f"Fingerprinting: {t_fp_nv:.2f}s")

# Step 2: Pairwise Tanimoto similarity -> distances
t_start = time.time()
nvmolkit_similarities = crossTanimotoSimilarity(nvmolkit_fps)
torch.cuda.synchronize()
t_sim_nv = time.time() - t_start
print(f"Similarity matrix: {t_sim_nv:.2f}s")

# Step 3: Butina clustering
t_start = time.time()
nvmolkit_distances = 1.0 - nvmolkit_similarities.torch()
cluster_ids = butina(nvmolkit_distances, distance_threshold).torch()
torch.cuda.synchronize()
t_clust_nv = time.time() - t_start
print(f"Butina clustering: {t_clust_nv:.2f}s")

nvmolkit_total_time = t_fp_nv + t_sim_nv + t_clust_nv
print(f"\nTotal nvMolKit time: {nvmolkit_total_time:.2f}s")

Fingerprinting: 0.05s
Similarity matrix: 0.02s
Butina clustering: 0.02s

Total nvMolKit time: 0.09s

Performance Comparison - RTX 5080 GPU vs Ryzen 9 9950X

Let’s visualize the speedup for each step of the workflow.

import matplotlib.pyplot as plt
import numpy as np

steps = ['Fingerprinting', 'Similarity', 'Clustering', 'Total']
rdkit_times = [t_fp, t_sim, t_clust, rdkit_total_time]
nvmolkit_times = [t_fp_nv, t_sim_nv, t_clust_nv, nvmolkit_total_time]

x = np.arange(len(steps))
width = 0.35

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))

# Time comparison
bars1 = ax1.bar(x - width/2, rdkit_times, width, label='RDKit', color='#1f77b4')
bars2 = ax1.bar(x + width/2, nvmolkit_times, width, label='nvMolKit', color='#2ca02c')
ax1.set_ylabel('Time (seconds)')
ax1.set_title('Execution Time by Step')
ax1.set_xticks(x)
ax1.set_xticklabels(steps)
ax1.legend()
ax1.set_yscale('log')
ax1.grid(axis='y', alpha=0.3)

# Speedup
speedups = [r/n for r, n in zip(rdkit_times, nvmolkit_times)]
colors = ['#ff7f0e' if s > 1 else '#d62728' for s in speedups]
bars3 = ax2.bar(steps, speedups, color=colors)
ax2.axhline(y=1, color='gray', linestyle='--', alpha=0.7)
ax2.set_ylabel('Speedup (RDKit time / nvMolKit time)')
ax2.set_title('nvMolKit Speedup vs RDKit')
ax2.grid(axis='y', alpha=0.3)

# Add speedup labels
for bar, speedup in zip(bars3, speedups):
    ax2.text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.5, 
             f'{speedup:.1f}x', ha='center', va='bottom', fontweight='bold')

plt.tight_layout()
plt.show()

print(f"\nOverall speedup: {rdkit_total_time / nvmolkit_total_time:.1f}x")


Overall speedup: 218.8x

RDKit fingerprinting benefits from being the lowest compute workload and from native RDKit multithreading. On this hardware, the performance is about equivalent. We get large speedups from putting similarity and clustering on the GPU. On NVIDIA datacenter GPUs, speedups can be well over 1000x.

Comparing Clustering Results

nvMolKit clusters may or may not be identical to RDKit, but are valid Butina clusters. Potential differences can happen when there are multiple candidate clusters of the same size.

# Convert cluster IDs to cluster tuples for comparison
n_clusters = cluster_ids.max().item() + 1
nvmolkit_clusters = [tuple(torch.where(cluster_ids == i)[0].tolist()) for i in range(n_clusters)]
print(f"Number of clusters: {len(nvmolkit_clusters)}")

rdkit_cluster_sizes = sorted([len(c) for c in rdkit_clusters], reverse=True)
nvmolkit_cluster_sizes = sorted([len(c) for c in nvmolkit_clusters], reverse=True)

plt.figure()
plt.plot(range(len(rdkit_cluster_sizes)), rdkit_cluster_sizes, 
         label='RDKit', linewidth=2.5, alpha=0.9)
plt.plot(range(len(nvmolkit_cluster_sizes)), nvmolkit_cluster_sizes, 
         label='nvMolKit', linewidth=2.5, linestyle='--', alpha=0.9)
plt.xlabel('Cluster Rank')
plt.ylabel('Cluster Size (molecules)')
plt.title('Cluster Size Distribution')
plt.legend()
plt.ylim(0, max(rdkit_cluster_sizes) * 1.05)

plt.tight_layout()
plt.show()

print(f"RDKit: {len(rdkit_clusters)} clusters, largest has {max(rdkit_cluster_sizes)} molecules")
print(f"nvMolKit: {len(nvmolkit_clusters)} clusters, largest has {max(nvmolkit_cluster_sizes)} molecules")

Number of clusters: 14164

RDKit: 14162 clusters, largest has 14 molecules
nvMolKit: 14164 clusters, largest has 14 molecules

Links

For more information, see the nvMolKit documentation and GitHub repository.

Creating a patent data set

Fri, 23 Jan 2026 23:00:00 GMT

For a while I’ve been meaning to put together a collection of sets of related compounds for use in teaching and some other experiments. I’ll do that by taking compounds from patent data found in ChEMBL.

I could also have worked with SureChEMBL for this, but ChEMBL is easier for me to work with (since I always have a copy of ChEMBL arounnd) and contains enough patent data for my purposes. I will probably also take advantage of the fact that there is bioactivity data associated with these compounds in ChEMBL at some point in the future.

In this post I put together the queries, grab the data, and do a bit of initial exploration. I’ll do more with these in the future

If you’re interested in the data sets themselves, you can find them in the source blog repo.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole
import rdkit
import pandas as pd
from matplotlib import pyplot as plt
plt.style.use('tableau-colorblind10')

%load_ext sql
%matplotlib inline
%config SqlMagic.feedback=0

Creating the data sets

Look at the number of compounds in the ChEMBL patents:

%sql postgresql://localhost/chembl_36 \
  select patent_id,chembl_id,count(distinct molregno) mrn_count from docs join compound_records using (doc_id) \
   where patent_id is not null \
   group by (patent_id,chembl_id)\
   order by mrn_count desc limit 10;

patent_id	chembl_id	mrn_count
US-9718790-B2	CHEMBL5725546	2236
US-20130252896-A1	CHEMBL4420062	1901
US-11286268-B1	CHEMBL5729040	1775
US-8466108-B2	CHEMBL4419170	1689
US-11618753-B2	CHEMBL5729657	1530
US-9302989-B2	CHEMBL3886617	1518
US-9796708-B2	CHEMBL5726550	1352
US-11124486-B2	CHEMBL5728711	1227
US-10774051-B2	CHEMBL5727936	1204
US-9303033-B2	CHEMBL3886195	1175

Truncated to displaylimit of 10.

I’m not looking for patents with that many compounds… let’s limit it to between 50 and 300:

%sql postgresql://localhost/chembl_36 \
  select count(*) from (select patent_id,chembl_id,count(distinct molregno) mrn_count\
                        from docs join compound_records using (doc_id) \
                        where patent_id is not null \
                        group by (patent_id,chembl_id)\
                        order by mrn_count desc) tmp \
    where mrn_count>50 and mrn_count<300;

count
2721

That’s a lot

Create a temporary table with the count of the number of unique compounds in each of the patents that contains between 50 and 300 compounds.

%sql postgresql://localhost/chembl_36 \
  drop table if exists patents

%sql postgresql://localhost/chembl_36 \
  select * into temporary table patents from \
    (select patent_id,chembl_id,doc_id,count(distinct molregno) mrn_count\
     from docs join compound_records using (doc_id) \
     where patent_id is not null \
     group by (patent_id,chembl_id,doc_id)\
     order by mrn_count desc) tmp \
  where mrn_count>50 and mrn_count<300;

%sql postgresql://localhost/chembl_36 \
   select count(*) from patents;

count
2721

%sql postgresql://localhost/chembl_36 \
   select count(*) from (select distinct on (doc_id) doc_id from patents) tmp;

count
2721

For each of those patents, get the counts of the number of activity values for each single-protein assay in the patent. Also pulls info about the assay itself

%sql postgresql://localhost/chembl_36 \
  drop table if exists patent_all_targets

%sql postgresql://localhost/chembl_36 \
   select assays.chembl_id assay_chembl_id,target_dictionary.chembl_id target_chembl_id,\
          pref_name,target_type,count(distinct molregno) act_count, tid, assay_id,assays.doc_id\
   into temporary table patent_all_targets \
   from patents join assays using (doc_id) join activities using (assay_id) \
       join target_dictionary using (tid) \
   where pchembl_value is not null and target_type='SINGLE PROTEIN'\
   group by (assays.chembl_id,target_dictionary.chembl_id,pref_name,target_type,tid, assay_id,assays.doc_id) \
   order by act_count desc;

%sql postgresql://localhost/chembl_36 \
   select count(*) from (select distinct on (doc_id) doc_id from patent_all_targets) tmp;

count
2334

Get the most populated target for each patent. We will call this the target for the patent (though this is clearly a crude assumption).

The distinct on (doc_id) thing combined with the order by doc_id, act_count desc is the way to get the row for each doc_id that has the highest act_count in postgresql.

%sql postgresql://localhost/chembl_36 \
  drop table if exists patent_targets

%sql postgresql://localhost/chembl_36 \
   select distinct on (doc_id) \
      assay_chembl_id,target_chembl_id,pref_name,act_count,tid,assay_id,doc_id \
    into temporary table patent_targets \
    from patent_all_targets order by doc_id, act_count desc;

%sql select count(distinct target_chembl_id) from patent_targets;

count
641

Find the most popular targets and the number of patents assigned to each:

%sql postgresql://localhost/chembl_36 \
 select target_chembl_id,pref_name,count(distinct doc_id) num_patents,tid \
  from patent_targets \
  group by (target_chembl_id,pref_name,tid) \
  order by num_patents desc limit 10;

target_chembl_id	pref_name	num_patents	tid
CHEMBL5251	Tyrosine-protein kinase BTK	61	100097
CHEMBL6136	Lysine-specific histone demethylase 1A	56	102439
CHEMBL1741186	Nuclear receptor ROR-gamma	40	103982
CHEMBL2000	Plasma kallikrein	32	23
CHEMBL1163125	Bromodomain-containing protein 4	30	103454
CHEMBL4409	cAMP and cAMP-inhibited cGMP 3',5'-cyclic phosphodiesterase 10A	29	100010
CHEMBL2815	High affinity nerve growth factor receptor	28	11902
CHEMBL3130	Phosphatidylinositol 4,5-bisphosphate 3-kinase catalytic subunit delta isoform	26	11177
CHEMBL6175	Lysine-specific demethylase 4C	25	102452
CHEMBL2973	Rho-associated protein kinase 2	24	11149

Truncated to displaylimit of 10.

d = %sql postgresql://localhost/chembl_36 \
 select target_chembl_id,num_patents from \
    (select target_chembl_id,pref_name,count(distinct doc_id) num_patents,tid \
     from patent_targets \
     group by (target_chembl_id,pref_name,tid) ) tmp\
  where num_patents >20 \
  order by num_patents desc;
d

target_chembl_id	num_patents
CHEMBL5251	61
CHEMBL6136	56
CHEMBL1741186	40
CHEMBL2000	32
CHEMBL1163125	30
CHEMBL4409	29
CHEMBL2815	28
CHEMBL3130	26
CHEMBL6175	25
CHEMBL2973	24

Truncated to displaylimit of 10.

len(d)

Write out the compounds for those 13 targets:

%config SqlMagic.named_parameters="enabled" 
for cid,_ in d:
    targetd = %sql postgresql://localhost/chembl_36 \
      select patent_id, patents.chembl_id patent_chembl_id, chembl_id_lookup.chembl_id compound_chembl_id,\
           canonical_smiles \
        from patents join patent_targets using (doc_id) \
         join compound_records using (doc_id) join molecule_hierarchy using (molregno) \
         join compound_structures cs on (cs.molregno=parent_molregno) \
         join chembl_id_lookup on (parent_molregno=entity_id and entity_type='COMPOUND')\
        where target_chembl_id=:cid
    
    targetd = targetd.DataFrame()
    targetd.to_csv(f'../data/patent_datasets/target_{cid}.csv',index=False)

!head ../data/patent_datasets/target_CHEMBL1163125.csv

patent_id,patent_chembl_id,compound_chembl_id,canonical_smiles
US-8975417-B2,CHEMBL3638496,CHEMBL3650854,COc1ccc(Cn2nc3c(c2C)C(c2ccc(Cl)cc2)N(c2ccc(=O)n(C)c2)C3=O)cc1
US-8975417-B2,CHEMBL3638496,CHEMBL3650855,Cc1[nH]nc2c1C(c1ccc(Cl)cc1)N(c1ccc(=O)n(C)c1)C2=O
US-8975417-B2,CHEMBL3638496,CHEMBL3650856,Cc1c2c(nn1C)C(=O)N(c1ccc(=O)n(C)c1)C2c1ccc(Cl)cc1
US-8975417-B2,CHEMBL3638496,CHEMBL3650857,CC1=NN(C)C2C(=O)N(c3ccc(=O)n(C)c3)C(c3ccc(Cl)cc3)C12
US-8975417-B2,CHEMBL3638496,CHEMBL3650858,Cc1[nH]nc2c1C(c1ccc(Cl)cc1)N(c1cc(Cl)c(=O)n(C)c1)C2=O
US-8975417-B2,CHEMBL3638496,CHEMBL3650859,Cc1c2c(nn1C)C(=O)N(c1cc(Cl)c(=O)n(C)c1)C2c1ccc(Cl)cc1
US-8975417-B2,CHEMBL3638496,CHEMBL3650860,Cc1c2c(nn1C)C(=O)N(c1cc(Cl)c(=O)n(C)c1)[C@@H]2c1ccc(Cl)cc1
US-8975417-B2,CHEMBL3638496,CHEMBL3650861,Cc1c2c(nn1C)C(=O)N(c1cc(Cl)c(=O)n(C)c1)[C@H]2c1ccc(Cl)cc1
US-8975417-B2,CHEMBL3638496,CHEMBL3650862,Cn1cc(N2C(=O)c3n[nH]cc3C2c2ccc(Cl)cc2)cc(Cl)c1=O

Looking at the some results with UMAP

Let’s do a bit of visualization workn by grabbing the data for a single target:

target_id = 'CHEMBL5251'
%sql postgresql://localhost/chembl_36 \
  drop table if exists patent_compounds

%sql postgresql://localhost/chembl_36 \
  select patent_id,patents.chembl_id patent_chembl_id,chembl_id_lookup.chembl_id compound_chembl_id,canonical_smiles \
    into temporary table patent_compounds \
    from patents join patent_targets using (doc_id) \
     join compound_records using (doc_id) join molecule_hierarchy using (molregno) join compound_structures cs on (cs.molregno=parent_molregno) \
     join chembl_id_lookup on (parent_molregno=entity_id and entity_type='COMPOUND')\
    where target_chembl_id=:target_id;

%sql select count(*) from patent_compounds

count
8450

%sql select * from patent_compounds limit 10;

patent_id	patent_chembl_id	compound_chembl_id	canonical_smiles
US-8618107-B2	CHEMBL3639120	CHEMBL3670078	Cc1c(-c2cc(Nc3cc4n(n3)CCN(C)C4)c(=O)[nH]n2)cccc1N1CCn2c(cc3c2CCCC3)C1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670079	Cc1c(-c2cc(Nc3cc4n(n3)CCN(C)C4)c(=O)[nH]n2)cccc1N1CCn2c(cc3ccccc32)C1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670080	Cc1c(-c2cc(Nc3cc4n(n3)CCN(C)C4)c(=O)n(C)n2)cccc1N1Cc2c(sc3ccccc23)C1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670081	Cn1cc(-c2cccc(N3CCc4c(sc5c4CCCC5)C3=O)c2CO)cc(Nc2ccncn2)c1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670082	Cn1cc(-c2cccc(N3CCc4c(sc5c4CC(C)(C)C5)C3=O)c2CO)cc(Nc2ccncn2)c1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670083	Cn1cc(-c2cccc(N3CCn4c(cc5c4CCCC5)C3=O)c2CO)cc(Nc2cc(C3CC3)n[nH]2)c1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670084	Cn1cc(-c2cccc(N3CCn4c(cc5c4CCCC5)C3=O)c2CO)cc(Nc2ccc(N3CCOCC3)cn2)c1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670085	CC(=O)N1CCn2nc(Nc3cc(-c4cccc(N5CCn6c(cc7c6CCCC7)C5=O)c4CO)cn(C)c3=O)cc2C1
US-8618107-B2	CHEMBL3639120	CHEMBL3670086	Cn1cc(-c2cccc(N3CCn4c(cc5c4CCCC5)C3=O)c2CO)cc(Nc2ccncn2)c1=O
US-8618107-B2	CHEMBL3639120	CHEMBL3670087	Cn1cc(-c2cccc(N3CCn4c(cc5c4CCCC5)C3=O)c2CO)cc(Nc2cc3n(n2)CCOC3)c1=O

Truncated to displaylimit of 10.

Start with a subset of 12 patents, sorted by number of compounds they contain.

We will have a bunch of duplicate compounds. For this exercise, pick each compound only once:

d = %sql \
  select distinct on (compound_chembl_id) * from patent_compounds \
   join (select patent_id from patent_compounds group by patent_id \
         order by count(distinct compound_chembl_id) desc \
         limit 12) tmp \
   using (patent_id)\
   order by compound_chembl_id, patent_id;

df = d.DataFrame()
df.head()

	patent_id	patent_chembl_id	compound_chembl_id	canonical_smiles
0	US-10828300-B2	CHEMBL5728085	CHEMBL2216827	C=CC(=O)Nc1cccc(Nc2nc(Nc3ccc(Oc4ccnc(C(=O)NC)c...
1	US-10828300-B2	CHEMBL5728085	CHEMBL3301625	C=CC(=O)Nc1cccc(Nc2nc(Nc3ccc(OCCOC)cc3)ncc2F)c1
2	US-9447106-B2	CHEMBL3886959	CHEMBL3889791	C=CC(=O)Nc1cc(C2CCNc3c(C(N)=O)c(-c4ccc(Oc5cccc...
3	US-9447106-B2	CHEMBL3886959	CHEMBL3890068	C=CC(=O)Nc1ccccc1C1CCNc2c(C(N)=O)c(-c3ccc(OCCO...
4	US-9447106-B2	CHEMBL3886959	CHEMBL3890197	NC(=O)c1c(-c2ccc(Oc3ccccc3)cc2)nn2c1NC(=O)C21C...

df.shape,len(set(df['canonical_smiles']))

((1656, 4), 1656)

Add fingerprints:

df['Mol'] = df['canonical_smiles'].apply(Chem.MolFromSmiles)

from rdkit.Chem import rdFingerprintGenerator
fpg = rdFingerprintGenerator.GetMorganGenerator(radius=3,fpSize=2048)
df['cmfp3'] = df['Mol'].apply(fpg.GetCountFingerprintAsNumPy)

Now do UMAP using the Dice similarity metric. I did a bit of exploration to come up with the values of n_neighbors and min_dist used here.

import umap
fitter = umap.UMAP(n_neighbors=8,min_dist=0.4,metric='dice',n_jobs=4)
pts = fitter.fit_transform(df['cmfp3'].tolist())
df['X'] = pts[:,0]
df['Y'] = pts[:,1]

/home/glandrum/mambaforge/envs/rdkit_blog/lib/python3.12/site-packages/sklearn/utils/deprecation.py:151: FutureWarning: 'force_all_finite' was renamed to 'ensure_all_finite' in 1.6 and will be removed in 1.8.
  warnings.warn(
/home/glandrum/mambaforge/envs/rdkit_blog/lib/python3.12/site-packages/umap/umap_.py:1887: UserWarning: gradient function is not yet implemented for dice distance metric; inverse_transform will be unavailable
  warn(

And do a scatter plot of the results:

plt.figure(figsize=(6,6))
markers = 'ovs*+d'
for i,pid in enumerate(set(df['patent_chembl_id'])):
    tdf = df[df['patent_chembl_id']==pid]
    plt.scatter(tdf['X'],tdf['Y'],label=pid,marker=markers[i%len(markers)],s=20,linewidth=0);

Look at a couple of different targets together

dfs = []
for target_id in ('CHEMBL3864','CHEMBL3130','CHEMBL4409'):
    %sql postgresql://localhost/chembl_36 \
      drop table if exists patent_compounds

    %sql postgresql://localhost/chembl_36 \
      select patent_id,patents.chembl_id patent_chembl_id,chembl_id_lookup.chembl_id compound_chembl_id,canonical_smiles \
        into temporary table patent_compounds \
        from patents join patent_targets using (doc_id) \
         join compound_records using (doc_id) join molecule_hierarchy using (molregno) join compound_structures cs on (cs.molregno=parent_molregno) \
         join chembl_id_lookup on (parent_molregno=entity_id and entity_type='COMPOUND')\
        where target_chembl_id=:target_id;
    d = %sql \
  select distinct on (compound_chembl_id) * from patent_compounds \
   join (select patent_id from patent_compounds group by patent_id \
         order by count(distinct compound_chembl_id) desc \
         limit 12) tmp \
   using (patent_id)\
   order by compound_chembl_id, patent_id;
    df = d.DataFrame()
    df['target_chembl_id'] = target_id
    dfs.append(df)
hybrid = pd.concat(dfs)

hybrid.shape

(4642, 5)

hybrid['Mol'] = hybrid['canonical_smiles'].apply(Chem.MolFromSmiles)
fpg = rdFingerprintGenerator.GetMorganGenerator(radius=3,fpSize=2048)
hybrid['cmfp3'] = hybrid['Mol'].apply(fpg.GetCountFingerprintAsNumPy)

fitter = umap.UMAP(n_neighbors=8,min_dist=0.4,metric='dice',n_jobs=4)
pts = fitter.fit_transform(hybrid['cmfp3'].tolist())
hybrid['X'] = pts[:,0]
hybrid['Y'] = pts[:,1]

/home/glandrum/mambaforge/envs/rdkit_blog/lib/python3.12/site-packages/sklearn/utils/deprecation.py:151: FutureWarning: 'force_all_finite' was renamed to 'ensure_all_finite' in 1.6 and will be removed in 1.8.
  warnings.warn(
/home/glandrum/mambaforge/envs/rdkit_blog/lib/python3.12/site-packages/umap/umap_.py:1887: UserWarning: gradient function is not yet implemented for dice distance metric; inverse_transform will be unavailable
  warn(
/home/glandrum/mambaforge/envs/rdkit_blog/lib/python3.12/site-packages/umap/spectral.py:548: UserWarning: Spectral initialisation failed! The eigenvector solver
failed. This is likely due to too small an eigengap. Consider
adding some noise or jitter to your data.

Falling back to random initialisation!
  warn(

plt.figure(figsize=(6,6))
markers = 'ovs*+d'
for i,pid in enumerate(set(hybrid['patent_chembl_id'])):
    tdf = hybrid[hybrid['patent_chembl_id']==pid]
    plt.scatter(tdf['X'],tdf['Y'],label=pid,marker=markers[i%len(markers)],s=20,linewidth=0);

Color by target id instead of patent:

plt.figure(figsize=(6,6))
markers = 'ovs*+d'
for i,tid in enumerate(set(hybrid['target_chembl_id'])):
    d = %sql postgresql://localhost/chembl_36 \
      select pref_name from target_dictionary where chembl_id=:tid
    print(f'{tid}: {d[0][0]}')
    tdf = hybrid[hybrid['target_chembl_id']==tid]
    for j,pid in enumerate(set(tdf['patent_chembl_id'])):
        pdf = tdf[tdf['patent_chembl_id']==pid]
        plt.scatter(pdf['X'],pdf['Y'],label=pid,marker=markers[j%len(markers)],s=20,linewidth=0,c=f'C{i}');

CHEMBL4409: cAMP and cAMP-inhibited cGMP 3',5'-cyclic phosphodiesterase 10A
CHEMBL3130: Phosphatidylinositol 4,5-bisphosphate 3-kinase catalytic subunit delta isoform
CHEMBL3864: Tyrosine-protein phosphatase non-receptor type 11

That’s enough for now… these compound sets are sure to show up in future blog posts.

Wrapping up 2025

Fri, 26 Dec 2025 23:00:00 GMT

For some random reason, at the beginning of 2025 I decided that I was going to do an RDKit blog post every week in 2025. It was a struggle at times, but if you include updates of old blog posts (which I definitely do!), I made it! :tada: For the last post of the year, I’m going to do a short look back at 2025.

The RDKit

For me the biggest RDKit thing of each year is the UGM, and in 2025 there were actually two. The first North American edition of the UGM took place in April in Cambridge, MA. I didn’t manage the travel for that, but it sounded like it went really well and was a great success. Here’s hoping that we can make that a regular thing! The European UGM took place in September in Prague. I did a recap post back in September, but the UGM was great. I love the chance to spend a couple of days surrounded by the RDKit community. Registration for the 2026 European UGM, which will take place from 16-18 September in Darmstadt, Germany, will open early next year.

Some numbers

(GitHub could make this easier…):

two major releases
ten minor releases (well, nine so far, the tenth comes early next week, before the year ends)
348 commits
The two largest contributions (in terms of lines of code) were the SCSR parser from Tad Hurst at CDD and the CDX/CDXML parser from Brian Kelley at Glysade (using code contributed by Revvity)
83 contributors (counting pull requests and resolved issues and enhancements)
147 closed bugs
122 completed enhancements
39 cleanup PRs
17 documentation PRs
I’d include something here about the number of discussions posts, but I’m not willing to invest the time to figure out the GraphQL API for doing that.

Things that should go better

Reviewing pull requests, particularly larger ones, can take much longer than it really should. We need to figure out a better way to handle these.

Next year I would like to be a bit better at paying attention to the Discussions tab in GitHub. I’m not 100% sure that GitHub Discussions is the best option for questions and general discussion, so I’m also trying an experiment with zulip. If you’re interested, this link should work to allow you to join automatically for the next 30 days. There’s a chicken and egg problem with this (we need participants for it to be useful, but it needs to be useful to attract participants), but we’ll see how it goes.

Because of all the other stuff going on, I didn’t manage to complete (or even make a lot of progress on) some of my longer-term RDKit projects this year; let’s see what 2026 brings there.

Personal things

Its my blog, so I’ll go ahead and include some non-RDKit stuff too.

According to Garmin I have (so far, there are still a few days left in the year) run 2563km with about 71km of elevation gain (that doesn’t count elevation when doing uphill treadmill workouts). Over the summer I ran two ultra-marathons: a 77km race that I finished and a 170km race that I DNF’ed after 100km (it turns out that running that distance is even more complicated than I thought it would be). Getting ready for those took a lot of time, but much of it was fun days moving “fast” in the mountains, so I enjoyed it. I also ran my usual stage for the research group’s team in the Zurich SOLA relay race and managed a PR there. Finally, at the end of the year I set myself the random goal of running a fast 5km (not a race, just a fast time on a stretch I run a few times a week). I didn’t manage to hit my time goal of 20 minutes, but I only missed by 16 seconds, so I’m still pretty happy with the time. It was definitely entertaining to change things up and do some focused training for going faster.

I hiked 653km with about 40km of elevation gain; this is down from a normal year because of the amount of running and because our summer vacation didn’t end up going as planned.

The climbing doesn’t lend itself to quantification (at least not the way we do it) but we ended up doing a fair amount this year. I did considerably less bouldering than last year since I mainly went running on the days I normally would have gone to the bouldering gym, so I didn’t make as much progress on technique as last year, but it looks like I didn’t lose too much.

Ok, it’s time to wrap this up and get back to my holiday project.

About tetrahedral chirality in the RDKit

Sat, 20 Dec 2025 23:00:00 GMT

This week, to try and find inspiration for a short, or at least quick, blog post, I did an “AI Search” and asked for some frequently un-answered RDKit questions. One of those was related to understanding stereochemistry. Then, this morning, I noticed a question in GitHub Discussions asking why the chirality on a tertiary amine wasn’t being preserved. My post topic was clear!

The contents of this post will end up, in an edited form, in the RDKit Book

from rdkit import Chem
from rdkit.Chem.Draw import IPythonConsole
IPythonConsole.molSize = 300,250

Intro

In this post I am going to focus solely on stereochemistry/chirality around tetrahedral atoms. I am aware that there are non-tetrahedral forms of stereochemistry for three- and four-coordinate atoms (and the RDKit supports some of them). Non-tetrahedral stereochemistry, double-bond stereochemistry, and atropisomerism are topics for a possible future post.

What can be chiral?

I will borrow the language used in the InChI technical manual and use the term “stereogenic” to refer to atoms which can be chiral if all of their substituents are different. The RDKit uses the same conditions to determine which atoms can be involved in ring stereochemistry (see below).

If you ever want to go straight to the source to see what could be a potential stereocenter, the function to look for is isAtomPotentialChiralCenter() .

In general, the following conditions have to be met for an atom to be considered a potential stereocenter: - It must have a total nonzero-degree of three or four. I.e. three or four neighbors not connected via zero-order bonds. - It cannot have two attached hydrogen atoms (explicit or implicit). For the purposes of this count, only H atoms with unspecified isotopes count. - Three coordinate atoms cannot have an H atom attached, except for phosphines, arsines, S and Se with an explicit valence of four, and S+ and Se+ with an explicit valence of three. - Degree three N atoms cannot have an H atom attached and must be involved in either a three-membered ring or be a bridgehead

Because arbitrary decisions have to be made when dealing with this topic - for example, are tertiary amines stereogenic? - we choose to (try and) follow InChI and use Table 8 from the InChI technical manual here. If an element is not present in that InChI table, we use the general rules above. One notable exception for main-group atoms is that the RDKit allows trivalent N atoms that are in a bridgehead to be chiral centers.

An example chiral phosphine (invented):

Chem.MolFromSmiles('C12C[N@](C)C(C2)C[P@H]1')

That really should have the [H] drawn as a separate atom

Notice that, though the chirality on the N was specified in the input, it was removed when the molecule was parsed: the N is neither in a three-membered ring nor a bridgehead.

Here’s an example of a ChEMBL molecule with a chiral bridgehead N:

Chem.MolFromSmiles('COC(=O)[C@@H]1C[N@@]2CCC[C@@H]1C2')

The RDKit’s representation of atomic stereochemistry

IPythonConsole.drawOptions.addAtomIndices = True
m = Chem.MolFromSmiles('C[C@](C(=O)O)(CCC)CC')
m

We can see the internal representation in the output of Debug() for the chiral center, atom 1:

m.Debug()

Atoms:
    0 6 C chg: 0  deg: 1 exp: 1 imp: 3 hyb: SP3
    1 6 C chg: 0  deg: 4 exp: 4 imp: 0 hyb: SP3 chi: CCW nbrs:[0 2 5 8]
    2 6 C chg: 0  deg: 3 exp: 4 imp: 0 hyb: SP2
    3 8 O chg: 0  deg: 1 exp: 2 imp: 0 hyb: SP2
    4 8 O chg: 0  deg: 1 exp: 1 imp: 1 hyb: SP2
    5 6 C chg: 0  deg: 2 exp: 2 imp: 2 hyb: SP3
    6 6 C chg: 0  deg: 2 exp: 2 imp: 2 hyb: SP3
    7 6 C chg: 0  deg: 1 exp: 1 imp: 3 hyb: SP3
    8 6 C chg: 0  deg: 2 exp: 2 imp: 2 hyb: SP3
    9 6 C chg: 0  deg: 1 exp: 1 imp: 3 hyb: SP3
Bonds:
    0 0->1 order: 1
    1 1->2 order: 1
    2 2->3 order: 2 conj?: 1
    3 2->4 order: 1 conj?: 1
    4 1->5 order: 1
    5 5->6 order: 1
    6 6->7 order: 1
    7 1->8 order: 1
    8 8->9 order: 1

The important information here is chi and nbrs, which tell us that if we look from atom 0 to atom 1, we need to rotate counter-clockwise to go from atom 2 to atom 5 (or from atom 5 to atom 8).

Building synthon spaces with combinatorial reactions

Sat, 13 Dec 2025 23:00:00 GMT

Last week’s blog post looked at using BRICS to build a synthon search space. This post builds on that and creates a search space using some combichem reactions from a set published by Hartenfeller et al.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem import rdChemReactions
from rdkit.Chem.Draw import IPythonConsole
from rdkit.Chem import rdSynthonSpaceSearch

import rdkit
print(rdkit.__version__)

2025.09.3

I’m going to use a set of Enamine building blocks that I have on my machine. I loaded these into a SubstructLibrary to make them fast and easy to search.

from rdkit.Chem import rdSubstructLibrary
import pickle
sslib = pickle.load(open('/scratch/Data/Enamine/real_reagents.sslib.pkl','rb'))

The first reaction

I’ll start with the Pictet-Spengler reaction from the Hartenfeller paper. It’s the first reaction in the SI for that paper and there’s something the name that I really like.

Here’s the definition of the reaction from the paper

sma = '[cH1:1]1:[c:2](-[CH2:7]-[CH2:8]-[NH2:9]):[c:3]:[c:4]:[c:5]:[c:6]:1.[#6:11]-[CH1;R0:10]=[OD1]>>[c:1]12:[c:2](-[CH2:7]-[CH2:8]-[NH1:9]-[C:10]-2(-[#6:11])):[c:3]:[c:4]:[c:5]:[c:6]:1'

rxn = rdChemReactions.ReactionFromSmarts(sma)
rxn

These are the two prototype educts from the SI:

r1 = Chem.MolFromSmiles('c1cc(CCN)ccc1')
r2 = Chem.MolFromSmiles('CC(=O)')
Draw.MolsToGridImage((r1,r2))

To make what’s going on in the reaction a bit easier to identify, here I add the atom map numbers from the reaction to the sample educts:

r_queries = [rxn.GetReactantTemplate(x) for x in range(rxn.GetNumReactantTemplates())]

for r,q in zip([r1,r2],r_queries):
    match = r.GetSubstructMatch(q)
    assert match
    for i,midx in enumerate(match):
        mnum = q.GetAtomWithIdx(i).GetAtomMapNum()
        if mnum:
            r.GetAtomWithIdx(midx).SetAtomMapNum(mnum)
Draw.MolsToGridImage((r1,r2))

I’m going to encode the reaction as three synthons: a core and two “sidechains”

core = Chem.MolFromSmiles('[7*]CCNC([11*])[1*]')
core

Here’s what I get for the two sample educts:

chains = [Chem.MolFromSmiles(x) for x in ('[7*]c1ccccc1[1*]','C[11*]')]
Draw.MolsToGridImage([core,chains[0],chains[1]],legends=['core','educt1','educt2'])

We can put these together using molzip (which is what the sython search code uses) to get the product:

tm = Chem.RWMol(core)
tm.InsertMol(chains[0])
tm.InsertMol(chains[1])

ps = Chem.MolzipParams()
ps.label = Chem.MolzipLabel.Isotope
tm = Chem.molzip(tm,ps)
tm

Now we need reactions that transform the building blocks that you would find in a chemical catalog into the synthons we need for the two educts.

I will do this using reactions.

r1_prep = rdChemReactions.ReactionFromSmarts('[cH1:1]1:[c:2](-[CH2:7]-[CH2:8]-[NH2:9]):[c:3]:[c:4]:[c:5]:[c:6]:1\
>>[1*]-[c:1]1:[c:2](-[7*])[c:3]:[c:4]:[c:5]:[c:6]1')
p = r1_prep.RunReactant(r1,0)

[15:59:19] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 7 8 9

Draw.MolsToGridImage([r1,p[0][0]],legends=['building block','synthon'])

r2_prep = rdChemReactions.ReactionFromSmarts('[#6:11]-[CH1;R0:10]=[OD1]>>[#6:11]-[11*]')
p = r2_prep.RunReactant(r2,0)

Draw.MolsToGridImage([r2,p[0][0]],legends=['building block','synthon'])

[15:59:41] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 10

More complex reactions (see the next example) may have multiple cores. I want to try and have the code for creating the space be reasonably generic, so I’ll put the preparation reactions and core into a list:

r_queries = [rxn.GetReactantTemplate(x) for x in range(rxn.GetNumReactantTemplates())]
possibles = [(core,(r1_prep,r2_prep))]

def enumerate_synthons(possibles,outf,writeHeader=True,startIdx=1,nWritten=1):
    if writeHeader:
        outf.write('\t'.join(['SMILES','synton_id','synton#','reaction_id','release'])+'\n')
    for rxnidx,poss in enumerate(possibles,start=startIdx):
        # get the core and preparation reactions
        core,preps = poss
        # write out the core:
        outf.write(f'{Chem.MolToSmiles(core)}\t{nWritten}\t1\tr{rxnidx}\t1\n')

        # now prepare each of the sidechain synthons:
        nWritten += 1
        for ridx,prep in enumerate(preps):
            if not prep.GetNumProductTemplates():
                continue
            # find possible building blocks that could work here:
            poss = sslib.GetMatches(r_queries[ridx],maxResults=10000)
            print(rxnidx,ridx,len(poss),nWritten)
            seen = set()
            # now loop over those, prepare them, and write them to the output:
            for mol in (sslib.GetMol(x) for x in poss):
                # we use an R0 primitive in the query, so make sure
                # we have ring presence
                Chem.FastFindRings(mol)
                # run our prep reaction:
                ps = prep.RunReactant(mol,0)
                # write each product (in case there's more than one)
                for p in ps:
                    smi= Chem.MolToSmiles(p[0])
                    # don't write duplicates:
                    if smi in seen:
                        continue
                    outf.write(f'{smi}\t{nWritten}\t{ridx+2}\tr{rxnidx}\t1\n')
                    seen.add(smi)
                    nWritten += 1
            print('\t',nWritten)
    return rxnidx+1,nWritten

with open('./space1.txt','w+') as outf:
    nextRxn,nextIdx = enumerate_synthons(possibles,outf)
            
                
!head space1.txt

1 0 264 2
     333
1 1 4456 333
     4778
SMILES  synton_id   synton# reaction_id release
[1*]C([11*])NCC[7*] 1   1   r1  1
[1*]c1ccccc1[7*]    2   2   r1  1
[1*]c1cc(OC)ccc1[7*]    3   2   r1  1
[1*]c1cc(S(N)(=O)=O)ccc1[7*]    4   2   r1  1
[1*]c1cc(Cl)ccc1[7*]    5   2   r1  1
[1*]c1cc(OC)c(OC)cc1[7*]    6   2   r1  1
[1*]c1c([7*])ccc(OC)c1OC    7   2   r1  1
[1*]c1cc(Cl)cc(Cl)c1[7*]    8   2   r1  1
[1*]c1cc(F)ccc1[7*] 9   2   r1  1

spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile('space1.txt')
spc.GetNumProducts()

q2 = Chem.MolFromSmarts('')
q1 = Chem.MolFromSmarts('c1cccc(C(F)(F)F)c1C')
#q = Chem.MolFromSmarts('O=c1ncncc1')

params = rdSynthonSpaceSearch.SynthonSpaceSearchParams()
params.randomSample = True
params.randomSeed = 0xf00d

res = spc.SubstructureSearch(q1,params=params)
resMols = list(sorted(res.GetHitMolecules(),key=lambda x:x.GetNumHeavyAtoms()))
# resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

1000 results

Second reaction

The second reaction is Niementowski_quinazoline, also from the Hartenfeller paper. I’ve used this reaction before in a blog post showing how to extract information from reaction products.

This one is a bit trickier to encode.

Start with the reaction definition:

sma = '[c:1](-[C;$(C-c1ccccc1):2](=[OD1:3])-[OH1]):[c:4](-[NH2:5]).[N;!H0;!$(N-N);!$(N-C=N);!$(N(-C=O)-C=O):6]-[C;H1,$(C-[#6]):7]=[OD1]>>[c:4]2:[c:1]-[C:2](=[O:3])-[N:6]-[C:7]=[N:5]-2'

rxn = rdChemReactions.ReactionFromSmarts(sma)
rxn

These are the two educts from the SI:

r1 = Chem.MolFromSmiles('c1c(C(=O)O)c(N)ccc1')
r2 = Chem.MolFromSmiles('C(=O)N')
Draw.MolsToGridImage((r1,r2))

Here are the educts with atom map info:

r_queries = [rxn.GetReactantTemplate(x) for x in range(rxn.GetNumReactantTemplates())]

for r,q in zip([r1,r2],r_queries):
    match = r.GetSubstructMatch(q)
    assert match
    for i,midx in enumerate(match):
        mnum = q.GetAtomWithIdx(i).GetAtomMapNum()
        if mnum:
            r.GetAtomWithIdx(midx).SetAtomMapNum(mnum)
Draw.MolsToGridImage((r1,r2))

#-----
# Educt 2 is a primary amine
#   core1: no substituent on educt 2 carbon
core1 = Chem.MolFromSmiles('O=C1-N-C=N-C([1*]):C1[2*]',sanitize=True)
#   core3: educt2 has a substituent on the carbon
core3 = Chem.MolFromSmiles('O=C1-N-C([3*])=N-C([1*]):C1[2*]',sanitize=True)

# Educt 2 is a secondary amine
#   core2: no substituent on educt 2 carbon
core2 = Chem.MolFromSmiles('O=C1-N([4*])-C=N-C([1*]):C1[2*]',sanitize=True)
#   core4: educt2 has a substituent on the carbon
core4 = Chem.MolFromSmiles('O=C1-N([4*])-C([3*])=N-C([1*]):C1[2*]',sanitize=True)

Draw.MolsToGridImage([core1,core2,core3,core4],legends=['core1','core2','core3','core4'],molsPerRow=4)

r1_prep = rdChemReactions.ReactionFromSmarts('[c:1]1(-[C:2](=[OD1:3])-[OH1]):[c:4](-[NH2:5])[c:6][c:7][c:8][c:9]1\
>>[2*][c:6][c:7][c:8][c:9][1*]')
p = r1_prep.RunReactant(r1,0)

[05:23:12] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 1 2 3 4 5

p[0][0]

Make sure we form the correct ring:

ps = Chem.MolzipParams()
ps.label = Chem.MolzipLabel.Isotope
tm = Chem.molzip(core1,p[0][0],ps)
tm

Now do the preparation reactions for R2, taking the four scenarios into account:

r2_prep1 = rdChemReactions.ReactionFromSmarts('[NH2;!$(N-N);!$(N-C=N);!$(N(-C=O)-C=O):6]-[CH1:7]=[OD1]\
>>')
r2_prep3 = rdChemReactions.ReactionFromSmarts('[NH2;!$(N-N);!$(N-C=N);!$(N(-C=O)-C=O):6]-[C:7](-[#6:3])=[OD1]\
>>[3*][#6:3]')
r2_prep2 = rdChemReactions.ReactionFromSmarts('[*:8][NH1;!$(N-N);!$(N-C=N);!$(N(-C=O)-C=O):6]-[CH1:7]=[OD1]\
>>[4*][*:8]')
r2_prep4 = rdChemReactions.ReactionFromSmarts('[*:8][NH1;!$(N-N);!$(N-C=N);!$(N(-C=O)-C=O):6]-[C;R0:7](-[#6:3])=[OD1]\
>>([4*][*:8].[3*][*:7])')

possibles = [(core1,(r1_prep,r2_prep1)),(core2,(r1_prep,r2_prep2)),
             (core3,(r1_prep,r2_prep3)),(core4,(r1_prep,r2_prep4))]

Create the synthon space:

with open('./space2.txt','w+') as outf:
    nextRxn,nextIdx = enumerate_synthons(possibles,outf)
            
                
!head space2.txt

1 0 189 2
     188
2 0 189 189
     375
2 1 8173 375

[05:38:56] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 6 7

     383
3 0 189 384
     570
3 1 8173 570

[05:38:57] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 6 7

     1690
4 0 189 1691
     1877
4 1 8173 1877

[05:38:57] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 6 3

     3769
SMILES  synton_id   synton# reaction_id release
[1*]c1nc[nH]c(=O)c1[2*] 1   1   r1  1
[1*]ccc1c([2*])C(=O)c2ccccc2C1=O    2   2   r1  1
[1*]cc(Cl)cc([2*])Cl    3   2   r1  1
[1*]ccc(c[2*])C(F)(F)F  4   2   r1  1
[1*]ccc(Cl)c[2*]    5   2   r1  1
[1*]cc1ccccc1c[2*]  6   2   r1  1
[1*]cc(OC)c(c[2*])OC    7   2   r1  1
[1*]cc(cc[2*])S(=O)(=O)Nc1ccccc1OC  8   2   r1  1
[1*]cc(cc[2*])S(=O)(=O)Nc1ccc(OC)cc1    9   2   r1  1

Read it in

spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile('space2.txt')
spc.GetNumProducts()

q2 = Chem.MolFromSmarts('cC(F)(F)F')
params = rdSynthonSpaceSearch.SynthonSpaceSearchParams()
params.randomSample = True
params.randomSeed = 0xf00d

res = spc.SubstructureSearch(q2,params=params)
resMols = list(sorted(res.GetHitMolecules(),key=lambda x:x.GetNumHeavyAtoms()))
# resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

1000 results

We have results here for cores 2-4, but none for core 1. Let’s confirm that there are core1 results in the set

q2 = Chem.MolFromSmarts('[#6](=O)@[#7H1]@[#6H1]@[#7]')
params = rdSynthonSpaceSearch.SynthonSpaceSearchParams()
params.randomSample = True
params.randomSeed = 0xf00d

res = spc.SubstructureSearch(q2,params=params)
resMols = list(sorted(res.GetHitMolecules(),key=lambda x:x.GetNumHeavyAtoms()))
# resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

[05:41:48] Complex queries can be slow.

186 results

Combine the two reaction spaces

We now add these synthons to the other space. We could do this by sequentially enumerating the spaces into the same output file, but it’s quicker to re-enumerate the second space onto the bottom of the output file from the first one:

!cp space1.txt combined.txt
!tail combined.txt

[11*]c1ccnn1-c1cccnc1   4768    3   r1  1
[11*]C1CC2(C1)CC(OC)C2  4769    3   r1  1
[11*]c1c(Cl)cnc(F)c1Cl  4770    3   r1  1
[11*]c1cc(Cl)c(C(=O)O)s1    4771    3   r1  1
[11*]C1COC2CC1C2    4772    3   r1  1
[11*]C12C3CCC(CC31)C2(F)F   4773    3   r1  1
[11*]c1ccc(C(F)(F)C(F)(F)F)s1   4774    3   r1  1
[11*]C12C3CC(CC31)C2NC(=O)OC(C)(C)C 4775    3   r1  1
[11*]C1OCCOC1(C)C   4776    3   r1  1
[11*]c1cc(C)cc(C(F)F)c1 4777    3   r1  1

Now re-enumerate the second space, start the synthon IDs at 4778 and the reaction IDs at 2:

with open('combined.txt','a') as outf:
    enumerate_synthons(possibles,outf,writeHeader=False,nWritten=4778,startIdx=2)
spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile('combined.txt')
spc.GetNumProducts()

2 0 189 4779
     4965
3 0 189 4966
     5152
3 1 8173 5152
     5160
4 0 189 5161
     5347
4 1 8173 5347
     6467
5 0 189 6468
     6654
5 1 8173 6654
     8546

!tail combined.txt

[3*]C.[4*]CC1(N)CCC1    8536    3   r5  1
[3*]C.[4*]C1CCCCC1CN    8537    3   r5  1
[3*]C.[4*]CC1(N)CCCCC1  8538    3   r5  1
[3*]C.[4*]CC1(N)CCCC1   8539    3   r5  1
[3*]C.[4*]C1CC(N)C12CCC2    8540    3   r5  1
[3*]C.[4*]C1CCOCC1N 8541    3   r5  1
[3*]C.[4*]CC(N)c1ccccc1OC   8542    3   r5  1
[3*]C.[4*]C(CN)C(C)(C)C 8543    3   r5  1
[3*]C.[4*]C1Cc2ccccc2C1N    8544    3   r5  1
[3*]C.[4*]C1CC2(C1)CC(N)C2  8545    3   r5  1

Verify that searches return results from both spaces:

q2 = Chem.MolFromSmarts('cC(F)(F)F')
params = rdSynthonSpaceSearch.SynthonSpaceSearchParams()
params.randomSample = True
params.randomSeed = 0xf00d

res = spc.SubstructureSearch(q2,params=params)
resMols = list(sorted(res.GetHitMolecules(),key=lambda x:x.GetNumHeavyAtoms()))
# resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

1000 results

Reaction 3

For the last example, I’ll add the reaction definition for spiro chromanone from the Hartenfeller paper. This one is fun because it forms a spiro linkage.

sma = '[c:1](-[C;$(C-c1ccccc1):2](=[OD1:3])-[CH3:4]):[c:5](-[OH1:6]).[C;$(C1-[CH2]-[CH2]-[N,C]-[CH2]-[CH2]-1):7](=[OD1])>>[O:6]1-[c:5]:[c:1]-[C:2](=[OD1:3])-[C:4]-[C:7]-1'

rxn = rdChemReactions.ReactionFromSmarts(sma)
rxn

These are the two educts from the SI:

r1 = Chem.MolFromSmiles('c1cc(C(=O)C)c(O)cc1')
r2 = Chem.MolFromSmiles('C1(=O)CCNCC1')
Draw.MolsToGridImage((r1,r2))

Here’s what the prototype product looks like:

p = rxn.RunReactants([r1,r2])
p[0][0]

Here are the educts with atom map info:

r_queries = [rxn.GetReactantTemplate(x) for x in range(rxn.GetNumReactantTemplates())]

for r,q in zip([r1,r2],r_queries):
    match = r.GetSubstructMatch(q)
    assert match
    for i,midx in enumerate(match):
        mnum = q.GetAtomWithIdx(i).GetAtomMapNum()
        if mnum:
            r.GetAtomWithIdx(midx).SetAtomMapNum(mnum)
Draw.MolsToGridImage((r1,r2))

core = Chem.MolFromSmiles('O=C(-[C][4*])c([1*]):c([5*])[O][6*]',sanitize=False)
core

r1_prep = rdChemReactions.ReactionFromSmarts('[c:1]1(-[C:2](=[OD1:3])-[CH3:4]):[c:5](-[OH1:6]):[c:7]:[c:8]:[c:9]:[c:10]1\
>>[*1]:[c:7]:[c:8]:[c:9]:[c:10]:[5*]')
p1 = r1_prep.RunReactant(r1,0)
p1[0][0]

[06:13:20] mapped atoms in the reactants were not mapped in the products.
  unmapped numbers are: 1 2 3 4 5 6

'[C;$(C1-[CH2]-[CH2]-[N,C]-[CH2]-[CH2]-1):7](=[OD1])'
r2_prep = rdChemReactions.ReactionFromSmarts('O=[C:11]1-[CH2:12]-[CH2:13]-[N,C:14]-[CH2:15]-[CH2:16]-1\
>>[4*][C:11]1([6*])-[CH2:12]-[CH2:13]-[N,C:14]-[CH2:15]-[CH2:16]-1')
p2 = r2_prep.RunReactant(r2,0)
p2[0][0]

Make sure we form the correct ring:

tm = Chem.RWMol(core)
tm.InsertMol(p1[0][0])
tm.InsertMol(p2[0][0])
ps = Chem.MolzipParams()
ps.label = Chem.MolzipLabel.Isotope
tm = Chem.molzip(tm,ps)
tm

Draw.MolsToGridImage([core,p1[0][0],p2[0][0],tm],legends=['synthon1','synthon2','synthon3','product'],molsPerRow=4)

possibles = [(core,(r1_prep,r2_prep))]

Create the synthon space:

with open('./space3.txt','w+') as outf:
    nextRxn,nextIdx = enumerate_synthons(possibles,outf)
            
                
!head space2.txt

1 0 58 2
     60
1 1 241 60
     301
SMILES  synton_id   synton# reaction_id release
[1*]c1nc[nH]c(=O)c1[2*] 1   1   r1  1
[1*]ccc1c([2*])C(=O)c2ccccc2C1=O    2   2   r1  1
[1*]cc(Cl)cc([2*])Cl    3   2   r1  1
[1*]ccc(c[2*])C(F)(F)F  4   2   r1  1
[1*]ccc(Cl)c[2*]    5   2   r1  1
[1*]cc1ccccc1c[2*]  6   2   r1  1
[1*]cc(OC)c(c[2*])OC    7   2   r1  1
[1*]cc(cc[2*])S(=O)(=O)Nc1ccccc1OC  8   2   r1  1
[1*]cc(cc[2*])S(=O)(=O)Nc1ccc(OC)cc1    9   2   r1  1

Read it in

spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile('space3.txt')
spc.GetNumProducts()

q2 = Chem.MolFromSmarts('cC(F)(F)F')
params = rdSynthonSpaceSearch.SynthonSpaceSearchParams()
params.randomSample = True
params.randomSeed = 0xf00d

res = spc.SubstructureSearch(q2,params=params)
resMols = list(sorted(res.GetHitMolecules(),key=lambda x:x.GetNumHeavyAtoms()))
# resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

778 results

And combine it with the other two:

!tail combined.txt

[3*]C.[4*]CC1(N)CCC1    8536    3   r5  1
[3*]C.[4*]C1CCCCC1CN    8537    3   r5  1
[3*]C.[4*]CC1(N)CCCCC1  8538    3   r5  1
[3*]C.[4*]CC1(N)CCCC1   8539    3   r5  1
[3*]C.[4*]C1CC(N)C12CCC2    8540    3   r5  1
[3*]C.[4*]C1CCOCC1N 8541    3   r5  1
[3*]C.[4*]CC(N)c1ccccc1OC   8542    3   r5  1
[3*]C.[4*]C(CN)C(C)(C)C 8543    3   r5  1
[3*]C.[4*]C1Cc2ccccc2C1N    8544    3   r5  1
[3*]C.[4*]C1CC2(C1)CC(N)C2  8545    3   r5  1

with open('combined.txt','a') as outf:
    enumerate_synthons(possibles,outf,writeHeader=False,nWritten=8546,startIdx=6)
spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile('combined.txt')
print(spc.GetNumProducts())

!tail combined.txt

6 0 58 8547
     8605
6 1 241 8605
     8846
6077623
[4*]C1([6*])CCC(=C(F)F)CC1  8836    3   r6  1
[4*]C1([6*])CCN(S(=O)(=O)F)CC1  8837    3   r6  1
[4*]C1([6*])CCN(c2cc(=O)[nH]cn2)CC1 8838    3   r6  1
[4*]C1([6*])CCN(OCc2ccc(Cl)cc2)CC1  8839    3   r6  1
[4*]C1([6*])CCC(N2CCCC2)CC1 8840    3   r6  1
[4*]C1([6*])CCC(N2CCOCC2)CC1    8841    3   r6  1
[4*]C1([6*])CCN(c2cc[nH]c(=O)c2)CC1 8842    3   r6  1
[4*]C1([6*])CCC([N+](=O)[O-])CC1    8843    3   r6  1
[4*]C1([6*])CCC2(CC1)CC2C(=O)OC 8844    3   r6  1
[4*]C1([6*])CCC2(CCC(CBr)O2)CC1 8845    3   r6  1

Do a search to be sure we get results from multiple reactions:

q2 = Chem.MolFromSmarts('cC(=O)C')
params = rdSynthonSpaceSearch.SynthonSpaceSearchParams()
params.randomSample = True
params.randomSeed = 0xf00d

res = spc.SubstructureSearch(q2,params=params)
resMols = list(sorted(res.GetHitMolecules(),key=lambda x:x.GetNumHeavyAtoms()))
# resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

1000 results

There are many more reactions in the Hartenfeller paper, but I’m going to stop here.

Building synthon spaces with BRICS fragments

Sat, 06 Dec 2025 23:00:00 GMT

Last year we added functionality to the RDKit to allow searching in synthon, or combinatorial library spaces. Dave Cosgrove did a blog post on this a while ago and there also a tutorial in the docs.

I’ve been asked a few times how one can create a synthon space from either a set of compounds or a set of reactions and building blocks. This post will focus on the first use case: creating a synthon space from the fragments created by applying BRICS fragmentation to a set of ChEMBL compounds. I will try to do a post covering the other use case in the not-too-distant future.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem import BRICS
from rdkit.Chem.Draw import IPythonConsole
from rdkit.Chem import rdSynthonSpaceSearch

import rdkit
print(rdkit.__version__)

2025.09.3

The synthon space input file

Let’s start by looking at what the synthon space input file looks like:

# snipped from the freedom space input
open('./blah.txt','w+').write('''SMILES synton_id   synton# reaction_id release
Clc1ccc(-c2cnc(N[U])s2)cc1  6   1   a1  3
O=[N+]([O-])c1cccc(N[U])c1O 10  1   a1  3
O=C([U])c1sc2cc(Cl)ccc2c1Cl 31  2   a1  3
COc1cccc(O)c1C(=O)[U]   86  2   a1  3
''')

There’s more information about the format and how searches work in the docs.

Here’s a quick demo on using the file:

spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile('blah.txt')
spc.GetNumProducts()

res = spc.SubstructureSearch(Chem.MolFromSmarts('CN'))
Draw.MolsToGridImage(res.GetHitMolecules())

BRICS decomposition:

I did a post earlier this year with a tutorial on BRICS decomposition, so I won’t get into a lot of detail here.

Here’s a ChEMBL molecule we’ll work with:

m = Chem.MolFromSmiles('C1=C(c2ccc(CCN3CCCCC3)cc2)C2CN(Cc3ccccc3)CC2C1 CHEMBL256225')
m

And this is what we get when we do a BRICS decomposition:

BRICS.BRICSDecompose(m)

{'[16*]c1ccccc1',
 '[4*]CC[8*]',
 '[4*]C[8*]',
 '[5*]N1CC2CC=C(c3ccc([16*])cc3)C2C1',
 '[5*]N1CCCCC1'}

The synthon space search code works by combining multiple building blocks to form a compound in a single “reaction” step, and it’s unhappy if you have dummy atoms/attachment points leftover after that step, So these fragments are too small to be useful in sython searching.

Fortunately, we can have the BRICS decomposition give us partial decomposition results along with the final results:

BRICS.BRICSDecompose(m,keepNonLeafNodes=True,minFragmentSize=2)

{'C1=C(c2ccc(CCN3CCCCC3)cc2)C2CN(Cc3ccccc3)CC2C1',
 '[16*]c1ccc(C2=CCC3CN(Cc4ccccc4)CC23)cc1',
 '[16*]c1ccccc1',
 '[4*]CC[8*]',
 '[4*]CCc1ccc(C2=CCC3CN(C[8*])CC23)cc1',
 '[4*]CCc1ccc(C2=CCC3CN(Cc4ccccc4)CC23)cc1',
 '[4*]CCc1ccc(C2=CCC3CN([5*])CC23)cc1',
 '[4*]Cc1ccccc1',
 '[5*]N1CC2CC=C(c3ccc(CCN4CCCCC4)cc3)C2C1',
 '[5*]N1CC2CC=C(c3ccc([16*])cc3)C2C1',
 '[5*]N1CCCCC1',
 '[8*]CCN1CCCCC1',
 '[8*]CN1CC2CC=C(c3ccc(CCN4CCCCC4)cc3)C2C1',
 '[8*]CN1CC2CC=C(c3ccc([16*])cc3)C2C1'}

What we’ll do here is pick out all of the fragments that have two attachment points to use as core synthons for a reaction and then provide all the single-attachment point fragments that are compatible with those two attachment points as the other partners in a reaction.

So in this case one “reaction” would be around the core [4*]CCc1ccc(C2=CCC3CN(C[8*])CC23)cc1 as synthon 1. We would we’d provide all of the single-attachment fragments that can connect to attachment point [4*] (in this set of fragments that’s only [5*]) as synthon 2, and all of the single-attachment fragnets that can connect to attachment point [8*] (here that’s [16*]) as synthon 3.

Because the synthon code connects attachment points with the same label, we need to transform all of the [5*]s into [4*]s and all of the [16*]s into [8*]s to give this:

SMILES  synton_id   synton# reaction_id release
[4*]CCc1ccc(C2=CCC3CN(C[8*])CC23)cc1    1   1   r1  1
[4*]N1CCCCC1    2   2   r1  1
[4*]Cc1ccccc1   3   2   r1  1
[4*]N1CC2CC=C(c3ccc(CCN4CCCCC4)cc3)C2C1 4   2   r1  1
[4*]CCc1ccc(C2=CCC3CN(Cc4ccccc4)CC23)cc1    5   2   r1  1
[8*]c1ccc(C2=CCC3CN(Cc4ccccc4)CC23)cc1  6   3   r1  1
[8*]CN1CC2CC=C(c3ccc(CCN4CCCCC4)cc3)C2C1    7   3   r1  1
[8*]CCN1CCCCC1  8   3   r1  1
[8*]c1ccccc1    9   3   r1  1

Automating this to handle a set of molecules doesn’t require a massive amount of code:

from collections import defaultdict
import re
from rdkit.Chem import BRICS
import copy
import logging

def get_connectors():
    ''' read all the BRICS "reaction" definitions and build a dict mapping
      connector type -> types it can connect to
    '''
    connectors = defaultdict(list)
    for defs in BRICS.reactionDefs:
        for i,j,b in defs:
            if b!='-':
                continue
            connectors[i+'*'].append(j+'*')
            connectors[j+'*'].append(i+'*')
    # remove duplicates and convert to a standard dict
    res = {}
    for k,v in connectors.items():
        res[k] = list(set(v))
    return res

def get_synthons(mol):
    frags = BRICS.BRICSDecompose(mol,keepNonLeafNodes=True,minFragmentSize=2)
    # remove anything with 7 in it since those are attached with double bonds
    # and we don't handle those:
    frags = [x for x in frags if '[7*' not in x]
    cnts = [(x,x.count('*')) for x in frags]
    ones = list(sorted([x for x,cnt in cnts if cnt==1],key=lambda x:len(x)))
    twos = list(sorted([x for x,cnt in cnts if cnt==2],key=lambda x:len(x)))
    return twos,ones

def get_possibles(rlabel,frags,connectors):
    res = []
    poss = connectors[rlabel]
    rlabel = '['+rlabel
    for smi in frags:
        if rlabel in smi:
            res.append(smi)
            continue
        for lbl in poss:
            lbl = '['+lbl
            if lbl in smi:
                nsmi = smi.replace(lbl,rlabel)
                res.append(nsmi)
                break
    return res

def get_rs(core):
    return re.findall(r'\[(.*?\*)\]',core)

def generate_space(mols,fname):
    connectors = get_connectors()
    with open(fname,'w+') as outf:
        nWritten = 0
        nRxns = 0
        for i,mol in enumerate(mols):
            twos,ones = get_synthons(mol)
            if not len(twos):
                continue
            nWritten,nRxns = generate_reactions(outf,twos,ones,connectors,nWritten,nRxns,writeHeader=(not nWritten))
    return nWritten,nRxns


def generate_reactions(outf,allTwos,allOnes,connectors,nWritten=0,nRxns=0,writeHeader=True):
    if writeHeader:
        outf.write('\t'.join(['SMILES','synton_id','synton#','reaction_id','release'])+'\n')
    for rs,twos in allTwos.items():
        assert len(rs)==2
        # do we have the same label twice?
        newrs = rs[:]
        if len(set(rs)) != 2:
            rs = list(rs)
            rlabel = rs[0]
            newlabel = rlabel.replace('*','00*')
            newrs = (newlabel,rs[1])
            connectors = copy.deepcopy(connectors)
            connectors[newlabel] = connectors[rlabel]
            rs = tuple(rs)
        for core in twos:
            if newrs != rs:
                core = core.replace(rs[0],newrs[0],1)
            outf.write(f'{core}\t{nWritten+1}\t1\tr{nRxns+1}\t1\n')
            nWritten += 1
        ones = allOnes[rs]
        for j,r in enumerate(newrs):
            for p in get_possibles(r,ones,connectors):
                outf.write(f'{p}\t{nWritten+1}\t{j+2}\tr{nRxns+1}\t1\n')
                nWritten += 1
        nRxns += 1
    return nWritten,nRxns

def generate_space(mols,fname):
    connectors = get_connectors()
    allOnes = defaultdict(set)
    allTwos = defaultdict(set)
    for mol in mols:
        twos,ones = get_synthons(mol)
        for two in twos:
            rs = tuple(sorted(get_rs(two)))
            if len(rs) != 2:
                logger.warning(f'core {two} does not have two attachment points')
            allTwos[rs].add(two)
            allOnes[rs] = allOnes[rs].union(ones)
    with open(fname,'w+') as outf:
        nWritten,nRxns = generate_reactions(outf,allTwos,allOnes,connectors)
    return nWritten,nRxns

Try that out on two molecules:

sample1 = Chem.MolFromSmiles('C1=C(c2ccc(CCN3CCCCC3)cc2)C2CN(Cc3ccccc3)CC2C1 CHEMBL256225')
sample2 = Chem.MolFromSmiles('C1=C(c2ccc(CCN3CCCCCC3)cc2)C2CN(Cc3ccccc3)CC2C1 madeup')
nWritten,nRxns=generate_space([sample1,sample2],'blah2.txt')
print(nWritten,nRxns)

65 4

!head blah2.txt

SMILES  synton_id   synton# reaction_id release
[4*]CCc1ccc(C2=CCC3CN(C[8*])CC23)cc1    1   1   r1  1
[4*]CC[8*]  2   1   r1  1
[4*]N1CCCCC1    3   2   r1  1
[4*]Cc1ccccc1   4   2   r1  1
[4*]N1CC2CC=C(c3ccc(CCN4CCCCC4)cc3)C2C1 5   2   r1  1
[4*]N1CCCCCC1   6   2   r1  1
[4*]CCc1ccc(C2=CCC3CN(Cc4ccccc4)CC23)cc1    7   2   r1  1
[4*]N1CC2CC=C(c3ccc(CCN4CCCCCC4)cc3)C2C1    8   2   r1  1
[8*]CCN1CCCCC1  9   3   r1  1

Notice in the above that we have combined the two cores that have attachment points [4*] and [8*] into a single reaction, r1. This makes the resulting synthon space smaller and more efficient to search.

Ok, let’s do more molecules. I’ll use the set of very active molecules from ChEMBL36 that I put together in an earlier blog post.

lines = open('../data/chembl36_very_active.txt','r').readlines()
# header:
lines.pop(0)
keep = []
for l in lines:
    if '[' not in l:
        keep.append(l.strip().split()[1])
len(keep)

Draw.MolsToGridImage([Chem.MolFromSmiles(smi) for smi in keep[:12]],molsPerRow=4)

Generate a synthon space for the first 300 of the ChEMBL compounds:

first = [Chem.MolFromSmiles(x) for x in keep[:300]]

fname = './results/actives_space.txt'
nwritten,nrxns = generate_space(first,fname)
print(f'{nwritten} synthons in {nrxns} reactions')

spc = rdSynthonSpaceSearch.SynthonSpace()
spc.ReadTextFile(fname)
print(f'{rdSynthonSpaceSearch.FormattedIntegerString(spc.GetNumProducts())} products in space')

23789 synthons in 79 reactions
253 702 889 products in space

Here’s what the file looks like:

!head ./results/actives_space.txt

SMILES  synton_id   synton# reaction_id release
[500*]NC(=O)N[5*]   1   1   r1  1
[500*]NC(=O)NC1CCN(C(=O)c2ccc(C(=O)N[5*])cc2)CC1    2   1   r1  1
[500*]Nc1nc2c(nc1N1CCC(Oc3ccc(F)cc3F)CC1)C(C)N([5*])CC2 3   1   r1  1
[500*]Nc1cc2cc(C3CCN([5*])CC3)c(C)cc2cn1    4   1   r1  1
[500*]Nc1nc2c(nc1N1CCC(Oc3ccc(F)cc3F)CC1)CN([5*])C(C)C2 5   1   r1  1
[500*]NCCCN1CCN(CCCN[5*])CC1    6   1   r1  1
[500*]Nc1cccc(CCN2CCN([5*])CC2)c1   7   1   r1  1
[500*]Nc1ccc(S(=O)(=O)N[5*])cc1F    8   1   r1  1
[500*]NC(=O)c1ccc(-c2cnc3c(N[5*])cc(C(F)(F)c4cccc(F)c4)nn23)cc1C    9   1   r1  1

Let’s do a few searches:

res = spc.SubstructureSearch(Chem.MolFromSmarts('c1cccs1'))
resMols = res.GetHitMolecules()
print(f'{len(resMols)} results')
Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

1000 results

res = spc.SubstructureSearch(Chem.MolFromSmarts('c1ncncc1'))
resMols = res.GetHitMolecules()

Draw.MolsToGridImage(resMols[:12],subImgSize=(250,200)) if len(resMols) else None

It’s nicer to sort by increasing molecular size:

tms = list(sorted(resMols,key=lambda x:x.GetNumHeavyAtoms()))
Draw.MolsToGridImage(tms[:12],subImgSize=(250,200)) if len(tms) else None

As we saw in the blog post on BRICS, you can get some odd molecules by putting together BRICS fragments, but there are plenty of results in there that still look reasonable.

Thresholds for “random” with 3D similarity methods

Sat, 29 Nov 2025 23:00:00 GMT

Intro

In this one I’m generating some reference data for 3D similarity approaches. The idea is inspired by this blog post, where I figured out noise thresholds for similarity calculations with a bunch of 2D fingerprints. Here I do basically the same thing: calculating a number of different 3D similarity (or distance) metrics on random pairs of molecules in order to establish noise thresholds for those metrics.

I compare results from two different data sets for this: 1. 25000 random pairs of molecules with crystal structures from the LOBSTER data set. The last three blog posts have looked at LOBSTER. I used the crystal structures from the LOBSTER data set for these molecules. 2. 50000 random pairs of molecules from the ChEMBL set I used in the fingerprint thresholds post. I used ETKDGv3 conformers for these molecules.

The first data set is considerably less diverse, the pairs are formed from only 3583 unique molecules, but the 3D structures are from crystals so I think it’s worth considering (even though we know that ETKDG does generally produce reasonable structures). For cases where the values disagree, I think it’s probably better to use the ChEMBL results since they come from a larger data set.

Here’s the summary of the results.

Alignment based approaches

Here some of the metrics are similarity based, where the thresholds are lower bounds, and some are distance based, where the thresholds are upper bounds. Rather than transform the distance into similarity, I think it’s more useful to report the values that actually come from the RDKit function. If this turns out to be wrong, I will update the blog post.

For example, if you do a shape-based alignment of two molecules to each other and get a shape Tanimoto score of 0.80, the LOBSTER data would say that the value is larger than 95% of the random pairs while the ChEMBL data says it’s larger (more significant) than 99% of the random pairs. Similarly, if the shape-based alignment produces a shape Tanimoto distance of 0.30, both data sets would say that the distance is smaller (more significant) than 99% of the random pairs.

LOBSTER Set

metric	70%	80%	90%	95%	99%	type
baseline_ShapeTanimoto	>0.49	>0.54	>0.62	>0.69	>0.81	SIMILARITY
shape_align_ShapeTanimoto	>0.62	>0.66	>0.71	>0.76	>0.85	SIMILARITY
baseline_TanimotoDist	<0.61	<0.57	<0.52	<0.48	<0.39	DISTANCE
shape_align_TanimotoDist	<0.53	<0.50	<0.46	<0.42	<0.34	DISTANCE
shape_align_noc_TanimotoDist	<0.52	<0.49	<0.45	<0.42	<0.34	DISTANCE
o3a_align_TanimotoDist	<0.58	<0.55	<0.51	<0.46	<0.36	DISTANCE
crippeno3a_align_TanimotoDist	<0.59	<0.56	<0.51	<0.47	<0.37	DISTANCE

ChEMBL Set

metric	70%	80%	90%	95%	99%	type
baseline_ShapeTanimoto	>0.44	>0.48	>0.54	>0.59	>0.69	SIMILARITY
shape_align_ShapeTanimoto	>0.58	>0.61	>0.65	>0.69	>0.76	SIMILARITY
baseline_TanimotoDist	<0.64	<0.61	<0.57	<0.54	<0.47	DISTANCE
shape_align_TanimotoDist	<0.55	<0.53	<0.50	<0.47	<0.42	DISTANCE
shape_align_noc_TanimotoDist	<0.55	<0.53	<0.50	<0.47	<0.41	DISTANCE
o3a_align_TanimotoDist	<0.61	<0.59	<0.55	<0.52	<0.46	DISTANCE
crippeno3a_align_TanimotoDist	<0.61	<0.59	<0.55	<0.52	<0.46	DISTANCE

Non-alignment based approaches

LOBSTER Set

metric	70%	80%	90%	95%	99%	type
USR_score	>0.66	>0.71	>0.76	>0.80	>0.87	SIMILARITY
noh_USR_score	>0.66	>0.71	>0.76	>0.80	>0.86	SIMILARITY
AP3D_DiceSimilarity	>0.49	>0.54	>0.60	>0.63	>0.70	SIMILARITY
noh_AP3D_DiceSimilarity	>0.29	>0.33	>0.38	>0.43	>0.51	SIMILARITY
E3FP_DiceSimilarity	>0.28	>0.31	>0.34	>0.37	>0.43	SIMILARITY
noh_E3FP_DiceSimilarity	>0.23	>0.26	>0.29	>0.32	>0.38	SIMILARITY

ChEMBL Set

metric	70%	80%	90%	95%	99%	type
USR_score	>0.65	>0.69	>0.74	>0.78	>0.84	SIMILARITY
noh_USR_score	>0.65	>0.69	>0.74	>0.78	>0.84	SIMILARITY
AP3D_DiceSimilarity	>0.57	>0.60	>0.65	>0.68	>0.73	SIMILARITY
noh_AP3D_DiceSimilarity	>0.34	>0.37	>0.42	>0.45	>0.51	SIMILARITY
E3FP_DiceSimilarity	>0.30	>0.33	>0.35	>0.38	>0.42	SIMILARITY
noh_E3FP_DiceSimilarity	>0.26	>0.28	>0.30	>0.32	>0.36	SIMILARITY

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole
IPythonConsole.ipython_3d = True

from matplotlib import pyplot as plt
plt.style.use('tableau-colorblind10')
plt.rcParams['font.size'] = '16'
%matplotlib inline

%load_ext sql
%config SqlMagic.feedback=0

import rdkit
print(rdkit.__version__)

2025.09.3

Getting started

import lwreg
from lwreg import utils

Load our lwreg configuration from the database we created before:

config = utils.configure_from_database(dbname='lobster_112024',dbtype='postgresql')
lwreg.set_default_config(config)

config

{'dbname': 'lobster_112024',
 'dbtype': 'postgresql',
 'cacheConnection': True,
 'standardization': 'none',
 'removeHs': 1,
 'useTautomerHashv2': 0,
 'registerConformers': 1,
 'numConformerDigits': 3,
 'lwregSchema': ''}

Random pairs from LOBSTER

Get a map from (nm,pdb) tuples to (molregno,confid,molblock):

d = %sql postgresql://localhost/lobster_112024 \
    select ligname,pdb,molregno,conf_id,molblock \
    from lobster_data.all_ligands join conformers using (molregno,conf_id);
ligs = {}
for nm,pdb,mrn,cid,mb in d:
    mol = Chem.MolFromMolBlock(mb,removeHs=False)
    mol_noh = Chem.MolFromMolBlock(mb)
    ligs[(mrn,cid)] = (nm,pdb,mb,mol,mol_noh)

Create a bunch of random pairs:

import random
random.seed(0xa100f)

ks = list(ligs.keys())
base = []
while len(base)<25000:
    t = ks[:]
    random.shuffle(t)
    base.extend(t)
tbase = base[:]
random.shuffle(tbase)
pairs = set((min(x,y),max(x,y)) for x,y in zip(base,tbase) if x!=y)
len(pairs)

pairs = list(pairs)[:25000]
pairs[:5]

[((453, 453), (925, 925)),
 ((1574, 1574), (2733, 2733)),
 ((175, 175), (3247, 3247)),
 ((1719, 1719), (3118, 3118)),
 ((3054, 3054), (3458, 3458))]

Shape-based alignment

Let’s see what we get when we perform shape-based alignment using the crystal conformers.

Start by aligning the crystal conformers.

import gzip,pickle
res_accum = pickle.load(open('./results/3d_random_distances.pkl','rb'))
from rdkit.Chem import rdShapeAlign
from rdkit.Chem import rdShapeHelpers
from rdkit.Chem import rdMolTransforms

from collections import defaultdict
res_accum = defaultdict(list)

from rdkit.Chem import rdShapeAlign
from rdkit.Chem import rdShapeHelpers
from rdkit.Chem import rdMolTransforms

from tqdm import tqdm
for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    
    st,ct = rdShapeAlign.AlignMol(m1,m2,opt_param=0.5)
    res_accum['shape_align_ShapeTanimoto'].append(st)
    res_accum['shape_align_ColorTanimoto'].append(ct)
    res_accum['shape_align_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))

    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    st,ct = rdShapeAlign.AlignMol(m1,m2,opt_param=1.0,useColors=False)
    res_accum['shape_align_noc_ShapeTanimoto'].append(st)
    res_accum['shape_align_noc_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))

100%|███████████████████████████████████████████████████████████████████████| 25000/25000 [00:47<00:00, 530.84it/s]

Open3DAlign

What about an alternative 3D alignment algorithm? Let’s try aligning with Paolo Tosco’s Open3DAlign:

from rdkit.Chem import rdMolAlign

for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    
    try:
        o3a = rdMolAlign.GetO3A(m2,m1)
        rmsd = o3a.Align()
        score = o3a.Score()
        res_accum['o3a_align_rmsd'].append(rmsd)
        res_accum['o3a_align_scpre'].append(score)
        res_accum['o3a_align_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))
    except ValueError:
        pass

100%|███████████████████████████████████████████████████████████████████████| 25000/25000 [02:06<00:00, 198.35it/s]

The RDKit has a variation on Open3DAlign that uses atomic contributions to the MolLogP value instead of MMFF94 atom types

from rdkit.Chem import rdMolAlign

for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    
    try:
        o3a = rdMolAlign.GetCrippenO3A(m2,m1)
        rmsd = o3a.Align()
        score = o3a.Score()
        res_accum['crippeno3a_align_rmsd'].append(rmsd)
        res_accum['crippeno3a_align_scpre'].append(score)
        res_accum['crippeno3a_align_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))
    except ValueError:
        pass

100%|███████████████████████████████████████████████████████████████████████| 25000/25000 [01:08<00:00, 366.77it/s]

Baseline: canonical alignment

Just put the molecules in their principle-axis frame

res_accum['baseline_TanimotoDist'] = []
from tqdm import tqdm
for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    
    res_accum['baseline_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))

100%|██████████████████████████████████████████████████████████████████████| 25000/25000 [00:07<00:00, 3483.93it/s]

Baseline: tanimoto shape score in the canonical alignment

In the v2025.09.3 RDKit release (released the day before I wrote this post), Dave Cosgrove added a function allowing two molecules to be scored using the Pubchem shape alignment code without doing the alignment first. This gives a score that is directly comparable to what you get when you do an alignment.

res_accum['baseline_ShapeTanimoto'] = []
from tqdm import tqdm
for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    opts = rdShapeAlign.ShapeInputOptions()
    res_accum['baseline_ShapeTanimoto'].append(rdShapeAlign.ScoreMol(m1,m2,opts,opts))

100%|██████████████████████████████████████████████████████████████████████| 25000/25000 [00:05<00:00, 4227.54it/s]

import pickle
pickle.dump(res_accum,open('./results/3d_random_distances.pkl','wb+'))

plt.figure(figsize=(6,4))
plt.hist([[x[0] for x in res_accum['baseline_ShapeTanimoto']],res_accum['shape_align_ShapeTanimoto']],
         bins=20,label=['baseline','aligned'])
plt.xlabel('shape tanimoto score');
plt.legend();

plt.figure(figsize=(6,4))
plt.hist([res_accum['baseline_TanimotoDist'],],bins=20,label=['baseline',])
plt.xlabel('shape tanimoto distance');
plt.legend();

plt.figure(figsize=(6,4))
plt.hist([res_accum['baseline_TanimotoDist'],res_accum['shape_align_TanimotoDist'],res_accum['shape_align_noc_TanimotoDist']],bins=20,label=['baseline','color','no color'])
plt.xlabel('shape tanimoto distance');
plt.legend();

plt.figure(figsize=(6,4))
plt.hist([res_accum['baseline_TanimotoDist'],res_accum['o3a_align_TanimotoDist'],res_accum['crippeno3a_align_TanimotoDist']],
         bins=20,label=['baseline','O3A','CrippenO3A'])
plt.xlabel('shape tanimoto distance');
plt.legend();

Non-alignment methods

from rdkit.Chem import rdMolDescriptors
res_accum['USR_score'] = []
res_accum['noh_USR_score'] = []


for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    usr1 = rdMolDescriptors.GetUSR(m1)
    usr2 = rdMolDescriptors.GetUSR(m2)
    res_accum['USR_score'].append(rdMolDescriptors.GetUSRScore(usr1,usr2))
    
    m1 = Chem.Mol(ligs[c1][4])
    m2 = Chem.Mol(ligs[c2][4])
    usr1 = rdMolDescriptors.GetUSR(m1)
    usr2 = rdMolDescriptors.GetUSR(m2)
    res_accum['noh_USR_score'].append(rdMolDescriptors.GetUSRScore(usr1,usr2))

100%|██████████████████████████████████████████████████████████████████████| 25000/25000 [00:03<00:00, 6754.96it/s]

fpg = rdFingerprintGenerator.GetAtomPairGenerator(use2D=False)
res_accum['AP3D_DiceSimilarity'] = []
res_accum['noh_AP3D_DiceSimilarity'] = []
for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    fp1 = fpg.GetCountFingerprint(m1)
    fp2 = fpg.GetCountFingerprint(m2)
    res_accum['AP3D_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

    m1 = Chem.Mol(ligs[c1][4])
    m2 = Chem.Mol(ligs[c2][4])
    fp1 = fpg.GetCountFingerprint(m1)
    fp2 = fpg.GetCountFingerprint(m2)
    res_accum['noh_AP3D_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

100%|██████████████████████████████████████████████████████████████████████| 25000/25000 [00:21<00:00, 1148.58it/s]

import logging
from e3fp.pipeline import fprints_from_mol
logging.disable(logging.INFO)

def get_fp(m):
    fp = fprints_from_mol(m,fprint_params={'counts':True})[0]
    rdkfp = DataStructs.ULongSparseIntVect(2**32)
    for k,v in fp.counts.items():
        rdkfp[int(k)] = v 
    return rdkfp

res_accum['E3FP_DiceSimilarity'] = []
res_accum['noh_E3FP_DiceSimilarity'] = []
for c1,c2 in tqdm(pairs):
    m1 = Chem.Mol(ligs[c1][3])
    m2 = Chem.Mol(ligs[c2][3])
    fp1 = get_fp(m1)
    fp2 = get_fp(m2)
    res_accum['E3FP_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))
    m1 = Chem.Mol(ligs[c1][4])
    m2 = Chem.Mol(ligs[c2][4])
    fp1 = get_fp(m1)
    fp2 = get_fp(m2)
    res_accum['noh_E3FP_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

100%|████████████████████████████████████████████████████████████████████████| 25000/25000 [39:01<00:00, 10.68it/s]

import pickle
pickle.dump(res_accum,open('./results/3d_random_distances.pkl','wb+'))

Random ChEMBL molecules

Using the pairs of random molecules I used for the fingerprint thresholds post and other blog posts about similarity.

import gzip
ind = [x.split(b'\t') for x in gzip.open('../data/chembl35_50K.mfp0.pairs.txt.gz')]
ms1 = []
ms2 = []
for i,row in enumerate(ind):
    m1 = Chem.AddHs(Chem.MolFromSmiles(row[1]))
    ms1.append(m1)
    m2 = Chem.AddHs(Chem.MolFromSmiles(row[3]))
    ms2.append(m2)

random.seed(23)
random.shuffle(ms2)

len(ms1)

try:
    import ipyparallel as ipp
    rc = ipp.Client()
    dview = rc[:]
    dview.execute('from rdkit import Chem')
    dview.execute('from rdkit.Chem import rdDistGeom')
    dview.execute('from rdkit.Chem import rdMolAlign')
    dview.execute('from rdkit.Chem import rdMolTransforms')
    dview.execute('from rdkit.Chem import rdShapeHelpers')
    dview.execute('from rdkit.Chem import rdShapeAlign')
    dview.execute('from rdkit.Chem import rdFingerprintGenerator')
    dview.execute('from rdkit import DataStructs')
    dview.execute('from e3fp.pipeline import fprints_from_mol')
    dview.execute('import logging;logging.disable(logging.INFO)')
except:
    print("could not use ipyparallel")
    dview = None

Generate one ETKDGv3 conformer per molecule:

from rdkit.Chem import rdDistGeom
ms1c = dview.map_sync(lambda x:(rdDistGeom.EmbedMolecule(x,randomSeed=0xf00d),x), ms1)
ms2c = dview.map_sync(lambda x:(rdDistGeom.EmbedMolecule(x,randomSeed=0xf00d),x), ms2)

import pickle
import gzip
pickle.dump((ms1c,ms2c),gzip.open('./results/random_pairs_confs.pkl.gz','wb+'))

res_accum2 = defaultdict(list)

for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())

    res_accum2['baseline_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))

    st,ct = rdShapeAlign.AlignMol(m1,m2,opt_param=0.5)
    res_accum2['shape_align_ShapeTanimoto'].append(st)
    res_accum2['shape_align_ColorTanimoto'].append(ct)
    res_accum2['shape_align_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))

    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    st,ct = rdShapeAlign.AlignMol(m1,m2,opt_param=1.0,useColors=False)
    res_accum2['shape_align_noc_ShapeTanimoto'].append(st)
    res_accum2['shape_align_noc_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))

res_accum2 = pickle.load(open('./results/3d_random_distances2.pkl','rb'))

res_accum2['baseline_ShapeTanimoto'] = []
for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    opts = rdShapeAlign.ShapeInputOptions()
    res_accum2['baseline_ShapeTanimoto'].append(rdShapeAlign.ScoreMol(m1,m2,opts,opts))

50000it [00:12, 3934.62it/s]

for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    try:
        o3a = rdMolAlign.GetO3A(m2,m1)
        rmsd = o3a.Align()
        score = o3a.Score()
        res_accum2['o3a_align_rmsd'].append(rmsd)
        res_accum2['o3a_align_scpre'].append(score)
        res_accum2['o3a_align_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))
    except ValueError:
        pass

50000it [07:06, 117.30it/s]

for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    
    try:
        o3a = rdMolAlign.GetCrippenO3A(m2,m1)
        rmsd = o3a.Align()
        score = o3a.Score()
        res_accum2['crippeno3a_align_rmsd'].append(rmsd)
        res_accum2['crippeno3a_align_scpre'].append(score)
        res_accum2['crippeno3a_align_TanimotoDist'].append(rdShapeHelpers.ShapeTanimotoDist(m1,m2))   
    except ValueError:
        pass

50000it [03:43, 223.62it/s]

from rdkit.Chem import rdMolDescriptors

res_accum2['USR_score'] = []
res_accum2['noh_USR_score'] = []
for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    
    usr1 = rdMolDescriptors.GetUSR(m1)
    usr2 = rdMolDescriptors.GetUSR(m2)
    res_accum2['USR_score'].append(rdMolDescriptors.GetUSRScore(usr1,usr2))
    
    m1 = Chem.RemoveHs(m1)
    m2 = Chem.RemoveHs(m2)
    usr1 = rdMolDescriptors.GetUSR(m1)
    usr2 = rdMolDescriptors.GetUSR(m2)
    res_accum2['noh_USR_score'].append(rdMolDescriptors.GetUSRScore(usr1,usr2))

50000it [00:18, 2744.31it/s]

fpg = rdFingerprintGenerator.GetAtomPairGenerator(use2D=False)

res_accum2['AP3D_DiceSimilarity'] = []
res_accum2['noh_AP3D_DiceSimilarity'] = []
for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    fp1 = fpg.GetCountFingerprint(m1)
    fp2 = fpg.GetCountFingerprint(m2)
    res_accum2['AP3D_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

    m1 = Chem.RemoveHs(m1)
    m2 = Chem.RemoveHs(m2)
    fp1 = fpg.GetCountFingerprint(m1)
    fp2 = fpg.GetCountFingerprint(m2)
    res_accum2['noh_AP3D_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

50000it [01:12, 688.09it/s]

res_accum2['E3FP_DiceSimilarity'] = []
res_accum2['noh_E3FP_DiceSimilarity'] = []

for m1,m2 in tqdm(zip(ms1c,ms2c)):
    m1 = Chem.Mol(m1[1])
    m2 = Chem.Mol(m2[1])
    if not m1.GetNumConformers() or not m2.GetNumConformers():
        continue
    fp1 = get_fp(m1)
    fp2 = get_fp(m2)
    res_accum2['E3FP_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

    m1 = Chem.RemoveHs(m1)
    m2 = Chem.RemoveHs(m2)
    fp1 = get_fp(m1)
    fp2 = get_fp(m2)
    res_accum2['noh_E3FP_DiceSimilarity'].append(DataStructs.DiceSimilarity(fp1,fp2))

50000it [1:49:13,  7.63it/s]

import pickle
pickle.dump(res_accum2,open('./results/3d_random_distances2.pkl','wb+'))

plt.figure(figsize=(6,4))
plt.hist([[x[0] for x in res_accum2['baseline_ShapeTanimoto']],res_accum2['shape_align_ShapeTanimoto']],
         bins=20,label=['baseline','aligned'])
plt.xlabel('shape tanimoto score');
plt.legend();

plt.figure(figsize=(6,4))
plt.hist([res_accum2['baseline_TanimotoDist'],res_accum2['shape_align_TanimotoDist'],res_accum2['shape_align_noc_TanimotoDist']],bins=20,label=['baseline','color','no color'])
plt.xlabel('shape tanimoto distance');
plt.legend();

plt.figure(figsize=(6,4))
plt.hist([res_accum2['baseline_TanimotoDist'],res_accum2['o3a_align_TanimotoDist'],res_accum2['crippeno3a_align_TanimotoDist']],
         bins=20,label=['baseline','O3A','CrippenO3A'])
plt.xlabel('shape tanimoto distance');
plt.legend();

res_accum2.keys()

dict_keys(['baseline_TanimotoDist', 'shape_align_ShapeTanimoto', 'shape_align_ColorTanimoto', 'shape_align_TanimotoDist', 'shape_align_noc_ShapeTanimoto', 'shape_align_noc_TanimotoDist', 'o3a_align_rmsd', 'o3a_align_scpre', 'o3a_align_TanimotoDist', 'crippeno3a_align_rmsd', 'crippeno3a_align_scpre', 'crippeno3a_align_TanimotoDist', 'USR_score', 'noh_USR_score', 'AP3D_DiceSimilarity', 'noh_AP3D_DiceSimilarity', 'E3FP_DiceSimilarity', 'noh_E3FP_DiceSimilarity', 'baseline_ShapeTanimoto'])

plt.figure(figsize=(6,4))
plt.hist([res_accum2['o3a_align_scpre'],res_accum2['crippeno3a_align_scpre'],],
         bins=20,label=['O3A','CrippenO3A'])
plt.xlabel('alignment score');
plt.legend();

Summary stats

import numpy as np

print('****   LOBSTER Set ****')
print('| baseline_ShapeTanimoto\t|',
      ' | '.join([">%.2f"%x for x in np.quantile([x[0] for x in res_accum['baseline_ShapeTanimoto']],[0.7, 0.8, 0.9,0.95,0.99])]),
     '\t| SIMILARITY |')
print('| shape_align_ShapeTanimoto\t|',
      ' | '.join([">%.2f"%x for x in np.quantile([x for x in res_accum['shape_align_ShapeTanimoto']],[0.7, 0.8, 0.9,0.95,0.99])]),
     '\t| SIMILARITY |')
for k in ['baseline_TanimotoDist','shape_align_TanimotoDist','shape_align_noc_TanimotoDist',
         'o3a_align_TanimotoDist','crippeno3a_align_TanimotoDist']:
    print(f'| {k}\t|',' | '.join(["<%.2f"%x for x in np.quantile(res_accum[k],[0.3,0.2,0.1,0.05,0.01])]),
         '\t| DISTANCE |')
print('****   ChEMBL Set ****')
print('| baseline_ShapeTanimoto\t|',
      ' | '.join([">%.2f"%x for x in np.quantile([x[0] for x in res_accum2['baseline_ShapeTanimoto']],[0.7, 0.8, 0.9,0.95,0.99])]),
     '\t| SIMILARITY |')
print('| shape_align_ShapeTanimoto\t|',
      ' | '.join([">%.2f"%x for x in np.quantile([x for x in res_accum2['shape_align_ShapeTanimoto']],[0.7, 0.8, 0.9,0.95,0.99])]),
     '\t| SIMILARITY |')
for k in ['baseline_TanimotoDist','shape_align_TanimotoDist','shape_align_noc_TanimotoDist',
         'o3a_align_TanimotoDist','crippeno3a_align_TanimotoDist']:
    print(f'| {k}\t|',' | '.join(["<%.2f"%x for x in np.quantile(res_accum2[k],[0.3,0.2,0.1,0.05,0.01])]),
         '\t| DISTANCE |')

print('\n\n\n ------------------   No alignment  --------------------')
print('****   LOBSTER Set ****')
for k in ['USR_score','noh_USR_score','AP3D_DiceSimilarity','noh_AP3D_DiceSimilarity',
          'E3FP_DiceSimilarity','noh_E3FP_DiceSimilarity']:
    print(f'| {k}\t|',' | '.join([">%.2f"%x for x in np.quantile(res_accum[k],[0.7, 0.8, 0.9,0.95,0.99])]),
         '\t| SIMILARITY |')
    
print('****   ChEMBL Set ****')
for k in ['USR_score','noh_USR_score','AP3D_DiceSimilarity','noh_AP3D_DiceSimilarity',
          'E3FP_DiceSimilarity','noh_E3FP_DiceSimilarity']:
    print(f'| {k}\t|',' | '.join([">%.2f"%x for x in np.quantile(res_accum2[k],[0.7, 0.8, 0.9,0.95,0.99])]),
         '\t| SIMILARITY |')

****   LOBSTER Set ****
| baseline_ShapeTanimoto    | >0.49 | >0.54 | >0.62 | >0.69 | >0.81     | SIMILARITY |
| shape_align_ShapeTanimoto | >0.62 | >0.66 | >0.71 | >0.76 | >0.85     | SIMILARITY |
| baseline_TanimotoDist | <0.61 | <0.57 | <0.52 | <0.48 | <0.39     | DISTANCE |
| shape_align_TanimotoDist  | <0.53 | <0.50 | <0.46 | <0.42 | <0.34     | DISTANCE |
| shape_align_noc_TanimotoDist  | <0.52 | <0.49 | <0.45 | <0.42 | <0.34     | DISTANCE |
| o3a_align_TanimotoDist    | <0.58 | <0.55 | <0.51 | <0.46 | <0.36     | DISTANCE |
| crippeno3a_align_TanimotoDist | <0.59 | <0.56 | <0.51 | <0.47 | <0.37     | DISTANCE |
****   ChEMBL Set ****
| baseline_ShapeTanimoto    | >0.44 | >0.48 | >0.54 | >0.59 | >0.69     | SIMILARITY |
| shape_align_ShapeTanimoto | >0.58 | >0.61 | >0.65 | >0.69 | >0.76     | SIMILARITY |
| baseline_TanimotoDist | <0.64 | <0.61 | <0.57 | <0.54 | <0.47     | DISTANCE |
| shape_align_TanimotoDist  | <0.55 | <0.53 | <0.50 | <0.47 | <0.42     | DISTANCE |
| shape_align_noc_TanimotoDist  | <0.55 | <0.53 | <0.50 | <0.47 | <0.41     | DISTANCE |
| o3a_align_TanimotoDist    | <0.61 | <0.59 | <0.55 | <0.52 | <0.46     | DISTANCE |
| crippeno3a_align_TanimotoDist | <0.61 | <0.59 | <0.55 | <0.52 | <0.46     | DISTANCE |



 ------------------   No alignment  --------------------
****   LOBSTER Set ****
| USR_score | >0.66 | >0.71 | >0.76 | >0.80 | >0.87     | SIMILARITY |
| noh_USR_score | >0.66 | >0.71 | >0.76 | >0.80 | >0.86     | SIMILARITY |
| AP3D_DiceSimilarity   | >0.49 | >0.54 | >0.60 | >0.63 | >0.70     | SIMILARITY |
| noh_AP3D_DiceSimilarity   | >0.29 | >0.33 | >0.38 | >0.43 | >0.51     | SIMILARITY |
| E3FP_DiceSimilarity   | >0.28 | >0.31 | >0.34 | >0.37 | >0.43     | SIMILARITY |
| noh_E3FP_DiceSimilarity   | >0.23 | >0.26 | >0.29 | >0.32 | >0.38     | SIMILARITY |
****   ChEMBL Set ****
| USR_score | >0.65 | >0.69 | >0.74 | >0.78 | >0.84     | SIMILARITY |
| noh_USR_score | >0.65 | >0.69 | >0.74 | >0.78 | >0.84     | SIMILARITY |
| AP3D_DiceSimilarity   | >0.57 | >0.60 | >0.65 | >0.68 | >0.73     | SIMILARITY |
| noh_AP3D_DiceSimilarity   | >0.34 | >0.37 | >0.42 | >0.45 | >0.51     | SIMILARITY |
| E3FP_DiceSimilarity   | >0.30 | >0.33 | >0.35 | >0.38 | >0.42     | SIMILARITY |
| noh_E3FP_DiceSimilarity   | >0.26 | >0.28 | >0.30 | >0.32 | >0.36     | SIMILARITY |

Working with the LOBSTER Data set III

Sat, 22 Nov 2025 23:00:00 GMT

This is my third post looking at the LOBSTER data set published last year by the Rarey and BioSolveIT group. Here are links for Part 1 and Part 2.

This time I look at applying the different 3D alignment algorithms the RDKit includes to the data.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole
IPythonConsole.ipython_3d = True

from matplotlib import pyplot as plt
plt.style.use('tableau-colorblind10')
plt.rcParams['font.size'] = '16'
%matplotlib inline

%load_ext sql
%config SqlMagic.feedback=0

import rdkit
print(rdkit.__version__)

2025.09.2

Getting started

import lwreg
from lwreg import utils

Load our lwreg configuration from the database we created before:

config = utils.configure_from_database(dbname='lobster_112024',dbtype='postgresql')
lwreg.set_default_config(config)

config

{'dbname': 'lobster_112024',
 'dbtype': 'postgresql',
 'cacheConnection': True,
 'standardization': 'none',
 'removeHs': 1,
 'useTautomerHashv2': 0,
 'registerConformers': 1,
 'numConformerDigits': 3,
 'lwregSchema': ''}

Get a map from (nm,pdb) tuples to (molregno,confid,molblock):

d = %sql postgresql://localhost/lobster_112024 \
    select ligname,pdb,molregno,conf_id,molblock \
    from lobster_data.all_ligands join conformers using (molregno,conf_id);
ligs = {}
for nm,pdb,mrn,cid,mb in d:
    mol = Chem.MolFromMolBlock(mb,removeHs=False)
    mol_noh = Chem.MolFromMolBlock(mb)
    ligs[(nm,pdb.lower())] = (mrn,cid,mb,mol,mol_noh)

pairstats = %sql postgresql://localhost/lobster_112024 \
    select * from lobster_data.pair_stats
pairstats = list(pairstats.dicts())

from scipy import stats
def comparison_plot(pairstats,metric1,metric2,invert1=False,invert2=False,includeLine=False):
    x1 = [x[metric1] for x in pairstats]
    if invert1:
        x1 = [1-x for x in x1]
        metric1 = f'1-{metric1}'
    x2 = [x[metric2] for x in pairstats]
    if invert2:
        x2 = [1-x for x in x2]
        metric2 = f'1-{metric2}'
    r,_ = stats.spearmanr(x1,x2)
    tau,_ = stats.kendalltau(x1,x2)

    plt.figure(figsize=(10,5))
    plt.subplot(1,2,1)
    plt.scatter(x1,x2,alpha=0.5,s=5)
    if includeLine:
        plt.plot((0,1),(0,1),'k-')
    plt.xlabel(metric1)
    plt.ylabel(metric2)
    plt.title(f'rho={r:.2f}, tau={tau:.2f}');
    plt.subplot(1,2,2)
    plt.hexbin(x1,x2,cmap='Blues');
    plt.tight_layout()

Comparing shape-similarity metrics in the LOBSTER data set

First let’s look at the metrics in the LOBSTER data set:

comparison_plot(pairstats,'shape_tversky_index','shape_tanimoto_distance',invert1=True,includeLine=True)

In the LOBSTER paper they use an asymmetric definition of the Tversky metric to detect how much of the template is covered by the probe. This yield small Tversky distances (large Tversky index values) when a small template is completely covered by a large probe. The Tanimoto distance, on the other hand, is symmetric, so it’s always greater than 1-Tversky here.

And now the protrude distance, which looks at how much of the larger shape is not overlapping the smaller shape.

comparison_plot(pairstats,'shape_tversky_index','shape_protrude_distance',invert1=True,includeLine=True)

The protrude distance is, by default, symmetric: it is the fraction of the smaller of the two shapes that protrudes outside the larger of the two.

Shape-based alignment

Let’s see what we get when we perform shape-based alignment using the crystal conformers.

Start by aligning the crystal conformers.

from rdkit.Chem import rdShapeAlign
from rdkit.Chem import rdShapeHelpers
from rdkit.Chem import rdMolTransforms

from tqdm import tqdm
for d in tqdm(pairstats):
    m1 = Chem.Mol(ligs[(d['ligname1'],d['pdb1'])][3])
    m2 = Chem.Mol(ligs[(d['ligname2'],d['pdb2'])][3])
    rdMolTransforms.CanonicalizeConformer(m1.GetConformer())
    rdMolTransforms.CanonicalizeConformer(m2.GetConformer())
    
    st,ct = rdShapeAlign.AlignMol(m1,m2,opt_param=0.5)
    d['shape_align_ShapeTanimoto'] = st
    d['shape_align_ColorTanimoto'] = ct
    d['shape_align_ShapeTversky'] = rdShapeHelpers.ShapeTverskyIndex(m1,m2,0,1)

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 72592/72592 [01:15<00:00, 964.83it/s]

comparison_plot(pairstats,'shape_tversky_index','shape_align_ShapeTversky',invert1=False,invert2=False,
                includeLine=True)

Look at examples where the ShapeTversky in the shape alignment is significantly lower than that in the crystal alignment:

pruned = [r for r in pairstats if r['shape_tversky_index']>0.9 and r['shape_align_ShapeTversky']<0.8]
len(pruned)

d = pruned[0]
print((d['ligname1'],d['pdb1']),(d['ligname2'],d['pdb2']))
m1 = ligs[(d['ligname1'],d['pdb1'])][3]
m2 = ligs[(d['ligname2'],d['pdb2'])][3]
m1c = Chem.Mol(m1)
m2c = Chem.Mol(m2)
rdMolTransforms.CanonicalizeConformer(m1c.GetConformer())
rdMolTransforms.CanonicalizeConformer(m2c.GetConformer())
st,ct = rdShapeAlign.AlignMol(m1c,m2c,opt_param=0.5)
tversky = rdShapeHelpers.ShapeTverskyIndex(m1c,m2c,0,1)

print(st,ct)
print(d['shape_tversky_index'],tversky)

('1D1_A_401', '4i5p') ('11G_A_401', '4i6b')
0.6968245697702556 0.11721759959746417
0.951 0.7622862091361766

IPythonConsole.drawMols3D([m1,m2])

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

IPythonConsole.drawMols3D([m1c,m2c])

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

This is a case where the asymmetry of the Tversky Index used in the Lobster paper shows. Here in the crystal alignment the second of the two molecules is more or less completely contained within the first one; that leads to a very high Tversky index that drops significantly if we re-order the molecules:

tversky12 = rdShapeHelpers.ShapeTverskyIndex(m1,m2,0,1)
tversky21 = rdShapeHelpers.ShapeTverskyIndex(m2,m1,0,1)
print(f'{tversky12:.2f} -> {tversky21:.2f}')

0.95 -> 0.76

The shape alignment is symmetrical because it maximizes the overall overlap:

m1c = Chem.Mol(m1)
m2c = Chem.Mol(m2)
rdMolTransforms.CanonicalizeConformer(m1c.GetConformer())
rdMolTransforms.CanonicalizeConformer(m2c.GetConformer())
st2,ct2 = rdShapeAlign.AlignMol(m2c,m1c,opt_param=0.5)
tversky2 = rdShapeHelpers.ShapeTverskyIndex(m1c,m2c,0,1)

print(f'{tversky:.2f} -> {tversky2:.2f}')

0.76 -> 0.76

What about molecules with a very low crystal Tversky index and a high one from the shape alignment?

pruned = [r for r in pairstats if r['shape_tversky_index']<0.1 and r['shape_align_ShapeTversky']>0.8]
len(pruned)

d = pruned[0]
m1 = ligs[(d['ligname1'],d['pdb1'])][3]
m2 = ligs[(d['ligname2'],d['pdb2'])][3]
m1c = Chem.Mol(m1)
m2c = Chem.Mol(m2)
rdMolTransforms.CanonicalizeConformer(m1c.GetConformer())
rdMolTransforms.CanonicalizeConformer(m2c.GetConformer())
st,ct = rdShapeAlign.AlignMol(m1c,m2c,opt_param=0.5)
tversky = rdShapeHelpers.ShapeTverskyIndex(m1c,m2c,0,1)

print(st,ct)
print(d['shape_tversky_index'],tversky)

0.3489983628032094 0.08050058859625212
0.0 0.8264984227129337

IPythonConsole.drawMols3D([m1,m2])

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

IPythonConsole.drawMols3D([m1c,m2c])

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

This is a situation described in the LOBSTER paper: the crystal ligands are in very different parts of the pocket that was identified by SIENA.

We can figure out what’s going on by looking at the PDB entries:

{'ligname1': 'N4Z_A_401',
 'pdb1': '6tel',
 'ligname2': '5JT_A_402',
 'pdb2': '5mw3',
 'ensemble': '5JU_A_401-5mw4',
 'morgan_fp_tanimoto': 0.115,
 'gobbi_2d_pharmacophore_fp_tanimoto': 0.102,
 'hac_difference': 29,
 'shape_tversky_index': 0.0,
 'shape_tanimoto_distance': 1.0,
 'shape_protrude_distance': 1.0,
 'shape_align_ShapeTanimoto': 0.3489983628032094,
 'shape_align_ColorTanimoto': 0.08050058859625212,
 'shape_align_ShapeTversky': 0.8264984227129337,
 'O3A_RMSD': 0.18706387903535077,
 'O3A_score': 99.22402124958128,
 'O3A_ShapeTversky': 0.6229205175600739,
 'CrippenO3A_RMSD': 0.3775024004623698,
 'CrippenO3A_score': 64.29333441352107,
 'CrippenO3A_ShapeTversky': 0.7638085218306154}

From looking at the PDBe pages for ligand N4Z in 6tel and ligand 5JT in 5mw3 it looks like the automatic curation procedure for LOBSTER may have picked the wrong ligand for 5mw3 (which has two bound ligands), ligand 5JJ looks like it may be a more similar pocket (or part of the pocket) to ligand N4Z.

Open3DAlign

What about an alternative 3D alignment algorithm? Let’s try aligning with Paolo Tosco’s Open3DAlign:

from rdkit.Chem import rdMolAlign

from tqdm import tqdm
for d in tqdm(pairstats):
    m1 = Chem.Mol(ligs[(d['ligname1'],d['pdb1'])][3])
    m2 = Chem.Mol(ligs[(d['ligname2'],d['pdb2'])][3])
    
    try:
        o3a = rdMolAlign.GetO3A(m2,m1)
        rmsd = o3a.Align()
        score = o3a.Score()
        tversky = rdShapeHelpers.ShapeTverskyIndex(m1,m2,0,1)
    except ValueError:
        # we get failures for molecules that don't have MMFF94 parameters.
        # set the results for those to zero
        rmsd = 0
        score = 0
        tversky = 0
    d['O3A_RMSD'] = rmsd
    d['O3A_score'] = score
    d['O3A_ShapeTversky'] = tversky

comparison_plot(pairstats,'shape_tversky_index','O3A_ShapeTversky',invert1=False,invert2=False,
                includeLine=True)

Ignoring the cases where Open3DAlign didn’t have parameters, that’s pretty similar to what we saw with the shape alignment, though there are points with a high Tversky index in the crystal structure and low Open3D align result.

ligname1,pdb1 = ('1D1_A_401', '4i5p') 
ligname2,pdb2 = ('11G_A_401', '4i6b')
d = pruned[0]
m1 = ligs[(ligname1,pdb1)][3]
m2 = ligs[(ligname2,pdb2)][3]
m1c = Chem.Mol(m1)
m2c = Chem.Mol(m2)
rdMolTransforms.CanonicalizeConformer(m1c.GetConformer())
rdMolTransforms.CanonicalizeConformer(m2c.GetConformer())
st,ct = rdShapeAlign.AlignMol(m1c,m2c,opt_param=0.5)
stversky = rdShapeHelpers.ShapeTverskyIndex(m1c,m2c,0,1)

m1c2 = Chem.Mol(m1)
m2c2 = Chem.Mol(m2)
o3a = rdMolAlign.GetO3A(m2c2,m1c2)
rmsd = o3a.Align()
o3atversky = rdShapeHelpers.ShapeTverskyIndex(m1c2,m2c2,0,1)

print(stversky,o3atversky)

0.7622862091361766 0.9846320346320346

The shape-based alignment (we saw this above):

IPythonConsole.drawMols3D([m1c,m2c])

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

And the Open3D alignment:

IPythonConsole.drawMols3D([m1c2,m2c2])

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

Here we can see that Open3DAlign can quite happily align a smaller molecule to a larger one.

Directly compare the O3A results to the shape alignment results:

comparison_plot(pairstats,'shape_align_ShapeTversky','O3A_ShapeTversky',invert1=False,invert2=False,
                includeLine=True)

There’s a reasonable number of pairs with high O3A similarities and low shape-align similarities. These are probably cases like we just saw: alignments of a small ligand to a larger one where there’s a good feature overlap between the small and large structures.

Aside from the alignments that didn’t work due to missing parameters (where the O3A Tversky is 0), the correlation is quite good

The RDKit has a variation on Open3DAlign that uses atomic contributions to the MolLogP value instead of MMFF94 atom types

from rdkit.Chem import rdMolAlign

from tqdm import tqdm
for d in tqdm(pairstats):
    m1 = Chem.Mol(ligs[(d['ligname1'],d['pdb1'])][3])
    m2 = Chem.Mol(ligs[(d['ligname2'],d['pdb2'])][3])
    
    try:
        o3a = rdMolAlign.GetCrippenO3A(m2,m1)
        rmsd = o3a.Align()
        score = o3a.Score()
        tversky = rdShapeHelpers.ShapeTverskyIndex(m1,m2,0,1)
    except ValueError:
        rmsd = 0
        score = 0
        tversky = 0
    d['CrippenO3A_RMSD'] = rmsd
    d['CrippenO3A_score'] = score
    d['CrippenO3A_ShapeTversky'] = tversky

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 72592/72592 [04:25<00:00, 273.70it/s]

It took a while to get here, so save the results:

import pickle
pickle.dump(pairstats,open('./results/pairstats.pkl','wb+'))

Compare the Crippen alignment to the shape one

comparison_plot(pairstats,'shape_align_ShapeTversky','CrippenO3A_ShapeTversky',invert1=False,invert2=False,
                includeLine=True)

This looks quite similar to the last results, but this time we don’t have failed alignments due to missing atom types.

Compare the two different Open3DAlign results to each other:

comparison_plot(pairstats,'O3A_ShapeTversky','CrippenO3A_ShapeTversky',invert1=False,invert2=False,
                includeLine=True)

And, finally, compare the Crippen O3A results to the crystal alignments:

comparison_plot(pairstats,'shape_tversky_index','CrippenO3A_ShapeTversky',invert1=False,invert2=False,
                includeLine=True)

Comparing performance on “easy” cases

The LOBSTER paper suggests that ligand pairs that have a Tversky overlap of 0.9 or higher are good starting points for studying superposition algorithms.

Let’s see how well the three alignment approaches we applied here do:

pruned = [r for r in pairstats if r['shape_tversky_index']>=0.9]
len(pruned)

comparison_plot(pruned,'shape_tversky_index','shape_align_ShapeTversky',invert1=False,invert2=False)

comparison_plot(pruned,'shape_tversky_index','CrippenO3A_ShapeTversky',invert1=False,invert2=False)

Make the same plot with the default O3A alignments that did not fail due to missing parameters:

comparison_plot([x for x in pruned if x['O3A_ShapeTversky']>0],'shape_tversky_index','O3A_ShapeTversky',invert1=False,invert2=False)

Here’s what that looks like with the failed alignments included:

comparison_plot(pruned,'shape_tversky_index','O3A_ShapeTversky',invert1=False,invert2=False)

Based on this simple analysis, it looks like the Open3DAlign methods are doing better than the shape-based alignment. Given the differences in size between some of the ligands, this isn’t terribly surprising.

To really dig into alignment quality, we’d probably also want to look beyond just shape overlap and considere things like how far the aligned poses are from what is observed in the crystal using some kind of RMSD measure, but that’s for a possible future post.

Working with the LOBSTER Data set II

Sat, 15 Nov 2025 23:00:00 GMT

This post builds on the last post and does a little bit of work with the LOBSTER data set.

I had planned something a bit more ambitious for this post, but while writing it I discovered a bug in the conformer generator that it took me a while to track down (it’s still not fixed) and that ate up all the time I had set aside for working on the blog this week. I’ll do another post soon.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole
IPythonConsole.ipython_3d = True

from matplotlib import pyplot as plt
plt.style.use('tableau-colorblind10')
plt.rcParams['font.size'] = '16'
%matplotlib inline

%load_ext sql
%config SqlMagic.feedback=0

Generating conformers for the LOBSTER compounds

import lwreg
from lwreg import utils

Load our lwreg configuration from the database we created before:

config = utils.configure_from_database(dbname='lobster_112024',dbtype='postgresql')
lwreg.set_default_config(config)

config

{'dbname': 'lobster_112024',
 'dbtype': 'postgresql',
 'cacheConnection': True,
 'standardization': 'none',
 'removeHs': 1,
 'useTautomerHashv2': 0,
 'registerConformers': 1,
 'numConformerDigits': 3,
 'lwregSchema': ''}

d = %sql postgresql://localhost/lobster_112024 \
    select molregno,ntabs from rdk.descriptors where ntabs<1000;
mrns,ntabs = zip(*d)
len(mrns)

cn = utils.connect(config=config) #< lwreg provides a convenience function to get a database connection
curs = cn.cursor()

curs.execute('create schema generated_data')
curs.execute('''create table generated_data.confgen
   (molregno integer references hashes, 
    conf_id integer references conformers,
    method text)''')
cn.commit()

mbs = utils.retrieve(ids=mrns)

from tqdm import tqdm
from rdkit.Chem import rdDistGeom
from rdkit import rdBase

Generate and store conformers

cn = utils.connect(config=config) #< lwreg provides a convenience function to get a database connection
curs = cn.cursor()

ps = rdDistGeom.ETKDGv3()
ps.randomSeed = 0xf00d
ps.pruneRmsThresh = 0.5
ps.numThreads = 8

for mrn,nt in tqdm(zip(mrns,ntabs)):
    mb = mbs[mrn][0]
    mol = Chem.MolFromMolBlock(mb,removeHs=False)
    if not mol:
        print(mrn)
        continue
    with rdBase.BlockLogs():
        cids = rdDistGeom.EmbedMultipleConfs(mol,2*nt,ps)
    reg = utils.register_multiple_conformers(mol=mol,fail_on_duplicate=False)
    
    rows = [(x,y,'ETKDGv3') for x,y in reg]
    if rows:
        curs.executemany('insert into generated_data.confgen values (%s,%s,%s)',rows)
        cn.commit()

2226it [21:11,  1.75it/s]

d = %sql postgresql://localhost/lobster_112024 \
    select molregno,nconfs,ntabs from \
    (select molregno, count(conf_id) nconfs from generated_data.confgen group by (molregno)) t1 \
      join rdk.descriptors using (molregno) where ntabs<1000;

Number of conformers generated per compound:

plt.Figure(figsize=(5,5))
plt.hist([x[2]-1 for x in d],bins=20);  #< it's -1 because we also have the crystal conformer
#plt.plot((0,1000),(0,1000),'k-');
plt.xlabel('nConfs generated');

nTABS is designed to be an upper limit on the number of conformers; let’s check to confirm that this works:

plt.Figure(figsize=(5,5))
plt.scatter([x[2]-1 for x in d],[x[1] for x in d]);
plt.plot((0,1000),(0,1000),'k-');
plt.xlabel('nTABS')
plt.ylabel('nConfs generated');

That works quite well in general, but let’s look at the two outliers

d = %sql postgresql://localhost/lobster_112024 \
    select molregno,canonical_smiles,nconfs,ntabs from \
    (select molregno, count(conf_id) nconfs from generated_data.confgen group by (molregno)) t1 \
      join rdk.descriptors using (molregno) \
      join hashes using (molregno) \
    where ntabs<1000 and ntabs>600 and nconfs-ntabs>100;

len(d)

Draw.MolsToGridImage([Chem.MolFromSmiles(x[1]) for x in d],subImgSize=(300,250),
                    legends=[f'mrn:{x[0]}: {x[2]}/{x[3]}' for x in d])

The underestimate of the number of conformers for compound 2980 are due to the triple bond. This is a known limitation of the TABS algorithm that was discussed in the paper and that we’re planning on working on.

The “extra” conformers observed in compound 298 are due to a bug in the conformer generation code that leads to non-physical conformers, I hope to have this one fixed in a future RDKit release.

Actually retrieve all of the conformers to see how we did at finding the crystal conformer in our conformer ensembles:

d = %sql postgresql://localhost/lobster_112024 \
    select molregno,conf_id,molblock from conformers order by (molregno,conf_id) asc;

Find the conformer for each molecule that’s closest to the crystal structure using RMSD as the metric:

from rdkit.Chem import rdMolAlign
accums = {}
r_mrn,r_cid,r_mb = d.pop(0)
r_mol = Chem.MolFromMolBlock(r_mb)
best = 1e8
best_cid = -1
for (mrn,cid,mb) in tqdm(d):
    mol = Chem.MolFromMolBlock(mb)
    if mrn==r_mrn:
        rms = rdMolAlign.GetBestRMS(r_mol,mol)
        if rms<best:
            best = rms
            best_cid = cid
    else:
        if best_cid > 0:
            accums[r_mrn] = (best,best_cid)
        r_mrn = mrn
        r_cid = cid
        r_mol = mol
        best = 1e8
        best_cid=-1

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 81315/81315 [00:14<00:00, 5507.62it/s]

plt.figure(figsize=(8,5))
ax = plt.subplot(1,1,1)
plt.hist([x[0] for x in accums.values()],bins=20);
plt.plot((0.5,0.5),(0,600),'k--');
plt.xlabel('RMSD ($\\AA$)')
plt.ylabel('count')
ax2 = ax.twinx()
ax2.ecdf([x[0] for x in accums.values()],c='C1');
ax2.grid();

import numpy as np
np.quantile([x[0] for x in accums.values()],[0.6,0.8,0.9])

array([0.50160024, 0.69941346, 0.88421735])

60% of molecules are within 0.50A of the crystal structure, 80% are within 0.70A, and 90% are within 0.88A. That’s pretty good.

Comparing shape-similarity approaches

The RDKit has a couple of built-in methods for doing shape similarity. Let’s compare those to each other on this data set:

Get a map from (nm,pdb) tuples to (molregno,confid,molblock):

d = %sql postgresql://localhost/lobster_112024 \
    select ligname,pdb,molregno,conf_id,molblock \
    from lobster_data.all_ligands join conformers using (molregno,conf_id);
ligs = {}
for nm,pdb,mrn,cid,mb in d:
    mol = Chem.MolFromMolBlock(mb,removeHs=False)
    mol_noh = Chem.MolFromMolBlock(mb)
    ligs[(nm,pdb.lower())] = (mrn,cid,mb,mol,mol_noh)

pairstats = %sql postgresql://localhost/lobster_112024 \
    select * from lobster_data.pair_stats
pairstats = list(pairstats.dicts())

Note that the scores for the ligand pairs in the LOBSTER data set are each present twice: 1. Ligand1 as reference, Ligand2 as probe 1. Ligand2 as reference, Ligand1 as probe

We can see this here:

len(pairstats)

%sql postgresql://localhost/lobster_112024 \
    select count(*) from lobster_data.pair_stats where (ligname1,pdb1)>(ligname2,pdb2)

count
36296

from scipy import stats

def comparison_plot(pairstats,metric1,metric2,invert1=False,invert2=False):
    x1 = [x[metric1] for x in pairstats]
    if invert1:
        x1 = [1-x for x in x1]
        metric1 = f'1-{metric1}'
    x2 = [x[metric2] for x in pairstats]
    if invert2:
        x2 = [1-x for x in x2]
        metric2 = f'1-{metric2}'
    r,_ = stats.spearmanr(x1,x2)
    tau,_ = stats.kendalltau(x1,x2)

    plt.figure(figsize=(10,5))
    plt.subplot(1,2,1)
    plt.scatter(x1,x2,alpha=0.5,s=5)
    plt.xlabel(metric1)
    plt.ylabel(metric2)
    plt.title(f'rho={r:.2f}, tau={tau:.2f}');
    plt.subplot(1,2,2)
    plt.hexbin(x1,x2,cmap='Blues');
    plt.tight_layout()

Calculate USR scores:

from rdkit.Chem import rdMolDescriptors
for d in pairstats:
    m1 = ligs[(d['ligname1'],d['pdb1'])][3]
    m2 = ligs[(d['ligname2'],d['pdb2'])][3]
    usr1 = rdMolDescriptors.GetUSR(m1)
    usr2 = rdMolDescriptors.GetUSR(m2)
    d['USR'] = rdMolDescriptors.GetUSRScore(usr1,usr2)
    
    m1 = ligs[(d['ligname1'],d['pdb1'])][4]
    m2 = ligs[(d['ligname2'],d['pdb2'])][4]
    usr1 = rdMolDescriptors.GetUSR(m1)
    usr2 = rdMolDescriptors.GetUSR(m2)
    d['USR_noh'] = rdMolDescriptors.GetUSRScore(usr1,usr2)

Calculate atom pair fingerprints using 3D distances instead of topological distances:

from rdkit.Chem import rdFingerprintGenerator
from rdkit import DataStructs
fpg = rdFingerprintGenerator.GetAtomPairGenerator(use2D=False)

for d in pairstats:
    m1 = ligs[(d['ligname1'],d['pdb1'])][3]
    m2 = ligs[(d['ligname2'],d['pdb2'])][3]
    fp1 = fpg.GetCountFingerprint(m1)
    fp2 = fpg.GetCountFingerprint(m2)
    d['AP3D'] = DataStructs.DiceSimilarity(fp1,fp2,returnDistance=True)
    
    m1 = ligs[(d['ligname1'],d['pdb1'])][4]
    m2 = ligs[(d['ligname2'],d['pdb2'])][4]
    fp1 = fpg.GetCountFingerprint(m1)
    fp2 = fpg.GetCountFingerprint(m2)
    d['AP3D_noh'] = DataStructs.DiceSimilarity(fp1,fp2,returnDistance=True)

comparison_plot(pairstats,'USR','AP3D',invert1=True,invert2=False)

comparison_plot(pairstats,'USR_noh','AP3D_noh',invert1=True,invert2=False)

Finally, try the E3FP fingerprint (a 3D analog of the Morgan fingerprint) from this paper. The code is easily pip installable directly from the github repo. Thanks to the Keiser lab team for making this so easy!

It takes a while to run this one:

import logging
from e3fp.pipeline import fprints_from_mol
logging.disable(logging.INFO)

def get_fp(m):
    fp = fprints_from_mol(m,fprint_params={'counts':True})[0]
    rdkfp = DataStructs.ULongSparseIntVect(2**32)
    for k,v in fp.counts.items():
        rdkfp[int(k)] = v 
    return rdkfp

for d in pairstats:
    m1 = ligs[(d['ligname1'],d['pdb1'])][3]
    m2 = ligs[(d['ligname2'],d['pdb2'])][3]
    fp1 = get_fp(m1)
    fp2 = get_fp(m2)
    d['E3FP'] = DataStructs.DiceSimilarity(fp1,fp2,returnDistance=True)
    
    m1 = ligs[(d['ligname1'],d['pdb1'])][4]
    m2 = ligs[(d['ligname2'],d['pdb2'])][4]
    fp1 = get_fp(m1)
    fp2 = get_fp(m2)
    d['E3FP_noh'] = DataStructs.DiceSimilarity(fp1,fp2,returnDistance=True)

comparison_plot(pairstats,'AP3D','E3FP',invert1=False,invert2=False)

comparison_plot(pairstats,'AP3D_noh','E3FP_noh',invert1=False,invert2=False)

Save the results in the database so that we can work with them later:

cn = utils.connect(config=config) #< lwreg provides a convenience function to get a database connection
curs = cn.cursor()

curs.execute('create schema if not exists results')
curs.execute('create table if not exists results.shape_scores \
  (ligname1 text, pdb1 text, ligname2 text, pdb2 text,\
   USR float, USR_noh float, AP3D float, AP3D_noh float, E3P float, E3P_noh float)')
curs.execute('delete from results.shape_scores')
rows = [[r['ligname1'],r['pdb1'],r['ligname2'],r['pdb2'],\
         r['USR'],r['USR_noh'],r['AP3D'],r['AP3D_noh'],r['E3FP'],r['E3FP_noh']] for r in pairstats]
curs.executemany('insert into results.shape_scores values (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)',rows)
cn.commit()

That’s it for this week

Working with the LOBSTER Data set I

Fri, 07 Nov 2025 23:00:00 GMT

Last year the Rarey and BioSolveIT groups published a paper describing LOBSTER, a data set of small molecule overlays drawn from the PDB. The carefully curated and constructed data set is intended to be a new benchmarking set for testing molecular alignment (superposition) tools. I’m really happy to have an up-to-date replacement for the older “AZ set”, which is what I’ve previously always used when looking at alignment. The well-written paper (as one expects from the Rarey group!) is definitely worth reading in order to understand the decisions made in the curation workflow. The code used for the curation is available in GitHub and the data set itself can be downloaded from Zenodo. There are links in the (open access) paper to both places.

I have a number of ideas for experiments to do using the LOBSTER set, so I wanted to get it loaded up into a database I could use for those experiments. This blog post demonstrates how I did that.

I’m using our lwreg tool to handle registering the compound structures and to provide the basic schema for the database. You can pip install it from our github repo I’ve blogged about using lwreg together with the RDKit PostgreSQL cartridge before.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole
IPythonConsole.ipython_3d = True
%load_ext sql
%config SqlMagic.feedback=0

The sql extension is already loaded. To reload it, use:
  %reload_ext sql

Registering the structures and populating the data base

import lwreg
from lwreg import utils

At the command line I created the postgres database:

% createdb lobster_112024

config = utils.defaultConfig()
config

{'dbname': './testdb.sqlt',
 'dbtype': 'sqlite3',
 'standardization': 'fragment',
 'removeHs': 1,
 'useTautomerHashv2': 0,
 'registerConformers': 0,
 'numConformerDigits': 3,
 'lwregSchema': '',
 'cacheConnection': True}

Configure lwreg to work with the database I created. We’ll turn off standardization and switch in to “registerConformers” mode.

config['dbtype'] = 'postgresql'
config['dbname'] = 'lobster_112024'
config['standardization'] = 'none'
config['registerConformers'] = 1
lwreg.set_default_config(config)

lwreg.initdb()

This will destroy any existing information in the registration database.
  are you sure? [yes/no]: yes

True

I downloaded the LOBSTER data from zenodo and extracted the zipfile locally.

Read in the SDFs containing the ligand structures and registering each 3D structure in lwreg:

import glob
sdfs = glob.glob('/scratch/Data/LOBSTER_112024/all_ligands/*.sdf')
registered = []
for sdf in sdfs:
    ms = [x for x in Chem.SDMolSupplier(sdf,removeHs=False) if x is not None]
    assert len(ms)==1
    m = ms[0]
    ligname = m.GetProp('_Name')
    ligpdb = m.GetProp('pdb_id')
    naomi_smis = m.GetProp('usmiles')
    mrn,confid = lwreg.register(mol=m)
    registered.append(((mrn,confid),ligname,ligpdb,naomi_smis,sdf))

Let’s store the additional information from the LOBSTER data set in a separate table in the database:

cn = utils.connect(config=config) #< lwreg provides a convenience function to get a database connection
curs = cn.cursor()

curs.execute('create schema lobster_data')
curs.execute('''create table lobster_data.all_ligands
   (molregno integer references hashes, 
    conf_id integer references conformers,
    ligname text, pdb text, naomi_smiles text, filename text)''')
rows = [(x[0][0],x[0][1],x[1],x[2],x[3],x[4]) for x in registered]
curs.executemany('insert into lobster_data.all_ligands values (%s,%s,%s,%s,%s,%s)',rows)
cn.commit()

%sql postgresql://localhost/lobster_112024 \
    select count(*) from lobster_data.all_ligands;

1 rows affected.

count
3583

Install the RDKit catridge, load the molecules, and create an index on the molecule column so that we can do efficient substructure searching. More information about that in this blog post

%sql postgresql://localhost/lobster_112024 \
  create extension if not exists rdkit; \
    create schema if not exists rdk; \
    drop table if exists rdk.mols cascade;\
    select molregno,mol_from_ctab(molblock::cstring,false) m into rdk.mols from molblocks;\
    create index molidx on rdk.mols using gist(m);

[]

Let’s add some fingerprints so that we can do similarity searches later. We’ll add both bit- and count-based Morgan fingerprints with radius 3:

%sql postgresql://localhost/lobster_112024 \
    drop table if exists rdk.fps;\
  select molregno,morganbv_fp(m,3) as mfp3, morgan_fp(m,3) cfp3 into rdk.fps from rdk.mols;\
    create index fps_mfp3_idx on rdk.fps using gist(mfp3); \
    create index fps_cfp3_idx on rdk.fps using gist(cfp3);

[]

And now add some descriptors we might want to use:

%sql postgresql://localhost/lobster_112024 \
    drop table if exists rdk.descriptors;\
  select molregno,mol_numheavyatoms(m) nhvy, mol_amw(m) amw, mol_numrotatablebonds(m) nrot into rdk.descriptors from rdk.mols;

[]

Add a column with nTABS, a measure of molecular flexibility developed by Jessica Braun in our lab (Jessica is also the other author/developer of lwreg). There’s a paper and a github repo from which you can install the code.

import tabs
import tqdm
d = %sql postgresql://localhost/lobster_112024 \
  select molregno,molblock from molblocks

ntabs = []
for mrn,mb in tqdm.tqdm(d):
    m = Chem.MolFromMolBlock(mb,removeHs=False)
    nt = tabs.GetnTABS(m)
    ntabs.append((mrn,nt))

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 3218/3218 [00:13<00:00, 245.40it/s]

Add the nTABS values to the database:

cn = utils.connect(config=config)
curs = cn.cursor()

intabs = [(y,x) for x,y in ntabs]
curs.execute('alter table rdk.descriptors add column ntabs int')
curs.executemany('update rdk.descriptors set ntabs=%s where molregno=%s',intabs)
cn.commit()

Load data about pairs of aligned ligands from the LOBSTER data:

ind = [x.strip().split(';') for x in open('/scratch/Data/LOBSTER_112024/stats/pair_stats.csv','r')]
len(ind)

ind.pop(0)

['query_file',
 'query',
 'pdb',
 'template_file',
 'template',
 'template_pdb',
 'ensemble',
 'morgan_fp_tanimoto',
 'gobbi_2D_pharmacophore_fp_tanimoto',
 'hac_difference',
 'shape_tversky_index',
 'shape_tanimoto_distance',
 'shape_protrude_distance',
 '']

And add that to the database too:

cn = utils.connect(config=config)
curs = cn.cursor()

curs.execute('''create table lobster_data.pair_stats
   (ligname1 text, pdb1 text, ligname2 text, pdb2 text, ensemble text,
    morgan_fp_tanimoto float, gobbi_2D_pharmacophore_fp_tanimoto float,
    hac_difference int, shape_tversky_index float, shape_tanimoto_distance float,
    shape_protrude_distance float)''')
rows = [(x[1],x[2],x[4],x[5],x[6],float(x[7]),float(x[8]),int(float(x[9])),float(x[10]),\
         float(x[11]),float(x[12]),) for x in ind]
curs.executemany('insert into lobster_data.pair_stats values (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)',rows)
cn.commit()

Composition of the database

Now let’s look at what we’ve got:

len(registered)

Number of unique molregnos:

%sql postgresql://localhost/lobster_112024 \
  select count(*) from hashes

count
3218

Number of conformers (this is the number we registered):

%sql postgresql://localhost/lobster_112024 \
  select count(*) from conformers

count
3583

Number of pairs of aligned molecules:

%sql postgresql://localhost/lobster_112024 \
  select count(*) from lobster_data.pair_stats

count
72592

Number of unique naomi_smiles:

%sql postgresql://localhost/lobster_112024 \
  select count(distinct naomi_smiles) from lobster_data.all_ligands

count
3212

We’ll look at the mismatch between the number of naomi_smiles and the number of molecule hashes below

Retrieving info from the database

Get all of the molecule hashes for one particular molregno using lwreg functions:

lwreg.retrieve(id=12,as_hashes=True)

{12: {'fullhash': '6d22ca45dd6fb8f29d263532db4f913e7da47d6a',
  'formula': 'C18H16N6',
  'canonical_smiles': 'CN(c1cnc2nc(N)nc(N)c2c1)c1cccc2ccccc12',
  'no_stereo_smiles': 'CN(c1cnc2nc(N)nc(N)c2c1)c1cccc2ccccc12',
  'tautomer_hash': 'CN([C]1[CH][N][C]2[N][C]([N])[N][C]([N])[C]2[CH]1)[C]1[CH][CH][CH][C]2[CH][CH][CH][CH][C]21_4_0',
  'no_stereo_tautomer_hash': 'CN([C]1[CH][N][C]2[N][C]([N])[N][C]([N])[C]2[CH]1)[C]1[CH][CH][CH][C]2[CH][CH][CH][CH][C]21_4_0',
  'escape': '',
  'sgroup_data': '[]'}}

We can see here that the Hs have been removed by lwreg when calculating the hashes used for comparing 2D molecules to each other.

The molecules stored in the database still have their Hs. We can see this by retrieving the registered mol block for the molecule:

l = Chem.MolFromMolBlock(lwreg.retrieve(id=12)[12][0],removeHs=False)
l

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

Ligands that appear multiple times in the data set will have the same molregno but different confids. Let’s find those:

from collections import defaultdict

repeated_ligands = defaultdict(list)

seen = set()
for tpl in registered:
    mrn,cid = tpl[0]
    if mrn in seen:
        repeated_ligands[mrn].append(cid)
    seen.add(mrn)
len(repeated_ligands)

Get the molregnos for the molecules that have more than two conformers present:

[k for k,v in repeated_ligands.items() if len(v)>2]

Each entry in the repeated_ligands dictionary includes the conf_ids of the conformers:

repeated_ligands[84]

[207, 215, 802, 968, 2150]

We can get the conformers themselves by calling lwreg.retrieve() with a list of (molregno,conf_id) tuples:

k = 84
confs = lwreg.retrieve(ids=[(k,x) for x in repeated_ligands[k]])

confs[(84,207)]

('AZM_E_303\n     RDKit          3D\n\n  0  0  0  0  0  0  0  0  0  0999 V3000\nM  V30 BEGIN CTAB\nM  V30 COUNTS 19 19 0 0 0\nM  V30 BEGIN ATOM\nM  V30 1 S 9.311200 -40.854300 -76.602200 0\nM  V30 2 S 10.494600 -38.554800 -74.957200 0\nM  V30 3 O 9.730100 -37.268900 -74.826900 0\nM  V30 4 O 10.160500 -39.565300 -73.911800 0\nM  V30 5 O 8.149900 -43.309900 -77.530300 0\nM  V30 6 N 12.092600 -38.206500 -74.914100 0\nM  V30 7 N 10.151000 -38.844000 -77.734300 0\nM  V30 8 N 9.709100 -39.647400 -78.701300 0\nM  V30 9 N 8.662000 -41.847200 -79.186200 0\nM  V30 10 C 7.602600 -44.129200 -79.665200 0\nM  V30 11 C 10.029000 -39.323500 -76.470700 0\nM  V30 12 C 9.187000 -40.841600 -78.320400 0\nM  V30 13 C 8.151000 -43.081400 -78.698500 0\nM  V30 14 H 12.620400 -39.051600 -74.999600 0\nM  V30 15 H 12.320600 -37.593200 -75.670300 0\nM  V30 16 H 8.653100 -41.677800 -80.171700 0\nM  V30 17 H 7.673700 -43.787600 -80.602400 0\nM  V30 18 H 8.131400 -44.973300 -79.577000 0\nM  V30 19 H 6.643600 -44.312500 -79.449200 0\nM  V30 END ATOM\nM  V30 BEGIN BOND\nM  V30 1 1 1 11\nM  V30 2 1 1 12\nM  V30 3 2 2 3\nM  V30 4 2 2 4\nM  V30 5 1 2 6\nM  V30 6 1 2 11\nM  V30 7 2 5 13\nM  V30 8 1 7 8\nM  V30 9 2 7 11\nM  V30 10 2 8 12\nM  V30 11 1 9 12\nM  V30 12 1 9 13\nM  V30 13 1 10 13\nM  V30 14 1 6 14\nM  V30 15 1 6 15\nM  V30 16 1 9 16\nM  V30 17 1 10 17\nM  V30 18 1 10 18\nM  V30 19 1 10 19\nM  V30 END BOND\nM  V30 END CTAB\nM  END\n',
 'mol')

Convert all of those mol blocks into RDKit molecules:

conf_mols = [Chem.MolFromMolBlock(confs[x][0],removeHs=False) for x in confs]

And look at the first couple:

conf_mols[0]

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

conf_mols[1]

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

Let’s get all the pairs of conformers that have a shape Tversky score of between 0.8 and 0.9:

d = %sql select ligname1,pdb1,ligname2,pdb2 from lobster_data.pair_stats \
  where shape_tversky_index<=0.9 and shape_tversky_index>0.8;
d[:5]

 * postgresql://localhost/lobster_112024

[('03V_A_2002', '3u3k', 'NPO_A_300', '3u3r'),
 ('NPO_A_300', '3u3r', '03V_A_2002', '3u3k'),
 ('052_A_809', '4zeg', 'O23_A_901', '3wzk'),
 ('HS4_A_0', '3f17', 'KLG_A_0', '3rts'),
 ('NGH_A_306', '5lab', 'HS3_A_0', '3f16')]

accum = []
for ln1,pdb1,ln2,pdb2 in d[:10]:
    d1 = %sql postgresql://localhost/lobster_112024\
        select molregno,conf_id from lobster_data.all_ligands where \
      ligname=:ln1 and upper(pdb)=upper(:pdb1);
    d2 = %sql postgresql://localhost/lobster_112024\
        select molregno,conf_id from lobster_data.all_ligands where \
      ligname=:ln2 and upper(pdb)=upper(:pdb2);
    accum.append((tuple(d1[0]),tuple(d2[0])))

Look at a couple of pairs:

confs = lwreg.retrieve(ids=accum[0])
mols = [Chem.MolFromMolBlock(v[0],removeHs=False) for v in confs.values()]
IPythonConsole.drawMols3D(mols)

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

confs = lwreg.retrieve(ids=accum[2])
mols = [Chem.MolFromMolBlock(v[0],removeHs=False) for v in confs.values()]
IPythonConsole.drawMols3D(mols)

3Dmol.js failed to load for some reason. Please check your browser console for error messages.

Look at duplicate mismatches

According to lwreg we have 3218 unique structures, while the canonical SMILES provided with the Lobster data set (generated using the Hamburg group’s Naomi package) says we have 3212. Let’s look at the mismatches

%sql postgresql://localhost/lobster_112024 \
  select count(distinct molregno) from hashes join lobster_data.all_ligands using (molregno);

count
3218

%sql postgresql://localhost/lobster_112024 \
  select naomi_smiles,count(distinct molregno) cnt \
     from hashes join lobster_data.all_ligands using (molregno)\
    group by (naomi_smiles) order by cnt desc limit 10;

naomi_smiles	cnt
S=P(OP(=O)(OP(=O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O)O)(O)O	3
P(=O)(OP(=O)(O)O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O	2
P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4N=C(NC(=O)C4N=C3)N)[C@@H]2O)CO1)O	2
P(=O)(OP(=O)(O)O)(OCC(=O)N1[C@H](C(=O)O)CCC1)O	2
P(=O)(O)([C@@H](N)C)C[C@H](C(=O)N[C@H](C(=O)O)C)Cc1ccc(c2ccccc2)cc1	2
P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4NC(=NC(=O)C4N=C3)N)[C@@H]2O)CO1)O	2
Brc1c2OC(=O)C(Br)=C(c2ccc1O)C	1
BrC=1C2=NC(=O)C(C=3Nc4c(cc(cc4)C(=O)O)C3NO)=C2C=CC1	1
Brc1c2c(OC(=O)C=C2CP(=O)(O)O)cc(c1)C	1
Brc1c(Br)c(Br)c2N(C(=Nc2c1Br)N(C)C)CC(=O)O	1

d = %sql postgresql://localhost/lobster_112024 \
  select molregno,canonical_smiles,naomi_smiles from \
      (select naomi_smiles,count(distinct molregno) cnt \
         from hashes join lobster_data.all_ligands using (molregno)\
         group by (naomi_smiles)) downsel join lobster_data.all_ligands using (naomi_smiles) \
          join hashes using (molregno)\
    where cnt>1 order by naomi_smiles asc;
d

molregno	canonical_smiles	naomi_smiles
927	Nc1nc2c(ncn2[C@@H]2O[C@@H]3CO[P@@](=O)(O)O[C@H]3[C@H]2O)c(=O)[nH]1	P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4N=C(NC(=O)C4N=C3)N)[C@@H]2O)CO1)O
1363	Nc1nc2c(ncn2[C@@H]2O[C@@H]3CO[P@](=O)(O)O[C@H]3[C@H]2O)c(=O)[nH]1	P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4N=C(NC(=O)C4N=C3)N)[C@@H]2O)CO1)O
1363	Nc1nc2c(ncn2[C@@H]2O[C@@H]3CO[P@](=O)(O)O[C@H]3[C@H]2O)c(=O)[nH]1	P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4N=C(NC(=O)C4N=C3)N)[C@@H]2O)CO1)O
927	Nc1nc2c(ncn2[C@@H]2O[C@@H]3CO[P@@](=O)(O)O[C@H]3[C@H]2O)c(=O)[nH]1	P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4N=C(NC(=O)C4N=C3)N)[C@@H]2O)CO1)O
2226	Nc1nc(=O)c2ncn([C@@H]3O[C@@H]4CO[P@](=O)(O)O[C@H]4[C@H]3O)c2[nH]1	P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4NC(=NC(=O)C4N=C3)N)[C@@H]2O)CO1)O
91	Nc1nc(=O)c2ncn([C@@H]3O[C@@H]4CO[P@@](=O)(O)O[C@H]4[C@H]3O)c2[nH]1	P1(=O)(O[C@@H]2[C@H](O[C@@H](N3C=4NC(=NC(=O)C4N=C3)N)[C@@H]2O)CO1)O
2558	C[C@H](NC(=O)[C@H](Cc1ccc(-c2ccccc2)cc1)C[P@](=O)(O)[C@H](C)N)C(=O)O	P(=O)(O)([C@@H](N)C)C[C@H](C(=O)N[C@H](C(=O)O)C)Cc1ccc(c2ccccc2)cc1
82	C[C@H](NC(=O)[C@H](Cc1ccc(-c2ccccc2)cc1)C[P@@](=O)(O)[C@H](C)N)C(=O)O	P(=O)(O)([C@@H](N)C)C[C@H](C(=O)N[C@H](C(=O)O)C)Cc1ccc(c2ccccc2)cc1
627	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@](=O)(O)OP(=O)(O)O)[C@@H](O)[C@H]1O	P(=O)(OP(=O)(O)O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O
1957	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@@](=O)(O)OP(=O)(O)O)[C@@H](O)[C@H]1O	P(=O)(OP(=O)(O)O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O
2256	O=C(O)[C@@H]1CCCN1C(=O)CO[P@@](=O)(O)OP(=O)(O)O	P(=O)(OP(=O)(O)O)(OCC(=O)N1[C@H](C(=O)O)CCC1)O
892	O=C(O)[C@@H]1CCCN1C(=O)CO[P@](=O)(O)OP(=O)(O)O	P(=O)(OP(=O)(O)O)(OCC(=O)N1[C@H](C(=O)O)CCC1)O
2877	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@@](=O)(O)O[P@](=O)(O)OP(O)(O)=S)[C@@H](O)[C@H]1O	S=P(OP(=O)(OP(=O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O)O)(O)O
1559	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@@](=O)(O)O[P@@](=O)(O)OP(O)(O)=S)[C@@H](O)[C@H]1O	S=P(OP(=O)(OP(=O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O)O)(O)O
1559	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@@](=O)(O)O[P@@](=O)(O)OP(O)(O)=S)[C@@H](O)[C@H]1O	S=P(OP(=O)(OP(=O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O)O)(O)O
2139	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@](=O)(O)O[P@](=O)(O)OP(O)(O)=S)[C@@H](O)[C@H]1O	S=P(OP(=O)(OP(=O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O)O)(O)O
2877	Nc1ncnc2c1ncn2[C@@H]1O[C@H](CO[P@@](=O)(O)O[P@](=O)(O)OP(O)(O)=S)[C@@H](O)[C@H]1O	S=P(OP(=O)(OP(=O)(OC[C@H]1O[C@@H](N2c3ncnc(N)c3N=C2)[C@H](O)[C@@H]1O)O)O)(O)O

These are all because the RDKit recognizes chiral phosphates and the Naomi SMILES seem not to.

Here are the first few molecules demonstrating this:

ms = []
for row in d:
    rm = Chem.MolFromSmiles(row[1])
    nm = Chem.MolFromSmiles(row[2])
    ms.extend((rm,nm))
Draw.MolsToGridImage(ms[:8],molsPerRow=2,subImgSize=(250,200))

I’m going to wrap this up here. I’ll come back to the LOBSTER data set and this database again in future posts.

How long does it take to do common tasks in the RDKit?

Thu, 30 Oct 2025 23:00:00 GMT

Years ago (back in the sourceforge days), I used to maintain a page with info about how long it took to do common operations in the RDKit. This was useful for both reference purposes and to track the evolution of RDKit performance over time. At some point I stopped doing this, but a recently merged PR got me thinking about this again (@Andrew: thanks for that contribution!).

I’m going to use this notebook to explain and run some new benchmarks (these are different from the PR mentioned above, which is meant to run as part of the RDKit build process). The results, including historical results, are tabulated in the wiki I will update this post as I add more benchmarks.

Let me know if you have ideas for interesting and useful benchmarks I should add!

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole

%load_ext sql

Get the SMILES we will be working with:

Get 10000 random compounds from ChEMBL that have a pchembl_value >= 9 and don’t have multiple components.

This stuff doesn’t need to be run every time, so I’m not saving the cells as code.

d = %sql postgresql://localhost/chembl_36 \
  select distinct(canonical_smiles) canonical_smiles,chembl_id from compound_structures tablesample bernoulli(20) repeatable (123892) \
    join chembl_id_lookup on (molregno=entity_id and entity_type='COMPOUND') \
    join activities using (molregno) \
    where activities.pchembl_value>=9 and \
    position('.' in canonical_smiles)=0 \
    limit 10000;

The distinct(canonical_smiles) query I did orders the results, so it looks like all of the compounds have isotopes specified. This is not actually the case:

Make sure all of those convert cleanly into molecules:

sum(1 for x,y in d if Chem.MolFromSmiles(x) is not None)

with open('../data/chembl36_very_active.txt','w+') as outf:
    outf.write('chembl_id canonical_smiles\n')
    for smi,cid in d:
        outf.write(f'{cid} {smi}\n')

Run the benchmarks

import rdkit
print(rdkit.__version__)

2025.09.1

with open('../data/chembl36_very_active.txt','r') as inf:
    ls = [x.strip().split() for x in inf]
    ls.pop(0)
    data = [(smi,cid) for cid,smi in ls]
len(data)

Construct molecule from SMILES

%timeit ms = [Chem.MolFromSmiles(smi) for smi,cid in data]

1.2 s ± 11.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

ms = [Chem.MolFromSmiles(smi) for smi,cid in data]

Generate canonical SMILES

%timeit [Chem.MolToSmiles(m) for m in ms]

658 ms ± 2.92 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

from rdkit.Chem import rdDepictor

Generating 2D coordinates

rdDepictor.SetPreferCoordGen(False)
%timeit [rdDepictor.Compute2DCoords(m) for m in ms]

47.4 s ± 482 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

rdDepictor.SetPreferCoordGen(True)
%timeit [rdDepictor.Compute2DCoords(m) for m in ms]

1min 55s ± 334 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

ms2d = [Chem.Mol(m) for m in ms]
_ = [rdDepictor.Compute2DCoords(m) for m in ms2d]

Writing mol blocks

%timeit [Chem.MolToMolBlock(m) for m in ms2d]

877 ms ± 9.83 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit [Chem.MolToV3KMolBlock(m) for m in ms2d]

1.08 s ± 11.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

mbs = [Chem.MolToMolBlock(m) for m in ms2d]
mbs3k = [Chem.MolToV3KMolBlock(m) for m in ms2d]

Parsing mol blocks

%timeit [Chem.MolFromMolBlock(m) for m in mbs]

[08:31:09] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:11] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:12] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:14] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:16] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:17] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:19] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:21] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored

1.71 s ± 17.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit [Chem.MolFromMolBlock(m) for m in mbs3k]

[08:31:23] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:24] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:26] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:28] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:30] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:32] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:34] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored
[08:31:36] Warning: ambiguous stereochemistry - zero final chiral volume - at atom 54 ignored

1.87 s ± 16.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Adding/removing Hs

%timeit [Chem.AddHs(m) for m in ms]

456 ms ± 2.34 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

mhs = [Chem.AddHs(m) for m in ms]

%timeit [Chem.RemoveHs(m) for m in mhs]

1.52 s ± 5.47 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Conformer generation

from rdkit.Chem import rdDistGeom

ps = rdDistGeom.EmbedParameters()
ps.randomSeed = 0xf00d
%timeit [rdDistGeom.EmbedMolecule(m,ps) for m in mhs[:1000]]

[08:14:36] UFFTYPER: Unrecognized charge state for atom: 38
[08:15:19] UFFTYPER: Unrecognized charge state for atom: 38
[08:16:03] UFFTYPER: Unrecognized charge state for atom: 38
[08:16:48] UFFTYPER: Unrecognized charge state for atom: 38
[08:17:32] UFFTYPER: Unrecognized charge state for atom: 38
[08:18:16] UFFTYPER: Unrecognized charge state for atom: 38
[08:19:00] UFFTYPER: Unrecognized charge state for atom: 38
[08:19:45] UFFTYPER: Unrecognized charge state for atom: 38

44.1 s ± 170 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

ps = rdDistGeom.ETKDGv3()
ps.randomSeed = 0xf00d
%timeit [rdDistGeom.EmbedMolecule(m,ps) for m in mhs[:1000]]

[08:36:46] UFFTYPER: Unrecognized charge state for atom: 38
[08:37:52] UFFTYPER: Unrecognized charge state for atom: 38
[08:38:58] UFFTYPER: Unrecognized charge state for atom: 38
[08:40:03] UFFTYPER: Unrecognized charge state for atom: 38
[08:41:09] UFFTYPER: Unrecognized charge state for atom: 38
[08:42:14] UFFTYPER: Unrecognized charge state for atom: 38
[08:43:20] UFFTYPER: Unrecognized charge state for atom: 38
[08:44:25] UFFTYPER: Unrecognized charge state for atom: 38

1min 5s ± 245 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Generate fingerprints

from rdkit.Chem import rdFingerprintGenerator

fpg = rdFingerprintGenerator.GetMorganGenerator(radius=3)
%timeit fpg.GetFingerprints(ms)

497 ms ± 3.53 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

fpg = rdFingerprintGenerator.GetMorganGenerator(radius=2)
%timeit fpg.GetFingerprints(ms)

379 ms ± 4.65 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

fpg = rdFingerprintGenerator.GetRDKitFPGenerator()
%timeit fpg.GetFingerprints(ms)

13.1 s ± 71.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

fpg = rdFingerprintGenerator.GetRDKitFPGenerator(maxPath=5)
%timeit fpg.GetFingerprints(ms)

4.48 s ± 27.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

fpg = rdFingerprintGenerator.GetAtomPairGenerator()
%timeit fpg.GetFingerprints(ms)

1.65 s ± 8.14 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

fpg = rdFingerprintGenerator.GetTopologicalTorsionGenerator()
%timeit fpg.GetFingerprints(ms)

1.36 s ± 4.22 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit [Chem.PatternFingerprint(m) for m in ms]

2.49 s ± 39.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Displaying atom maps and highlighting with reactions

Thu, 16 Oct 2025 22:00:00 GMT

This one was inspired by an example I did this week for the class I’m teaching: I wanted to apply an RDKit reaction to a set of reactants and then draw the reactants and the products of the reaction as a normal reaction with the mapped atoms indicated. Doing that made me realize that, with a bit more code, I could produce some other useful views of a reaction.

In addition to the visualizations themselves, this post has some potentially useful details about what kind of extra information is available in the products of reactions.

from rdkit import Chem
from rdkit.Chem import Draw
from rdkit.Chem.Draw import IPythonConsole
from rdkit.Chem import rdChemReactions

import rdkit
print(rdkit.__version__)

2025.09.1

The first example reaction, adapted from the SI for http://pubs.acs.org/doi/abs/10.1021/ci200379p. I frequently use this paper as a source of reaction SMARTS definitions for real reactions; left to my own devices I would just use amide bond formation all the time, and that gets pretty boring pretty quick.

# adapted example from the SI for: http://pubs.acs.org/doi/abs/10.1021/ci200379p
sma = '[Br,I;$(*c1ccccc1)]-[c:1]:[c:2]-[OH1:3].[CH1:5]#[C;$(C-[#6]):4]>>[c:1]1:[c:2]-[O:3]-[C:4]=[C:5]-1'
rxn = rdChemReactions.ReactionFromSmarts(sma)
rxn

A set of reactants for the reaction:

r1 = Chem.MolFromSmiles('c1cc(I)c(O)cc1')
r2 = Chem.MolFromSmiles('C1CCC1C#C')
Draw.MolsToGridImage([r1,r2],molsPerRow=2)

Run the reaction and show the product:

reactants = (r1,r2)

ps = rxn.RunReactants(reactants)
ps[0][0]

Look at the properties set on one of the atoms in the product that came from a mapped atom:

prod = ps[0][0]
print('mapped atom:',prod.GetAtomWithIdx(0).GetPropsAsDict(includePrivate=False))
print('unmapped atom:',prod.GetAtomWithIdx(8).GetPropsAsDict(includePrivate=False))

mapped atom: {'old_mapno': 1, 'react_atom_idx': 2, 'react_idx': 0}
unmapped atom: {'react_atom_idx': 6, 'react_idx': 0}

Here’s what those mean: 1. old_mapno: the atom map number for the atom (obviously only present on mapped atoms) 2. react_idx: which reactant the atom came from 3. react_atom_idx: the index of the atom in its reactant

What I did for the course was set the atom map numbers on the reactants and products, combine them into a new reaction, and then display that reaction.

Start by setting the atom map numbers:

# since we're going to modify things, copy them first:
prod = Chem.Mol(ps[0][0])
reactants = (Chem.Mol(r1),Chem.Mol(r2))

for at in prod.GetAtoms():
    pd = at.GetPropsAsDict()
    if 'old_mapno' not in pd:
        continue
    r = reactants[pd['react_idx']]
    rat = r.GetAtomWithIdx(pd['react_atom_idx'])
    rat.SetAtomMapNum(pd['old_mapno'])
    at.SetAtomMapNum(pd['old_mapno'])

Now create the reaction and display it:

nrxn = rdChemReactions.ChemicalReaction()
nrxn.AddReactantTemplate(reactants[0])
nrxn.AddReactantTemplate(reactants[1])
nrxn.AddProductTemplate(prod)

IPythonConsole.molSize = 600,150
nrxn

Once the atom-mapping information is there, The reaction drawing code can also highlight the atoms based upon which reactant they came from. Unfortunately this causes the atom-mapping information to not be displayed:

IPythonConsole.highlightByReactant = True
nrxn

IPythonConsole.highlightByReactant = False

At this point it makes sense to take what we know and write a function to draw the highlighted reaction. By working directly with a MolDraw2DCairo object instead of using the notebook integration we more easily control what’s going on.

from IPython.display import Image
def drawHighlightedReaction(rxn, reacts, prods, 
                            includeAtomMaps=True, highlightAllAtoms=True,
                            mapAllAtoms=False,
                            highlightColors=None,
                            size=(900,200), annotationFontScale=0.74,
                            drawOptions=None):
    ''' draws a specific reaction with the reactants and products highlighted
    Returns an Image object with the drawing.    
    
    Arguments
    rxn: the reaction object (not currently used)
    reacts: a sequence of molecules. The reactants used in the reaction
    prods: a sequence of molecules. The products from the reaction
    includeAtomMaps: bool. Whether or not atom map numbers should be included in the output
    highlightAllAtoms: bool. Whether or not to highlight all reactant/product atoms in the output. 
                           If True, non-mapped atoms will be highlighted. 
                           If False, only the mapped atoms will be highlighted
    mapAllAtoms: bool. Whether or not to include atom mapping numbers on all atoms.
                           If True, non-mapped atoms will have negative atom map numbers displayed
    highlightColors: sequence of 3-tuples. Controls the colors used for highlighting the reactants.
                           The values should go from 0-1. The sequence should have (at least) len(reacts) 
                           elements.
    size: tuple. Controls the size of the output image.
    annotationFontScale: float. Controls the size of the atom map notes (if being drawn)
    drawOptions: a MolDraw2DOptions object. Used as the draw options for the rendering. 
                           Overrides annotationFontScale if provided.
    
    '''
    # make copies of all the reactants and the products since we will modify them
    reacts = [Chem.Mol(r) for r in reacts]
    prods = [Chem.Mol(p) for p in prods]
    
    # find the largest atom map number, used to initialize the negative atom map numbers
    #  when we are doing highlightAllAtoms
    negVal = 0
    if mapAllAtoms:
        for prod in prods:
            for at in prod.GetAtoms():
                if at.HasProp('old_mapno'):
                    negVal = min(negVal,-1 * at.GetIntProp('old_mapno'))
    negVal -= 1

    # loop over each of the products and set the atom map and note information
    #  in both the product atoms and corresponding reactant atoms.
    for prod in prods:
        for at in prod.GetAtoms():
            pd = at.GetPropsAsDict()
            mno = pd.get('old_mapno',negVal)
            if mno<0:
                if not highlightAllAtoms:
                    continue
                else:
                    negVal -= 1
            
            r = reacts[pd['react_idx']]
            rat = r.GetAtomWithIdx(pd['react_atom_idx'])
            for tat in at,rat:
                tat.SetAtomMapNum(mno)
                if includeAtomMaps and (mno>0 or mapAllAtoms):
                    tat.SetProp('atomNote',str(mno))    
    
    # create the reaction we'll actually render:
    nrxn = rdChemReactions.ChemicalReaction()
    for react in reacts:
        nrxn.AddReactantTemplate(react)
    for prod in prods:
        nrxn.AddProductTemplate(prod)

    # and draw it
    d2d = Draw.MolDraw2DCairo(size[0],size[1])
    if drawOptions is not None:
        d2d.SetDrawOptions(drawOptions)
    else:
        d2d.drawOptions().annotationFontScale=annotationFontScale
    d2d.DrawReaction(nrxn, highlightByReactant=True, highlightColorsReactants=highlightColors)
    d2d.FinishDrawing()
    return Image(d2d.GetDrawingText())

r1 = Chem.MolFromSmiles('c1cc(I)c(O)cc1')
r2 = Chem.MolFromSmiles('C1CCC1C#C')
reactants = (r1,r2)
ps = rxn.RunReactants(reactants)

drawHighlightedReaction(rxn,reactants,ps[0])

drawHighlightedReaction(rxn,reactants,ps[0],highlightAllAtoms=False)

Include negative atom map numbers for atoms that were not in the reaction definition;

drawHighlightedReaction(rxn,reactants,ps[0],mapAllAtoms=True)

drawHighlightedReaction(rxn,reactants,ps[0],includeAtomMaps=False)

Change the highlighting

drawHighlightedReaction(rxn,reactants,ps[0],highlightColors=[(0.3, 0.7, 0.9), (0.6, 0.9, 0.3)])

Provide our own draw options. Here we play with dark mode:

from rdkit.Chem import rdDepictor
for r in reactants:
    rdDepictor.Compute2DCoords(r)
dopts = Draw.MolDrawOptions()
Draw.SetDarkMode(dopts)
dopts.annotationFontScale = 0.8
drawHighlightedReaction(rxn,reactants,ps[0],drawOptions=dopts,
                       highlightColors=[(0.3, 0.7, 0.9), (0.3, 0.6, 0.1)])

Do another reaction from the same paper:

rxn = rdChemReactions.ReactionFromSmarts('[c:1](-[C;$(C-c1ccccc1):2](=[OD1:3])-[OH1]):[c:4](-[NH2:5]).[N;!H0;!$(N-N);!$(N-C=N);!$(N(-C=O)-C=O):6]-[C;H1,$(C-[#6]):7]=[OD1]>>[c:4]2:[c:1]-[C:2](=[O:3])-[N:6]-[C:7]=[N:5]-2')
reactants = [Chem.MolFromSmiles(x) for x in ('c1c(C(=O)O)c(N)ccc1','CCC(=O)Nc1ccccc1')]
prods = rxn.RunReactants(reactants)
drawHighlightedReaction(rxn,reactants,prods[0])

Similarity search time and thresholds

Thu, 09 Oct 2025 22:00:00 GMT

This is an updated and modified version of a post I wrote back in 2015. # Impact of the threshold on similarity search times

A ChEMBL blog post included the (to me) somewhat surprising result that similarity search times while using the RDKit cartridge did not show much of a dependency on the similarity threshold being used. I wanted to investigate this a bit more closely.

I’m using a local install of ChEMBL35 for these tests. Since I don’t have the patience to wait for searches to complete with 1000 molecules, I will randomly select just 10:

chembl_35=# select * into temporary table foo from rdk.fps order by random() limit 10;
SELECT 10
chembl_35=# select molregno,m from rdk.mols join foo using (molregno);
molregno |                                                      m                                                      
----------+-------------------------------------------------------------------------------------------------------------
    4111 | C=C1CC2(CCCCCCCCC2)OC1=O
    4131 | CC1Cc2cc3c(cc2C(c2ccc(N)cc2)=NN1C=O)OCO3
    4185 | CCOC(=O)/C=C(C)/C(F)=C/C=C(C)/C=C/c1c(C)cc(OC)c(C)c1C
    4256 | CC(C)(C)c1cc(CCc2cccnc2)cc(C(C)(C)C)c1O
    4292 | O=C(CCCCCCCCCBr)CC(=O)N[C@H]1CCOC1=O
    4334 | CCCC[C@@H](C[C@@H](CCc1ccc(-c2ccc(F)cc2)cc1)C(=O)N[C@H](C(=O)Nc1ccccc1)C(C)(C)C)C(=O)O
    4335 | COc1ccc(/C=C/c2cc(C(C)(C)C)c(O)c(C(C)(C)C)c2)cc1
    4389 | Cc1ccc2c(c1)C(=O)N(CC(C)(C)C[N+](C)(C)CCCCCC[N+](C)(C)CC(C)(C)CN1C(=O)c3cccc4cccc(c34)C1=O)C2=O.[Br-].[Br-]
    4591 | O=C(NCCCn1ccnc1)c1cc2ccccc2[nH]1
    4599 | O=Cc1ccn(-c2cc3c(cc2Cl)nc(O)n2nc(C(=O)O)cc32)c1
(10 rows)

And now look at timing results for a number of threshold values:

chembl_35=# set rdkit.tanimoto_threshold=0.4;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.202 ms
count 
-------
3431
(1 row)

Time: 2446.118 ms (00:02.446)
chembl_35=# set rdkit.tanimoto_threshold=0.5;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.245 ms
count 
-------
814
(1 row)

Time: 2258.420 ms (00:02.258)
chembl_35=# set rdkit.tanimoto_threshold=0.6;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.146 ms
count 
-------
261
(1 row)

Time: 2005.068 ms (00:02.005)
chembl_35=# set rdkit.tanimoto_threshold=0.7;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.304 ms
count 
-------
122
(1 row)

Time: 1702.164 ms (00:01.702)
chembl_35=# set rdkit.tanimoto_threshold=0.8;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.147 ms
count 
-------
    35
(1 row)

Time: 1272.436 ms (00:01.272)
chembl_35=# set rdkit.tanimoto_threshold=0.9;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.207 ms
count 
-------
    21
(1 row)

Time: 645.774 ms
chembl_35=# set rdkit.tanimoto_threshold=0.95;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.101 ms
count 
-------
    17
(1 row)

Time: 268.827 ms
chembl_35=# set rdkit.tanimoto_threshold=0.99;select count(*) from rdk.fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.242 ms
count 
-------
    14
(1 row)

Time: 54.065 ms

In the previous version of the post, there were some interesting differences here. That’s no longer the case. this is probably at least partially because both Postgres and the cartridge have seen some improvements in the last ten years (for example, index-only scans are now possible). I guess the bigger part is because I’m using a more capable machine with a bunch more RAM.

A larger test: ZINC

To see if there are any trends for a larger dataset, I grabbed a copy of the ZINC All Clean set. After removing the molecules that the RDKit doesn’t like (there are actually only 14 molecules in the full set that the RDKit rejected), this leaves 16.4 million molecules.

For the record, here’s how I built the database:

(rdkit_build) glandrum@stoat:/scratch/RDKit_git/LocalData$ createdb zinc
(rdkit_build) glandrum@stoat:/scratch/RDKit_git/LocalData$ psql -c 'create table raw_data (id SERIAL, smiles text, zinc_id char(12))' zinc
CREATE TABLE
(rdkit_build) glandrum@stoat:/scratch/RDKit_git/LocalData$ cd Zinc
(rdkit_build) glandrum@stoat:/scratch/RDKit_git/LocalData/Zinc$ zcat zinc_all_clean.smi.gz | sed '1d; s/\\/\\\\/g' |psql -c "copy raw_data (smiles,zinc_id) from stdin with delimiter ' '" zinc
COPY 16403864
(rdkit_build) glandrum@stoat:/scratch/RDKit_git/LocalData/Zinc$ psql zinc
psql (14.19 (Ubuntu 14.19-0ubuntu0.22.04.1))
Type "help" for help.

zinc=# \timing
Timing is on.
zinc=# create extension rdkit;
CREATE EXTENSION
Time: 21.754 ms
zinc=# select * into mols from (select id,mol_from_smiles(smiles::cstring) m from raw_data) tmp where m is not null;
  ... snip ...
SELECT 16403848
Time: 945195.267 ms (15:45.195)
zinc=# select id,morganbv_fp(m) as mfp2 into fps from mols;
SELECT 16403848
Time: 155185.639 ms (02:35.186)
zinc=# create index fps_mfp2_idx on fps using gist(mfp2);
CREATE INDEX
Time: 140542.861 ms (02:20.543)
zinc=# select * into temporary table foo from fps tablesample bernoulli (1) repeatable (123456) limit 10;
SELECT 10
Time: 11.977 ms

Here are the 10 random molecules:

zinc=# select id,m from mols join foo using (id);
id    |                             m                             
------+-----------------------------------------------------------
12317 | CCc1nnc(NC(=O)[C@H](CC)Oc2ccccc2)s1
385   | CCOC(=O)c1ccccc1C(=O)OCC
24317 | CC(C)(C)c1ccc([C@@]23OC(=O)C(C)(C)[C@@H]2OC(=O)C3(C)C)cc1
48    | CC(C)(C)[NH2+]C[C@H](O)COc1cc(Cl)ccc1Cl
225   | CC[C@@]1(CO)CCC[NH+]2CCc3c([nH]c4ccccc34)[C@@H]21
90    | C[NH2+]C[C@@H](OC)c1cccc(C(F)(F)F)c1
24406 | CCOC1(OCC)[NH+]=C(N)[C@@]2(C#N)C3(CCCCC3)[C@@]12C#N
24738 | Cc1ccc(S(=O)(=O)Nc2ccccc2C#N)cc1
356   | CCc1c(O)c(=O)ccn1CC
8     | O=C([O-])[C@@H](O)c1ccccc1
(10 rows)

Time: 5167.539 ms (00:05.168)

Now the searches:

zinc=# set rdkit.tanimoto_threshold=0.4; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.133 ms
count 
-------
55762
(1 row)

Time: 15931.816 ms (00:15.932)
zinc=# set rdkit.tanimoto_threshold=0.5; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.183 ms
count 
-------
4486
(1 row)

Time: 13616.273 ms (00:13.616)
zinc=# set rdkit.tanimoto_threshold=0.6; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.205 ms
count 
-------
623
(1 row)

Time: 11555.706 ms (00:11.556)
zinc=# set rdkit.tanimoto_threshold=0.7; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.213 ms
count 
-------
165
(1 row)

Time: 8509.171 ms (00:08.509)
zinc=# set rdkit.tanimoto_threshold=0.8; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.097 ms
count 
-------
    50
(1 row)

Time: 4717.467 ms (00:04.717)
zinc=# set rdkit.tanimoto_threshold=0.9; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.097 ms
count 
-------
    20
(1 row)

Time: 1026.567 ms (00:01.027)
zinc=# set rdkit.tanimoto_threshold=0.95; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.103 ms
count 
-------
    20
(1 row)

Time: 195.971 ms
zinc=# set rdkit.tanimoto_threshold=0.99; select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
SET
Time: 0.096 ms
count 
-------
    20
(1 row)

Time: 34.866 ms

I’m going to look at two interesting steps in this data: from a threshold of 0.4 to 0.6, and from 0.6 to 0.99. In the first case, the number of hits goes from 55762 to 623, while the time goes from 15.9s to 11.6s. In the second case, the number of hits goes from 623 to 20, while the time goes from 11.6s to 34.9ms. In the first case the number if hits drops by almost two orders of magnitude while the time only drops by about 25%. In the second case, the number of hits drops by a factor of 30, while the time drops by a factor of almost 39.

Look at the output from EXPLAIN ANALYZE for these queries to see if we can figure out what’s going on:

zinc=# set rdkit.tanimoto_threshold=0.4;
SET
Time: 0.152 ms
zinc=# explain (analyze on, buffers on) select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
                                                                    QUERY PLAN                                                                      
------------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate  (cost=1376580.94..1376580.95 rows=1 width=8) (actual time=16239.641..16239.642 rows=1 loops=1)
Buffers: shared hit=9573 read=2374633, local hit=1
->  Nested Loop  (cost=0.42..1324498.62 rows=20832924 width=0) (actual time=8.908..16236.122 rows=55762 loops=1)
        Buffers: shared hit=9573 read=2374633, local hit=1
        ->  Seq Scan on foo  (cost=0.00..22.70 rows=1270 width=32) (actual time=0.004..0.016 rows=10 loops=1)
            Buffers: local hit=1
        ->  Index Only Scan using fps_mfp2_idx on fps fps1  (cost=0.42..878.85 rows=16404 width=65) (actual time=6.231..1622.310 rows=5576 loops=10)
            Index Cond: (mfp2 % foo.mfp2)
            Heap Fetches: 0
            Buffers: shared hit=9573 read=2374633
Planning Time: 0.052 ms
JIT:
Functions: 5
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.126 ms, Inlining 1.992 ms, Optimization 3.165 ms, Emission 3.565 ms, Total 8.848 ms
Execution Time: 16239.872 ms
(16 rows)

Time: 16240.250 ms (00:16.240)
zinc=# set rdkit.tanimoto_threshold=0.6;
SET
Time: 0.114 ms
zinc=# explain (analyze on, buffers on) select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
                                                                    QUERY PLAN                                                                      
------------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate  (cost=1376580.94..1376580.95 rows=1 width=8) (actual time=13299.969..13299.970 rows=1 loops=1)
Buffers: shared hit=4685 read=1995775, local hit=1
->  Nested Loop  (cost=0.42..1324498.62 rows=20832924 width=0) (actual time=112.152..13299.772 rows=758 loops=1)
        Buffers: shared hit=4685 read=1995775, local hit=1
        ->  Seq Scan on foo  (cost=0.00..22.70 rows=1270 width=32) (actual time=0.004..0.013 rows=10 loops=1)
            Buffers: local hit=1
        ->  Index Only Scan using fps_mfp2_idx on fps fps1  (cost=0.42..878.85 rows=16404 width=65) (actual time=112.027..1329.058 rows=76 loops=10)
            Index Cond: (mfp2 % foo.mfp2)
            Heap Fetches: 0
            Buffers: shared hit=4685 read=1995775
Planning Time: 0.052 ms
JIT:
Functions: 5
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.126 ms, Inlining 2.130 ms, Optimization 3.170 ms, Emission 3.601 ms, Total 9.027 ms
Execution Time: 13300.146 ms
(16 rows)

Time: 13300.681 ms (00:13.301)
zinc=# set rdkit.tanimoto_threshold=0.99;
SET
Time: 0.253 ms
zinc=# explain (analyze on, buffers on) select count(*) from fps fps1 cross join foo where fps1.mfp2%foo.mfp2;
                                                                QUERY PLAN                                                                   
------------------------------------------------------------------------------------------------------------------------------------------------
Aggregate  (cost=1376580.94..1376580.95 rows=1 width=8) (actual time=50.800..50.801 rows=1 loops=1)
Buffers: shared hit=14558 read=5872, local hit=1
->  Nested Loop  (cost=0.42..1324498.62 rows=20832924 width=0) (actual time=11.722..50.787 rows=12 loops=1)
        Buffers: shared hit=14558 read=5872, local hit=1
        ->  Seq Scan on foo  (cost=0.00..22.70 rows=1270 width=32) (actual time=0.003..0.010 rows=10 loops=1)
            Buffers: local hit=1
        ->  Index Only Scan using fps_mfp2_idx on fps fps1  (cost=0.42..878.85 rows=16404 width=65) (actual time=2.392..4.188 rows=1 loops=10)
            Index Cond: (mfp2 % foo.mfp2)
            Heap Fetches: 0
            Buffers: shared hit=14558 read=5872
Planning Time: 0.052 ms
JIT:
Functions: 5
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 0.123 ms, Inlining 2.134 ms, Optimization 3.203 ms, Emission 3.554 ms, Total 9.014 ms
Execution Time: 50.972 ms
(16 rows)

Time: 51.323 ms

` The big difference I see here is the number of buffers that actually need to be read when executing the queries. At the 0.4 threshold, we need to read 2.37 million buffers, at 0.7 it’s just under 2 million, and at 0.99 we only need to read 5872. This is presumably because the index is pruning so many rows at the higher thresholds. I’m definitely not an expert at interpreting this output, so if anyone has other ideas or explanations, please let me know!

An aside: chemfp performance

The previous version of this post included a performance comparison with chemfp. Since I no longer have access to the licensed version of chemfp, I’m going to skip that here. I assume it’s dramatically quicker than the cartridge. :-)

Rendering intramolecular H bonds in 2D

Fri, 03 Oct 2025 22:00:00 GMT

I had the idea for this short post while working on last week’s post about drawing simple protein–ligand interaction diagrams. It shows a quick way to force the RDKit’s 2D coordinate generator to put non-bonded atoms close to each other in a drawing. I use the examples of rendering intramolecular H bonds and chelators.

from rdkit import Chem
from rdkit.Chem.Draw import IPythonConsole
from rdkit.Chem import Draw
from rdkit.Chem import rdDepictor

IPythonConsole.drawOptions.addAtomIndices = True
IPythonConsole.molSize = 350,300

Let’s start with a simple molecule that forms intramolecular hydrogen bonds, malonaldehyde (I also used this as the example in the blog post on generating 3D conformers which include intramolecular H bonds).

The 2D coordinate generation code generates an extended 2D conformer for malonaldehyde:

m = Chem.MolFromSmiles('O=CCCO')
m

This type of extended conformer looks completely reasonable and is, for most molecules, almost certainly the right way to draw chains, but in this case I want to actually show the H bond.

To do so, I start by adding the H which will be involved in the H bond to the molecule:

m2 = Chem.RWMol(m)
aid = m2.AddAtom(Chem.Atom(1))
m2.AddBond(4,aid,Chem.BondType.SINGLE)
m2

We still have an extended conformer. I can force this to change by doing something non-physical and adding a single bond between the H and O0:

bid1 = m2.AddBond(0,aid,Chem.BondType.SINGLE)
m2

That has the coordinates I wanted, but having the H bond drawn as a single bond is definitely not desirable.

To solve this I’m going to explicitly generate the coordinates for the molecule with the single bond present (instead of relying up on the notebook code to generate a temporary set of coordinates), then remove the single bond and render the molecule:

rdDepictor.Compute2DCoords(m2)
m2.RemoveBond(0,aid)
m2

That’s what I was looking for.

If I want to actually include the H bond in the drawing in a way that won’t make chemists cringe, I can add a hydrogen bond:

bid1 = m2.AddBond(0,aid,Chem.BondType.HYDROGEN)
m2

Note that simply adding the hydrogen bond to start with would not have worked since zero-order bonds are not included in the RDKit’s ring-finding code by default:

m3 = Chem.RWMol(m)
aid = m3.AddAtom(Chem.Atom(1))
m3.AddBond(4,aid,Chem.BondType.SINGLE)
m3.AddBond(0,aid,Chem.BondType.HYDROGEN)
m3

Here’s a more complex molecule where I want to show two intramolecular H bonds:

m = Chem.MolFromSmiles('c1cccc(C(=O)OC)c1NC(=O)C1CCCCC1(=O)')
m

In this case I would like to indicate H bonds between the H on N10 and O6 as well as the H on C0 and O12.

# Add the Hs:
m2 = Chem.RWMol(m)
aid1 = m2.AddAtom(Chem.Atom(1))
m2.AddBond(0,aid1,Chem.BondType.SINGLE)
aid2 = m2.AddAtom(Chem.Atom(1))
m2.AddBond(10,aid2,Chem.BondType.SINGLE)

# Add single bonds for the intramolecular H bonds:
bid1 = m2.AddBond(aid1,12,Chem.BondType.SINGLE)
bid2 = m2.AddBond(aid2,6,Chem.BondType.SINGLE)

# generate coordinates:
rdDepictor.Compute2DCoords(m2)

# convert to H bonds:
m2.GetBondWithIdx(bid1-1).SetBondType(Chem.BondType.HYDROGEN)
m2.GetBondWithIdx(bid2-1).SetBondType(Chem.BondType.HYDROGEN)

m2

We can use a similar trick to get conformers of chelators that have the chelating atoms in the expected geometry.

m = Chem.MolFromSmiles('NCCN')
m

This time I’m going to add an extra atom (to stand in for the atom which will eventually be chelated), form bonds to that, and then remove that atom after generating 2D coordinates:

m2 = Chem.RWMol(m)
aid = m2.AddAtom(Chem.Atom(0))
m2.AddBond(0,aid,Chem.BondType.SINGLE)
m2.AddBond(3,aid,Chem.BondType.SINGLE)
rdDepictor.Compute2DCoords(m2)
m2.RemoveAtom(aid)
m2

Drawing simple protein–ligand interaction diagrams with the RDKit

Thu, 25 Sep 2025 22:00:00 GMT

I’m a big fan of 2D protein–ligand interaction diagrams; when well done these plots can provide an information-dense view of the structure that is still easy to understand.

A really nice example of this is the work on PoseView from Matthias Rarey’s group in Hamburg. Here’s an example from that publication:

image

Though I would love to have an RDKit implementation of PoseView, actually implementing something like that is deeply nontrivial, so I’ve never really done anything in that direction. This week I had a random idea for a way to provide basic protein–ligand interaction diagrams using existing RDKit functionality. This post is an exploration of that.

Here’s an example of what you get with this:

image-2.png

To be clear: I am fully aware that this is not nearly as good as what PoseView and similar tools can do, but I think it’s still quite useful. I have a few ideas for straightforward changes to the backend code to improve the plots that I’m also going to take a look at.

from rdkit import Chem
from rdkit.Chem import rdDepictor
from rdkit.Chem import Draw
from IPython.display import SVG
from rdkit.Chem.Draw import IPythonConsole
IPythonConsole.molSize = 400,400
import rdkit
print(rdkit.__version__)

2025.03.6

Initial exploration

Start with the ligand for a recent PDB structure, 8yqe:

# from https://www.ebi.ac.uk/pdbe/entry/pdb/8yqe?activeTab=ligands
lig = Chem.MolFromSmiles('CCS(=O)(=O)N1CCC(CC1)NC(=O)c2c(cn[nH]2)NC(=O)c3c(ccc(c3F)C4CCOCC4)F')
IPythonConsole.drawOptions.addAtomIndices = True
lig

The strategy to draw the interaction diagrams is to add a new dummy atom to the ligand molecule for each residue it’s interacting with and then connect that to the interacting ligand atom with a zero-order bond. The standard RDKit 2D coordinate generation code will then do something sensible with this.

I include hbond interactions that are around 3.0 or less from the list of interactions on the ligand page

lig_with_interactions = Chem.RWMol(lig)
leu83 = Chem.Atom(0)
leu83.SetProp('atomLabel','Leu 83')
aid = lig_with_interactions.AddAtom(leu83)
lig_with_interactions.AddBond(18,aid,Chem.BondType.ZERO)
lig_with_interactions.AddBond(11,aid,Chem.BondType.ZERO)

hoh407 = Chem.Atom(0)
hoh407.SetProp('atomLabel','H2O 407')
aid = lig_with_interactions.AddAtom(hoh407)
lig_with_interactions.AddBond(21,aid,Chem.BondType.ZERO)

hoh441 = Chem.Atom(0)
hoh441.SetProp('atomLabel','H2O 441')
aid = lig_with_interactions.AddAtom(hoh441)
lig_with_interactions.AddBond(13,aid,Chem.BondType.ZERO)

# this doesn't work because the ring centroid doesn't get put in the middle of the ring
# ring_center = Chem.Atom(0)
# aid = lig_with_interactions.AddAtom(ring_center)
# for rid in (22,23,24,25,26,27):
#     lig_with_interactions.AddBond(rid,aid,Chem.BondType.ZERO)

lig_with_interactions

Making it easier to use

That doesn’t look terrible. Write a function to automate the process and include highlights around the residue pseudo-atoms:

def draw_ligand_with_interactions(lig,lig_name,interactions,size=(400,400)):   
    lig_with_interactions = Chem.RWMol(lig)
    
    # add pseudo-atoms (and bonds to them) for the interacting residues:
    pts = []
    clrs = {}
    for (aname,oaids) in interactions:
        res = Chem.Atom(0)
        res.SetProp('atomLabel',aname)
        aid = lig_with_interactions.AddAtom(res)
        pts.append(aid)
        clrs[aid] = (1,.2,1,.3)
        for oaid in oaids:
            lig_with_interactions.AddBond(aid,oaid,Chem.BondType.ZERO)
   
    d2d = Draw.MolDraw2DSVG(size[0],size[1])
    
    # set the draw options so that we end up with circles under the pseudo-atoms:
    d2d.drawOptions().circleAtoms = True
    d2d.drawOptions().fillHighlights = True
    d2d.drawOptions().continuousHighlight = False
    d2d.drawOptions().highlightRadius = 0.5
    
    # now draw and return the result
    d2d.DrawMolecule(lig_with_interactions,legend=lig_name,
                     highlightAtoms=pts,highlightAtomColors=clrs)
    d2d.FinishDrawing()
    return SVG(d2d.GetDrawingText())

# from https://www.ebi.ac.uk/pdbe/entry/pdb/8yqe?activeTab=ligands
lig = Chem.MolFromSmiles('CCS(=O)(=O)N1CCC(CC1)NC(=O)c2c(cn[nH]2)NC(=O)c3c(ccc(c3F)C4CCOCC4)F')
interactions = (
    ('LYS 89',(3,)),
    ('ASP 86',(4,)),
    ('HOH 444',(3,)),
    ('HOH 441',(13,)),
        ('HOH 407',(21,)),
        ('LEU 83',(18,11)),
)
draw_ligand_with_interactions(lig,'A1D60 from 8yqe',interactions)

Do a couple more examples using other recent PDB structures:

# from: https://www.ebi.ac.uk/pdbe/entry/pdb/9uhc?activeTab=ligands
lig = Chem.MolFromSmiles('CCC(=O)Nc1ccc2c(c1)n(cc2c3c[nH]c4c3nc(cn4)c5cnn(c5)CCN6CCOCC6)C')
interactions = (
    ('ALA 564',(22,)),
    ('GLU 562',(16,)),
    ('ASP 641',(4,)),
    ('CYS 488',(3,)),
)
draw_ligand_with_interactions(lig,'A1EPE from 9uhc',interactions)

# from https://www.ebi.ac.uk/pdbe/entry/pdb/9vci?activeTab=ligands
lig = Chem.MolFromSmiles('CC(=O)NCc1cc(c(cc1F)O)Oc2ccc(cc2)N3c4c(ncnc4N(C3=O)C5CCNCC5)N')
interactions = (
        ('GLY 9',(2,)),
        ('ASP 135',(12,)),
        ('THR 48',(23,)),
        ('HOH 637',(29,)),
)
draw_ligand_with_interactions(lig,'A1ERR from 9vci',interactions)

There’s clearly room for improvement here (and I have a couple of ideas already), but I think this is already useful as-is.

2025 RDKit UGM Recap

Sat, 13 Sep 2025 22:00:00 GMT

The 2025 RDKit UGM took place last week (September 10-12, 2025) in Prague. This year’s installment of the meeting was organized by Martin Šícho at the University of Chemistry and Technology, Prague. From my perspective the meeting went very well: the logistics were smooth, the technology mostly worked - we had some problems with sound on the first day, but got past those (why is it always sound?), and the program had a good balance of talks and breaks. Many thanks to Martin and the Prague group for putting together a great meeting and to the companies that sponsored the meeting (listed in the github repo) for enabling us to continue to run the meeting without having to charge registration fees.

We had 120 registered attendees; as usual we were at capacity (the limiting factor was the size of the lecture hall). This year we only had a couple of no-shows (a common problem with free meetings). We had the usual good mix of academic and industrial attendees and of people in different stages of their careers. The meeting was live streamed over zoom and there were typically 40-60 people watching the talks remotely and participating via the discord server.

There were 19 standard talks (we did a mix of 20- and 30-minute slots this year and I think that worked pretty well), 10 lightning talks, and 20 posters. On Friday we had the hackathon and three workshops. I haven’t heard much feedback about the workshops yet, but there were a number of pull requests submitted to the RDKit repo during the hackathon and in the next couple of days, so it looks like that was productive.

The github repo is here: https://github.com/rdkit/UGM_2025. The slides from the presentations will show up in the repo as speakers send them to me or submit PRs themselves. I will be uploading the videos of the talks to the YouTube playlist over the next few weeks as I have time to process them.

I, unsurprisingly, really enjoyed the meeting. I was exhausted at the end of each day, and totally wiped out on Friday, but that’s 100 percent expected. Some personal thoughts/impressions:

I had never been to Prague before and very much enjoyed the bits of it I saw. The UGM itself kept me busy during the days, but the area around the campus was quite lively and I did get to see some other parts while running (pro tip: don’t try and run along the river in the afternoon… there are way too many people out and about! Early mornings were a lot more runnable).
As always, it was great to see old friends and meet new people. My tendency, particularly during meetings like this, is to spend most of my time talking to people I already know, but I think I did a decent job this year of tempering that.
It was really nice to have all four RDKit maintainers (Brian, Paolo, Ricardo, and myself) together in one place. I’m not sure when the last time that happened was.
The quality of the talks was good. We had a couple that were more marketing-heavy than I would of liked, but they were definitely the exception. As usual, there was a nice mix of academic and industrial, method-development and application, deeply technical and high-level.
One recurring theme this year was the handling of sequence-based entities and/or polymers. There were several talks that touched on this topic from different angles and I’m really curious to see how this will evolve in the RDKit. This is an area where I don’t have a lot of personal experience, so it’s going to be fun for me to see how the RDKit community tackles this.

I came back from the meeting with a long list of things I want to work on in the RDKit, and that’s a really good sign. I hope that other attendees came away similarly inspired!

I will announce the dates and locations of the 2026 European UGM as soon as they are finalized. If you are interested in hosting a future UGM, please get in touch with me.

The 2025 RDKit UGM is next week

Sat, 06 Sep 2025 22:00:00 GMT

The 2025 RDKit UGM takes place next week (September 10-12, 2025) in Prague. The github repo is here: https://github.com/rdkit/UGM_2025.

The UGM is “sold out”, so we can’t accept any last-minute registrations, but we will be live streaming it over zoom. I will be posting the links to the zoom sessions in the repo as well as in the UGM discord server.

The upcoming meeting is my official excuse for why there is no real blog post this week (I also took a few days off to run an ultra-marathon, but I’m not allowed to use that as an excuse for skipping a blog post. ;-)). Hopefully I’ll manage to put something real together next week after the UGM.

Scaling conformer generation

Fri, 29 Aug 2025 22:00:00 GMT

This week the question came up in the lab of how well the conformer generation code scales with the number of threads used. Since generating conformers is embarassingly parallel - the conformers don’t depend on each other (this isn’t quite true if you are doing RMS pruning) - in a perfect world you’d expect more or less linear scaling. So theoretically using 4 times as many threads should take 1/4 as much time as long as the number of threads is less than the number of conformers being generated. In reality, things don’t quite work out this way since the individual conformers can take different amounts of time to generate due to the stochastic nature of the RDKit’s distance-geometry-based algorithm.

I decided to do a quick test to see how well things scale on my machine.

Aside: this post is looking at runtimes of conformer generation for individual molecules. We can further increase the speed of conformer generation for large sets of molecules by splitting the work across multiple machines. That’s a topic for a different post

import gzip
from rdkit import Chem
from rdkit.Chem import rdDistGeom

import time
from tqdm import tqdm

from matplotlib import pyplot as plt
%matplotlib inline
plt.style.use('tableau-colorblind10')

import rdkit
print(rdkit.__version__)

2025.03.5

Start by loading some molecules. This is a set of 370 molecules that are present in both the COD and ChEMBL. I use these (as well as some other COD subsets) a lot for confgen testing:

ms = [x for x in Chem.ForwardSDMolSupplier(gzip.open('../data/COD_2025Jan13.organic.chembl_selected.sdf.gz'),
                                          removeHs=False) if x is not None]
len(ms)

For the purposes of this post we’ll just use 100 of the COD molecules. Add Hs to the mols and create the subset:

ms = [Chem.AddHs(m) for m in ms]
ms = ms[:100]

My linux box has 8 performance cores (each of which can run two hyperthreads) and 8 “efficient” cores, so python thinks I have 24 CPUs available:

import multiprocessing
multiprocessing.cpu_count()

I will try thread counts up to 10 to check scaling; a bit beyond the number of physical performance cores on my machine.

Now try generating both 100 conformer and 400 conformer sets for each molecule using different thread counts and track how long it takes:

threadCounts = [1,2,4,6,8,10]

params = rdDistGeom.ETKDGv3()
params.randomSeed = 0xa100f

tgtConfs = (100,400)
accum = {}
for tgt in tgtConfs:
    print(f"Doing {tgt} conformers")
    accum[tgt] = {}
    for tc in threadCounts:
        params.numThreads = tc
        accum[tgt][tc] = []
        print(f"Doing {tc} threads")
        for m in tqdm(ms):
            t1 = time.time()
            rdDistGeom.EmbedMultipleConfs(m,tgt,params)
            t2 = time.time()
            accum[tgt][tc].append(t2-t1)
        print(f'\t{sum(accum[tgt][tc]):.1f}')

Doing 100 conformers
Doing 1 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [02:25<00:28,  1.65s/it][16:37:31] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [02:37<00:00,  2.19it/s][16:37:43] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [02:39<00:00,  1.59s/it]

    159.0
Doing 2 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [01:17<00:14,  1.15it/s][16:39:02] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [01:24<00:00,  3.86it/s][16:39:08] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:25<00:00,  1.18it/s]

    85.0
Doing 4 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [00:42<00:08,  2.12it/s][16:39:51] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [00:45<00:00,  6.30it/s][16:39:55] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:46<00:00,  2.17it/s]

    46.1
Doing 6 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [00:29<00:05,  2.93it/s][16:40:25] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [00:32<00:00,  9.02it/s][16:40:27] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:32<00:00,  3.09it/s]

    32.3
Doing 8 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [00:24<00:04,  3.57it/s][16:40:52] UFFTYPER: Unrecognized charge state for atom: 0
 96%|██████████████████████████████████████████████████████████████████████████████████████████████    | 96/100 [00:26<00:00,  8.34it/s][16:40:54] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:26<00:00,  3.74it/s]

    26.7
Doing 10 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [00:25<00:04,  3.44it/s][16:41:19] UFFTYPER: Unrecognized charge state for atom: 0
 96%|██████████████████████████████████████████████████████████████████████████████████████████████    | 96/100 [00:26<00:00,  7.85it/s][16:41:21] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:27<00:00,  3.67it/s]

    27.2
Doing 400 conformers
Doing 1 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [09:37<01:48,  6.40s/it][16:50:59] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [10:25<00:03,  1.82s/it][16:51:47] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [10:29<00:00,  6.30s/it]

    629.9
Doing 2 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [05:04<00:57,  3.37s/it][16:56:56] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [05:29<00:01,  1.04it/s][16:57:21] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [05:32<00:00,  3.32s/it]

    332.1
Doing 4 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [02:38<00:29,  1.76s/it][17:00:02] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [02:51<00:00,  2.01it/s][17:00:15] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [02:52<00:00,  1.72s/it]

    172.4
Doing 6 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [01:48<00:20,  1.22s/it][17:02:04] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [01:57<00:00,  2.87it/s][17:02:14] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:58<00:00,  1.18s/it]

    118.4
Doing 8 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [01:30<00:17,  1.01s/it][17:03:44] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [01:37<00:00,  3.39it/s][17:03:52] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:38<00:00,  1.02it/s]

    98.1
Doing 10 threads

 83%|█████████████████████████████████████████████████████████████████████████████████▎                | 83/100 [01:36<00:18,  1.10s/it][17:05:29] UFFTYPER: Unrecognized charge state for atom: 0
 98%|████████████████████████████████████████████████████████████████████████████████████████████████  | 98/100 [01:44<00:00,  3.17it/s][17:05:37] UFFTYPER: Unrecognized charge state for atom: 8
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [01:45<00:00,  1.05s/it]

    105.4

Calculate the relative runtime for each molecule with each threadcount. If everything is scaling perfectly, we’d expect doubling the number of threads to cut the runtime in half.

factors = {}
for nc in accum:
    factors[nc] = {}
    for tc in accum[nc]:
        if tc==1:
            factors[nc][tc] = [1]*len(ms)
        else:
            factors[nc][tc] = []
            for i,m in enumerate(ms):
                factors[nc][tc].append(accum[nc][tc][i]/accum[nc][1][i])

Now plot the results.

plt.figure(figsize=(10,5))
xp = [1+x-0.2 for x in range(len(threadCounts))]
widths = [0.3 for x in range(len(threadCounts))]
plt.boxplot([factors[100][tc] for tc in threadCounts],positions=xp,widths=widths,label='100 conformers');
xp = [1+x+0.2 for x in range(len(threadCounts))]
plt.boxplot([factors[400][tc] for tc in threadCounts],positions=xp,widths=widths,label='400 conformers');
plt.xticks(range(1, len(threadCounts) + 1),[str(x) for x in threadCounts]);
plt.xlabel('number of threads');
plt.ylabel('scaling factor')
plt.plot((1.7,2.3),(0.5,0.5),'k--');
plt.plot((2.7,3.3),(0.25,0.25),'k--');
plt.plot((3.7,4.3),(1/6,1/6),'k--');
plt.plot((4.7,5.3),(1/8,1/8),'k--');
plt.plot((5.7,6.3),(1/10,1/10),'k--');

The left-hand boxplot in each column is the results for 100 conformers, the right-hand boxplot is for 400 conformers. The dashed line in each column shows what you would expect for perfect scaling.

We can see that on my machine with these molecules the scaling is pretty good up to about 6 threads but that by the time we get to 10 threads we’re deviating pretty strongly from perfect scaling. In fact, the runtime for 10 threads isn’t massively better than it is for 8 (the number of physical performance cores in my machine). Unsurprisingly, the scaling is better for 400 conformers than it is for 100, but even with 400 conformers it’s not really worth increasing from 8 to 10 threads on my machine.

Based on this analysis, I’d probably go with using either 6 or 8 threads for future confgen work on this machine.

How the 2D/3D flag in Mol blocks is used

Thu, 21 Aug 2025 22:00:00 GMT

[This is an expanded version of a section which will be added to the RDKit documentation to clarify an important detail about the way Mol and SD files are parsed.]

Background

Mol blocks, Mol files, and SD files can describe 2D or 3D molecules and include a flag (called “dimensional code” in the spec) to indicate whether the coordinates are 2D or 3D. The flag shows up in the second line of the Mol block, and is present in both V2000:

nitrogen
     RDKit          2D

  2  1  0  0  0  0  0  0  0  0999 V2000
    0.0000    0.0000    0.0000 N   0  0  0  0  0  0  0  0  0  0  0  0
   -0.0000   -1.5000    0.0000 N   0  0  0  0  0  0  0  0  0  0  0  0
  1  2  3  0
M  END

and V3000 mol blocks:

nitrogen
     RDKit          2D

  0  0  0  0  0  0  0  0  0  0999 V3000
M  V30 BEGIN CTAB
M  V30 COUNTS 2 1 0 0 0
M  V30 BEGIN ATOM
M  V30 1 N 0.000000 0.000000 0.000000 0
M  V30 2 N -0.000000 -1.500000 0.000000 0
M  V30 END ATOM
M  V30 BEGIN BOND
M  V30 1 3 1 2
M  V30 END BOND
M  V30 END CTAB
M  END

The CTFile specification says the following about this flag: > The “dimensional code” is maintained explicitly. Thus “3D” really means 3D, > although “2D” will be interpreted as 3D if any non-zero Z-coordinates are > found

That’s simple (and logical) enough, but of course in the real world things are more complicated. Files found “in the wild” include every possible combination of 2D/3D flag and 2D/3D coordinates, so we need to decide how to interpret these combinations. Things are made more complicated by the possible presence of wedged bonds in the Mol block. Wedged bonds are typically a signal that something is known about the stereochemistry of the molecule; how do we combine this information with the coordinates?

Aside on 2D vs 3D coordinates: in Mol blocks X, Y, and Z coordinates are always provided (if any of them are missing, they are set to zero). It’s also possible to have 3D coordinates for planar molecules. For the purposes of this discussion, “2D” coordinates means that all Z coordinates are zero, and a “2D” RDKit molecule is one where the (default) conformer is marked as being 2D (i.e. the conformer’s Is3D() method returns False).

What the RDKit does

The following table describes how the RDKit interprets all possible combinations of dimensionality flag, coordinate dimensionality, and the presence or absence of wedged bonds:

flag	coords	wedging	result	notes
2D	2D	no	2D	no chirality
3D	2D	no	3D	no chirality
3D	3D	no	3D	chirality from coords
2D	3D	no	3D	chirality from coords
2D	2D	yes	2D	chirality from wedging
3D	2D	yes	2D	chirality from wedging
3D	3D	yes	3D	chirality from coords
2D	3D	yes	3D	chirality from coords

This is consistent with what the specification says except for the case where the 3D flag is set for 2D coordinates and a wedge is present, in which case we ignore the 3D flag, mark the conformer as 2D, and set the stereochemistry based on the wedging.

In cases where no 2D/3D flag is provided, the default value of the flag is 2D.

In a 2D structure, wedging is interpreted as a signal to indicate that stereochemistry is present and to indicate what the stereochemistry is (i.e. the stereochemistry is determined based on the 2D coordinates and the direction of the wedge).

When 3D coordinates are provided, stereochemistry is perceived from the coordinates themselves. If wedging is also present it will be ignored; the 3D signal is “stronger” than the wedging signal. The exception to this rule is atropisomeric bonds, where the wedged bond indicates that atropisomerism should be perceived, but the direction of the wedge is ignored.

A BRICS tutorial

Thu, 14 Aug 2025 22:00:00 GMT

This post is a short tutorial on using the RDKit’s BRICS implementation, expanding a bit on what’s in the documentation.

BRICS is a method for fragmenting molecules into smaller pieces along bonds which are likely to be synthetically accessible. The original paper describing the method is:

Degen, J.; Wegscheid-Gerlach, C.; Zaliani, A.; Rarey, M. On the Art of Compiling and Using “Drug-Like” Chemical Fragment Spaces. ChemMedChem 2008, 3 (10), 1503–1507. https://chemistry-europe.onlinelibrary.wiley.com/doi/full/10.1002/cmdc.200800178

from rdkit import Chem
from rdkit.Chem import rdDepictor
rdDepictor.SetPreferCoordGen(True)
from rdkit.Chem.Draw import IPythonConsole
from rdkit.Chem import Draw
from rdkit.Chem import BRICS
import rdkit
print(rdkit.__version__)

2025.03.5

BRICS basics

esomeprazole = Chem.MolFromSmiles('COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(OC)c3C)nc2c1')
rdDepictor.Compute2DCoords(esomeprazole)
esomeprazole

pieces = BRICS.BRICSDecompose(esomeprazole)
pieces

{'[14*]c1ncc(C)c([16*])c1C',
 '[3*]OC',
 '[8*]C[S+]([O-])c1nc2cc([16*])ccc2[nH]1'}

Draw.MolsToGridImage([Chem.MolFromSmiles(x) for x in pieces])

The atom environments and connection rules can be found in Scheme 2 of the BRICS paper:

BRICS connection rules

Note that the RDKit BRICS implementation does not contain a definition of L2, a three connected N with a lone pair. We incorporated that with L5 in a more general definition of amine. In the RDKit implementation L5 can connect with L1, L4, L12, L13, L14, L15, and L16. We also added a few additional connection definitions. The actual SMARTS used in the RDKit can be found in MolFragmenter.cpp along with the and the connection rules. There is also a Python version of the definitions in BRICS.py, but this is no longer used and may not exactly the match the C++ implementation.

The RDKit’s BRICS implementation includes the ability to create new molecules by stitching fragments together according to the connection rules above. Here’s an illustration of that showing all the molecules that can be formed by combining the three fragments from esomeprazole using the BRICS connection rules with a maximum enumeration depth of 2:

piecems = [Chem.MolFromSmiles(smi) for smi in pieces]
builder = BRICS.BRICSBuild(piecems, maxDepth=2, scrambleReagents=False)
ms = list(builder)
Draw.MolsToGridImage(ms)

Enumerating a larger set of molecules

Here I grab the ChEMBL molecules that were most similar to esomeprazole, fragment them together with esomeprazole, and then explore some of the resulting molecules.

Aside: grabbing similar molecules from ChEMBL

Here’s the query I executed and the results returned:

chembl_35=# select chembl_id,m,similarity from get_mfp2_neighbors('COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(OC)c3C)nc2c1') join chembl_id_lookup on (molregno=entity_id and entity_type='COMPOUND') join molecule_hierarchy using (molregno) where molregno=parent_molregno and similarity<1 order by similarity desc limit 20;
   chembl_id   |                              m                              |     similarity     
---------------+-------------------------------------------------------------+--------------------
 CHEMBL9890    | COc1c(C)cnc(C[S+]([O-])c2nc3cc(OC(F)F)ccc3[nH]2)c1C         | 0.8333333333333334
 CHEMBL5089043 | COc1c(C)cnc(C[S@@+]([O-])c2nc3cc(OC(F)F)ccc3[nH]2)c1C       | 0.8333333333333334
 CHEMBL5076667 | COc1c(C)cnc(C[S@+]([O-])c2nc3cc(OC(F)F)ccc3[nH]2)c1C        | 0.8333333333333334
 CHEMBL3527071 | COc1ccc2[nH]c([S+]([O-])Cc3ncc(CO)c(OC)c3C)nc2c1            | 0.8148148148148148
 CHEMBL4525760 | COc1ccc2[nH]c([S+]([O-])Cc3ncc(C(=O)O)c(OC)c3C)nc2c1        |                0.8
 CHEMBL10184   | COc1c(C)cnc(C[S+]([O-])c2nc3ccccc3[nH]2)c1C                 | 0.7692307692307693
 CHEMBL9430    | COc1cnc(C[S+]([O-])c2nc3cc(OC(F)F)ccc3[nH]2)c(C)c1OC        | 0.7288135593220338
 CHEMBL59068   | COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N(C)C)c3Cl)nc2c1         | 0.7192982456140351
 CHEMBL138250  | COc1c(C)cnc(C[S+]([O-])c2nc3cscc3[nH]2)c1C                  | 0.7169811320754716
 CHEMBL5070031 | COc1ccc2[nH]c([S@@+]([O-])Cc3nccc(OC)c3C)nc2c1              | 0.7090909090909091
 CHEMBL10061   | COc1cnc(C[S+]([O-])c2nc3cc(OC(F)(F)C(F)F)ccc3[nH]2)c(C)c1OC | 0.6935483870967742
 CHEMBL1475252 | COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(OC)c3C)nc2n1             | 0.6909090909090909
 CHEMBL9992    | COc1cnc(C[S+]([O-])c2nc3cc(OCC(F)(F)F)ccc3[nH]2)c(C)c1OC    | 0.6885245901639344
 CHEMBL440676  | COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N4CCCC4)c3Cl)nc2c1       | 0.6721311475409836
 CHEMBL20315   | COc1ccc2[nH]c([S+]([O-])Cc3c(N)cc(C)c(OC)c3C)nc2c1          | 0.6666666666666666
 CHEMBL59784   | COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N4CCCCC4)c3Cl)nc2c1      | 0.6612903225806451
 CHEMBL144285  | COc1ccc2[nH]c([S+]([O-])Cc3nccc(OC(C)C)c3C)nc2c1            |               0.65
 CHEMBL341550  | COc1ccc(-c2scc3[nH]c([S+]([O-])Cc4ncc(C)c(OC)c4C)nc23)cc1   |               0.65
 CHEMBL1796802 | COc1cc2nc([S+]([O-])Cc3ncc(C)c(OC)c3C)[nH]c2cc1NC(C)=O      | 0.6451612903225806
 CHEMBL58833   | COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N4CCOCC4)c3Cl)nc2c1      |           0.640625
(20 rows)

the very useful get_mfp2_neighbors() function is from the cartridge documentation.

Here’s the set of molecules we’ll use:

smis = '''COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(OC)c3C)nc2c1
COc1c(C)cnc(C[S+]([O-])c2nc3cc(OC(F)F)ccc3[nH]2)c1C
COc1ccc2[nH]c([S+]([O-])Cc3ncc(CO)c(OC)c3C)nc2c1
COc1ccc2[nH]c([S+]([O-])Cc3ncc(C(=O)O)c(OC)c3C)nc2c1
COc1c(C)cnc(C[S+]([O-])c2nc3ccccc3[nH]2)c1C
COc1cnc(C[S+]([O-])c2nc3cc(OC(F)F)ccc3[nH]2)c(C)c1OC
COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N(C)C)c3Cl)nc2c1
COc1c(C)cnc(C[S+]([O-])c2nc3cscc3[nH]2)c1C
COc1ccc2[nH]c([S+]([O-])Cc3nccc(OC)c3C)nc2c1
COc1cnc(C[S+]([O-])c2nc3cc(OC(F)(F)C(F)F)ccc3[nH]2)c(C)c1OC
COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(OC)c3C)nc2n1
COc1cnc(C[S+]([O-])c2nc3cc(OCC(F)(F)F)ccc3[nH]2)c(C)c1OC
COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N4CCCC4)c3Cl)nc2c1
COc1ccc2[nH]c([S+]([O-])Cc3c(N)cc(C)c(OC)c3C)nc2c1
COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N4CCCCC4)c3Cl)nc2c1
COc1ccc2[nH]c([S+]([O-])Cc3nccc(OC(C)C)c3C)nc2c1
COc1ccc(-c2scc3[nH]c([S+]([O-])Cc4ncc(C)c(OC)c4C)nc23)cc1
COc1cc2nc([S+]([O-])Cc3ncc(C)c(OC)c3C)[nH]c2cc1NC(C)=O
COc1ccc2[nH]c([S+]([O-])Cc3ncc(C)c(N4CCOCC4)c3Cl)nc2c1'''.split('\n')
mols = [Chem.MolFromSmiles(smi) for smi in smis]
Draw.MolsToGridImage(mols,molsPerRow=4)

Fragment all the molecules and keep the unique fragments:

allfrags=set()
for m in mols:
    pieces = BRICS.BRICSDecompose(m)
    allfrags.update(pieces)
allfrags

{'[1*]C(C)=O',
 '[14*]c1ncc(C)c([16*])c1C',
 '[14*]c1ncc(C)c([16*])c1Cl',
 '[14*]c1ncc([16*])c([16*])c1C',
 '[14*]c1nccc([16*])c1C',
 '[16*]c1c(C)cc(N)c([16*])c1C',
 '[16*]c1ccc([16*])cc1',
 '[3*]OC',
 '[3*]OC(F)F',
 '[3*]O[3*]',
 '[4*]C(C)C',
 '[4*]C(F)(F)C(F)F',
 '[4*]CC(F)(F)F',
 '[5*]N(C)C',
 '[5*]N1CCCC1',
 '[5*]N1CCCCC1',
 '[5*]N1CCOCC1',
 '[5*]N[5*]',
 '[6*]C(=O)O',
 '[8*]CO',
 '[8*]C[S+]([O-])c1nc2c([14*])scc2[nH]1',
 '[8*]C[S+]([O-])c1nc2cc([16*])c([16*])cc2[nH]1',
 '[8*]C[S+]([O-])c1nc2cc([16*])ccc2[nH]1',
 '[8*]C[S+]([O-])c1nc2ccccc2[nH]1',
 '[8*]C[S+]([O-])c1nc2cscc2[nH]1',
 '[8*]C[S+]([O-])c1nc2nc([14*])ccc2[nH]1'}

We can then recombine fragments to produce new molecules using BRICS.BRICSBuild:

import random
random.seed(27)

fragms = [Chem.MolFromSmiles(x) for x in allfrags]
builder = BRICS.BRICSBuild(fragms)

The result is a generator:

builder

Here’s what the first 16 results look like:

newMols = [next(builder) for i in range(16)]
Draw.MolsToGridImage(newMols,molsPerRow=4)

Let’s filter out the everything that doesn’t have exactly one sulfoxide:

qry = Chem.MolFromSmarts('S-O')
def filter(gen):
    for res in gen:
        if len(res.GetSubstructMatches(qry)) ==1:
            yield res

random.seed(27)
builder = BRICS.BRICSBuild(fragms)
newMols = [next(filter(builder)) for i in range(16)]
Draw.MolsToGridImage(newMols,molsPerRow=4)

random.seed(42)
builder = BRICS.BRICSBuild(fragms)
newMols = [next(filter(builder)) for i in range(100)]
# order by number of atoms;
newMols = [newMols[y] for (x,y) in sorted([(m.GetNumAtoms(),i) for i,m in enumerate(newMols)])]
Draw.MolsToGridImage(newMols[:16],molsPerRow=4)

Providing a seed to the enumeration

Another option in the BRICS builder is to provide one or more seeds that must be present in every output molecule. Here’s an example of how to do that using the scaffold of esomeprazole along with a version of the scaffold where the N in the second ring is replaced by a C:

seeds = [Chem.MolFromSmiles(x) for x in ('c19ncc(C)c([16*])c1C.C9[S+]([O-])c1nc2cc([16*])ccc2[nH]1',
                                         'c19ccc(C)c([16*])c1C.C9[S+]([O-])c1nc2cc([16*])ccc2[nH]1',)]

Draw.MolsToGridImage(seeds)

Now do the enumeration using those seeds:

random.seed(127)
builder = BRICS.BRICSBuild(fragms,seeds=seeds)
newMols = [next(filter(builder)) for i in range(16)]
Draw.MolsToGridImage(newMols,molsPerRow=4)

Doing enumeration providing seeds that are this specific is similar in ways to doing enumeration using the results of R-group decomposition

RDKit blog

GPU-Accelerated Clustering with nvMolKit

GPU-Accelerated Molecular Clustering with nvMolKit

Installation

Configuration

Loading Molecules

Workflow Description

RDKit Workflow

nvMolKit Workflow

Performance Comparison - RTX 5080 GPU vs Ryzen 9 9950X

Comparing Clustering Results

Links

Creating a patent data set

Creating the data sets

Looking at the some results with UMAP

Wrapping up 2025

The RDKit

Some numbers

Things that should go better

Related stuff

Personal things

About tetrahedral chirality in the RDKit

Intro

What can be chiral?

The RDKit’s representation of atomic stereochemistry

Building synthon spaces with combinatorial reactions

The first reaction

Second reaction

Combine the two reaction spaces

Reaction 3

Building synthon spaces with BRICS fragments

The synthon space input file

BRICS decomposition:

Thresholds for “random” with 3D similarity methods

Intro

Alignment based approaches

LOBSTER Set

ChEMBL Set

Non-alignment based approaches

LOBSTER Set

ChEMBL Set

Getting started

Random pairs from LOBSTER

Shape-based alignment

Open3DAlign

Baseline: canonical alignment

Baseline: tanimoto shape score in the canonical alignment

Non-alignment methods

Random ChEMBL molecules

Summary stats

Working with the LOBSTER Data set III

Getting started

Comparing shape-similarity metrics in the LOBSTER data set

Shape-based alignment

Open3DAlign

Comparing performance on “easy” cases

Working with the LOBSTER Data set II

Generating conformers for the LOBSTER compounds

Comparing shape-similarity approaches

Working with the LOBSTER Data set I

Registering the structures and populating the data base

Composition of the database

Retrieving info from the database

Look at duplicate mismatches

How long does it take to do common tasks in the RDKit?

Get the SMILES we will be working with:

Run the benchmarks

Construct molecule from SMILES

Generate canonical SMILES

Generating 2D coordinates

Writing mol blocks

Parsing mol blocks

Adding/removing Hs

Conformer generation

Generate fingerprints

Displaying atom maps and highlighting with reactions

Similarity search time and thresholds

A larger test: ZINC

An aside: chemfp performance

Rendering intramolecular H bonds in 2D