Harmony v2.0.3 Fails to Integrate Data Effectively Compared to v1.2.4

[xiao00su]

I've encountered an issue where the new version of Harmony (v2.0.3) does not effectively integrate my single-cell data, even when forcing a high number of iterations.

Observed Behavior:
PCA (before integration):​ Shows clear batch separation between E12, E14, E16, and E18 (cyan and purple clusters are distinct).
Harmony v2.0.3 (default, stop @ 2 iter):​ Some mixing occurs but significant batch structure remains.
v2.0.3 (e=0.0001, stop @ 3 iter): slightly improved.
v2.0.3 (early_stop=False, max_iter=20):​ significantly improved, but the batches remain partially separated.
v2.0.3 (early_stop=False, max_iter=50):​ the integration effectiveness does not improve substantially compare to 20 iter.
Harmony v1.2.4 (default, stop @ 2 iter):​ Achieves much better mixing, with batches overlapping significantly more than in v2.0.3.
code for install harnomy v1.2.4:
remove.packages("harmony")
[install.packages('https://cran.r-project.org/src/contrib/Archive/harmony/harmony_1.2.4.tar.gz')](remotes::install_version("harmony", version = "1.2.4", repos = "https://cran.r-project.org"))

[pati-ni]

Hi @xiao00su

Thank you for reporting your issue. If there is a problem with the versions, we will investigate this thoroughly

Is the data you are working on publicly available? If not, we will have difficulty reproducing this behavior on our end.

Can you also post the exact RunHarmony call you are running in either case?

It looks like the coordinates are flipped in harmony 1.2.4 2 iter panel. Do you use the same PCA coordinates in either scenario?

[xiao00su]

Hi @pati-ni
The PCA embeddings and cell metadata are now accessible via the following link: https://drive.google.com/file/d/1RgLj5OulBkY80HubKaFEBfJMOzGbFfEC/view?usp=sharing

The coordinate flip issue observed previously arose from using different downsampled cell subsets for PCA in the initial runs. Specifically, I employed the "SketchData" function to select 100,000 cells from a total of approximately 2.6 million cells for performing PCA and Harmony integration. For consistency, I have now updated the panel for version 1.2.4 using the identical subsampled cells and the same PCA embeddings.

I executed the same code on my Seurat object for each version as follows:

brain <- RunHarmony(object = brain, group.by.vars = "orig.ident",
# max_iter = 20, early stop = F,
verbose = T, plot_convergence = T)

I also applied Harmony directly to the extracted PCA matrix and cell metadata. The results from this alternative approach aligned with those obtained from the Seurat object workflow.

har <- RunHarmony(data_mat = pca, meta_data = meta, vars_use = 'orig.ident',
# max_iter = 20, early_stop = F,
verbose = T, plot_convergence = T )

harX <- as.data.frame(har)
harX$batch <- gsub('_.*$', '', rownames(harX))
p2 <- harX %>% ggplot(aes(x=PC_1,y=PC_2, color=batch)) + geom_point(size=0.01) +
labs(title = 'harmony v2.0.3, default, 2 iter')

[pati-ni]

Hi @xiao00su

Thanks for sharing the data. We have identified an issue with the alpha parameter during lambda autotuning. Please adjust the parameter lambda=1 and early_stop=FALSE.

We will provide a proper fix down the line so I will leave this issue open for the time.