ENH: tracking changes to ML manuscript (la-2026-00306e)

[adamwitmer]

This issue is to be used to track changes associated with review comments from the manuscript la-2026-00306e, specifically those having to deal with improvements related to questions about the machine-learning (some review comments have been omitted here because Mihee will address them separately).

Reviewer 1:

• Systems that may have irregular shaped droplets or precipitates and droplets present: Peacock et al. [https://doi.org/10.1021/acsami.0c21036] describes some mixtures where the system is immiscible but does not necessarily form clearly visible droplets. Some of the systems formed with highly branched polysaccharides, led to structures observed by brightfield microscopy that would be interesting to test the performance on. For example, the system formed from chondroitin sulfate and hydroxypropyl cellulose (see text for others). A simple demonstration that the approach works with systems like these to detect the presence of liquid-liquid phase separation would be a great addition.
• Effect of microscope resolution: It is mentioned in the text but not explored. How well the system works as a function of image resolution would be relatively easy to characterize and useful for a user when selecting a microscope system. It would also be great to see what would come out of a super resolution system, e.g., super-resolution dark field microscopy or phase-contrast super resolution if these microscopy systems are available.
• The effect of temperature: An easy to perform and useful demonstration would be to detect the transition from one phase to two phases, starting with a miscible composition and gradually lowering the temperature to the point where droplets become detectable, then raising the temperature again. It might even be possible to observe hysteresis in this experiment if temperature of the solution can be reliably recorded.

Most of the above have to do with performing analysis of additional images provided by Mihee

Reviewer 2

• For each feature, please plot their distributions in the SI, partitioned into each class (two phase vs one phase) to see if there are meaningful patterns we can spot with our eyes. I imagine that a simple rule based on the number and size of bubbles could suffice instead of the machine learning model. This will help us see which features tend to separate the class more. Or you can make a pairplot in seaborn where it shows the relationship between the features and color/shape the points according to class. I just think more insights into your data are needed. The feature importance study is helpful. I was going to suggest that but see you have it.
• Spell out how many training images you have in the Methods. Are they for one ATPS i.e. with fixed components? Do they span the whole space of concentrations of the two components? How did you choose the concentrations for the training examples? Clarify/explain in the text. Do the training images come for all depths and all positions? Can the same model see images at different depths (it doesn’t matter, does it?). What is the class balance? If the classes are imbalanced, area under precision-recall curve more appropriate than AUROC.
• Do you train one independent ML model per ATPS (i.e. for two given components) or does one ML model handle looking for ATPSs for ANY two components (polymers/salts)?
• In Figure 5 were these images included in the train set? State if so or if not. If so this is not a proper test/train split to evaluate the ML model. The ability of ML model to evaluate if an ATPS is present should be best on test images it didn’t see during training. It is unclear if you are following these standard and required practices of test/train splits/cross validation for the ML.
• Do you expect the classifier to transfer from one ATPS to another? Or for each ATPS you must train a model? Clarify in the text.
• Please concretely list the features input to the ML model in the text. E.g. “spatial distribution patterns” is very vague.

[adamwitmer]

For the question about Figure 5, I located the graphs used in the plot on glycan, under /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/legacy_ML_data, which contains the outputs of all the permutations of training/validation/inference for the three compositions. From what I can infer there, the point of Figure 5 is to show the results from training/validation on the DEX10/PEG10 image dataset, and inferring on the other compositions, which is why the "model" decision boundary is not shown for the DEX10/PEG10 phase diagrams. Instead, this figure should show representative phase diagram outputs for inference on all the compositions, and the caption should be updated to reflect the exact training/validation/testing split used for the specific output.

Also, I dont know what legacy_ML_data means, but there are several different folders with various versions of the phase diagram outputs in subfolders of nidhin_train_model/neat_ml/data/ so perhaps these figures need to be updated with the most recent model outputs as well.

[tylerjereddy]

Probably want to make sure you can reproduce the outplot plot(s) by making use of code that is available/reviewed on GitHub.

[tylerjereddy]

(FWIW, your explanation above still isn't particularly clear to me, and the caption is super confusing as written, so clarity is a big problem IMO)

[adamwitmer]

I sent an email to Mihee today (3/10/26) detailing the dataset breakdown in relation to the second point from Reviewer 2 above:

Time point/Composition	Phase Separation (0 == No, 1 == Yes)	PEO20/DEX500	PEO8/Sod. Cit.	PEO10/DEX10
1st	0	655	307	1069
1st	1	762	935	444
1st	Total Images	1417	1242	1996 (483)
2nd	0	624	286	729
2nd	1	780	961	780
2nd	Total Images	1404	1247	1977 (468)

Where the values in parentheses for "Total Images" in PEO10/DEX10 represent the discrepancy between the total number of images in the dataset folders on glycan and the number of labeled (phase_separation) images in the composition csv files on glycan. I validated the count for the 1st time point in PEO10/DEX10 but the value for the 2nd time point is lower than the count obtained by comparing the dataset (672). I have not determined why the values for this specific time point/composition differ.

[tylerjereddy]

We're getting too close to the deadline for me to provide much more feedback now. I'll provide a few more points below.

How well the system works as a function of image resolution

I take that to mean our best possible performance at a given resolution, rather than our performance using the same single set of naively hardcoded default (hyper)parameters for all resolutions. I believe that's what I conveyed in person as well.

For Figure 5:

Probably want to make sure you can reproduce the outplot plot(s) by making use of code that is available/reviewed on GitHub.

I don't know that I saw any code changes/additions presented for review, that should probably be clarified.

For the imbalanced data table/analysis there is a comment:

I have not determined why the values for this specific time point/composition differ.

Has this been sorted out yet? It will be confusing enough to additionally add parentheses for the aggregated counts, so not needing two sets of parentheses would be nice. I find it hard to follow the discussion below the table at #26 (comment) as well.

[tylerjereddy]

@adamwitmer you should probably be opening issues/PRs related to code that was used in the manuscript rebuttal but not shared/reviewed with the team yet.

For example, convex hull overlap code for feature pair plots, and so on.

Try to make sure you are responding to team queries with a reasonable sense of urgency, #26 (comment) is two weeks old now, and I still can't tell what is going on with code for figure 5, etc.

[tylerjereddy]

Also, should clarify if you reran analyses for plots/results in manuscript with SAM 2.1, etc. Need to document what was done with 2 vs. 2.1 somewhere for reproducibility... ideally you're on top of this stuff!

[adamwitmer]

Ok, I will respond to these queries this week.

[adamwitmer]

I am able to reproduce Figure 5 from the manuscript by generating the plots using the workflow branch in PR #28 with the following workflow incantation and input .yaml file:

python run_workflow.py --config PEO10k_Dex10k_BubbleSAM_Ph_train.yaml --steps train,infer,explain,plot

PEO10k_Dex10k_BubbleSAM_Ph_train.yaml

roots:
  work: PEO10k_DEX10k
  results: PEO10k_DEX10k
  model: PEO10k_DEX10k

datasets:

  - id: PEO10K_DEX10K_Ph_2nd                
    role: train
    method: BubbleSAM
    class: Ph
    time_label: 2nd
    composition_cols:
      - "Dextran 10 kg/mol (wt%)"
      - "PEO 10 kg/mol (wt%)"
    analysis:
      input_dir: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO10K_DEX10K/BubbleSAM/Ph/2nd/
      composition_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO10K_DEX10K/PEO10K_DEX10K_Image_Composition_Phase.csv
      per_image_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO10K_DEX10K/BubbleSAM/Ph/2nd/PEO10K_DEX10K_2nd_BubbleSAM_Analysis_Summary.csv
      aggregate_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO10K_DEX10K/BubbleSAM/Ph/2nd/PEO10K_DEX10K_2nd_BubbleSAM_Analysis_Summary_Aggregate.csv
      group_cols:
        - Group
        - Label
        - Time
        - Class
      graph_method: radius
      graph_param: 30

  - id: PEO10K_DEX10K_Ph_1st
    role: val
    method: BubbleSAM
    system: PEO10K_DEX10K
    class: Ph
    time_label: 1st
    composition_cols:
      - "Dextran 10 kg/mol (wt%)"
      - "PEO 10 kg/mol (wt%)"
    analysis:
      input_dir: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO10K_DEX10K/BubbleSAM/Ph/1st/
      composition_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO10K_DEX10K/PEO10K_DEX10K_Image_Composition_Phase.csv
      per_image_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO10K_DEX10K/BubbleSAM/Ph/1st/PEO10K_DEX10K_1st_BubbleSAM_Analysis_Summary.csv
      aggregate_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO10K_DEX10K/BubbleSAM/Ph/1st/PEO10K_DEX10K_1st_BubbleSAM_Analysis_Summary_Aggregate.csv
      group_cols:
        - Group
        - Label
        - Time
        - Class
      graph_method: radius
      graph_param: 30

  - id: PEO8K_Sodium_Citrate_Ph_2nd
    method: BubbleSAM 
    role: infer            
    class: Ph
    time_label: 2nd
    composition_cols:
      - "Sodium Citrate (wt%)"
      - "PEO 8 kg/mol (wt%)"
    analysis:
      input_dir: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO8K_Sodium_Citrate/BubbleSAM/Ph/2nd
      composition_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO8K_Sodium_Citrate/PEO8K_Sodium_Citrate_Image_Composition_Phase.csv
      per_image_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO8K_Sodium_Citrate/BubbleSAM/Ph/2nd/PEO8K_Sodium_Citrate_2nd_BubbleSAM_Analysis_Summary.csv
      aggregate_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO8K_Sodium_Citrate/BubbleSAM/Ph/2nd/PEO8K_Sodium_Citrate_2nd_BubbleSAM_Analysis_Summary_Aggregate.csv
      group_cols:
        - Group
        - Label
        - Time
        - Class
      graph_method: radius
      graph_param: 30

  - id: PEO8K_Sodium_Citrate_Ph_1st
    method: BubbleSAM 
    role: infer            
    class: Ph
    time_label: 1st
    composition_cols:
      - "Sodium Citrate (wt%)"
      - "PEO 8 kg/mol (wt%)"
    analysis:
      input_dir: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO8K_Sodium_Citrate/BubbleSAM/Ph/1st
      composition_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO8K_Sodium_Citrate/PEO8K_Sodium_Citrate_Image_Composition_Phase.csv
      per_image_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO8K_Sodium_Citrate/BubbleSAM/Ph/1st/PEO8K_Sodium_Citrate_1st_BubbleSAM_Analysis_Summary.csv
      aggregate_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO8K_Sodium_Citrate/BubbleSAM/Ph/1st/PEO8K_Sodium_Citrate_1st_BubbleSAM_Analysis_Summary_Aggregate.csv
      group_cols:
        - Group
        - Label
        - Time
        - Class
      graph_method: radius
      graph_param: 30

  - id: PEO20K_DEX500K_Ph_2nd                
    role: infer
    method: BubbleSAM
    class: Ph
    time_label: 2nd
    composition_cols:
      - "Dextran 500 kg/mol (wt%)"
      - "PEO 20 kg/mol (wt%)"
    analysis:
      input_dir: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO20K_DEX500K/BubbleSAM/Ph/2nd/
      composition_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO20K_DEX500K/PEO20K_DEX500K_Image_Files_Composition_Phase_New.csv
      per_image_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO20K_DEX500K/BubbleSAM/Ph/2nd/PEO20K_DEX500K_2nd_BubbleSAM_Analysis_Summary.csv
      aggregate_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO20K_DEX500K/BubbleSAM/Ph/2nd/PEO20K_DEX500K_2nd_BubbleSAM_Analysis_Summary_Aggregate.csv
      group_cols:
        - Group
        - Label
        - Time
        - Class
      graph_method: radius
      graph_param: 30

  - id: PEO20K_DEX500K_Ph_1st
    role: infer
    method: BubbleSAM
    system: PEO20K_DEX500K
    class: Ph
    time_label: 1st
    composition_cols:
      - "Dextran 500 kg/mol (wt%)"
      - "PEO 20 kg/mol (wt%)"
    analysis:
      input_dir: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO20K_DEX500K/BubbleSAM/Ph/1st/
      composition_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/LDRD_DR_NEAT_Dataset/PEO20K_DEX500K/PEO20K_DEX500K_Image_Files_Composition_Phase_New.csv
      per_image_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO20K_DEX500K/BubbleSAM/Ph/1st/PEO20K_DEX500K_1st_BubbleSAM_Analysis_Summary.csv
      aggregate_csv: /scratch2/LDRD_Chicoma_Backup/Final_Results_06092025/nidhin_train_model/neat_ml/data/PEO20K_DEX500K/BubbleSAM/Ph/1st/PEO20K_DEX500K_1st_BubbleSAM_Analysis_Summary_Aggregate.csv
      group_cols:
        - Group
        - Label
        - Time
        - Class
      graph_method: radius
      graph_param: 30

Which produces the phase diagrams for the inference on PEO8k/Sodium Citrate and PEO20k/DEX500k of the model trained on PEO10k/DEX10k. The workflow does not generate phase diagrams for the training/validation datasets, which are instead generated by calling python -m neat_ml.plot_manuscript_figures.

A mockup of Figure 5 using these outputs is shown below:

[tylerjereddy]

Ok, that's good. Do try to respond to such queries before we submit manuscripts--the original query #26 (comment) is more than a month old, which should have been enough buffer to share before submitting. Reviewing work before it is submitted for peer review is an important part of the scientific process/integrity.

[adamwitmer]

I updated PR #31 with the code that I used to generate the figures in the manuscript. I have not removed the WIP status there because I still need to address several TODO items before presenting it for formal review.