graph LR
Pipeline_Orchestrator["Pipeline Orchestrator"]
Image_Processing_Segmentation["Image Processing & Segmentation"]
Cellular_Data_Processing["Cellular Data Processing"]
Omics_Data_Analysis_Integration["Omics Data Analysis & Integration"]
Spatial_Analysis_Reporting["Spatial Analysis & Reporting"]
Pipeline_Orchestrator -- "Orchestrates" --> Image_Processing_Segmentation
Pipeline_Orchestrator -- "Initiates" --> Image_Processing_Segmentation
Pipeline_Orchestrator -- "Orchestrates" --> Cellular_Data_Processing
Pipeline_Orchestrator -- "Initiates" --> Cellular_Data_Processing
Pipeline_Orchestrator -- "Orchestrates" --> Omics_Data_Analysis_Integration
Pipeline_Orchestrator -- "Initiates" --> Omics_Data_Analysis_Integration
Pipeline_Orchestrator -- "Orchestrates" --> Spatial_Analysis_Reporting
Pipeline_Orchestrator -- "Initiates" --> Spatial_Analysis_Reporting
Image_Processing_Segmentation -- "Provides" --> Cellular_Data_Processing
Image_Processing_Segmentation -- "Outputs" --> Cellular_Data_Processing
Cellular_Data_Processing -- "Provides" --> Omics_Data_Analysis_Integration
Cellular_Data_Processing -- "Outputs" --> Omics_Data_Analysis_Integration
Omics_Data_Analysis_Integration -- "Provides" --> Spatial_Analysis_Reporting
Omics_Data_Analysis_Integration -- "Outputs" --> Spatial_Analysis_Reporting
click Pipeline_Orchestrator href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//spatialone-pipeline/Pipeline Orchestrator.md" "Details"
click Image_Processing_Segmentation href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//spatialone-pipeline/Image Processing & Segmentation.md" "Details"
click Cellular_Data_Processing href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//spatialone-pipeline/Cellular Data Processing.md" "Details"
click Omics_Data_Analysis_Integration href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//spatialone-pipeline/Omics Data Analysis & Integration.md" "Details"
click Spatial_Analysis_Reporting href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//spatialone-pipeline/Spatial Analysis & Reporting.md" "Details"
The spatialone-pipeline is a comprehensive system designed for processing spatial transcriptomics data, particularly for Visium platforms. Its main flow involves orchestrating a series of specialized modules: starting with image processing and cell segmentation, followed by cellular data processing including cell-to-spot assignment and morphological clustering. Subsequently, it performs omics data analysis, encompassing cell type deconvolution and quality control, and integrates all results into standardized AnnData objects. Finally, the pipeline conducts advanced spatial analyses and generates detailed reports and visualizations, providing a complete solution for spatial transcriptomics data interpretation.
Orchestrates the entire spatial transcriptomics data processing pipeline, coordinating various sub-pipelines from image segmentation to spatial analysis. It serves as the central control flow for the spatialone-pipeline.
Related Classes/Methods:
src.pipelines.visium_flow.VisiumFlow:start(54:60)src.pipelines.visium_flow.VisiumFlow:get_params(63:69)src.pipelines.visium_flow.VisiumFlow:image_seg(72:116)src.pipelines.visium_flow.VisiumFlow:cell_to_spot(119:153)src.pipelines.visium_flow.VisiumFlow:cell_deconv(156:230)src.pipelines.visium_flow.VisiumFlow:morph_cluster(233:265)src.pipelines.visium_flow.VisiumFlow:assign_celltype(268:306)src.pipelines.visium_flow.VisiumFlow:spot_qc(309:334)src.pipelines.visium_flow.VisiumFlow:data_merge(337:387)src.pipelines.visium_flow.VisiumFlow:spatial_analysis(390:449)src.pipelines.spatial_one_flow.SpatialOneFlow:spatial_one_flow_get_params(112:136)
Manages all image-related tasks, including preprocessing (tissue detection, padding, cropping, stain normalization), cell segmentation using models like Cellpose and HoverNet, and post-processing (stitching, reindexing). It also handles model loading and execution utilities for image segmentation.
Related Classes/Methods:
src.pipelines.imgseg_cellpose_pipeline.ImageSeg:load_unittest_data(54:63)src.pipelines.imgseg_cellpose_pipeline.ImageSeg:load_data(65:76)src.pipelines.imgseg_cellpose_pipeline.ImageSeg:pre_process(78:120)src.pipelines.imgseg_cellpose_pipeline.ImageSeg:predict(122:138)src.pipelines.imgseg_cellpose_pipeline.ImageSeg:post_process(140:171)src.pipelines.imgseg_cellpose_pipeline.ImageSeg:save_data(174:189)models.hovernet.src.infer.base.InferManager:__init__(31:63)models.hovernet.src.infer.tile.InferManager:process_file_list(147:394)models.hovernet.src.infer.wsi.InferManager:process_single_file(458:744)models.hovernet.src.infer.wsi.InferManager:process_wsi_list(746:803)src.pipelines.imgseg_hovernet_pipeline.HoverNet:load_model_configs_from_flow(21:34)src.pipelines.imgseg_hovernet_pipeline.HoverNet:predict(36:66)src.data_prep.postprocessing:stitching_instance_segmentation(16:104)src.data_prep.postprocessing:reindex_masks(129:156)src.data_prep.postprocessing:resolve_conflict(275:323)src.data_prep.postprocessing:match_input_shape(351:427)src.data_prep.preprocessing.Preprocessing:resize_img(45:59)src.data_prep.preprocessing.Preprocessing:detect_tissue_bbox(61:95)src.data_prep.preprocessing.Preprocessing:pad_all_around(97:115)src.data_prep.preprocessing.Preprocessing:adjust_tissue_boundary(131:142)src.data_prep.preprocessing.Preprocessing:crop_tissue_region(144:159)src.data_prep.preprocessing.Preprocessing:tissue_detector(161:180)src.data_prep.preprocessing.Preprocessing:norm_stain_remove(182:195)src.data_prep.preprocessing.Preprocessing:downsample(197:215)src.data_prep.preprocessing.Preprocessing:resize_img_dim(28:43)src.data_prep.preprocessing.Preprocessing:generate_patches(217:238)src.data_prep.preprocessing.Preprocessing:undo_padding(117:129)src.utils.image_processing_utils.ImageManipulation:crop_and_pad_img(13:61)src.utils.model_loader.cellpose_model(15:32)src.utils.model_loader.predict(36:66)models.hovernet.src.run_utils.engine.RunEngine:__init__(82:124)models.hovernet.src.run_utils.engine.RunEngine:__reset_state(126:134)models.hovernet.src.run_utils.engine.RunEngine:run(154:232)models.hovernet.src.run_utils.callbacks.base.TrackLr:__init__(40:49)models.hovernet.src.run_utils.callbacks.base.ScheduleLr:__init__(70:72)models.hovernet.src.run_utils.callbacks.base.PeriodicSaver:__init__(125:134)models.hovernet.src.run_utils.callbacks.base.ConditionalSaver:__init__(166:175)models.hovernet.src.run_utils.callbacks.base.ScalarMovingAverage:__init__(258:266)models.hovernet.src.run_utils.callbacks.base.ProcessAccumulatedRawOutput:__init__(302:312)models.hovernet.src.run_utils.callbacks.base.VisualizeOutput:__init__(338:348)models.hovernet.src.run_utils.callbacks.logging.LoggingEpochOutput:run(105:188)models.hovernet.src.run_utils.callbacks.serialize._ConfusionMatrix:to_tensorboard(127:221)models.hovernet.src.models.hovernet.net_desc.HoVerNet:__init__(19:142)models.hovernet.src.models.hovernet.net_desc:create_model(199:210)models.hovernet.src.models.hovernet.net_utils.ResidualBlock:__init__(203:289)models.hovernet.src.models.hovernet.post_proc:process(96:188)models.hovernet.src.models.hovernet.run_desc:train_step(16:122)models.hovernet.src.models.hovernet.run_desc:viz_step_output(231:286)models.hovernet.src.models.hovernet.run_desc:proc_valid_step_output(293:403)models.hovernet.src.models.hovernet.targets:gen_targets(100:114)models.hovernet.src.models.hovernet.targets:prep_sample(118:172)models.hovernet.src.models.hovernet.utils:crop_to_shape(33:51)models.hovernet.src.models.hovernet.utils:msge_loss(107:173)src.utils.data_loader.load_images(13:21)
Focuses on processing segmented cell data, including assigning cells to Visium spatial spots, converting masks to polygons, extracting morphological features, performing morphological clustering, and assigning cell types to spots using various classification methods.
Related Classes/Methods:
src.pipelines.cell2spot_pipeline.Cell2Spot:load_model_configs_from_flow(42:55)src.pipelines.cell2spot_pipeline.Cell2Spot:load_data(57:94)src.pipelines.cell2spot_pipeline.Cell2Spot:convert_mask_to_polygons(96:131)src.pipelines.cell2spot_pipeline.Cell2Spot:generate_visium_spot_polygons(156:171)src.pipelines.cell2spot_pipeline.Cell2Spot:create_cell_search_tree(173:181)src.pipelines.cell2spot_pipeline.Cell2Spot:get_cells_per_spot(183:201)src.pipelines.cell2spot_pipeline.Cell2Spot:save_data(203:207)src.pipelines.clustering_pipeline.MorphCluster:load_unittest_data(51:62)src.pipelines.clustering_pipeline.MorphCluster:load_data(67:75)src.pipelines.clustering_pipeline.MorphCluster:extract_features(77:80)src.pipelines.clustering_pipeline.MorphCluster:preprocess_data(82:87)src.pipelines.clustering_pipeline.MorphCluster:save_data(109:118)src.pipelines.clustering_pipeline.MorphCluster:load_model_configs_from_flow(36:49)src.pipelines.clustering_pipeline.MorphCluster:fit_cluster(89:107)src.pipelines.assign_pipeline.SpotCluster:load_unittest_data(54:65)src.pipelines.assign_pipeline.SpotCluster:load_data(70:78)src.pipelines.assign_pipeline.SpotCluster:extract_features(80:83)src.pipelines.assign_pipeline.SpotCluster:preprocess_data(85:90)src.pipelines.assign_pipeline.SpotCluster:spot_cluster(92:173)src.pipelines.assign_pipeline.SpotCluster:save_data(182:191)src.pipelines.assign_pipeline.SpotCluster:load_model_configs_from_flow(39:52)src.pipelines.assign_pipeline.SpotCluster:confidence_metrics(175:180)src.image_analysis.clustering.confidence_metrics:compute_confidence_metrics(53:69)src.image_analysis.clustering.constrained_clustering.DeterministicAnnealing:fit(62:116)src.image_analysis.clustering.constrained_clustering.DeterministicAnnealing:predict(118:123)src.image_analysis.clustering.constrained_clustering.DeterministicAnnealing:modify(125:155)src.image_analysis.clustering.global_assignment:estimate_cluster_probability(71:84)src.image_analysis.clustering.global_assignment:get_best_allocation(151:162)src.image_analysis.clustering.global_assignment.alloc_obj:copy(181:187)src.image_analysis.clustering.global_assignment:objective_min_score(224:230)src.image_analysis.clustering.global_assignment:get_best_allocation_simanel(233:257)src.image_analysis.clustering.spot_clustering:bead_clustering(65:100)src.image_analysis.clustering.spot_clustering:random_classifier(231:285)src.image_analysis.clustering.spot_clustering:probability_classifier(346:388)src.image_analysis.clustering.spot_clustering:assign_probabilistic_celltype(415:514)src.image_analysis.clustering.feature_extraction.extract_image_features(10:45)src.image_analysis.clustering.transform.scaling(10:20)src.image_analysis.clustering.transform.dimensionality_reduction(23:34)src.image_analysis.clustering.spot_clustering.create_barcode_dict(29:50)src.image_analysis.clustering.spot_clustering.assign_celltype(125:176)src.image_analysis.clustering.spot_clustering.assign_random_celltype(288:342)src.utils.data_loader.load_pos_list(35:55)src.utils.data_loader.load_segmentation(24:32)src.utils.data_saver.save_data(10:23)
Handles cell type deconvolution using models like Cell2Location and CARD to infer cell type proportions. It also performs quality control on gene expression data and integrates results from various upstream pipelines into unified AnnData objects for spots and cells, including gene filtering and TMAP file generation.
Related Classes/Methods:
src.pipelines.celldeconv_cell2location_pipeline.Cell2Location:save_train_sig(138:155)src.pipelines.celldeconv_cell2location_pipeline.Cell2Location:save_data(273:318)src.pipelines.celldeconv_cell2location_pipeline.Cell2Location:load_model_configs_from_flow(40:63)src.pipelines.celldeconv_cell2location_pipeline.Cell2Location:load_data(65:85)src.pipelines.celldeconv_cell2location_pipeline.Cell2Location:train_location_model(157:222)src.pipelines.celldeconv_cell2location_pipeline.Cell2Location:get_cell_type_proportion(224:257)src.pipelines.celldeconv_card_pipeline.CARD:load_model_configs_from_flow(31:56)src.pipelines.celldeconv_card_pipeline.CARD:predict(58:103)src.pipelines.qc_pipeline.QC:load_data(54:72)src.pipelines.qc_pipeline.QC:get_qc_metrics(74:145)src.pipelines.qc_pipeline.QC:save_data(263:272)src.pipelines.qc_pipeline.QC:load_model_configs_from_flow(39:52)src.pipelines.datamerge_pipeline.DataMerge:load_data(90:127)src.pipelines.datamerge_pipeline.DataMerge:load_gene_matrix(341:352)src.pipelines.datamerge_pipeline.DataMerge:filter_gene_data(380:443)src.pipelines.datamerge_pipeline.DataMerge:load_kmeans_clustering_results(493:515)src.pipelines.datamerge_pipeline.DataMerge:create_spots_adata(528:634)src.pipelines.datamerge_pipeline.DataMerge:create_cells_adata(660:689)src.pipelines.datamerge_pipeline.DataMerge:create_tmap(738:752)src.pipelines.datamerge_pipeline.DataMerge:save_data(754:771)src.pipelines.datamerge_pipeline.DataMerge:load_model_configs_from_flow(45:88)src.pipelines.datamerge_pipeline.DataMerge:create_cell_outlines_layer(445:467)src.pipelines.datamerge_pipeline.DataMerge:create_visium_spots_layer(469:491)src.pipelines.datamerge_pipeline.DataMerge:summarize_spots_adata_to_dataframe(691:713)src.pipelines.datamerge_pipeline.DataMerge:summarize_cells_adata_to_dataframe(715:736)src.utils.tmap_generator.tmapGenerator:__init__(17:44)src.utils.tmap_generator.tmapGenerator:update_tmap_markers(205:300)src.utils.tmap_generator.tmapGenerator:create_tmap_file(316:351)src.utils.data_loader.get_matrix_from_h5(58:95)src.utils.data_saver.save_data(10:23)
Conducts various spatial analyses, including neighborhood enrichment, co-occurrence, Moran's I, and SpatialDE analysis, to understand spatial patterns of cells and gene expression. It also generates comprehensive HTML reports and various plots for visualizing spatial analysis results.
Related Classes/Methods:
src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:load_data(61:147)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:load_region_level_data(186:227)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:cells_df_to_adata(250:306)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:spots_df_to_adata(308:387)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:infiltration_analysis(658:696)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:run_spatial_analysis(698:793)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:run_region_spatial_analysis(795:840)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:generate_spatial_analysis_report(853:920)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:save_data(953:976)src.pipelines.spatialanalysis_pipeline.SpatialAnalysis:load_model_configs_from_flow(46:59)src.spatial_report_generator.plot_generator.PlotGenerator:__init__(35:64)src.spatial_report_generator.plot_generator.PlotGenerator:add_qc_report_button(138:151)src.spatial_report_generator.plot_generator.PlotGenerator:cell_summary_table(153:186)src.spatial_report_generator.plot_generator.PlotGenerator:cell_count_plot(188:242)src.spatial_report_generator.plot_generator.PlotGenerator:cells_within_spot_plot(244:283)src.spatial_report_generator.plot_generator.PlotGenerator:avg_gene_exp_plot(285:322)src.spatial_report_generator.plot_generator.PlotGenerator:neighborhood_enrichment_plots(324:369)src.spatial_report_generator.plot_generator.PlotGenerator:chord_plot(371:451)src.spatial_report_generator.plot_generator.PlotGenerator:cooccurrence_plots(453:532)src.spatial_report_generator.plot_generator.PlotGenerator:morans_i_exp_plots(534:567)src.spatial_report_generator.plot_generator.PlotGenerator:morans_i_bar_plot(569:602)src.spatial_report_generator.plot_generator.PlotGenerator:spatialde_heatmap_plot(604:655)src.spatial_report_generator.plot_generator.PlotGenerator:spatialde_bar_plot(657:699)src.spatial_report_generator.plot_generator.PlotGenerator:volcano_plots(748:841)src.spatial_report_generator.plot_generator.PlotGenerator:abundance_vs_cells_plot(843:930)src.spatial_report_generator.plot_generator.PlotGenerator:abundance_vs_level(932:1000)src.spatial_report_generator.plot_generator.PlotGenerator:get_report(600:603)src.spatial_report_generator.report_generator.ReportGenerator:format_to_html(49:63)src.spatial_report_generator.comparative_analysis.ComparativeAnalysis:compare_gene_exp(107:147)src.utils.data_loader.load_data(full file reference)src.utils.data_saver.save_data(10:23)