Error when forming edge table

[JoshuaSiraj]

Hi,

I'm trying to preprocess the this dataset with Med-Imagetools.

Here is the pipeline call:

pipeline = AutoPipeline(input_directory = liver_path, output_directory = output_path, modalities = "CT,RTSTRUCT,RTDOSE", spacing = (5,5,5), n_jobs = 2, visualize = True, warn_on_error=True )

Here is the full error:

Edge table not present. Forming the edge table based on the crawl data...

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

[<ipython-input-7-c6d75188aadb>](https://localhost:8080/#) in <module>
      3 output_path = pathlib.Path(os.path.join(curr_path, "processed_dataset"))
      4 
----> 5 pipeline = AutoPipeline(input_directory = liver_path,
      6                         output_directory = output_path,
      7                         modalities = "CT,RTSTRUCT,RTDOSE",

6 frames

[/usr/local/lib/python3.9/dist-packages/imgtools/autopipeline.py](https://localhost:8080/#) in __init__(self, input_directory, output_directory, modalities, spacing, n_jobs, visualize, missing_strategy, show_progress, warn_on_error, overwrite, nnunet, train_size, random_state, read_yaml_label_names, ignore_missing_regex, roi_yaml_path, custom_train_test_split, nnunet_inference, dataset_json_path, continue_processing, dry_run, verbose, update, roi_select_first, roi_separate)
    320 
    321         # Input operations
--> 322         self.input = ImageAutoInput(input_directory, modalities, n_jobs, visualize, update)
    323         self.output_df_path = pathlib.Path(self.output_directory, "dataset.csv").as_posix()
    324 

[/usr/local/lib/python3.9/dist-packages/imgtools/ops/ops.py](https://localhost:8080/#) in __init__(self, dir_path, modalities, n_jobs, visualize, update)
    100         # Form the graph
    101         edge_path = pathlib.Path(self.parent,".imgtools",f"imgtools_{self.dataset_name}_edges.csv").as_posix()
--> 102         graph = DataGraph(path_crawl=path_crawl, edge_path=edge_path, visualize=visualize)
    103         print(f"Forming the graph based on the given modalities: {self.modalities}")
    104         self.df_combined = graph.parser(self.modalities)

[/usr/local/lib/python3.9/dist-packages/imgtools/modules/datagraph.py](https://localhost:8080/#) in __init__(self, path_crawl, edge_path, visualize)
     46         else:
     47             print("Edge table not present. Forming the edge table based on the crawl data...")
---> 48             self.form_graph()
     49         if visualize:
     50             self.visualize_graph()

[/usr/local/lib/python3.9/dist-packages/imgtools/modules/datagraph.py](https://localhost:8080/#) in form_graph(self)
     55         '''
     56         #Get reference_rs information from RTDOSE-RTPLAN connections
---> 57         df_filter = pd.merge(self.df, self.df[["instance_uid","reference_rs"]], 
     58                              left_on="reference_pl",
     59                              right_on="instance_uid",

[/usr/local/lib/python3.9/dist-packages/pandas/core/reshape/merge.py](https://localhost:8080/#) in merge(left, right, how, on, left_on, right_on, left_index, right_index, sort, suffixes, copy, indicator, validate)
    105     validate: str | None = None,
    106 ) -> DataFrame:
--> 107     op = _MergeOperation(
    108         left,
    109         right,

[/usr/local/lib/python3.9/dist-packages/pandas/core/reshape/merge.py](https://localhost:8080/#) in __init__(self, left, right, how, on, left_on, right_on, axis, left_index, right_index, sort, suffixes, copy, indicator, validate)
    702         # validate the merge keys dtypes. We may need to coerce
    703         # to avoid incompatible dtypes
--> 704         self._maybe_coerce_merge_keys()
    705 
    706         # If argument passed to validate,

[/usr/local/lib/python3.9/dist-packages/pandas/core/reshape/merge.py](https://localhost:8080/#) in _maybe_coerce_merge_keys(self)
   1259                     inferred_right in string_types and inferred_left not in string_types
   1260                 ):
-> 1261                     raise ValueError(msg)
   1262 
   1263             # datetimelikes must match exactly

ValueError: You are trying to merge on float64 and object columns. If you wish to proceed you should use pd.concat```

[skim2257]

Hi @JoshuaSiraj ,

We've made some edits to the AutoPipeline since March. Could you try again with the latest version?

Thanks,
Sejin