Issues when converting T1 MRI of the ADNI database.

[fyb99]

Description
Hi, recently I am trying to preprocess the T1 MRI and tabular data from the ADNI database. I download the raw data and install clinica as the official guideline. However, when I converted, only some samples can be converted. The converted folders' number is far less than the collected image folders.


The commands in the terminal indicates some paths can not be found. How to handle this problem. Appreciate it a lot for your solid neuroimaging works.

Pipeline
Which Clinica's Pipeline is concerned?

• t1 mri (description MPRAGE)
  ...

Converters

• adni-to-bids

[AliceJoubert]

Hi !

Thanks for reporting. I need a few more details to answer you properly :

• What version of Clinica are you using ?
• What subjects did you download from ADNI (if you have a list) ?
• What command line did you use ?

Assuming you are using Clinica v0.8.4 :

• the first log appears when writing the participants.tsv. As is, the converter makes the comparison between what is in the clinical data files (all ADNI) and what you actually have converted : the long list you see corresponds to what is not your dataset in the clinical data.
• the second log appears when parsing ADNI subjects from the clinical data (again, all) to find images paths. The converter looks for T1 files and informs the user when there is none. Since it parses clinical data there is possibly lots of files you did not download, so you get a lot these messages.

Did you find an example of subject that should have T1 in your raw data that does not have a folder in your BIDS ?

Best,

[fyb99]

Hi, thanks for your reply. My clinica version is 0.8.4. I load the T1 MRI subject from ADNI1, 2, 3 and Go with the description of MPRAGE. As given from the official guideline, I tick all original, pre- and post-processed buttons in the image type. The subjects are attached.
list.txt
My commancd is ''clinica convert adni-to-bids -m T1 data-path all-csv-path save-path''

For your last question, I found it does exist, for example, I have 002_S_0295 in my raw data folder, however, I can not found any sub-ADNI002S0295 folders in the converted results. By the way, is the .tsv file the exacted tabular data of the sample? I find the column names are not aligned with the adni tabular data.

[AliceJoubert]

Hi !

Thanks for the precisions, they were very helpful. What is happening is that the converter is selecting images ids based on the information it has in the clinical data (the csv) following a specific protocol. This protocol is described in our documentation here but basically it searches for the best possible image inside clinical data. If the image it finds was not actually downloaded then it won't be able to convert anything. My guess is that maybe you did not download all the data (I understand the ADNI download site is a bit tricky !!), and you should make sure to download all types of images :

I tested it with subject 002_S_0295 with base screening and it seems to work. Let me know if you encounter other issues !

For the tabular data we are following BIDS specifications so the format is a bit different from raw ADNI clinical data.

Best,

[fyb99]

Thanks for your reply. Actually, when I download the images, I tick all the three image types. But I set a "MPGAGE" in the image description part. Do you mean I should download all the MRI images without any description here? The attached image is my searching interface.

[AliceJoubert]

Yes I think you should not put any description, since the key is not always straight "MPRAGE" :

You are going to download all the other images but at least you will not miss what you want.

[fyb99]

Sorry for interrupting again. I have adopted your advice and encouter the same problem again. I have download all the images without setting any descriptions. The search interface are shown as attached. I choose 813 folders to convert, however, after running the clinica, I only get around 210 folders. For example, as you can see in the attached figures, the 002_S_0413 to _0685 seem not to be converted.

[AliceJoubert]

No worries, feel free to ask !

I cannot seem to reproduce this situation. Could you try to run the same command line with a subjects_list ? As described in the documentation : you can add --subjects_list=subjects.txt at the end of your command line. The file subjects.txt is a text file added to the clinica folder (from where you launch the command line) where you can indicate which subjects you want to convert (ex: 002_S_0413).

You can send me the logs you get when doing this so we can may be pinpoint the problem better.

[fyb99]

Thanks for your reply. I have relauch the converting process with the .txt file as you suggeted. The same issue happens again. The attahced files are the subject lists and the logs which is printed during the converting process.
subject.txt
output.txt

[AliceJoubert]

Thanks for the files ! From what I can see :

• Some subjects are actually not in ADNI metadata (ex : WARNING:Subject with PTID 031_S_0773 does not exist. Please check your subjects list.) so they are not converted. From your logs you get at least 20 subjects like this.
• Some subjects do not pass the quality check from ADNI (the process for image selection is described here in our documentation)

INFO:QC found but NOT passed
INFO:Subject 037_S_1225 for visit ADNI Screening - Series: 45506 - Study: 14580

So there is no associated file for these specific subject/session.

---

For these cases this behavior is expected, but I don't think that is all it is about. Do you mind doing the following ?

Relaunch the conversion with the following parameters :

• use another .txt file with only 002_S_0413 in it
• direct the BIDS output to another folder so you can show me the output easily

Send me :

• the logs once again
• the t1_paths.tsv you can find in the folder bids > conversion_info > v0

Best,

[fyb99]

Hi, I have run the command. This time the convertting reports an error. The attached files are the logs, t1_paths.tsv(I send the screenshot since github do not allow the upload of tsv file) and the result folder

output.txt

[AliceJoubert]

Thanks ! Clearly the converter cannot find the path to your raw images ; else in the .tsv you would have the actual path for the image in the column to the right end of the table. Could you show me how the raw data in this folder is organized ? I used the same settings as you for the image search so I guess it is not that the issue.

[fyb99]

Hi, this is the folder content. As you can see, I guess it provides virious equipments, in this subfolders, sometimes it has a nii file, sometimes numerous dcm file

[AliceJoubert]

I downloaded this subject file with the same parameters you indicated before and I got this :

The image selected by clinica is in the MPR_GradWarp_B1_Correction_N3_Scaled folder, so if you don't have it it makes sense for the software not to find it. I am not sure where the problem is though, do you select all images to add to your collection and download all the collection or do you select them by hand ?

[fyb99]

Hi, this time I search the sample again, The interface is attached (The sample in the screenshot is another one but all selections are the same.) This time I get more subfolders in 002_S_0413, specifically 34 subfolders. After comparsion, I find that last time I only choose the phase ''ADNI1'', and this time I do not select any specific phase. Will that be the reason (I guess some subfolders are stored in other phases)? And this time I successfully converting the sample, the outcome is also attached, which seems like a multi-visit results of the patient.
In summary, shall I select no specific phases during downloading data? As shown in the screenshots, only choose ''MRI'' and ''T1 weight'' while keeping others unselected?

[fyb99]

BTW, I guess the participants.tsv lists the tabular data of the converted samples from raw .csv files. But I find it only have around 20 columns. Using the R-based integration method mentioned by ADNI website, we can get a overall tabular data which has more than 100 columns. How to understand the participant.tsv file? The adnimerge file is attached
adnimerge.xlsx

[AliceJoubert]

Well I would say it depends on what you are aiming for : all the visits, only the first... what do you need ?

I had no issue getting folders with the same parameters you described before (ADNI1 only, T1 only, baseline screening only). If the subject has data from ADNI1 then you will get it by checking the case, I don't think you specifically need to add other phases. The problem maybe lies in the way the files are downloaded after. You need to add all images to the collection :

Then once again get all images when downloading the collection :

---

For the participants.tsv it is a metadata file based on what the BIDS specifications require and common useful parameters such as age, sex, ...

[fyb99]

Actually, I have selected all in the two shown screeplots. But as I have said, when I only choose the ADNI1 phase, I will only get around 10 subfolders of the 002_S_0413 sample and therefore the convertting failed. It seems that the converting process relies on some data which are available when selecting other phases. Thus, when I select no specific phases and only kick the ''ADNI'' button, I will get as many as 34 subfolders for this subject, and the coverting succeed. I just wonder whether it is the right way?

[AliceJoubert]

To be fair I am a bit confused ; for 002_S_0413 specifically I get hundreds of folders id I do not restrict the ADNI phase and get only T1. Even when I want the T1 for ADNI screening on phase 1 I get 23 folders. I would say the issue does not lie with our converter but maybe the ADNI website. For now I can only suggest clearing your cache for instance

[fyb99]

Hi, sorry to interrupt again, I have tried again. This time I want to convert 941_S_7106 and 941_S_7087. And according to you previous reply, this time, I get 2 subfolders of each sample, Field_Mapping and Accelerated_Sagittal_MPRAGE (Same for the two samples). However, the converter can not achieve any outcomes, lead to a empty output dir.
By the way, if I want to integate all the tabular features in the csv file to the tsv file, how should I modify the command. Current tsv file only contrain around 20 features of the sample.

[AliceJoubert]

Hi !

What do your logs tell you ? For subject 941_S_7106 for example it's due to missing clinical data so the subject will not be converted anyway, I described this earlier in the conversation.

To integrate more tabular features you could try using the --xml option from clinica, which is describe in the documentation in this section.

[fyb99]

output.txt
Hi, this is the log. By the way, when I use the --xml path, the generated tsv file contains more than 700 columns , I guess these are all the tabular features of this sample in ADNI dataset, although most of them are n/a, do i understand correctly?

[AliceJoubert]

In the version of clinica you are using (0.8.4) at the beginning, the converter parses all subjects from clinical data for image paths regardless if they are in your raw directory or not. It produces a lot of noise about subjects you are not even interested in. This was fixed in the latest release (0.9.1) if you want to update. This will not solve the issue but you should get better insight about what is going on !

---

Yes, that is exactly it !

[fyb99]

Hi, I find that I will get the same output whether I set the --xml_path or not (set as the path to the download metadata, i.e., the xml files) . each of the convertting lead to tsv flles with over 700 columns, but when I set the __xml_path, it reports an error? what is wrong with that? The report is attached.

[github-actions]

This issue is considered stale because it has not received further activity for the last 14 days. You may remove the inactive label or add a comment, otherwise it will be closed after the next 14 days.