"TMT-Integrator can't find the 'Purity' column"

[dnwhdals]

• Describe the issue or question: Hello,

I am struggling with an issue with FragPipe v16.0 while quantifies PSMs in TMT-10 dataset. It seems the tool is not able to generate the "Purity" column in the PSM tables. Please find attached the log file. How can I resolve the issue?

Thank you in advance.

Jongmin Woo

• Upload your log file (If a log file hasn't been generated, go to the 'Run' tab in FragPipe, click 'Export Log', zip the resulting "log_[date_time].txt" file to avoid truncation, then attach the zipped file by drag & drop here. log_2021-09-26_01-54-25.txt )

[huiyinc]

Hi Jongmin,

Can you please check if there is a column, Purity, in your PSM.tsv file? Thanks.

Huiyin

dnwhdals @.***> 於 2021年9月26日 下午2:04 寫道：

 Describe the issue or question: Hello, I am struggling with an issue with FragPipe v16.0 while quantifies PSMs in TMT-10 dataset. It seems the tool is not able to generate the "Purity" column in the PSM tables. Please find attached the log file. How can I resolve the issue?

Thank you in advance.

Jongmin Woo

Upload your log file (If a log file hasn't been generated, go to the 'Run' tab in FragPipe, click 'Export Log', zip the resulting "log_[date_time].txt" file to avoid truncation, then attach the zipped file by drag & drop here. log_2021-09-26_01-54-25.txt ) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[fcyu]

Hi Jongmin,

I think you didn't run Philosopher' label quant and free quant, which are required by TMT-Integrator. Can you load one of the TMT workflow and try again?

Best,

Fengchao

[dnwhdals]

I tried to re-load TMT10 workflow and re-run the search, but same issue was occurred. In the PSM.tsv file, no "purity" column was created during the process.

Could you more specific of the running Philosopher label quant and free quant? Where can I find the check box of the Philosopher quant?

[fcyu]

Hi Jongmin,

Can you send me your log file? Please zip it. Then, drag it to GitHub issue page, or email it to [email protected]. GitHub somehow won't show the zip attachment if directly replying the email. And GitHub truncates the text file if you attach it directly.

I just tried a dry-run and see those commands in my log, which indicates that freequant and labelquant will be run.

FreeQuant [Work dir: F:\dev\msfragger\dev\exp_1]
C:\Users\yufe\Desktop\bin\philosopher-v4.0.1-RC5.exe freequant --ptw 0.4 --tol 10 --dir E:\data\PXD001468
FreeQuant [Work dir: F:\dev\msfragger\dev\exp_2]
C:\Users\yufe\Desktop\bin\philosopher-v4.0.1-RC5.exe freequant --ptw 0.4 --tol 10 --dir E:\data\PXD001468
FreeQuant [Work dir: F:\dev\msfragger\dev\exp_3]
C:\Users\yufe\Desktop\bin\philosopher-v4.0.1-RC5.exe freequant --ptw 0.4 --tol 10 --dir E:\data\PXD001468
Quant (Isobaric) [Work dir: F:\dev\msfragger\dev\exp]
C:\Users\yufe\Desktop\bin\philosopher-v4.0.1-RC5.exe labelquant --tol 20 --level 2 --plex 10 --annot E:\data\PXD001468\11_annotation.txt --brand tmt --dir E:\data\PXD001468

Best,

Fengchao

[dnwhdals]

log_2021-09-26_01-54-25.zip

Hi Fengchao.

Please find the attached zip file. I can also see the free quant and label quant in a dry run console. FreeQuant [Work dir: E:\Jacob\NPTDAB_Raw_data\AB01] E:\Jacob\fragpipe\tools\philosopher\philosopher.exe freequant --ptw 0.4 --tol 10 --raw --dir E:\Jacob\NPTDAB_Raw_data Quant (Isobaric) [Work dir: E:\Jacob\NPTDAB_Raw_data\AB01] E:\Jacob\fragpipe\tools\philosopher\philosopher.exe labelquant --tol 20 --level 2 --plex 10 --annot E:\Jacob\NPTDAB_Raw_data\annotation_annotation.txt --brand tmt --raw --dir E:\Jacob\NPTDAB_Raw_data

Best regards, Jongmin

[fcyu]

Hi Jongmin,

Thanks for your log. But there is no free quant or label quant in your log. Can you double check?

Best,

Fengchao

[dnwhdals]

log_2021-09-27_13-11-48.zip Could you check out the new log file attached? I uploaded a log file that was created when I first noticed the problem, but strangely there was no labelquant on the log. But if you look at the log file I upload now, you will see that labelquant exists. But I still have the same issue.

Jongmin

[fcyu]

Thanks Jongmin,

Yes, the latest log has free quant and label quant, and both of them finished successfully. But TMT-Integer still says that can't find the "Purity" column. Can you help to check if there indeed no "Purity" column in psm.tsv?

Thanks,

Fengchao

[dnwhdals]

No, there is no 'Purity' column on the psm.tsv file. Please take a look the top row of psm file. If you need psm.tsv, I can send you via email because Github doesn't support .tsv file.

Spectrum	Spectrum File	Peptide	Modified Peptide	Prev AA	Next AA	Peptide Length	Charge	Retention	Observed Mass	Calibrated Observed Mass	Observed M/Z	Calibrated Observed M/Z	Calculated Peptide Mass	Calculated M/Z	Delta Mass	Expectation	Hyperscore	Nextscore	PeptideProphet Probability	Number of Enzymatic Termini	Number of Missed Cleavages	Protein Start	Protein End	Intensity	Assigned Modifications	Observed Modifications	Is Unique	Protein	Protein ID	Entry Name	Gene	Protein Description	Mapped Genes	Mapped Proteins	Quan Usage	TP01	TP02	TP03	TP04	TP05	TP06	TP07	TP08	TP09	pool

Thank you.

Jongmin

[fcyu]

Thanks Jongmin. Then, it is a Philosopher bug.

Felipe @prvst , can you take a look?

Thanks,

Fengchao

[sarah-haynes]

Hi Jongmin, can you upload the psm table and one of the raw files here? https://www.dropbox.com/request/dnXWcJre9FgQJeibzZEl

[dnwhdals]

Hi, Sarah.

I uploaded the files in the dropbox so you can take a look. The raw file I uploaded is only partial of entire dataset. We have currently 864 raw files, corresponding to 840 GB, for the entire dataset. I am indeed wondering the FragPipe can be handle the high-throughput dataset.

Best regards Jongmin Woo

[anesvi]

I should handle it except possibly one step in philosopher. But it is not needed, really. If you get a crash at that stage, you can rerun from TMT-Integrator step.

Fengchao – should we add an option not to run that step?

Jongmin, you can also contract us directly by email, as you have a special case

Best Alexey

From: dnwhdals @.> Sent: Tuesday, September 28, 2021 3:11 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Assign @.> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471)

External Email - Use Caution

Hi, Sarah.

I uploaded the files in the dropbox so you can take a look. The raw file I uploaded is only partial of entire dataset. We have currently 864 raw files, corresponding to 840 GB, for the entire dataset. I am indeed wondering the FragPipe can be handle the high-throughput dataset.

Best regards Jongmin Woo

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/471#issuecomment-929546437, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM6YOIOLMI56B5LCKGEDUEIHMPANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[fcyu]

Hi Alexey,

Yes, I will add a checkbox to FragPipe.

Hi Jongmin,

Sarah will contact you directly regarding the files you uploaded.

Thanks,

Fengchao

[dnwhdals]

Hi Alexey, Sounds good to me. I will try to rerun with marking the option of skip PSM quantification.

And thank you, Fengchao, for your help!

JW

[sarah-haynes]

Hi Jongmin, can you try uploading the raw file and psm.tsv file again? It looks like they didn't get uploaded the first time: https://www.dropbox.com/request/dnXWcJre9FgQJeibzZEl

(Still writing you here because the email didn't go through.)

Thanks! Sarah

[dnwhdals]

hi Sarah,

Sorry for your inconvenience, please double check the uploaded. You can also directly email me at [email protected] Thank you!

JW

[sarah-haynes]

Hi Jongmin, we are able to use other Thermo .raw files directly for TMT workflows, but it looks like these .raw files don't work, and it may take us a while to figure out why. I converted the file you sent to .mzML and that worked fine, so I would recommend converting to .mzML for now. You can do this in the command line if it's easier, here's an example: msconvert.exe --mzML --64 --zlib --filter "peakPicking true 1-" *.raw

Sarah

[dnwhdals]

Thank you for your suggestion. The files have been searched by MaxQuant before so I believe it would work on the FragPipe. As you suggested, I will try to convert the files and search it. Please keep an update if you figure it out.

JW

[anesvi]

Yes, you should convert from RAW to mzML. Philosopher is not optimized for RAW, so it would take a lot longer to run with RAW than mzML (although in your case it seems it is not even an option to use RAW)

From: dnwhdals @.> Sent: Wednesday, September 29, 2021 12:10 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Assign @.> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471)

External Email - Use Caution

Thank you for your suggestion. The files have been searched by MaxQuant before so I believe it would work on the FragPipe. As you suggested, I will try to convert the files and search it. Please keep an update if you figure it out.

JW

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/471#issuecomment-930321873, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM6YUB2ZBYTUJCJUR3I3UEM255ANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[dnwhdals]

Agreed, Alexey.

[anesvi]

Please make sure to sue a recent version of proteowizard/scovert, not more than 6 months old…

From: dnwhdals @.> Sent: Wednesday, September 29, 2021 12:16 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Assign @.> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471)

External Email - Use Caution

Agreed, Alexey.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/471#issuecomment-930327090, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM62ZYHGYQ4QR2ZMFI2LUEM3UTANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[surendradasari]

Hi, There. I ran into the same issue described here with FragPipe 16.0 (downloaded on 10/18/2021). TMT Integrator complains about "Purity" column missing in the PSM files. Just wondering if issue was fixed. Best, Surendra

[anesvi]

Can you check you are using the latest philosopher? And also converted to mzML using a recent version of msconvert

From: surendradasari @.> Sent: Monday, October 18, 2021 11:34 AM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Assign @.> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471)

External Email - Use Caution

Hi, There. I ran into the same issue described here with FragPipe 16.0 (downloaded on 10/18/2021). TMT Integrator complains about "Purity" column missing in the PSM files. Just wondering if issue was fixed. Best, Surendra

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/471#issuecomment-945899297, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM64XEWJMH2MLBR7SSJLUHQ45NANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[surendradasari]

Can you check you are using the latest philosopher? And also converted to mzML using a recent version of msconvert From: surendradasari @.> Sent: Monday, October 18, 2021 11:34 AM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Assign @.> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471) External Email - Use Caution Hi, There. I ran into the same issue described here with FragPipe 16.0 (downloaded on 10/18/2021). TMT Integrator complains about "Purity" column missing in the PSM files. Just wondering if issue was fixed. Best, Surendra — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#471 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM64XEWJMH2MLBR7SSJLUHQ45NANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

Hi, Alexey. Thank your for quick reply. Philosopher is latest. I updated it with the Fragpipe "Download" button. MSConvert maybe older. Please let me redo the conversion with latest PWIZ and see if I can make it through. Best, Surendra

[dnwhdals]

Hi, Surendra.

Recently we resolved the missing purity issue using conversion to mxML. I used this tool to convert the files. Hope this is helpful for you.

JM

[surendradasari]

Can you check you are using the latest philosopher? And also converted to mzML using a recent version of msconvert From: surendradasari @.> Sent: Monday, October 18, 2021 11:34 AM To: Nesvilab/FragPipe _@**._> Cc: Nesvizhskii, Alexey _@.>; Assign @.**_> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471) External Email - Use Caution Hi, There. I ran into the same issue described here with FragPipe 16.0 (downloaded on 10/18/2021). TMT Integrator complains about "Purity" column missing in the PSM files. Just wondering if issue was fixed. Best, Surendra — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#471 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM64XEWJMH2MLBR7SSJLUHQ45NANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

Hi, Alexey. Thank your for quick reply. Philosopher is latest. I updated it with the Fragpipe "Download" button. MSConvert maybe older. Please let me redo the conversion with latest PWIZ and see if I can make it through. Best, Surendra

[surendradasari]

Can you check you are using the latest philosopher? And also converted to mzML using a recent version of msconvert From: surendradasari @.> Sent: Monday, October 18, 2021 11:34 AM To: Nesvilab/FragPipe _@**._> Cc: Nesvizhskii, Alexey _@.>; Assign @.**_> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471) External Email - Use Caution Hi, There. I ran into the same issue described here with FragPipe 16.0 (downloaded on 10/18/2021). TMT Integrator complains about "Purity" column missing in the PSM files. Just wondering if issue was fixed. Best, Surendra — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#471 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM64XEWJMH2MLBR7SSJLUHQ45NANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

Hi, Alexey. Thank your for quick reply. Philosopher is latest. I updated it with the Fragpipe "Download" button. MSConvert maybe older. Please let me redo the conversion with latest PWIZ and see if I can make it through. Best, Surendra

Hi, Alexey. I ran with latest proteowizard and that seems to have fixed the issue. Also, I think the issue may be that I filtered out MS1 scans from the mzML files before giving them to FragPipe. That may have caused the issue I have seen. When I included both MS1 and MS2 in mzML, used latest PWIZ for conversion and latest FragPipe, things seem to have worked OK. Best, Surendra

[anesvi]

Good. Yes, you need MS1 scans too even though it is TMT analysis (that’s how the Purity score is calculated) Best, Alexey

From: surendradasari @.> Sent: Monday, October 18, 2021 3:21 PM To: Nesvilab/FragPipe @.> Cc: Nesvizhskii, Alexey @.>; Assign @.> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471)

External Email - Use Caution

Can you check you are using the latest philosopher? And also converted to mzML using a recent version of msconvert From: surendradasari @.> Sent: Monday, October 18, 2021 11:34 AM To: Nesvilab/FragPipe @**.mailto:_@**._> Cc: Nesvizhskii, Alexey @.*>; Assign @._> Subject: Re: [Nesvilab/FragPipe] "TMT-Integrator can't find the 'Purity' column" (#471https://github.com/Nesvilab/FragPipe/issues/471) External Email - Use Caution Hi, There. I ran into the same issue described here with FragPipe 16.0 (downloaded on 10/18/2021). TMT Integrator complains about "Purity" column missing in the PSM files. Just wondering if issue was fixed. Best, Surendra — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#471 (comment)https://github.com/Nesvilab/FragPipe/issues/471#issuecomment-945899297>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM64XEWJMH2MLBR7SSJLUHQ45NANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

Hi, Alexey. Thank your for quick reply. Philosopher is latest. I updated it with the Fragpipe "Download" button. MSConvert maybe older. Please let me redo the conversion with latest PWIZ and see if I can make it through. Best, Surendra

Hi, Alexey. I ran with latest proteowizard and that seems to have fixed the issue. Also, I think the issue may be that I filtered out MS1 scans from the mzML files before giving them to FragPipe. That may have caused the issue I have seen. When I included both MS1 and MS2 in mzML, used latest PWIZ for conversion and latest FragPipe, things seem to have worked OK. Best, Surendra

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/Nesvilab/FragPipe/issues/471#issuecomment-946089108, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIIMM64EG6XEUUU6DEY6RULUHRXS3ANCNFSM5EYNCRBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

---

Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

[rtool1]

Dear FragPipe team and Nesvizhskii Lab,

I'm really grateful for having FragPipe; the tsv outputs are fantastic, and your documentation and the active discussion on GitHub make this a better experience than with MQ.

Pardon me for posting in a closed thread. However, I've run into the "TMT-Integrator can't find the 'Purity' column" error.

• I have always converted SCIEX WIFF's into mzML, with vendor peak picking for msLevel 1-
• I was using the "TMT16" workflow through the GUI

Please find attached my log file, along with a zipped up psm.tsv file; I indeed don't have the Purity column

log_2022-08-22_23-26-25.txt psm.zip

For context, this bottom-up sample set was labeled with the TMTpro 16 reagent. However, to save time on our Eclipse, I'm trying to run a label efficiency test on our SCIEX 7600. With the reduced resolution of the 7600, I'm running the 16 channels in groups of 4. In this particular sample, I've grouped together 127C, 128N, 131C, 132N (i.e. samples-03, -04, -11, and -12). Quantification was done at the MS2 level, and I've set the quant level to MS2 in TMT-Integrator.

Is the "Purity" error due to the fact I have so many "missing" channels? Is there a way to specify just the 4 channels used, if that is indeed the underlying cause of the error?

Down the road, it would be great if we can indeed specify just the channels used, as that would allow us to just maintain the TMTpro16 reagents, while deciding later on if we want to assume the 8-plex vs. 16-plex format depending on our instrument availability and experimental design. Thank you again for your advice. -Eric