gatk icon indicating copy to clipboard operation
gatk copied to clipboard

Created FuncotatorDataSourceBundler Tool

Open haileypfox opened this issue 3 years ago • 14 comments

haileypfox avatar Aug 19 '21 18:08 haileypfox

Travis reported job failures from build 35614 Failures in the following jobs:

Test Type JDK Job ID Logs
unit openjdk11 35614.13 logs
integration openjdk11 35614.12 logs
unit openjdk8 35614.3 logs
integration openjdk8 35614.2 logs

gatk-bot avatar Aug 19 '21 19:08 gatk-bot

Travis reported job failures from build 35833 Failures in the following jobs:

Test Type JDK Job ID Logs
unit openjdk11 35833.13 logs
integration openjdk11 35833.12 logs
unit openjdk8 35833.3 logs
integration openjdk8 35833.2 logs

gatk-bot avatar Sep 01 '21 22:09 gatk-bot

Travis reported job failures from build 36902 Failures in the following jobs:

Test Type JDK Job ID Logs
unit openjdk11 36902.13 logs
unit openjdk8 36902.3 logs

gatk-bot avatar Nov 16 '21 18:11 gatk-bot

Travis reported job failures from build 36904 Failures in the following jobs:

Test Type JDK Job ID Logs
unit openjdk11 36904.13 logs
unit openjdk8 36904.3 logs

gatk-bot avatar Nov 16 '21 19:11 gatk-bot

Travis reported job failures from build 36908 Failures in the following jobs:

Test Type JDK Job ID Logs
cloud openjdk11 36908.14 logs

gatk-bot avatar Nov 16 '21 21:11 gatk-bot

Travis reported job failures from build 36910 Failures in the following jobs:

Test Type JDK Job ID Logs
unit openjdk11 36910.13 logs
unit openjdk8 36910.3 logs

gatk-bot avatar Nov 16 '21 21:11 gatk-bot

@droazen OK - I think it's ready for your review.

jonn-smith avatar Nov 17 '21 00:11 jonn-smith

Travis reported job failures from build 36989 Failures in the following jobs:

Test Type JDK Job ID Logs
unit openjdk11 36989.13 logs
unit openjdk8 36989.3 logs

gatk-bot avatar Nov 23 '21 20:11 gatk-bot

Is there still anything missing? It is not merged since quite some time. The funcotator data sources in version 1.7 are already lacking at least 2 years behind the current databases. I would like to see a possibility to create own and updated bundles merged. Thanks!

robby81 avatar May 24 '22 08:05 robby81

@robby81 There are still one or two issues with the tool that need to be resolved before this can be merged, unfortunately. @jonn-smith will be resuming work on these shortly -- we do want to get this tool into GATK soon to make it easier to create custom Funcotator datasources!

droazen avatar May 24 '22 18:05 droazen

@droazen We are still on hold for update the Funcotator datasources. Especially getting new versions of gencode would be desirable and does not work with the current scripts. We already started looking for alternatives for our routine diagnostics workflow.

robby81 avatar Aug 30 '22 08:08 robby81

@robby81 Which scripts are you running and what are the errors you see? The data sources scripts are unsupported, but should work out of the box (they did last time I tried them).

jonn-smith avatar Aug 30 '22 14:08 jonn-smith

you may want to have a look here: https://paste.opensuse.org/40b96df4

Before, I tried it with version 38 of Gencode. It is the indexing, that fails due to fixed fields, that changed looong time ago.

--

r-engelmann.de - Ihre Seite für die Auswertung und Visualisierung von Daten aus den Bereichen Biomedizin, Finanzen, Sozioökonomie und weitere.

On Dienstag, 30. August 2022 16:36:24 CEST Jonn Smith wrote:

@robby81 Which scripts are you running and what are the errors you see? The data sources scripts are unsupported, but should work out of the box (they did last time I tried them).

robby81 avatar Aug 30 '22 18:08 robby81

@robby81 Ah, OK. Yeah - this is a problem in the parser I wrote. I was trying to be very specific about what was allowed so users could not build erroneous data sources.

This is high on my list of fixes. I'm trying to get some time to do them.

jonn-smith avatar Aug 30 '22 20:08 jonn-smith

And for cosmic:

/home/robby/Tools/NGS/gatk-4.2.6.1-src/scripts/funcotator/data_sources/cosmic/ getCosmicDataSources.sh This script creates the cosmic data sources for the Funcotator GATK tool.

For usage information run with the '-h' option

To retrieve the COSMIC data sources you must have a COSMIC account. Please enter your COSMIC account credentials: Enter your email address: @.*** Enter your password: Creating folders: ... mkdir: created directory 'cosmic' mkdir: created directory 'cosmic/hg19' mkdir: created directory 'cosmic/hg38' mkdir: created directory 'cosmic_fusion' mkdir: created directory 'cosmic_fusion/hg19' mkdir: created directory 'cosmic_fusion/hg38' mkdir: created directory 'cosmic_tissue' mkdir: created directory 'cosmic_tissue/hg19' mkdir: created directory 'cosmic_tissue/hg38' Getting files ... get: cosmic/grch37/cosmic/v84/ CosmicCompleteTargetedScreensMutantExport.tsv.gz: ssh: Could not resolve hostname sftp-cancer.sanger.ac.uk: Name or service not known get: cosmic/grch37/cosmic/v84/CosmicFusionExport.tsv.gz: ssh: Could not resolve hostname sftp-cancer.sanger.ac.uk: Name or service not known get: cosmic/grch38/cosmic/v84/ CosmicCompleteTargetedScreensMutantExport.tsv.gz: ssh: Could not resolve hostname sftp-cancer.sanger.ac.uk: Name or service not known get: cosmic/grch38/cosmic/v84/CosmicFusionExport.tsv.gz: ssh: Could not resolve hostname sftp-cancer.sanger.ac.uk: Name or service not known @.***:~/Tools/NGS>

--

r-engelmann.de - Ihre Seite für die Auswertung und Visualisierung von Daten aus den Bereichen Biomedizin, Finanzen, Sozioökonomie und weitere.

On Dienstag, 30. August 2022 16:36:24 CEST Jonn Smith wrote:

@robby81 Which scripts are you running and what are the errors you see? The data sources scripts are unsupported, but should work out of the box (they did last time I tried them).

robby81 avatar Oct 11 '22 07:10 robby81

Whats the status here? Also the bundled sources got no updates since 2020.

robby81 avatar Apr 05 '23 14:04 robby81

@robby81 This still has to be merged. I've been pulled off onto some other projects for a bit. Some changes to the internals of Funcotator are needed for this tool to be most useful, so I was waiting until those updates were made to merge this.

I can compile a new release of the data sources, but I haven't heard from anyone in the community that it's a priority. Can you create a new issue for it? Some questions around a new release: If I were to create one, what would the new release include? Would there be any new data sources that were not included before? Are any included data sources no longer useful and should be removed?

jonn-smith avatar Apr 05 '23 14:04 jonn-smith

The biggest thing for a new data source would be the latest (or dare I say the ability to choose the version) of GENCODE.

On Wed, Apr 5, 2023, 10:06 AM Jonn Smith @.***> wrote:

@robby81 https://github.com/robby81 This still has to be merged. I've been pulled off onto some other projects for a bit. Some changes to the internals of Funcotator are needed for this tool to be most useful, so I was waiting until those updates were made to merge this.

I can compile a new release of the data sources, but I haven't heard from anyone in the community that it's a priority. Can you create a new issue for it? Some questions around a new release: If I were to create one, what would the new release include? Would there be any new data sources that were not included before? Are any included data sources no longer useful and should be removed?

— Reply to this email directly, view it on GitHub https://github.com/broadinstitute/gatk/pull/7427#issuecomment-1497550761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJMDRJS4JRSEURXGDDFK5TW7V36RANCNFSM5COZRAWA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

dheiman avatar Apr 05 '23 14:04 dheiman

Is there anything happening on this front?

robby81 avatar Jul 27 '23 14:07 robby81

@robby81 - we're finally getting around to fixing some of the underlying incompatibilities with this. We're working on updating some of the internals of Funcotator and there will be a new datasources release very soon. After that I'll test this branch again to see if it's functional.

jonn-smith avatar Jul 27 '23 15:07 jonn-smith