gatk icon indicating copy to clipboard operation
gatk copied to clipboard

VS-227 clean up temp tables

Open RoriCremer opened this issue 2 years ago • 5 comments

Previously, a temporary table is created as part of extract of the VQSR features, and it goes into a separate temp_tables dataset in the current project -- that is no longer true, and it now goes into the default dataset as a short living temp table with the task name and a hash

This pr should:

  • default to the current dataset (with the VET etc tables) rather than a different dataset

  • give a prefix to the temp tables so we know which one came from which step

  • temp table TTL---not a changeable option, but default to 24 hours across the board

Still to discuss: Parameterization of the location (dataset) to create the temp table in (default to the default dataset) manual clean up/TTL is a changeable option and TTL is parametrizable (currently the TTL is a parameter for the prepare step -- but then we set a default as 24 hrs in the WDL)

Screen Shot 2022-06-10 at 1 27 58 PM

RoriCremer avatar Mar 29 '22 20:03 RoriCremer

Github actions tests reported job failures from actions build 2315930821 Failures in the following jobs:

Test Type JDK Job ID Logs
cloud 8 2315930821.10 logs
cloud 11 2315930821.11 logs
cloud 11 2315930821.11 logs
cloud 8 2315930821.10 logs

gatk-bot avatar May 12 '22 21:05 gatk-bot

Codecov Report

:exclamation: No coverage uploaded for pull request base (ah_var_store@e4c4bfc). Click here to learn what that means. The diff coverage is n/a.

@@               Coverage Diff                @@
##             ah_var_store     #7742   +/-   ##
================================================
  Coverage                ?   86.282%           
  Complexity              ?     35195           
================================================
  Files                   ?      2170           
  Lines                   ?    164904           
  Branches                ?     17787           
================================================
  Hits                    ?    142283           
  Misses                  ?     16296           
  Partials                ?      6325           

codecov[bot] avatar May 12 '22 21:05 codecov[bot]

Github actions tests reported job failures from actions build 2320536513 Failures in the following jobs:

Test Type JDK Job ID Logs
cloud 8 2320536513.10 logs
cloud 11 2320536513.11 logs

gatk-bot avatar May 13 '22 16:05 gatk-bot

Github actions tests reported job failures from actions build 2347228211 Failures in the following jobs:

Test Type JDK Job ID Logs
cloud 8 2347228211.10 logs
cloud 11 2347228211.11 logs

gatk-bot avatar May 18 '22 18:05 gatk-bot

Github actions tests reported job failures from actions build 2347214817 Failures in the following jobs:

Test Type JDK Job ID Logs
cloud 11 2347214817.11 logs

gatk-bot avatar May 18 '22 18:05 gatk-bot