fgbio icon indicating copy to clipboard operation
fgbio copied to clipboard

[bug] ClipBam: remaining aligned read bitflag not properly set when mate is clipped to 0 and unaligned

Open eboyden opened this issue 2 years ago • 3 comments

When ClipBam clips one read in a pair to length 0, it properly sets that read's 0x4 "read unmapped" bitflag, and sets the remaining aligned read's 0x8 "mate unmapped" bitflag. However the remaining aligned read still has the 0x2 bitflag set for "read mapped in proper pair". If I'm not mistaken, the 0x2 bitflag should never be set if either the 0x4 or 0x8 bitflag is set.

eboyden avatar Feb 28 '22 00:02 eboyden

@eboyden unless @tfenne disagrees (feel free to reopen), we do not to touch that flag as it is completely determined by the aligner (or other upstream tools) and we tend to ignore it anyhow.

nh13 avatar Feb 28 '22 23:02 nh13

That may be, but it's an important flag for filtering. If ClipBam renders one of the two previously aligned reads unaligned, it is logically impossible for 0x2 to be set; no aligner would ever do that. So if I filter based on it, at a minimum I should only obtain pairs of aligned reads. (Different aligners may vary with regard to read orientation, interval size, etc.) So I think it's entirely appropriate to simply unset that flag for both reads in a pair (if it was set originally) if one or both reads is rendered unaligned.

eboyden avatar Feb 28 '22 23:02 eboyden

@eboyden I could be convinced with a PR.

nh13 avatar Mar 01 '22 00:03 nh13