About Circle-Map Realign calculation

[10KGenomics]

Dear Author： I encountered a problem when using Circle-Map. I found that when running Circle-Map Realign, the progress bar is always 0%, as shown below. I thought it was a problem of building a database, but later I found it was not. I followed the instructions in your tutorial, but it was not feasible. I want to ask you, is there any other reason? The key is that it does not report an error, I have no clue! Looking forward to your reply!

[iprada]

Hi caijingtao1993,

Thanks a lot for using Circle-Map.

The error you encounter is a little bit extrange. The error happens because Circle-Map fails to estimate the realignment prior parameters on the file Yin-cancer-50ng_qname.bam. And consequently, it is not able to start the realignment process. That is the reason you do not see any progress. I think that it might be that the file Yin-cancer-50ng_qname.bam is corrupted. Could you post the output of the following commands:

samtools quickcheck Yin-cancer-50ng_qname.bam

samtools view -H Yin-cancer-50ng_qname.bam

samtools view Yin-cancer-50ng_qname.bam | head -n 50

I think that the output of those commands might help me diagnose what is going on

Best wishes

Inigo

[10KGenomics]

Dear Author： the following commands: samtools view -bS Yin-cancer-50ng.sam > Yin-cancer-50ng.bam samtools sort -@ 48 -n Yin-cancer-50ng.bam >Yin-cancer-50ng_qname.bam

[10KGenomics]

But not all samples can't be run, and all samples can't build the index of qname.bam, so we didn't consider the problem in this direction at that time. And your tutorial did not specify the need to index qname.bam.

[iprada]

I do not see anything wrong from the error message. This is a extrange error. The queryname is used very little. I wonder what is going on. I would like to give a look to the bam file. Lets talk by mail

[iprada]

Hi

Sorry for the silence. I have been looking at this. This is an issue similar to #35

I have been looking a bit into the code to see what is going on and I have narrow it down to a few lines of code that perform poorly.

Briefly described, this is what is going on:

Circle-Map collects realigned intervals, denoted as n, and discordant read intervals, denoted as m, independently. Then, it will simply assign the m discordant intervals that are within a reasonable distance (insert size mean + 3 standard deviations) to the n realigned intervals.

The computation described above has a complexity of O(nm) ~ O(n**2). This has never been a problem for me in all the data has gone through Circle-Map. However, it seems reasonable that this can be a problem in some corner cases when n and m are very large. Which is probably your case and the case of #35.

I should of course fix this, but I can tell you that I am not going to have time to fix this over the next 2-3 months as I finishing my PhD. However, this is going to be one of the first things I will look once I am done.

I assume waiting a couple of months until I fix the issue an release a new version is not going to be a realistic thing for you I guess that you could try circle_finder: https://github.com/pk7zuva/Circle_finder. In my benchmarks they are a bit faster than Circle-Map. And if circle_finder code does not scale quadratically they should be much faster on your data.

[10KGenomics]

Dear Author: Thanks for your reply, I can also look forward to the follow-up to optimize the software, I used the 1.1.3 version found to be feasible, temporarily I can use this version, you can compare the differences between the two versions, hope you Know. Thank you very much for your guidance, thank you.You can always contact me later.I will learn more about your software!

[iprada]

1.1.3 should be much faster due to a bug in the software (too many graph paths are filtered out). However, it might still provide you reliable results.

Best,

Iñigo

[10KGenomics]

Received, author, thank you for your guidance.

发自我的iPhone

------------------ Original ------------------ From: iprada <[email protected]> Date: Wed,Jul 15,2020 6:50 PM To: iprada/Circle-Map <[email protected]> Cc: caijingtao1993 <[email protected]>, Author <[email protected]> Subject: Re: [iprada/Circle-Map] About Circle-Map Realign calculation (#37)

[iprada]

You are welcome :)

Feel free to open another issue if you have any other questions

[10KGenomics]

Ok，thanks a lot ！

发自我的iPhone

------------------ Original ------------------ From: iprada <[email protected]> Date: Wed,Jul 15,2020 6:55 PM To: iprada/Circle-Map <[email protected]> Cc: caijingtao1993 <[email protected]>, Author <[email protected]> Subject: Re: [iprada/Circle-Map] About Circle-Map Realign calculation (#37)

[kpkhushi15]

Hiii iprada!

I encountered the same problem as @caijingtao1993.

I followed the steps which you have mentioned in the tutorial. but still I was getting something like this:

2020-10-18 23:07:53: Realigning reads using Circle-Map

2020-10-18 23:07:53: Clustering structural variant reads

2020-10-18 23:09:09: Splitting clusters to to processors

[E::idx_find_and_load] Could not retrieve index file for 'SRR7642284.bam'

I kept it running for overnight and I got this:

Now I do not understand whether the bed file (SRR7642284.bed) contains the eccDNA or not. Bed File result looks like this:

Could you please look at the output of the bed file and suggest me regarding the same.

[iprada]

Hi @kpkhushi15

The output looks fine. You can read more about the output here. https://github.com/iprada/Circle-Map/wiki/Circle-Map-Realign-output-files

BEst,

Inigo