mtag icon indicating copy to clipboard operation
mtag copied to clipboard

Analysis terminated from error

Open 123569 opened this issue 5 years ago • 14 comments

The mean chi2 statistic of trait 1 is less than 1.02, which may lead to unstable estimates. To perform MTAG on your results anyways, include the --force option, though the estimates should be interpreted cautiously. Traceback (most recent call last): File "/public/home/course_users/c010/wc/mtag/mtag.py", line 1557, in mtag(args) File "/public/home/course_users/c010/wc/mtag/mtag.py", line 1327, in mtag DATA_U, DATA, args = load_and_merge_data(args) File "/public/home/course_users/c010/wc/mtag/mtag.py", line 280, in load_and_merge_data raise ValueError("The mean chi2 statistic of trait {} is less than 1.02, which may lead to unstable estimates. To perform MTAG on your results anyways, include the --force option, though the estimates should be interpreted cautiously.".format(p+1)) ValueError: The mean chi2 statistic of trait 1 is less than 1.02, which may lead to unstable estimates. To perform MTAG on your results anyways, include the --force option, though the estimates should be interpreted cautiously. Analysis terminated from error

123569 avatar Mar 09 '19 12:03 123569

Hi @123569 ,

Did you use the --force option? Can you attach the log file / Do you see the flag showing in the header? If so the flag should function properly.

Here is what I saw in a test that works:

...
<> Note:  It is recommended to run your own QC on the input before using this program. 
<> Software-related correspondence: [email protected] 
<> All other correspondence: [email protected] 
<><><<>><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>


Calling ./mtag.py \
--force  \
--stream-stdout  \
--n-min 0.0 \
--sumstats ./1_OA2016_hm3samp_NEUR.txt,./1_OA2016_hm3samp_SWB.txt_modZ \
--out ../output/test_force_small_chi2
...

huilisabrina avatar Mar 10 '19 19:03 huilisabrina

I did not use the --force option,Can I try to solve this mistake by adding FORCE option? my log file showing this :<> Note: It is recommended to run your own QC on the input before using this program. <> Software-related correspondence: [email protected] <> All other correspondence: [email protected] <><><<>><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

Calling ./mtag.py
--stream-stdout
--n-min 0.0
--sumstats KGW.txt,YD.txt
--out ./tutorial_results_1.1NS

123569 avatar Mar 11 '19 11:03 123569

The software may run if you use the force option, but you should be careful. This error gets thrown when one or more of the GWAS results are low powered, which makes the MTAG results potentially unstable.

On Mon, Mar 11, 2019 at 7:39 AM 123569 [email protected] wrote:

I did not use the --force option,Can I try to solve this mistake by adding FORCE option? my log file showing this :<> Note: It is recommended to run your own QC on the input before using this program. <> Software-related correspondence: [email protected] <> All other correspondence: [email protected]

<><><<>><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

Calling ./mtag.py --stream-stdout --n-min 0.0 --sumstats KGW.txt,YD.txt --out ./tutorial_results_1.1NS

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/omeed-maghzian/mtag/issues/63#issuecomment-471505691, or mute the thread https://github.com/notifications/unsubscribe-auth/AUNA9bsipZAtZWHezaYb-pXOtwaGs2B5ks5vVkCHgaJpZM4bmquv .

paturley avatar Mar 11 '19 11:03 paturley

Using 1638454 SNPs to estimate Omega (0 SNPs excluded due to strand ambiguity) Estimating sigma.. After merging with reference panel LD, 0 SNPs remain. Traceback (most recent call last): File "/public/home/course_users/c010/wc/mtag/mtag.py", line 1557, in mtag(args) File "/public/home/course_users/c010/wc/mtag/mtag.py", line 1342, in mtag args.sigma_hat = estimate_sigma(DATA[not_SA], args) File "/public/home/course_users/c010/wc/mtag/mtag.py", line 468, in estimate_sigma rg_results = sumstats_sig.estimate_rg(args_ldsc_rg, Logger_to_Logging()) File "/public/home/course_users/c010/wc/mtag/ldsc_mod/ldscore/sumstats.py", line 423, in estimate_rg M_annot, w_ld_cname, ref_ld_cnames, sumstats, _ = _read_ld_sumstats(args, log, None, alleles=True, dropna=True,sumstats=p1) File "/public/home/course_users/c010/wc/mtag/ldsc_mod/ldscore/sumstats.py", line 251, in _read_ld_sumstats sumstats = _merge_and_log(ref_ld, sumstats, 'reference panel LD', log) File "/public/home/course_users/c010/wc/mtag/ldsc_mod/ldscore/sumstats.py", line 236, in _merge_and_log raise ValueError(msg.format(N=len(sumstats), F=noun)) ValueError: After merging with reference panel LD, 0 SNPs remain. Analysis terminated from error at Tue Mar 12 16:30:49 2019 Total time elapsed: 2.0m:2.83s Metadata: Mean chi^2 = 1.254 Lambda GC = 1.181 Max chi^2 = 27.37 0 Genome-wide significant SNPs (some may have been removed by filtering)

What's the reference panel LD? I wonder if my "n" value is wrong. I have 189,4012 SNPs in total. The sample number is 535. My experimental material is rice diploid. So I set my "n" value to 1070. Is there a problem here or is the data not good?

123569 avatar Mar 12 '19 09:03 123569

Is the minimum P value of one of the two traits greater than a certain value determined as no significant SNP, and affect the subsequent operation, if so, can I manually modify this value?

123569 avatar Mar 12 '19 09:03 123569

I just verified the reason for the error, not because of the lack of significant SNPs for a single trait. Maybe it's related to my data or the value of N.

123569 avatar Mar 12 '19 12:03 123569

MTAG uses a reference genome to run LD score regression. If you are not using a human sample, you may need to calculate the LD scores yourself for MTAG to run. Might this be the problem?

On Tue, Mar 12, 2019 at 8:31 AM 123569 [email protected] wrote:

I just verified the reason for the error, not because of the lack of significant SNPs for a single trait. Maybe it's related to my data or the value of N.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/omeed-maghzian/mtag/issues/63#issuecomment-471980820, or mute the thread https://github.com/notifications/unsubscribe-auth/AUNA9TIqPbVMQpb1Q_7JxxfSa9vjRvFeks5vV542gaJpZM4bmquv .

paturley avatar Mar 12 '19 21:03 paturley

How can I bring my calculated LD score into the MTAG program?

123569 avatar Mar 13 '19 13:03 123569

It sounds as if I was mistaken and MTAG can't take user-generated LD scores. (Correct me if I'm wrong, Hui.) If you want to put in the work, I think your best shot would be first to run LD score regression yourself with your LD scores to estimate the sigma matrix (basically the intercepts from the set of LD score regressions), and then use the MTAG option to input your Sigma matrix yourself. I worry that this would be pretty complicated though, so you may decide that it's not worth the effort.

On Wed, Mar 13, 2019 at 9:21 AM 123569 [email protected] wrote:

How can I bring my calculated LD score into the MTAG program?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/omeed-maghzian/mtag/issues/63#issuecomment-472418923, or mute the thread https://github.com/notifications/unsubscribe-auth/AUNA9QZSGtPYKM7TADCPt0J_wLmV6Mc3ks5vWPtsgaJpZM4bmquv .

paturley avatar Mar 13 '19 15:03 paturley

Hi @123569 ,

I echo everything @paturley said. If you decide to estimate the sigma matrix via LDSC first, the flag to input it to MTAG is --residcov_path, and you can find more details in the wiki section here.

Best, Hui

huilisabrina avatar Mar 13 '19 15:03 huilisabrina

If the method is feasible, I have no reason to refuse it. What file format does sigma matrix need to upload?Is this Sigma matrix? Estimated Sigma: [[ 1.052 -0.167] [-0.167 1.016]]

123569 avatar Mar 14 '19 02:03 123569

It's the Sigma matrix that requires LD score regression. I don't recall the exact format the is required for the file, but Hui should be able to help there.

Will there also be problems with the munge step, Hui, or does that step get skipped if the user provides Sigma?

On Wed, Mar 13, 2019 at 10:50 PM 123569 [email protected] wrote:

If the method is feasible, I have no reason to refuse it. What file format does sigma matrix need to upload?Is this Sigma matrix? Estimated Sigma: [[ 1.052 -0.167] [-0.167 1.016]]

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/omeed-maghzian/mtag/issues/63#issuecomment-472684287, or mute the thread https://github.com/notifications/unsubscribe-auth/AUNA9c6gXG-KnCz0jb88JGGwaQh4Il_kks5vWbkKgaJpZM4bmquv .

paturley avatar Mar 14 '19 15:03 paturley

@123569 The format of the Sigma should be a whitespace-delimited .txt file containing just the numbers - so no brackets or the header.

@paturley that's a good point - munging does not get skipped if the user provides Sigma. Munging is integrated into the sumstats reading / QC process at the moment.

Best, Hui

huilisabrina avatar Mar 14 '19 15:03 huilisabrina

In that case, you may need to dig into the code and comment out the lines where the SNPs are compared to the reference SNP list. I imagine the code should run after that.

On Thu, Mar 14, 2019 at 11:53 AM huilisabrina [email protected] wrote:

@123569 https://github.com/123569 The format of the Sigma should be a whitespace-delimited .txt file containing just the numbers - so no brackets or the header.

@paturley https://github.com/paturley that's a good point - munging does not get skipped if the user provides Sigma. Munging is integrated into the sumstats reading / QC process at the moment.

Best, Hui

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/omeed-maghzian/mtag/issues/63#issuecomment-472931671, or mute the thread https://github.com/notifications/unsubscribe-auth/AUNA9SSye4ZaMIa6_zd7hWw-9aGVLozlks5vWnCWgaJpZM4bmquv .

paturley avatar Mar 14 '19 16:03 paturley