Uncertainty quantification for a temporal auto correlation function

[cmoecke]

Dear Sjoerd,

as stated in the title, I want to perform uncertainty quantification for an ACF that I calculate dependent on some lag time dt. The ACF f(dt) is implicitly given by

d(dt) = A(1-f(dt))+B

where A and B are educated estimations of time independent parameters (i.e. noise and amplitude estimates of a continuous signal) and d(dt) is some quantity that is calculated for said discrete lag times dt; these are the input parameters. Let's say for now that these parameters are independent of each other and normally distributed. Further, I have some prior knowledge about f(dt), namely that it should be (normally) distributed within the closed interval [0,1] (same goes for A and B). As a first step, in order to estimate the associated uncertainty of the ACF, I did Gaussian propagation of uncertainty obtaining df(dt). Now it might happen that within the frequentist uncertainty margins, this prior knowledge is violated and I am having trouble reasoning putting a hard constraint on it. That's why I thought this would be a great use case of Bayesian statistics. However, I am not quite sure how to implement it properly within this excellent pacelet. My initial idea was to perform the nested sampling for each dt step as follows (please also see the notebook attached, where I provide example data and the full code.)

Table[nestedSampling[
  defineInferenceProblem["Data" -> {d[[i]]}, 
   "GeneratingDistribution" -> 
    NormalDistribution[ai*(1 - fi) + bi, dd[[i]]], 
   "Parameters" -> {{fi, 0, 1}, {ai, 0, 1}, {bi, 0, 1}}, 
   "PriorDistribution" -> {NormalDistribution[f[[i]], df[[i]]], 
     NormalDistribution[A, dA], NormalDistribution[B, dB]}], 
  "SamplePoolSize" -> 1000,
  "MonteCarloSteps" -> 1000,
  "TerminationFraction" -> 0.01,
  "MinIterations" -> 10], {i, Length@dt}];

where I input the values of f(dt) and df(dt) from previous calculations and the Gaussian uncertainty propagation, respectively. Here I am not sure, whether it would be possible to do the analysis with the full list of d, instead of employing Table and with that, the calculations could be sped up a bit, since for a full data analysis I would need to run this procedure ~50 times. I then get the new median and .68 quantiles by

Table[Around[#[[2]], {#[[2]] - #[[1]], #[[3]] - #[[2]]}] &@
  Quantile[EmpiricalDistribution[
    Values@fs[[i]]["Samples", All, "Point"][[All, 1]]], {1 - 0.68, 
    0.5, 0.68}], {i, Length@combdts}]

The values I get in the end seem plausible to me, though I am unsure whether this approach is actually suited for this problem. Intuitively it makes sense to me, but maybe I am missing something crucial.

Thanks in advance!

UCP_Bayesian.nb.zip

[tasoo-oos]

I also tried with about 500 m3u8 links.

[tasoo-oos]

wait no i successfully downloaded a deleted vod

[ewang620]

hmm... ok the vod that i tried to revocer was from this one : https://twitchtracker.com/wkgml/streams/40201692781 I entered this in following order:

1. wkgml
2. 40201692781
3. 2021-01-16 11:50:00 and got this m3u8 link as a result: https://vod-secure.twitch.tv/43f8c7c4847ff8999389_wkgml_40201692781_1610797800/chunked/index-dvr.m3u8

But the link doesn't work..... Have I typed in any wrong info?

[tasoo-oos]

Ive tried the same And I failed to get recovered vod. I dont know what is wrong with it.....

https://twitchtracker.com/ma_mwa/streams/40190198189 i tried with this vod

[daylamtayari]

@dc-creator saw this thread because of the mention to another issue of my tool and directly using my tool I was able to recover that VOD you were looking for: https://vod-secure.twitch.tv/dcbf08bec1292ebb88ba_ma_mwa_40190198189_1610724826/chunked/index-dvr.m3u8

[tasoo-oos]

@daylamtayari I have tried the same And I failed to get recovered vod.

I tried to get m3u8 address with

wkgml
40201692781
2021-01-16 11:50:00

these information and failed

https://twitchtracker.com/ma_mwa/streams/40190198189 I tried with this vod

This means I previously tried to get this stream`s m3u8 address and write the post wait no i successfully downloaded a deleted vod 17 hours ago

sorry for not writing posts properly

[daylamtayari]

@dc-creator I am confused, what are you trying to recover?

For this VOD: https://twitchtracker.com/ma_mwa/streams/40190198189, the M3U8 URL is: https://vod-secure.twitch.tv/dcbf08bec1292ebb88ba_ma_mwa_40190198189_1610724826/chunked/index-dvr.m3u8

And for the other VOD from the streamer wkgml it appears that that streamer does not save their VODs. If a streamer does not enable their VODs to be saved, you cannot recover a VOD because no VOD exists.

[tasoo-oos]

@daylamtayari Right now i am not trying to recover something. I successfully recovered https://twitchtracker.com/ma_mwa/streams/40190198189 right after I found your code. And I tried to recover https://twitchtracker.com/wkgml/streams/40201692781 because jermia314 asked for it. Thank you for your kind service

[dnpthree]

Hello, is it possible to recover this vod? https://twitchtracker.com/metzy_b/streams/41319458494

I tried both tools and it says object not found.

[zenscope]

@dnpthree here it is https://vod.544146.workers.dev/41319458494

[daylamtayari]

@dnpthree just tried myself and cannot find a VOD that is currently still up on Twitch servers. The link @zenscope linked only generates all of the possible links but if you try any of them you will see there is nothing there.

[Sp3ncer03]

Guys, really works on deleted VODS? Im try this VOD https://twitchtracker.com/yoda/streams/36475526752

and the script returns: https://vod-secure.twitch.tv/a4b62df398446cf2b5fa_yoda_36475526752_1577069670/chunked/index-dvr.m3u8

This URL no open in VLC...

Help ps

[ewang620]

Twitch server started to permanently delete VODs in their servers since about one or two months ago. U can't use them anymore....

[Redfly020]

can u help me for this vod plz https://twitchtracker.com/aloonea/streams/42481365052

[tasoo-oos]

@Redfly020 Right now stream vods is deleted when vod is deleted in twitch server. This stream was streamed at last year, so There won`t be vod file in twitch server. Try finding other person having that vod

[tasoo-oos]

@Redfly020 vod is deleted in twitch server when streamer/twitch deletes vod (there are some time gap though) or automatically deletes in two month after stream(if streamer is twitch partner, it may be shorter).