allsky
allsky copied to clipboard
AllskyTeam
ZWO nighttime flickering
Hello good Morning. I have a problem that I cannot find where the solution can be ... Days when there is a Moon for example, the automatic exposure changes constantly producing a "flicker" in the sequences, such as: https://observatoridepujalt.cat/allsky/videos/allsky-20210925.mp4 This happens to me since I update to the latest version 0.8 In version 0.7 this did not happen to me.
Thanks!
Hi @brianboru82 -- Can you attach the /var/log/allsky.log file file to this issue. Note: Attach can be done by dragging the file into the compose window for your next comment, please don't copy/paste the file contents.
@EricClaeys will likely want to see that.
I'm wondering too, if you could upgrade to the latest code ( just merged last night ) and see if that helps at all.
Thanks!
Thank you! This is the file: allsky.log
How to update the latest code? Do I have to delete everything and reinstall?
You don't have to delete, you can just move the allsky directory out of the way. Additionally, if you're comfortable with git, you can do something like git stash && git pull && git stash pop (you might have to resolve some merge conflicts if you've made local changes).
@bribon2 We didn't do anything to the nighttime exposure per se, but it's possible the 0.8 exposure algorithm has a side effect. Try disabling the Version 0.8 Exposure - in the GUI, go to the bottom of the Camera Options and click on Show Advanced Options, then set the 0.8 Exposure Mode to No. If it's aldeady No I will have to look at your log file in a couple days when I have my PC.
Eric
@bribon2 It looks like auto gain is on, and I suspect auto exposure as well since the sleep times vary, and that only occurs when the exposure length changes. Try disabling auto gain, and set it to a manual 220 which is what the auto gain was using. Some people have reported problems when both auto gain and auto exposure are set. At night the camera handles the exposure time and gain value, our software doesn't. Let me know how it goes with manual gain. Thanks - Eric
Removing the automatic gain happens to me the same. The option to remove the algorithm 0.8 does not appear to me ... What's more, I can't apply the settings from the GUI, I have to do it from the config file and the default settings appear from the GUI, although it actually applies those from the config. I updated it from version 0.7 to 0.8 by deleting the files by hand and installing it from 0, I don't know if I did something wrong ...
Example of tonight: https://observatoridepujalt.cat/allsky/videos/allsky-20211003.mp4
I've also noticed this, but not mentioned it due to the ongoing daily changes, I was hoping that it would already be on the to-do list. You can see by the attached compressed video excerpt and keogram from last night, that the exposure is hunting when not at the maximum exposure value - zoom in and look at the keogram, the 2nd half of it where it's cloudy and the sky brightness changes continuously - it ramps up over 5 or so exposures, before reverting back and hunting again. You can also see this on the video, watch the exposure value on the overlay - you can see it in the numbers. In another allsky program I used in the past, you could set the number of exposures the software used for it's target exposure, the minimum being 2 frames, maximum, I don't remember. the idea was that it could be fine tuned by the user to filter out these ramps and settle on the best for their camera / setup.
I also remember there was an option to set the autoexposure minimum / maximum changes, for example during the night you might want the exposure to only change by 0.5s, to reduce the flickering, and only use the last 3 frames to settle on an exposure.
I'm using a version of this allsky that is probably only 3 or 4 days old (i.e. not the very latest).
https://user-images.githubusercontent.com/77614536/135823196-ae3ca0b2-fde0-4338-adb7-709166cecb66.mp4
@bribon2 Is the bright light on the right side the moon? It doesn't appear to be moving. Do you notice the "flicker" when there are no clouds? I would guess that some of the flicker is due to the clouds coming and going. When there are lots of clouds the image is dark, but when the clouds clear around the moon the next picture will be bright, then if the clouds come back before the picture after that, that picture will be dark. It takes the auto exposure algorithm some time to adjust to changes in brightness.
In order to see the option for disabling the 0.8 exposure method, you'll need to upgrade the GUI (allsky-portal). When you do that you'll see a "Show advanced options" button at the bottom of the Camera settings page. Click on that and you'll then see the version 0.8 option (it's considered an "advanced" option).
@Jonk2 Do you notice the "flicker" when there are no clouds? Does changing the version 0.8 exposure method make any difference?
I don't notice a flicker when the exposure is at maximum which I suppose is expected, as nothing's changing. It just appears to ramp up over 4 or 5 frames, then returns to the value at the beginning of the ramp when it's at anything less than the max (30 seconds in my case). I tend to do all my testing on the PI indoors, leaving the one outside as is, but I've just changed the one outside to v0.7 exposure mode and will report back tomorrow, to see if there are any differences.
@Jonk2. Thanks in advance. I'm curious what the outdoor one does with and without clouds, both with 0.7 and 0.8. It might be difficult to test for the next couple weeks since the moon isn't out.
@EricClaeys The 0.8 exposure mode was switched off and here are 3 segments of that night's timelapse (ignore the dew, it's internal, needs sorting out and the compression is to keep the filesize down).
1st one, sunset - if flickers like mad with the dereasing light levels but is fine once it settles.
https://user-images.githubusercontent.com/77614536/136443340-afa1283e-0eb6-43df-aacd-fe7c6ffc563d.mp4
2nd one, midway through the night with clouds - you can see it works well, the exposure value stays static enough not to flicker, but still changes a bit to suit. Video is smooth when cloudy and presumably will be the same for a moonlit sky with no or minimal clouds.
https://user-images.githubusercontent.com/77614536/136443903-6111c7bc-366b-40d8-ad48-636ed5c0cc91.mp4
3rd one, sunrise - behaves the same as sunset, lots of flicker as the light level increases.
https://user-images.githubusercontent.com/77614536/136444286-690dc295-fbd2-4173-a560-57e90d4a9004.mp4
I notice that the gain adjustment / ramp to follow the sunset works well for sunset (I find 120 minutes works well for me) but only starts after the sunrise period has ended, and therefore not part of the video, is this just a time adjustment in the code that's required? If it can work for sunrise too, i.e. ramp the gain back down to 0 over the same period that would be very good.
If you need me to do anything else, but for now I'll put the PI outdoors back to 0.8 exposure to give the sensor a breather and cool it slightly!
@Jonk2 if daytime and nighttime gain are different and aren't set to "auto", then gain should be adjusted from its night value to its day value when going from night to day, and from the day to night value when going from day to night. It begins at the day/night transition point as specified by your "angle". The transition length is user-settable. It doesn't take the image brightness into account when adjusting the gain, it just does simple math - I'm at 100 and need to get to 0 in 5 minutes so I'll decrease gain by 20 every minute. If you want to attach /var/log/allsky.log when this is happening I'll take a look. The exposure algorithm I think has a bug that may be contributing to the problem.
I'm not able to play the videos. Tried with 2 browsers on my PC and with my phone. Can you try attaching them rather than including the URL?
I assume you're "angle" is such that sunrise and sunset are during daytime. If so, the histogram exposure algorithm is being used. You can try adjusting where the histogram box is (the part of the image that's looked at to get the brightness), and adjust the size of the box. On smaller sensors the box size is too big (15% of the chip size in pixels is a good amount). In the WebUI on the Camera Settings page, enable "Show Histogram Box" and then adjust the "Histogram Box" numbers. My camera is positioned such that the sun and moon go right through my histogram box and when that happens the pictures get very dark, so I moved my box and on my ASI290 with a small chip, decreased the size.
@EricClaeys strange, must be the encoding. Try these?
Although they're quite compressed, you can see the numbers and sunset / sunset flickering.
https://user-images.githubusercontent.com/77614536/136597293-8641cdc3-69b4-4bf2-a81d-75316a68bd7f.mp4
https://user-images.githubusercontent.com/77614536/136597435-245445fa-048b-460b-9567-b22d4a1a02b1.mp4
https://user-images.githubusercontent.com/77614536/136597451-f6ea709b-e63e-477f-b650-0f00f2656cdb.mp4
My 'nighttime' is set to an angle of 0, so my timelapse will always show the transition from day / sunset into full night (which is usually over a 2 hour period and timelapses showing this to me look great - it's called the holy grail method). I get it's just a timer, but this timer starts at sunset in my case (angle of 0), so why doesn't it finish at angle 0 (i.e. sunrise) rather than start the transition back to gain 0 at sunrise? Does that make sense? I would have thought the transition back to 0 gain for daytime would start 120 minutes before sunrise (the angle of 0).
This is interesting. On the attached screenshot you can see on the left, a version 0.7 autoexposure keogram (camera in video mode, prior to the main change in 0.8). On the right you can see last night's keogram, which was set back to 0.8 (snapshot mode).
Zoomed in, you can see the difference between the 2 and it highlights on the 0.8 version, the constant changes of exposure value.
I know you know about this already and hopefully working on a solution, but the other thing I noticed was the keograms were a different physical image size, as can be seen in the bars at the top (0.7 version is 2154px wide, 0.8 version is 1519px wide. nothing else was changed and the only difference between the 2 images would have been a few minutes of daylight.) and the screenshot of the bottom of the 2 images.
Whether this helps or not I don't know, but I thought I'd mention it. Could be nothing.
@ckuethe Chris, can you take a look at the changes in keogram size above, between version 0.7 and 0.8? The version 0.8 one seems to be scrunched up in width.
Run with debugging and see if images are being rejected? The output is variable size depending on the number valid input images
Hi everyone,
I am experiencing a very similar issue. I have only ran 0.8.1 for one evening. Numerous clouds + the Moon were a factor. But, 0.7 seemed more stable when dealing with these factors, and holding the exposure time stable in general.
Other notes :
- I have New Exposure set to "Yes" due to that addressing the issue described in #655 / #660.
- ASI224MC
- Gain = 200 / Exposure = Auto
Examples:
-
V0.8.1 strobing http://www.thewanderers.xyz/allsky/videos/allsky-20211024.mp4
-
V0.8.1 Keogram http://www.thewanderers.xyz/allsky/keograms/keogram-20211024.jpg
-
V0.7 stable exposure in similar conditions http://www.thewanderers.xyz/allsky/videos/allsky-20211023.mp4
-
V0.7 Keogram http://www.thewanderers.xyz/allsky/keograms/keogram-20211023.jpg
Just posting this as a data point. I'll keep messing with settings over the next few nights.
Thanks!
@Apennine2, please try with the attached file using your v0.8.1 settings. It probably won't fix the strobing but will provide me with more debugging information.
Rename it to "capture.cpp" then move it to allsky/src, then:
cd allsky/src
make capture
sudo service allsky stop
cp capture ..
sudo truncate -s 0 /var/log/allsky.log
sudo service allsky start
Let it run for a couple minutes, then upload the log. Thanks much!
@Jonk2, there's a new option to the keogram program that causes it to always create an image that's as wide as your sensor, so you shouldn't get images like below.
I think it's the --extend option, but run ./keogram --help to be sure.
@Jonk2, have you had a chance to try the new keogram option to expand the image?
@Apennine2, have you tried the most recent capture.cpp and other files? I'm curious if they help with the nighttime strobing.
I haven't done anything manually, and I have only glanced at the automatically generated keograms every other day or so, but checking this morning's I notice 3 changes from the images above:
1 - The text is now white (better) 2 - There's a date (which I don't like personally) 3 - The top of the image is trimmed so it doesn't show the entire height.
This morning's is attached.
Regarding the strobing, it's still there and is caused by exposure hunting over 5, sometimes 6 frames for some reason. You can see it clearly in the keogram when you zoom in.
@Jonk2 For # 1, are you using the default settings in config.sh:
KEOGRAM_EXTRA_PARAMETERS="--rotate 0 --font-size 1.0 --font-line 1 --font-color '255 255 255'"
I removed the font-size and font-line and let the program use its default, which are 2.0 and 3, which I think make the text much easier to see.
For numbers 2 and 3, would you please enter a new Issue stating the top of the keogram image is trimmed, and include the picture above? Also request that displaying the date be user-configurable.
For the strobing, please see this Wiki page and attach your log file. Do you see the strobing during the day AND night?
@EricClaeys , I do have the latest software versions installed and I still see the strobing at night. I do not see the strobing during the day, although it's not making videos during the day. But, I say that it isn't since I always see a nicely exposed constant exposure during the day.
I keep the max night exposure at 60s, since that makes great videos on moonless clear nights. Here is last night's video where you still see the strobing caused by a ~+/-3s exposure bounce during an overcast night.
-Chris
@EricClaeys , the log is attached. This is for the subsequent (last) night. Also a constant cloudy/foggy night. I would have attached the log yesterday, but it was huge since I had not cleared it in a while. I truncated it yesterday, and collected data for about 24hr.
-Chris allsky.log
@Jonk2 For # 1, are you using the default settings in config.sh:
KEOGRAM_EXTRA_PARAMETERS="--rotate 0 --font-size 1.0 --font-line 1 --font-color '255 255 255'"I removed the font-size and font-line and let the program use its default, which are 2.0 and 3, which I think make the text much easier to see.For numbers 2 and 3, would you please enter a new Issue stating the top of the keogram image is trimmed, and include the picture above? Also request that displaying the date be user-configurable.
For the strobing, please see this Wiki page and attach your log file. Do you see the strobing during the day AND night?
Yes the keogram_extra_parameters is the same as you mention (default).
I will start a new issue about the keogram trimming.
The 'strobing' happening during the day, I don't know as it doesn't save the images to create a day timelapse.
@Apennine2 You are probably getting pretty bad strobing during the day as well. Try saving daytime pictures so they are included in a timelapse and look for daytime strobing. Or you may be able to see stobing by zooming in on the keogram.
Your camera appears to be reacting unpredictably to changes in exposure time. During the day the first picture gave a mean brightness of 250 (out of 255) at 500 microseconds (us) exposure so was way too bright. The next couple pictures used shorter exposure times and the brightness went down some, then all of a sudden it got too dark. Adding a little exposure then made it too bright. It then kept ping-ponging like that (which causes the strobing) for quite a while then settled down, then started again.
At night the camera uses the ZWO auto exposure so we have no control over it. It was also doing the ping-pong but also had shots of the same exposure but somewhat different brightness. That again could be caused by rain drops on the dome.
It's very possible that the strobing was caused by rain drops - slight movements in them can cause bright spots that cause the camera to try and darken the next picture, then the bright spot is gone and the camera brightens the next picture.
Do you see the strobing when it's clear outside? Or when it's fully cloudy but not raining?
I'm in a similar position as reported. I had to recently do a complete rebuild of the pi and upgrade from a version of allsky about a year ago to a fresh install. Now nighttime cloudy videos flicker as the exposure ramps up and back over a few frames as described, before they were nice and stable. This is with an asi224mc.
@ennui2342 Try turning "Version 0.8 Exposure Mode" on in the WebUI. It's an advanced options so you'll need to Display Advanced Options.
@ennui2342 Try turning "Version 0.8 Exposure Mode" on in the WebUI. It's an advanced options so you'll need to Display Advanced Options.
Thanks Eric - I'll set that and let you know how it goes tonight.