wz_mini_hacks icon indicating copy to clipboard operation
wz_mini_hacks copied to clipboard

Internet access required ?

Open rickgitdone opened this issue 2 years ago • 130 comments

I installed the SD CARD made all the rtsp edits and booted up ... seems this still requires access to wyze for authorization before anything else works?

Is this possible to run local only on home subnet w/o any Internet access to the CAM?

rickgitdone avatar May 15 '22 15:05 rickgitdone

as far as I know, yes internet access is required. I haven't done any testing without internet access at all.

gtxaspec avatar May 15 '22 23:05 gtxaspec

For setting up without the Wyze App, it may be possible. This thread has some details on the QR code setup process: https://github.com/HclX/WyzeHacks/issues/146

As for offline use, I've been working to set up my four v3 cams with wz_mini_hacks today. Unintentionally, they did have internet access until just about 30 minutes ago. I didn't realize that adding another interface to my router for this second network would automatically created PASS rules to WAN until this thread prompted me to double check.

I have my cameras on a dedicated wifi network that has no access to my main network or the internet. I plan to grab the most recent updates tomorrow (RTSP sub streams? Awesome!) but I'll report back if they get into reboot loops. My cameras currently have firmware 4.36.9.131 loaded internally.

shamlord avatar May 16 '22 06:05 shamlord

my hope is setup wifi with config file and launch the rtsp server v4l2rtspserver and ssh server ( dropbear) and leave iCamera and all the monitor processes down to allow local only system... wired ethernet is good also ...

rickgitdone avatar May 16 '22 15:05 rickgitdone

currently wz_mini needs iCamera. We hook the functions coded inside of iCamera to pass video to the rtsp server, so all the magic is done inside iCamera, without it nothing works. You could look to the T31 SDK to program your own video program, but that is outside the scope of this project.

gtxaspec avatar May 16 '22 23:05 gtxaspec

Just following up on my isolated camera network experience.. The cameras functioned overnight after being cut off of IP or DNS access. This afternoon I updated the cameras to use the more recent build of wz_mini_hacks available earlier today, and afterwards they cycled between streaming RTSP and not. Mostly not.

After giving the cameras IP and DNS access again, they stayed online.

If there is any way to spoof iCamera into not rebooting the cameras repeatedly without internet access that would be neat.

shamlord avatar May 16 '22 23:05 shamlord

Very interested in this as I'll be using this in a vehicle.

evanheckert avatar May 16 '22 23:05 evanheckert

I will run a test on a spare camera, and see if I can examine the logs to see what it uses as a check for internet access to prevent reboots.

gtxaspec avatar May 16 '22 23:05 gtxaspec

does anyone have an approximate time (hours?) of when the camera will reboot?

gtxaspec avatar May 16 '22 23:05 gtxaspec

quickly checking the logs...

May 16 16:42:45 iCamera: [init.c:565]gateway recovery ok...
May 16 16:42:46 iCamera: [DN:829]err: (getaddrinfo) fail:-2(Name or service not known), (domain: www.google.com)
May 16 16:42:46 iCamera: [netservice.c:896][test_conn_cloud_by_url]err: get_ip_addr_by_domain fail:1
May 16 16:42:46 iCamera: [DN:829]err: (getaddrinfo) fail:-2(Name or service not known), (domain: api.wyzecam.com)
May 16 16:42:46 iCamera: [netservice.c:896][test_conn_cloud_by_url]err: get_ip_addr_by_domain fail:1
May 16 16:42:48 iCamera: [DN:829]err: (getaddrinfo) fail:-2(Name or service not known), (domain: www.google.com)
May 16 16:42:48 iCamera: [netservice.c:896][test_conn_cloud_by_url]err: get_ip_addr_by_domain fail:1
May 16 16:42:48 iCamera: [DN:829]err: (getaddrinfo) fail:-2(Name or service not known), (domain: api.wyzecam.com)
May 16 16:42:48 iCamera: [netservice.c:896][test_conn_cloud_by_url]err: get_ip_addr_by_domain fail:1

so if anyone wants to try these commands via ssh:

echo -e "127.0.0.1 www.google.com\n127.0.0.1 api.wyzecam.com" > /tmp/hosts ; mount --bind /tmp/hosts /etc/hosts

and see if the camera stays up

gtxaspec avatar May 16 '22 23:05 gtxaspec

It's much less than an hour. If the camera starts up and does not have internet access, it seems to reboot almost constantly.

I just cut off access and rebooted an easy to access camera. Once I can get into SSH on the camera I'll give this a shot.

shamlord avatar May 16 '22 23:05 shamlord

I can't seem to get the camera to even respond to pings, let alone allow me to SSH in.

I just added the commands to the end of run_mmc.sh to see if that works.

shamlord avatar May 17 '22 00:05 shamlord

no pings even on LAN? hmm... can you share the run_mmc.log in the log dir when you have a chance?

gtxaspec avatar May 17 '22 00:05 gtxaspec

Its odd, after one power cycle the camera started responding to pings almost immediately. After another, it didn't start responding to pings for several minutes, responded to about 20 pings, then stopped responding again. I was able to SSH in long enough during another window of responsiveness, but the connection dropped before I could do anything.

I just pulled the SD card and grabbed the requested log: run_mmc.log

shamlord avatar May 17 '22 00:05 shamlord

I'm looking through the log and if I read this correctly:

+ grep 'inet addr'
+ ifconfig wlan0
          inet addr:10.42.0.190  Bcast:10.42.0.255  Mask:255.255.255.0

The camera is being configured with an IP not in my network's range. I'm using 192.168.2.0/24. This specific camera has a DHCP reservation for 192.168.2.17, which does show up later in the logs.

shamlord avatar May 17 '22 00:05 shamlord

can you check your router? maybe there is a stale dhcp static lease or something. perhaps you can check the logs too to see if and why it's giving out that 10.x ip.

gtxaspec avatar May 17 '22 00:05 gtxaspec

Its similar to an IP range I used years ago (10.42.69.0/24 .. heh) but I currently use 172.16.0.0/16 for my home network and 192.168.2.0/24 for my camera network. I have confirmed the ranges on my router's DHCP server (pfSense), and both of my Wifi APs are Asus routers in AP mode running the Merlin firmware.

A DHCP diagnostic utility on a windows laptop connected to my camera Wifi is not getting replies, interestingly, so I'm troubleshooting that now.

Edit: Still troubleshooting. I think it is an issue with the AP I'm using for the cameras.

Edit 2: Not my Wireless AP. On my Linux machine, apparently Network Manager will assign the address 10.42.0.1 to any network adapter that's active but not configured for DHCP or Static addressing, and then responds to DHCP requests from that address. This is without dhcpd installed. Jeez. I'll get back to you later regarding the camera rebooting issue.

shamlord avatar May 17 '22 01:05 shamlord

Its been a few months, but this thread is interesting with regard to the camera failing to stay online based on DNS and NTP connection failures: https://forums.wyzecam.com/t/wyze-cam-v3-connection-time-out-during-setup-when-ntp-being-forwarded/150908/3

shamlord avatar May 17 '22 09:05 shamlord

so if anyone wants to try these commands via ssh:

echo -e "127.0.0.1 www.google.com\n127.0.0.1 api.wyzecam.com" > /tmp/hosts ; mount --bind /tmp/hosts /etc/hosts

and see if the camera stays up

I added this to the end of run_mmc.sh on one camera, it caused it to frequently drop its connection. The camera would drop 2-3 pings once every 15 to 20 seconds. The log file run_mmc.log just shows Blue Iris repeatedly reconnecting.

I removed this line and rebooted the camera, it hasn't dropped since. My other cameras have not been restarting, but were not blocked from WAN.

shamlord avatar May 18 '22 05:05 shamlord

this was for a camera with LAN access, but no internet right?

gtxaspec avatar May 18 '22 05:05 gtxaspec

Yes. It has the same behavior with internet access as well.

shamlord avatar May 18 '22 05:05 shamlord

i'll do more testing, but heres what i did:

start camera up with network, then change dns server to 0.0.0.0 to simulate internet down

camera pings LAN, RTSP works, but no internet access in or out of the camera.

run echo -e "127.0.0.1 www.google.com\n127.0.0.1 api.wyzecam.com" > /tmp/hosts ; mount --bind /tmp/hosts /etc/hosts

check 4 hours later, camera still online, rtsp server still working.

gtxaspec avatar May 18 '22 05:05 gtxaspec

On my network, the camera seemed to drop its connection every 15-20 seconds, but I think now that it does not actually reboot. My SSH connection would freeze, but then seemed to start responding again after a few moments. The connection would not drop, and whatever I was doing was not lost (i.e. nano was still running with run_mmc.sh open). Its possible the camera may be disabling and reenabling its interface or something similar.

The way I blocked the camera's access was in the router, disallow everything from the camera network except the camera's access to the router's IP on that network (192.168.2.1). DNS would still be working, but the camera would not be able to make a ICMP, TCP or UDP connection to anything outside of the network.

I can do some more troubleshooting tomorrow.

shamlord avatar May 18 '22 05:05 shamlord

There is a huge thread on the Wyze Forums about using the RTSP firmware and isolating the cameras from the Internet, what they try to access, and the experiences people have had trying to run the cams without Internet access: https://forums.wyzecam.com/t/beta-testing-for-wyze-cam-v3-rtsp-firmware-now-available/196922/622

I tried the following so far today. The router interface the cameras are attached to is set to default DENY all traffic except for what I have allowed for each test. After making a change to my firewall rules, the camera was restarted with the reboot command via SSH. When the cameras disconnect, they are unreachable for about 3 to 5 seconds.

Test 1:

  • Allow only UDP 123 from the cameras to WAN (No DNS or anything else) Result: Cameras drop connection about every 60 to 70 seconds

Test 2:

  • Allow UDP 123 from cameras->WAN
  • Allow All ports/protocols including DNS to the router's IP (but nothing to WAN) Result: Cameras drop connection about every 4 minutes

FWIW, here are the NTP servers the camera tries to reach. I've not had success yet redirecting them to my router's NTP server via the hosts file on the camera. The hosts file does redirect the name resolution for the NTP servers but timesync fails to sync the time. The hostnames are hardcoded into the /system/bin/timesync binary:

clock.fmt.he.net
ntp0.cornell.edu
time-a.nist.gov
time.windows.com
time-b.nist.gov
  • I pkilled the timesync process and restarted it to capture the following output, ending with me allowing NTP access again and the process syncing the time. timesync.log

The camera apparently also tries to make connections to the following additional addresses based on the router/DNS log files of a few other users in the thread linked above:

api.wyzecam.com
wyze-membership-service.wyzecam.com
wyze-general-api.wyzecam.com
www.google.com
kinesisvideo.us-west-2.amazonaws.com - possibly for video processing or machine vision
wyze-iot.s3-us-west-2.amazonaws.com - retrieving/storing something in an S3 bucket
c1ybkrkbxxxxxx.credentials.iot.us-west-2.amazonaws.com
m-756xxxxx.kinesisvideo.us-west-2.amazonaws.com
r-fd5xxxxx.kinesisvideo.us-west-2.amazonaws.com

The addresses with the "xxxxx" at the beginning of the domain were partially redacted as the commenter was worried they might uniquely identify them.

I'm not super familiar with the Wyze cameras, but I'm learning as I go. If you have any insight that might help point me in a troubleshooting direction that would be neat. To get these camera's truly offline I think they will at least need to be redirectable to a local NTP server, and then their connections to other web services would need to be redirected to a lightweight web server (nginx, Lighttpd, etc) supplying the camera firmware with whatever response makes it happy.

shamlord avatar May 18 '22 23:05 shamlord

Something I forgot to add. According to Wyze documentation, the cameras need the following access to the Internet:

Port What it does What it's used for
TCP: 80 Local timelapse download Timelapse
TCP: 123 Internet time check Confirming the camera's time zone
TCP: 443 Cloud data transfer Uploading Events
TCP: 8443 Cloud API Keeping the connection with Cloud server
TCP: 8605 Upgrade package download Firmware Upgrades
TCP: 10001 P2P streaming connection Local live streaming over WiFi
TCP: 10002 LAN firmware upgrade Firmware Upgrades
TCP: 22345 TCP control Making sure the connection stays open when viewing the Live Stream
UDP: 80 Heart beat & streaming data For all UDP ports: Loading the Live Stream and keeping the connection between camera and server even when it's not actively being used
UDP: 443 Heart beat & streaming data  

shamlord avatar May 18 '22 23:05 shamlord

Interesting.. I see that according to that table they want TCP 123 not UDP. Oof. Let me try that again..

EDIT: Nope.. setting the following in my camera's hosts file:

192.168.2.1 clock.fmt.he.net
192.168.2.1 ntp0.cornell.edu
192.168.2.1 time-a.nist.gov
192.168.2.1 time.windows.com
192.168.2.1 time-b.nist.gov

And allowing both TCP and UDP access to port 123 causes the following output from timesync:

[root@WCV3-NUM4:etc]# pkill /system/bin/timesync; timesync
[timesync_utility.c:ip_validate::69] ip address obtained

[main.c:time_sync_process::276] time sync process running!
[main.c:time_sync_ntp::65] ntp_worker_handler time_zone:21600
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::163] two consecutive ntp requests differ more than 15 minutes, try next NTP server!
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_process::293]  try 1, ntp_worker_handler() failed:-1
[main.c:time_sync_ntp::65] ntp_worker_handler time_zone:21600
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_process::293]  try 2, ntp_worker_handler() failed:-1
[main.c:time_sync_process::308] ntp synchronization failed after 2 try(s), consecutive failure: 1
[main.c:time_sync_process::346] ntp failed, checking the time from the checkpoint file!
[main.c:time_sync_process::351] file_extracted_time: 1652912161 read from the checkpoint
[main.c:time_sync_ntp::65] ntp_worker_handler time_zone:21600
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_process::293]  try 1, ntp_worker_handler() failed:-1
[main.c:time_sync_ntp::65] ntp_worker_handler time_zone:21600
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_ntp::106] ntp ip: 192.168.2.1
[main.c:time_sync_ntp::134] select timeout
[main.c:time_sync_process::293]  try 2, ntp_worker_handler() failed:-1
[main.c:time_sync_process::308] ntp synchronization failed after 2 try(s), consecutive failure: 2
[main.c:time_sync_process::346] ntp failed, checking the time from the checkpoint file!
[main.c:time_sync_process::351] file_extracted_time: 1652912161 read from the checkpoint
[main.c:time_sync_ntp::65] ntp_worker_handler time_zone:21600

shamlord avatar May 18 '22 23:05 shamlord

Have you tried disabling all motion/sound detection/recording/notifications? Each of those events probably tries to phone home.

From what I can tell the problematic one is probably AWS IOT core (MQTT on 8443) which is also what all the unofficial wyze SDKs use to communicate with the camera over the wyze web api.

mrlt8 avatar May 19 '22 00:05 mrlt8

Thank you! I never set up any detection rules, but I have just gone through and checked/set the following on each camera:

Detection Settings -> Detection Zone: Off
Event Recording    -> Detects Motion: Off
Event Recording    -> Detects Sound: Off
Event Recording    -> Smart Detection -> All: Off
Notifications      -> Notifications: Off
Alarm Settings     -> All: Off
Rules              -> None
Sharing            -> None

Some of the cameras did have Event Recording and Smart Detection enabled. Annoying since with each new camera it prompts you to start a Wyze Plus trial and I don't see a way to opt out.

I'll try isolating the cameras again to see how they react with their new settings.

shamlord avatar May 19 '22 01:05 shamlord

Actually, I wonder if touch /tmp/aws_iot/rooCAok might be enough to keep it from looping?

mrlt8 avatar May 19 '22 02:05 mrlt8

So Just to clarify... Does this problem STILL affect the cameras EVEN if using them in WebCam mode???

capncybo avatar May 19 '22 17:05 capncybo

webcam mode disables everything. it's just a plain dumb webcam

gtxaspec avatar May 19 '22 19:05 gtxaspec