TPU USB accelerator

[gabeale]

Dear fellows, first of all many compliments for Frigate Project that is very cool.

I was wondering if it would be possible to use a different Tensorflow accelerator hardware because the indicated ones from Google Coral are not available on the market and they won't be before January 2023 (one year from now)!

This is an issue for many users like me because they cannot go further with Frigate (only option is to use CPU that is not recommended).

Being Tensorflow open source, in principle it should be able to run on hardware of different manufacturers. For instance browsing the Tensorflow website i understood that any TPU and GPU can be used.

What is your point of view?

Thanks in advance for the reply you'll give me. Ciao! :-)

[blakeblackshear]

I haven't seen any other AI accelerators that support tensorflow lite directly. They all require their own Sdk and model conversion.

[gabeale]

Thanks Blake for prompt reply. Have you an idea where purchase the Coral Accelerator? In EU, all the stores don't have it in stock.

[suth]

Not in the EU but I found them available on ebay, albeit at a large markup. Resisted paying that much for awhile but now that I have one I wish I had given in sooner.

[alfwro13]

They are available from Kamami store for about 100Euro. I bought mine from Germany few weeks ago for about 70Euros from this company: Berrybase

[Re4DeR]

Kamami store doesnt have any. they sent me mail "Coral USB Accelerator is out of stock."

[vakilando]

Yes, I ordered there and they sent me a mail that the coral will be in stock again in april. I did not cancel my order......hopefully

[ve6rah]

The complete inability to purchase a coral tpu anywhere is a real barrier to frigate adoption. I've been searching for months to try to find one, I've asked every site to notify me when they get any, and nothing. I refuse to pay $500 for a product advertised as 1/10th of that.

My cpu detector just isn't going to be able to handle my 4 camera setup. Realistically with stock estimates on most sites showing over a year delay to get one, coupled with Google's propensity for discontinuing products at random, trying yourself to only this one, impossible to get, option really limits the project.

I will shortly be forced to go with a competing project, that may not work quite as well, but doesn't require hardware that is unobtainable.

[jmorris644]

We have Frigate working on the Jetson Nano.

https://github.com/blakeblackshear/frigate/discussions/2752

[blakeblackshear]

Jetson Nano devices are also unobtainable for reasonable prices. GPUs are expensive right now. Even an overpriced Coral is comparable to either of these options. The only reasonable alternative would be the Intel compute stick, and I haven't had the time to look at integrating it yet.

[jmorris644]

My apologies, I had not recently looked. You are right, they are going for close to 4x.

[ve6rah]

Looks like a Jetson nano dev kit is about half the price of a coral, and is a full computer rather than just the TPU, still, I'd rather just a tpu to use with my existing setup.

An Intel compute sick is about 1/4 the price of a coral.

The coral may be best, but at $500 it's certainly not cheap! The advertising that says it's $25 makes the mark-up painful!

[blakeblackshear]

If you can find a Jetson Nano 4gb for less than $200 please send me a link. I have been looking for weeks. Everything is out of stock.

It's going to be months before the Intel compute stick could be integrated even if it was a priority.

[ve6rah]

$200 is less than half the cost of any coral I can find.

[blakeblackshear]

You can't find a jetson for less than $400-500 either. I see several listings for a Coral USB on ebay for less than that. I get that they are hard to come by, but so are most of the alternatives.

[ve6rah]

Google Coral $455 Jetson Nano $300 Intel Compute Stick $95

Best prices I can find for each.

[katsaplias]

+1 for NCS2

[stixpunk]

what about donation campaign for prioritizing intel ncs2 integration? after lot of searching, there is many people interested and ncs2 is only one available hardware on market for long term

[LaurenceGough]

There are some sites in the UK now where you can backorder the Coral USB accelerator - a few months away, one of them even advertises a 22 day lead time but I suspect it will be a few months too.

I'm sure other countries will also be getting stock soon.

[x9prototype]

I believe this could be a starting point for Tensorflow on NCS2 https://www.intel.com/content/www/us/en/support/articles/000055228/boards-and-kits.html

[gabeale]

Thanks x9prototype. I also believe that it is required to develop Tensorflow on different hardware than Coral USB. This is because, in general, is not advisable to be bounded on a single Manufacturer: for instance the availability problem we are facing with Coral is the proof. As somebody has suggested, I propose the crowdfunding for this project.

[bugsysiegals]

While the supply chain is disrupted, is it possible to leverage Nvidia GPUs for acceleration?

[NickM-27]

While the supply chain is disrupted, is it possible to leverage Nvidia GPUs for acceleration?

Check out #2548 and https://github.com/blakeblackshear/frigate/discussions/3016

[gabeale]

As far as I'm concerned, Nvidia Jetson nano is nowadays more expensive than Coral!

[NickM-27]

As far as I'm concerned, Nvidia Jetson nano is nowadays more expensive than Coral!

Sure, seems most products in that category are, but that PR also adds general support for Nvidia models and at least one user has it working with an nvidia GPU

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ve6rah]

Certainly still a big issue. I even bit the bullet and paid the ridiculous price for a coral, over a month later the seller admitted they didn't actually have any even at that price, and I got my money back. You simply cannot get a coral for any price at this point.

[victorhooi]

I was able to order a Intel Neural Compute Stick 2 (NCS2) which arrived in a few days - this appears to be an alternative to the Coral USB stick, and is more available.

I don't believe Frigate supports the NCS2 currently - there's a discussion about it here:

https://github.com/blakeblackshear/frigate/discussions/2918

From some naive Google searching, it seems the NCS2 is supported by the OpenVINO library, which does have some kind of tflite import support:

• https://docs.openvino.ai/latest/omz_tools_accuracy_checker_tf_lite_launcher.html
• https://discuss.tensorflow.org/t/convert-tflite-model-maker-object-detection-model-to-openvino/2057/2

What do people think?

Also, Deepstack already does support the Intel NCS2 already =), which helps there as well, I guess if people want to use the Frigate/Deepstack integration, right?

[kusold]

Also, Deepstack already does support the Intel NCS2 already

I might be wandering into the offtopic zone, but Deepstack's NCS2 support seems to be limited exclusively to rPis. There doesn't seem to be a way to run it on amd64.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ve6rah]

No change. There are still no available coral devices to purchase, nor will this project accept any alternatives.