Dual-Edge-TPU-Adapter icon indicating copy to clipboard operation
Dual-Edge-TPU-Adapter copied to clipboard

PCIe x1 adapter cooling options

Open magic-blue-smoke opened this issue 3 years ago • 29 comments

Option 1

Dual Edge TPU cards are distributed by Mouser, so I went through their catalog looking for heat sinks.

TPU_Adapter_PCIe_heat_sink

This one fits, with few thoughts in mind:

  1. Distance between mounting holes on PCB is 45.25mm, heat sink is 46.7mm. Pushpins are tilted a bit to compensate this difference, but locked securely
  2. TPU flipchips are not the highest components, that's why thermal grease is not recommended and elastic thermal pads should be used
  3. This heat sink is way too high and won't let another card in the next PCIe slot

update: 4. thermal pads have to be purchased separately, as existing thermal grease won't reach TPU and PMIC flipchips 5. Rev2 boards (with 4 mounting slots) solve distance issue (1) above, however would require spacers for pins to lock securely. 6. Rev2 board mounting slots accept heatsinks with square mounting holes arrangement and distance between holes 31.3-32.7mm

Heat sink Mfr. No: ATS-CPX040040020-115-C2-R0 Pushpins Mfr. No: ATS-HK127-R0 Also available form Digikey

Option 2

Jump to 3D printed Coral TPU Cooler Adapter for 40mm Heatsink by @ZCalilung image

magic-blue-smoke avatar Oct 19 '21 19:10 magic-blue-smoke

How hot do these dual TPUs get? Would it be feasible to 3D print an enclosure to house the board, a 40x40x10 heat sink, and 40x40x10 fan with a little thermal pad in between the TPU and heat sink? I can try to work on a 3D printed model if it doesn't get crazy hot

ZCalilung avatar Oct 21 '21 05:10 ZCalilung

How hot do these dual TPUs get? Would it be feasible to 3D print an enclosure to house the board, a 40x40x10 heat sink, and 40x40x10 fan with a little thermal pad in between the TPU and heat sink? I can try to work on a 3D printed model if it doesn't get crazy hot

@ZCalilung how hot it gets depends on heatsink itself. Worst case is 2 TPUs x 2W + DC-DC losses + margin = 5W If anyone old enough to remember Intel 486 DX2 66 - it has 5W TDP and can give you idea how big heatsink should be

Distance between mounting holes is exactly to mount 40x40mm fan

To stay within PCIe max component height and not to interfere with next slot, heatsink + fan should be not more than 10-11mm

For heatsink + fan configuration, I don’t think temperatures would go anywhere close to melt or soften ABS plastic, but it’s just a feeling

magic-blue-smoke avatar Oct 21 '21 07:10 magic-blue-smoke

@magic-blue-smoke since you have mounting holes, installing a heat sink may be as easy as getting something like this from amazon.

I was originally thinking of using a blower fan right next to a 40mm heat sink so that there is more surface area for heat dissipation and stronger airflow from; and then printing an enclosure to hold everything to the board.

I may have missed it somewhere in issue#4, but what is the voltage going to the "fan" pins on your board?

Also, do you have any boards that I could buy from you so that I can play around with different heat sinks configurations?

ZCalilung avatar Oct 21 '21 13:10 ZCalilung

@magic-blue-smoke since you have mounting holes, installing a heat sink may be as easy as getting something like this from amazon.

Options like this could be nice option with two thoughts in mind:

  • This item has 5V fan, while fan pins on PCB are 12V. One would need to find 12V version or get 5V from PSU/motherboard
  • Item description is missing distance between pushpins

To allow more choices of heat sinks I plan to have slots instead of holes for pushpins

I was originally thinking of using a blower fan right next to a 40mm heat sink so that there is more surface area for heat dissipation and stronger airflow from; and then printing an enclosure to hold everything to the board.

If blower fan can be placed near heat sink (not stacked), that would make sense, allowing PCIe card in the next slot

I may have missed it somewhere in issue#4, but what is the voltage going to the "fan" pins on your board?

12V, as PCIe slot only provides 3.3V and 12V

Also, do you have any boards that I could buy from you so that I can play around with different heat sinks configurations?

As of today, I have boards manufactured, tested and packaged. Next week I'm trying my last options for accepting USD to avoid multiple and unpredictable currency conversions.

magic-blue-smoke avatar Oct 22 '21 13:10 magic-blue-smoke

Kinda confused about these measurements you posted

Distance between mounting holes on PCB is 45.25mm, heat sink is 46.7mm.

The specsheet seems to imply the heatsink is 40x40x20mm and the distance between holes is 33mm. https://www.qats.com/pushPIN/DataSheet/ATS-CPX040040020-115-C2-R0

46.7mm would be the diagonal between holes.

If the specsheet is correct for the heatsink, it would appear this heatsink is almost perfect for a standard 40mm fan (holes are 32mm apart normally, and 46.7 diagonal)

j0rd avatar Oct 31 '21 10:10 j0rd

@j0rd Standard 40mm fan (and my adapter) has distance between holes: 32mm nearest/45.25mm diagonal Heatsink mentioned: 33mm nearest/46.7mm diagonal

magic-blue-smoke avatar Nov 01 '21 19:11 magic-blue-smoke

Ended up going with a tall cooling option for testing. Not designed for your pcie adapter unfortunately though. How would you recommend I look for signs of overheating during use? 20211104_151521 20211106_223935

ZCalilung avatar Nov 08 '21 15:11 ZCalilung

@ZCalilung nicely designed! As for temperature readings, currently I don't have access to hardware to check, but here is:

  • some documentation: https://coral.ai/docs/pcie-parameters/#pcie-parameters-overview
  • examples: https://github.com/google-coral/edgetpu/issues/102#issuecomment-635967304

magic-blue-smoke avatar Nov 08 '21 21:11 magic-blue-smoke

Ended up going with a tall cooling option for testing. Not designed for your pcie adapter unfortunately though. How would you recommend I look for signs of overheating during use? 20211104_151521 20211106_223935

any chance for the STL file of this 3d print? :)

krim404 avatar Nov 18 '21 10:11 krim404

@krim404 , Here is a thingiverse link to my design. https://www.thingiverse.com/thing:5140489 I also linked the non printed parts that I used from Amazon in the description.

I am able to monitor the TPU temp in UnRAID using the TPU Driver Community app. I am using the TPU for object detection on frigate for about ~25 cameras at 5 FPS. The cooler is able to keep the TPU around 40C. I think the cooler is a success!

ZCalilung avatar Nov 18 '21 18:11 ZCalilung

Just curious, how necessary is this? I have one sitting at 50 C and another at 60 C. Seems stable and naturally they will throttle if needed, if they stay at those temperatures is it necessary to have a heat sink?

NickM-27 avatar Jul 20 '22 03:07 NickM-27

Just curious, how necessary is this? I have one sitting at 50 C and another at 60 C. Seems stable and naturally they will throttle if needed, if they stay at those temperatures is it necessary to have a heat sink?

@NickM-27 Edge TPU max junction temperature is 115C (page4)

To get maximum performance heatsink is needed, as TPUs combined have 4W TDP. That's TDP of 486DX4 if you old enough or TDP of Raspberry Pi 4 if you young enough :)

magic-blue-smoke avatar Jul 24 '22 00:07 magic-blue-smoke

@magic-blue-smoke thanks, makes sense. I ordered this heatsink along with a noctua 40mm. Will try the heatsink alone first and worst case add the noctua fan and move it down to a different pcie slot

NickM-27 avatar Jul 24 '22 03:07 NickM-27

The heatsink's default thermal pad wasn't thick enough to actually touch the components so I don't think it would have done much if anything. I ended up just using the noctua 40 mm fan along with the anti-vibration mounts through the pcie adapters fan holes.

The noctua fan's back bracket touching the m.2 board likely isn't ideal but I didn't put the rubber anti-vibration mounts super tight so I think it should be okay, and so far it has made a big difference in temps

image

NickM-27 avatar Jul 24 '22 21:07 NickM-27

I commented in the main page a little while ago but I have an updated design of the heatsink specifically for @magic-blue-smoke 's dual tpu board. It only uses a 40mm heat sink and no fan. I havent pushed this one yet though. The instance of frigate that I run with the noctua fan has ~30 cameras running person detection. The one with only the heat sink only has ~3 cameras so far. https://www.printables.com/model/151840-coral-tpu-cooler-adapter-for-40mm-heatsink

image image

ZCalilung avatar Jul 25 '22 00:07 ZCalilung

The heatsink's default thermal pad wasn't thick enough to actually touch the components so I don't think it would have done much if anything. I ended up just using the noctua 40 mm fan along with the anti-vibration mounts through the pcie adapters fan holes.

@NickM-27 Thanks for your feedback, I've updated first post to explicitly state that separately purchased thermal pad is required for proper cooling

I commented in the main page a little while ago but I have an updated design of the heatsink specifically for @magic-blue-smoke 's dual tpu board. It only uses a 40mm heat sink and no fan. I havent pushed this one yet though. The instance of frigate that I run with the noctua fan has ~30 cameras running person detection. The one with only the heat sink only has ~3 cameras so far. https://www.printables.com/model/151840-coral-tpu-cooler-adapter-for-40mm-heatsink

@ZCalilung thanks for sharing your design, looks nice! I've updated first post with link to it

magic-blue-smoke avatar Jul 27 '22 23:07 magic-blue-smoke

I tried your design, I have 14 cams and it's running at 64C before 74C. I think will try to modify your design and add a blower fan to the side. Let's see how that turns out.

steppel avatar Sep 05 '22 02:09 steppel

I tried your design, I have 14 cams and it's running at 64C before 74C. I think will try to modify your design and add a blower fan to the side. Let's see how that turns out.

Alright, first attempt. Temp at 50.5C with 30mm blower fan.

image

image

w/o anything 74C~ w/ 40mm Heat sink 64C~ w/ 40mm heat sink + 30mm blower fan 50C~

steppel avatar Sep 06 '22 02:09 steppel

What's the recommended minimal thermal pad thickness? Both under and above the coral board.

4

And wouldn't a non-conductive thermal putty be better?

RedAnon avatar Dec 22 '22 18:12 RedAnon

@RedAnon

What's the recommended minimal thermal pad thickness? Both under and above the coral board.

Thermal pad thickness should be selected according to manufacturer recommendations, because it might need some tension to be most effective. Ie if the gap is 3mm, then manufacturer can recommend 3.5 or 4mm for example

  • Mounting standoff height and distance from adapter PCB to TPU card is 1.5mm
  • Distance from TPU to heatsink has to be calculated depending on cooling solution you plan to use

And wouldn't a non-conductive thermal putty be better?

Non-conductive soft thermal pad shall be used, as it will be touching SMD components

magic-blue-smoke avatar Dec 29 '22 00:12 magic-blue-smoke

Sorry to revive an older thread but on the REV2 boards what is the distance between the mounting holes? I found these on EBAY and wondered if they would work.

https://www.ebay.com/itm/254979067561?chn=ps&_trkparms=ispr%3D1&amdata=enc%3A1LQdsOAk_Q763Bj8xunMr-w56&norover=1&mkevt=1&mkrid=711-117182-37290-0&mkcid=2&mkscid=101&itemid=254979067561&targetid=1584739241414&device=c&mktype=&googleloc=9010767&poi=&campaignid=15275224983&mkgroupid=131097072938&rlsatarget=pla-1584739241414&abcId=9300697&merchantid=101686882

pyrodex avatar May 14 '23 18:05 pyrodex

no those ebay heatsinks have the holes way too far out, i have one in my hand. if you look at the blue digikey heat sink, the holes are inside the footprint, ebay ones are outside, unfortunately.

rball8985 avatar May 19 '23 17:05 rball8985

no those ebay heatsinks have the holes way too far out, i have one in my hand. if you look at the blue digikey heat sink, the holes are inside the footprint, ebay ones are outside, unfortunately.

https://www.digikey.com/en/products/detail/advanced-thermal-solutions-inc./ATS-05C-134-C2-R0/4656045?utm_adgroup=General&utm_source=google&utm_medium=cpc&utm_campaign=PMax%20Shopping_Product_Zombie%20SKUs&utm_term=&utm_content=General&gclid=CjwKCAjwvJyjBhApEiwAWz2nLUDqigvlaEiSWpSUT045cgmrAmn98iM-vE077OdXyV2P7pjbgSNXqRoCFAkQAvD_BwE ?

pyrodex avatar May 19 '23 17:05 pyrodex

looking at the ebay heatsink, I'm thinking about using a dremel cutting wheel and drill to make it fit. if I end up doing this, ill post a photo.

rball8985 avatar May 19 '23 18:05 rball8985

PXL_20230519_200715435 PXL_20230519_185253833

It should work nicely, fans too big now but I should have some 30mm that should work.

rball8985 avatar May 19 '23 20:05 rball8985

Sorry to revive an older thread but on the REV2 boards what is the distance between the mounting holes? I found these on EBAY and wondered if they would work.

@pyrodex I've updated first post here regarding distance. It's 31.3-32.7mm

looking at the ebay heatsink, I'm thinking about using a dremel cutting wheel and drill to make it fit. if I end up doing this, ill post a photo.

@rball8985 That's a lot of efforts :)

magic-blue-smoke avatar May 19 '23 23:05 magic-blue-smoke

PXL_20230520_005403450 It took about 15 minutes to cut, drill and paint it. Sure, its 3 dollars from digikey, but I wasn't going to pay 10 for shipping. I used 3mm thermal pad, 2mm would probably work. Now, I'm wondering if the 2pin holes labeled 12v will put out 12v to run a fan? That'd be awesome.

rball8985 avatar May 20 '23 00:05 rball8985

The PCI bracket that the designer magic-blue-smoke recommended elsewhere fits perfectly. This is the bracket for the Standard Full Height Bracket for LSI 9265-8i, and I got it here. Then I needed to go to the hardware store to get some M3 screws as the bracket did not come with any.

I tried a few different heatsinks, one with a fan, as well as the noctua 40mm fan. So far, the suggestion from the designer magic-blue-smoke seems to fit the best. I bought a 1mm heat transfer pad here and cut it to size, removed the cellophane cover, and put it over the coral chip. I bought the heatsink from mouser here and then fastened it to the board with the pushpins from mouser here. Note that the pushpins come in quantity 2, so you need to buy two sets. After the pushpins were through, I found it helpful to spread the anchoring barbs on their tips (look closely at pic of underside) with something small like a needle or a punch owl. This is helpful as the holes in the PCI-E adapter board are elongated and the pushpins can turn and come loose. Note also that I had to remove the two heatsink fins that were closest to the PCI holder bracket as the heatsink was bumping up against it and would not fit otherwise. So I just wiggled those two until they broke off. So far this is enough of a heatsink for me and is keeping temps around 40 deg C. If I get more heat I'll consider adding a fan.

20230629_181711 20230629_181555

adoucette avatar Jun 30 '23 00:06 adoucette

Hi,

I chose a slightly different path to mount a heatsink - I 3D printed a jig for a 40x40x11mm heatsink and drilled and tapped holes for M3 screwed and then used the back plate @ZCalilung posted above to clamp it from the back.

Some of the heatsink fins got bent drilling through the heatsink but small price to pay. I also had to grind a bit of the bracket because it hit the heatsink, probably would have been easier to just mount the headsink rotated by 90 degrees and bend a row of fins.

Working fine so far, no fan but temps are definitely lower.

Photo1 Photo2

jnicolson avatar Jul 02 '23 02:07 jnicolson