blog icon indicating copy to clipboard operation
blog copied to clipboard

Poking at the Tesla Model 3 MCU and a closer look at its eMMC

Open lewurm opened this issue 4 years ago • 38 comments

Previous related posts: #1 and #3.

As I mentioned in the last post I got some LEDs blinking. Before I present the findings, I should first outline some parts on the MCU board:

  • Ethernet switch: Marvell 88EA6321. Not sure what's connected to it. At least the CID and the ethernet port J15 of the MCU board (justification see below).
  • CID: Intel Atom E3950 with some RAM chips (4GB?).
  • eMMC: MTFC64GJVDN-4M with 64GB in a 169 BGA package connected to Intel SoC.
  • gateway: SPC5748GSMMJ6, a smaller chip with a PowerPC e200 core. It bridges the CAN interfaces to the ethernet world.
  • FTDI FT4232HQ: Establishes serial interface between Intel SoC and port J21 "FTDI USB DEBUG"

Enter the serial console to the CID

Connecting the FT4232HQ to a computer will yield four serial devices. It looks like only the first device is hooked up with the Intel SoC, the others seem disconnected. Why Tesla hasn't chosen a cheaper FT232R is beyond my understanding (maybe the other three are connected?!). Anyway, here's the first stuff I saw:

Screenshot 2019-08-10 at 18 04 59

WOOT!

And also: Amazingly I get a Google hit for it: https://pastebin.com/DbPAunKV Unfortunately I don't know the CID credentials. That would have been too easy, right? 😉

Wiring up the Ethernet port

I connected the ethernet port to a PC; the last time (see #1) I did that on my actual car, I wasn't aware of the apparently common network scheme used by Tesla vehicles:

CID:     192.168.90.100
IC:      192.168.90.101   # only Model S/X
Gatway:  192.168.90.102
AP:      192.168.90.103

Since I'm dealing with a Model 3 board, I set the IP of my ethernet adapter to 192.168.90.101. I was a bit surprised, something is happening:

$ sudo nmap -A 192.168.90.100
Starting Nmap 7.70 ( https://nmap.org ) at 2019-08-10 21:11 CEST
Nmap scan report for 192.168.90.100
Host is up (0.00024s latency).
Not shown: 998 filtered ports
PORT     STATE SERVICE VERSION
22/tcp   open  ssh     OpenSSH 7.5 (protocol 2.0)
| ssh-hostkey:
|   2048 e7:b5:43:77:d2:81:11:1f:db:fd:61:66:f4:0e:cf:06 (RSA)
|   256 33:f5:c6:85:a0:bc:bb:18:02:66:52:53:f7:20:aa:dc (ECDSA)
|_  256 ca:2e:2f:9f:6f:e6:a8:ee:bd:e7:66:97:b6:1a:58:9c (ED25519)
8080/tcp open  http    aiohttp 2.1.0 (Python 3.6)
|_http-server-header: Python/3.6 aiohttp/2.1.0
|_http-title: Site doesn't have a title (text/html).
MAC Address: A4:34:D9:01:02:03 (Intel Corporate)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running: Linux 3.X|4.X
OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4
OS details: Linux 3.10 - 4.11, Linux 3.2 - 4.9
Network Distance: 1 hop

TRACEROUTE
HOP RTT     ADDRESS
1   0.24 ms 192.168.90.100

Okay, there is an open SSH open which doesn't even prompt for a password. That's a dead end for now. BUT WAIT, WHAT IS HAPPENING ON 8080?!

Screenshot 2019-08-10 at 21 13 06

That is interesting. Unfortunately each item fails and doesn't do anything interesting. On the developer network tab I see this JSON response:

{"error": "You do not have command permission to perform this task"}

Still, that could be an attack vector, although it's probably hard even when having the server side Python program available. I wonder why Tesla left that port open. Maybe the MCU was put into some kind of reset after the car crash happened?

Anyway, here is the client side source code if anyone is interested: https://gist.github.com/lewurm/63fc5bd4e2591a677b298e2bc277d28a

Circling back to my actual car

With that gathered knowledge I went back. Remember this post #1? I have an ethernet port conveniently exposed on the passenger side. Again, I configured my ethernet adapter to 192.168.90.101/24, and here we go:

IMG_0914

The same test menu is available there too! (Software version 2019.24.4 73fb1ab)

WOAH. So now I really wonder what this is all about.

Obviously I tried clicking something; as you can see something harmless like "Calibrate Camera". Nothing happened: A look at the developer network tab gave me this::

{error: "Token 2.0 not found."}

That is weird. Why is the response different here to what we have seen on the salvaged MCU? The excited part in me want the following to be true: There was an vulnerability that has been fixed in newer software versions. (Meh, but why would they still leave that port open?!).

Here is a port scan (alas only as a picture 😴 ): Screenshot 2019-08-14 at 22 58 06

There is a slight difference: The car has one hostkey less. Wut?

No idea, moving on for now.

Poking at the gatway (unsuccessfully)

I remembered reading this instruction at https://github.com/Lunars/tesla/wiki/Gateway-Shell

$ printf "\x12\x01" |socat - udp:gateway:3500

# Once the port is open, Use netcat (or equivalent tool) to 
# create a connection to the newly opened telnet port. If 
# successful, you will be prompted with ?.

$ nc gateway 23
?

# Use the password 1q3e5t7u for authentication. All gateways 
# use the same static password. if successful, you will be 
# prompted with gw>.

Unfortunately this didn't work on the both MCUs. There was however some weird behavior: If I would do this socat command right after the reset of the MCU it would wait for some timeout. After the MCU was up, the socat command was immediately rejected; maybe some rules on the ethernet switch have been set up by this point? Does that make sense?

No idea, but it was worth a try.

Where do I go from here?

Initially I didn't want to dump the eMMC, but having this login prompt on the serial console only a /etc/shadow-crack away it is really intruding now.

One way to attempt an eMMC dump would be to figure out the pins (see datasheet too):

Screenshot 2019-08-14 at 18 16 38

There are eight data pins (D0 would be sufficient. Slower, but who cares for a one-time dump?). CMD, CLK and Vcc. Figuring out those pins is not exactly straight forward, and as I already said, I'm kind of a hardware noob.

At the same time I was following a thread in the German "TFF" forum about replacing the eMMC on the Model S/X MCU. As commonly known Tesla's software does not treat them nicely and they can wear out after some years leading to corrupted blocks. The problem as of today is that the service center won't replace just the eMMC but the whole MCU and that's expensive. Bottom line: People successfully attempted to swap the eMMC on their own.

So it cannot be that hard, right? After googling around I found a shop not far away from my place who are happily dealing with BGA soldering. I gave them a call and they were up for it; cool folks!

IMG_0926 IMG_0931

😬

My plan now:

  • Get a reader, around 80-100 bucks from AliExpress, and DUMP THAT PUPPY.
  • Get a replacement eMMC for my salvaged board. I was told that re-balling the BGA package is quite time consuming, so it's much simpler to use a fresh eMMC chip.

That said, there is still a slight change that the eMMC is encrypted, so better not getting too excited yet (although rumors say it's not). In the meanwhile I will practice my patience until the eMMC reader arrives from China.

Bonus: Pinmapping for eMMC storage

With the eMMC storage removed I was able to trace the pin layout. Maybe it's useful for someone out there.

front

back

lewurm avatar Aug 14 '19 21:08 lewurm

A fresh eMMC chip is the way to go. Excited to see updates.

PuppyLover101 avatar Aug 14 '19 23:08 PuppyLover101

reballing emmc chip is easy-peasy if you have the right tools. unfortunately I am not aware ofthe right tools for this particular size.

For different sizes - this is great: https://www.aliexpress.com/item/32848783273.html?spm=2114.12010608.0.0.60c223f29ykGr5

verygreen avatar Aug 15 '19 01:08 verygreen

Also, now do the MCU2 ;)

verygreen avatar Aug 15 '19 02:08 verygreen

Interesting, waiting for your update

mikealanni avatar Aug 15 '19 05:08 mikealanni

If you can find the Intel SOC reset pin then you will be able to do in-circuit access.

pk-mdt avatar Aug 15 '19 15:08 pk-mdt

Nice excited here :)

haffi78 avatar Oct 16 '19 00:10 haffi78

Wow! exciting! great work and looking forward to the next bits!

Have you already extracted the eMMC dump?

deppymon avatar Nov 13 '19 14:11 deppymon

Have you already extracted the eMMC dump?

No. There is a longer answer to it, I should flush my brain into another post... 😅

lewurm avatar Nov 18 '19 12:11 lewurm

Have you already extracted the eMMC dump?

No. There is a longer answer to it, I should flush my brain into another post... 😅

Please do!

davidh- avatar Nov 19 '19 09:11 davidh-

If you can find the Intel SOC reset pin then you will be able to do in-circuit access.

Did you manage to trigger it in-circuit?

bequa avatar Dec 05 '19 09:12 bequa

Apparently there is an easier way to access the ethernet port: https://teslaownersonline.com/threads/ethernet-port-in-driver-footwell.15045/

lewurm avatar Jan 13 '20 12:01 lewurm

Did anyone connect the hw3.0 mcu to the display yet? Thanks.

olealgoritme avatar Jun 24 '20 16:06 olealgoritme

Did anyone connect the hw3.0 mcu to the display yet? Thanks.

yes

davidh- avatar Jul 27 '20 22:07 davidh-

I've just plugged my laptop into the ethernet port on the cid/ice, and set my network adapter's IP with sudo ipconfig set en21 INFORM 192.168.90.101. Running arp -na gives me back nothing, opening 192.168.90.101:8080 in Chrome doesn't connect, and sudo nmap -A 192.168.90.100 Says 0 IPs (same story with -Pn). I'm running 2020.32.3. Does anyone know if this has been patched or if I've missed a step or something?

Un3qual avatar Sep 02 '20 06:09 Un3qual

I'm running 2020.32.5 Model X MCU2 - all ok! And Service Mode available... IMG_8435

Igor251263 avatar Sep 08 '20 14:09 Igor251263

I'm running 2020.32.5 Model X MCU2 - all ok! And Service Mode available... IMG_8435

How were you able to unlock the diag port under the MCU2? Good to see curl commands still work on newer FW versions.

dockramer avatar Sep 14 '20 00:09 dockramer

unfortunately only port 8080 is open

Igor251263 avatar Sep 14 '20 07:09 Igor251263

unfortunately only port 8080 is open

Can enter factory mode ?

zrachwal84 avatar Sep 30 '20 13:09 zrachwal84

Hi,could you please tell me where is the vccq in PCB?

lemuellew avatar Jan 12 '21 09:01 lemuellew

After some work I found out the CMD, VCC (3.3V) and VCCQ (1.8V) on the same side of the rest of the other useful pins (with D0 & CLK).

Some more poking needed if you want more that 1 bit r/w. As D3 is still to be found on that side.

image

jeankalud avatar Jan 20 '21 14:01 jeankalud

Hi @lemuellew,

it is my understanding that the SPI flash is also encrypted as of 2020.40, but I need to confirm this with another source. If you plan to change the configuration of the car, things are getting more and more complicated, specially for the Intel MCU. The nVidia MCU is still accessible, but not for long: Tesla is quite determined to close the holes.

Good luck in your endeavours.

AlexVaq avatar Jan 27 '21 18:01 AlexVaq

Thank for your reply.

I tried to find where the SPI flash on PCB, and I found the chip(RW229) on the right of eMMC. I know it is MT25QU128ABA8E12-0AAT via micron FBGA decoder.

image

Because MT25QU128ABA8E12-0AAT matches MT25QXXXXXXX8E12-XXXX, I found the pinout of SPI flash in datasheet. According to the data pins, you can use flashrom or other flash programmers to read the SPI Flash. Could you please tell me where are the data pins on pcb? @jeankalud @lewurm

image

what do you want to do in this way?

stiffler-m avatar Jan 28 '21 08:01 stiffler-m

Hi @lemuellew,

it is my understanding that the SPI flash is also encrypted as of 2020.40, but I need to confirm this with another source. If you plan to change the configuration of the car, things are getting more and more complicated, specially for the Intel MCU. The nVidia MCU is still accessible, but not for long: Tesla is quite determined to close the holes.

Good luck in your endeavours.

starting from 2020.36, all firmware is encrypted, including tegra. starting from 2020.48, gateway has received additional "internal.dat" protection. there are still a lot of loopholes left. and new ones appear).

stiffler-m avatar Jan 28 '21 08:01 stiffler-m

Tegra never crypt anything, daont have enough power to make that.

zrachwal84 avatar Jan 28 '21 08:01 zrachwal84

There is a cryptographic lock in some entries of the gw configuration, even in tegra. Rooting with full access to the gw is possible up to 2020.36 on tegra. From that on you might no be able to modify all the options in the gw.

The filesystem in the eMMC is not encrypted in tegra, as far as I know.

El ene. 28, 2021, a las 01:56, zrachwal84 [email protected] escribió:

 Tegra never crypt anything, daont have enough power to make that.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

AlexVaq avatar Jan 28 '21 09:01 AlexVaq

I think you should be able to do that. I don't know where in the board you need to pit your probes.

Maybe you could find an update and extract from there the gw firmware. I'm feeling that would be easier.

El ene. 28, 2021, a las 02:24, Lemuel Lew [email protected] escribió:

 Thank for your reply. I tried to find where the SPI flash on PCB, and I found the chip(RW229) on the right of eMMC. I know it is MT25QU128ABA8E12-0AAT via micron FBGA decoder.

Because MT25QU128ABA8E12-0AAT matches MT25QXXXXXXX8E12-XXXX, I found the pinout of SPI flash in datasheet. According to the data pins, you can use flashrom or other flash programmers to read the SPI Flash. Could you please tell me where are the data pins on pcb? @jeankalud @lewurm

what do you want to do in this way?

Hi, I want to read the firmware of SPI Flash.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

AlexVaq avatar Jan 28 '21 16:01 AlexVaq

@lewurm do you remember whether you were able to make outbound requests to the internet via LTE modem while connected to the car's ethernet port, and properly configured the local gateway and subnet?

joeyx22lm avatar Mar 18 '21 05:03 joeyx22lm

I'm running 2020.32.5 Model X MCU2 - all ok! And Service Mode available... IMG_8435

How to open service mode?

7752669 avatar Oct 09 '21 10:10 7752669

@7752669 enter "service" into the access code box at a service center (or anywhere in europe)

AnalyticETH avatar Oct 10 '21 15:10 AnalyticETH

Service not working in my Tesla m3 21y

7752669 avatar Oct 10 '21 16:10 7752669