SdFat-beta

SdFat-beta copied to clipboard

Hi Mr. Greiman, thank you for your efforts, I appreciate you. I am working on a wrist watch project that will use an ESP32 (for the Bluetooth and Wifi capabilities and 2 processors), a TFT display, and SD card (to hold the video images), and a VS1053 audio chip with a second SD card holding the audio. Some questions:

Will SdFat-beta support ESP32? I read a prior post that you plan no support for ESP32.

Will SdFat-beta support multiple cards? I read how to do it on other posts, and the solutions ask to edit source and header files.

The ESP32 has multiple SPI interfaces, will SdFat-beta support multiple SPI configurations? I'm not sure if changing the SPI.begin() to identify the MOSI/MISO/CLK pins is the best practice.

I would be glad to support/contribute/test SdFat-beta for ESP32 if that interests you.

-Frank

There is a custom driver in SdFat for ESP processors but it only supports the library standard SPI port. The driver is old and was written by a user. I will update it if we can get the following to work.

You could try the following in the release version of SdFat or SdFat-beta. Edit SdFatConfig.h and set SPI_DRIVER_SELECT to one like this:

/**
 * If the symbol SPI_DRIVER_SELECT is:
 *
 * 0 - An optimized custom SPI driver is used if it exists
 *     else the standard library driver is used.
 *
 * 1 - The standard library driver is always used.
 *
 * 2 - An external SPI driver of SoftSpiDriver template class is always used.
 *
 * 3 - An external SPI driver derived from SdSpiBaseClass is always used.
 */
#define SPI_DRIVER_SELECT 1  // Was 0 <<======== 

choose the correct pin numbers to match these GPIO numbers.

SPI | MOSI | MISO | CLK | CS
VSPI | GPIO 23 | GPIO 19 | GPIO 18 | GPIO 5
HSPI | GPIO 13 | GPIO 12 | GPIO 14 | GPIO 15

Then try this to use HSPI in place of the standard VSPI.

#include "SdFat.h"

SdFat sd;

const uint8_t SD_CS_PIN = 5; // replace with correct pin for the SD card on HSPI.
#define SPI_CLOCK SD_SCK_MHZ(50)  // replace with reasonable speed.
SPIClass SPI1(HSPI);
#define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SPI_CLOCK, &SPI1)


void setup() {
  Serial.begin(9600);
  if (!sd.begin(SD_CONFIG)) {
    sd.initErrorHalt(&Serial);
  }
  Serial.println("ls:");
  sd.ls();
}

void loop() {
}

I found an ESP32 DEV module and ran this program:

#include "SdFat.h"

SdFat sd;

const uint8_t SD_CS_PIN = 15; // replace with correct pin for the SD card on HSPI.
#define SPI_CLOCK SD_SCK_MHZ(20)  // replace with reasonable speed.
SPIClass SPI1(HSPI);
#define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SPI_CLOCK, &SPI1)


void setup() {
  Serial.begin(9600);
  while(!Serial) {}
  Serial.println("Type any character");
  while(!Serial.available()){}
  if (!sd.begin(SD_CONFIG)) {
    sd.initErrorHalt(&Serial);
  }
  Serial.println("ls:");
  sd.ls();
}

I used the above pins for HSPI.

I got it to work but only after I loaded/started the program without the SD in the socket. Is HSPI used for something else?

Here is the output.

Type any character
ls:
TestESP32.txt

Thanks, Bill. I'm glad to set-up the test examples. To be clear, you want me to test where one sd card is on VSPI and a second card is on HSPI? -Frank

You could test with two cards I don't think two cards will be a problem, I do that on many systems. with a sd1 and sd2.

My worry is HSPI seems flaky. I modified the custom driver so it is multi port. I ran the bench example on the two ports and got fair performance.

The SPI driver seem to fail for high SPI rates. A standard SPI driver should set the rate to the highest rate supported by the chip that is less than or equal to the value in SpiSettings. All Arduino boards work this way.

Here are the results from bench with the custom driver at 20 MHz.

I defined SPI1 like this in bench:

#define SPI_CLOCK SD_SCK_MHZ(20)
SPIClass SPI1(HSPI);
#define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SPI_CLOCK, &SPI1)

Here are the results:

VSPI port as SPI

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1913.51,188574,227,266
2212.39,5454,227,230

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
824.67,798,480,619
825.08,797,480,619

-------------------------
HSPI port as SPI1

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2194.91,9682,227,232
2207.51,5374,227,231

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
825.08,795,479,619
825.22,797,480,619

Here is the modified ESP driver. You could replace the contents of SdFat/src/SpiDriver/SdSpiESP.cpp to test the new driver. SPI_DRIVER_SELECT must be zero in SdFatConfig.h to use this driver.

/**
 * Copyright (c) 2011-2020 Bill Greiman
 * This file is part of the SdFat library for SD memory cards.
 *
 * MIT License
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included
 * in all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
 * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 */

#include "SdSpiDriver.h"
#if defined(SD_USE_CUSTOM_SPI) && (defined(ESP8266) || defined(ESP32))
#define ESP_UNALIGN_OK 1
//------------------------------------------------------------------------------
void SdSpiArduinoDriver::activate() {
  m_spi->beginTransaction(m_spiSettings);
}
//------------------------------------------------------------------------------
void SdSpiArduinoDriver::begin(SdSpiConfig spiConfig) {
  if (spiConfig.spiPort) {
    m_spi = spiConfig.spiPort;
#if defined(SDCARD_SPI) && defined(SDCARD_SS_PIN)
  } else if (spiConfig.csPin == SDCARD_SS_PIN) {
    m_spi = &SDCARD_SPI;
#endif  // defined(SDCARD_SPI) && defined(SDCARD_SS_PIN)
  } else {
    m_spi = &SPI;
  }
  m_spi->begin();
}
//------------------------------------------------------------------------------
void SdSpiArduinoDriver::deactivate() {
  m_spi->endTransaction();
}
//------------------------------------------------------------------------------
uint8_t SdSpiArduinoDriver::receive() {
  return m_spi->transfer(0XFF);
}
//------------------------------------------------------------------------------
uint8_t SdSpiArduinoDriver::receive(uint8_t* buf, size_t count) {
#if defined(ESP_UNALIGN_OK) && ESP_UNALIGN_OK
  m_spi->transferBytes(nullptr, buf, count);
#else  // ESP_UNALIGN_OK
  // Adjust to 32-bit alignment.
  while ((reinterpret_cast<uintptr_t>(buf) & 0X3) && count) {
    *buf++ = m_spi->transfer(0xff);
    count--;
  }
  // Do multiple of four byte transfers.
  size_t n4 = 4*(count/4);
  if (n4) {
    m_spi->transferBytes(nullptr, buf, n4);
  }
  // Transfer up to three remaining bytes.
  for (buf += n4, count -= n4; count; count--) {
    *buf++ = m_spi->transfer(0xff);
  }
#endif  // ESP_UNALIGN_OK
  return 0;
}
//------------------------------------------------------------------------------
void SdSpiArduinoDriver::send(uint8_t data) {
  m_spi->transfer(data);
}
//------------------------------------------------------------------------------
void SdSpiArduinoDriver::send(const uint8_t* buf , size_t count) {
#if defined(ESP_UNALIGN_OK) && !ESP_UNALIGN_OK
  // Adjust to 32-bit alignment.
  while ((reinterpret_cast<uintptr_t>(buf) & 0X3) && count) {
    SPI.transfer(*buf++);
    count--;
  }
#endif  // #if ESP_UNALIGN_OK
 m_spi->transferBytes(const_cast<uint8_t*>(buf), nullptr, count);
}
#endif  // defined(SD_USE_CUSTOM_SPI) && defined(ESP8266)

I edited the driver above, it was the wrong version.

I found out why HSPI is flaky with an SD card.

The device on the natural GPIO pins of the ESP32 must not send or receive any data during program load. GPIO13, which is used as the HSPI MOSI is also used as the JTAG TCK pin.

The SD card looks for commands on MOSI so it probably goes active during program load on the ESP32. Maybe a pullup on the SD CS pin would help.

I have seen this on other boards. In SPI mode SD command are not CRC checked. I had a board that some downloads would erase the SD or corrupt the file system. Using HSPI for an SD looks like a bad idea.

To avoid any conflict you have to change your ESP32 program: #define SD_FAT_TYPE 3//0 in bench add SPIClass SD_SPI(VSPI); or SPIClass SD_SPI(HSPI); #define SPI_CLOCK SD_SCK_MHZ(25) // Try to select the best SD card configuration. //#if HAS_SDIO_CLASS //#define SD_CONFIG SdioConfig(FIFO_SDIO) //#elif ENABLE_DEDICATED_SPI //#define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SPI_CLOCK) #define SD_CONFIG SdSpiConfig(SD_CS_PIN, DEDICATED_SPI, SPI_CLOCK, &SD_SPI) //#else // HAS_SDIO_CLASS //#define SD_CONFIG SdSpiConfig(SD_CS_PIN, SHARED_SPI, SPI_CLOCK) //#endif // HAS_SDIO_CLASS File file_;//to avoid conflict with FS and SPIFFSS file in main program make sure line 124 in sdfatconfig.h is #define SPI_DRIVER_SELECT 0//must be 0 for ESP32 Change line 44 in the SdSpiESP.cpp for custom VSPI/HSPI pin assignment // m_spi->begin(); I don't know why this line never executed so I did it manually in the setup() m_spi->begin(SDCARD_SCK_PIN,SDCARD_MISO_PIN,SDCARD_MOSI_PIN,SDCARD_SS_PIN); in setup before start add this line: SdFat-beta_ESP32.zip

SD_SPI.begin(SDCARD_SCK_PIN,SDCARD_MISO_PIN,SDCARD_MOSI_PIN,SDCARD_SS_PIN); After searching in https://github.com/greiman/SdFat-beta I couldn't find where defined SS

Hi, I could use some help with a similar issue.

I tried the changes proposed by @greiman on Jan21, but they didn't cut it.

My issue is the following:

I am currently working with the Lilygo T3S3. It has booth LoRa and a SD interface. LINK

The manufactor uses SD library on his example code. I gather it is because SD allows for a call like: "SD.begin(CS_PIN, SPI_CLASS)". See HERE

For compatibility issues with other sections of my code, I have to use SdFat lib, but SdFat does not allow this sort of calling with the SPI_class as second parameter. I wonder why @greiman suggestion didn't work for me.

Current code is variant of StuartsProjects example. LoRa works, but SD doesn't. REFFERENCE

---

`/******************************************************************************************************* Details of the packet identifiers, header and data lengths and formats used are in the file; 'Data transfer packet definitions.md' in the \SX128X_examples\DataTransfer\ folder.

Max segment size (defined by DTSegmentSize) is 245 bytes for LoRa. *******************************************************************************************************/

#define USELORA //enable this define to use LoRa packets

#include <SPI.h>

#include <SX128XLT.h> #include <ProgramLT_Definitions.h> #include "DTSettings.h" //LoRa settings etc.

SX128XLT LoRa; //create an SX128XLT library instance called LoRa, required by SDtransfer.h

#define ENABLEMONITOR //enable monitor prints #define PRINTSEGMENTNUM //enable this define to print segment numbers #define ENABLEFILECRC //enable this define to uses and show file CRCs //#define DISABLEPAYLOADCRC //enable this define if you want to disable payload CRC checking //#define DEBUG //see additional debug info

//#define SDLIB //define SDLIB for SD.h or SDFATLIB for SDfat.h #define SDFATLIB

#include <DTSDlibrary.h> //library of SD functions #include <SDtransfer.h> //library of data transfer functions

//choice of files to send char FileName[] = "/$50SATS.JPG"; //file length 6880 bytes, file CRC 0x0281

SPIClass SDSPI(HSPI);

#define SD_CONFIG SdSpiConfig(SDCS, DEDICATED_SPI, SDSCLK, &SDSPI)

void setup() {

//SDsetLED(LED1); //setup LED pin for data transfer indicator

#ifdef ENABLEMONITOR

Monitorport.begin(115200);

long time = millis();

while (!Serial && ( millis() < time + 10000) );

Monitorport.println();
Monitorport.println(F(__FILE__));
Monitorport.flush();

#endif

SPI.begin(LoRa_SCLK, LoRa_MISO, LoRa_MOSI, NSS);

if (LoRa.begin(NSS, NRESET, RFBUSY, DIO1, LORA_DEVICE)) {

#ifdef ENABLEMONITOR

    Monitorport.println(F("LoRa device OK"));

#endif

} else {

#ifdef ENABLEMONITOR

    Monitorport.println(F("LoRa device error"));

#endif

while (1)
{

}

}

#ifdef USELORA

LoRa.setupLoRa(Frequency, Offset, SpreadingFactor, Bandwidth, CodeRate);
Serial.println(F("Using LoRa packets"));

#endif

#ifdef ENABLEMONITOR

Monitorport.println();
Monitorport.print(F("Initializing SD card..."));

#endif

pinMode(SDMISO, INPUT_PULLUP);

SDSPI.begin(SDSCLK, SDMISO, SDMOSI, SDCS);

if (SD.begin(SD_CONFIG)) {

#ifdef ENABLEMONITOR

  Monitorport.println(F("SD Card initialized."));

#endif

} else {

Monitorport.println(F("SD Card failed, or not present."));

while (1)
{

}

}

#ifdef ENABLEMONITOR

Monitorport.println();

#endif

#ifdef DISABLEPAYLOADCRC

LoRa.setReliableConfig(NoReliableCRC);

#endif

if (LoRa.getReliableConfig(NoReliableCRC)) {

#ifdef ENABLEMONITOR

  Monitorport.println(F("Payload CRC disabled"));

#endif

} else {

#ifdef ENABLEMONITOR

  Monitorport.println(F("Payload CRC enabled"));
    
#endif

}

SDDTFileTransferComplete = false;

#ifdef ENABLEMONITOR

Monitorport.println(F("SDfile transfer ready"));
Monitorport.println();

#endif

}

void loop() {

uint32_t filelength;

#ifdef ENABLEMONITOR

Monitorport.println(F("Transfer started"));

#endif

filelength = SDsendFile(FileName, sizeof(FileName));

if (filelength) {

#ifdef ENABLEMONITOR

    Monitorport.println(F("Transfer finished"));

#endif

} else {

#ifdef ENABLEMONITOR

    Monitorport.println(F("Transfer failed"));
    Monitorport.println();

#endif

}

delay(15000);

} `

User calls to SPI begin like this:

SDSPI.begin(SDSCLK, SDMISO, SDMOSI, SDCS);

only work in SdFat-beta. See this example.

Install SdFat-beta and modify this line:

#define SD_CONFIG SdSpiConfig(SDCS, DEDICATED_SPI, SDSCLK, &SDSPI)

to:

#define SD_CONFIG SdSpiConfig(SDCS, USER_SPI_BEGIN | DEDICATED_SPI, SDSCLK, &SDSPI)

This will prevent SdFat from calling SPI begin and overriding your options.

Hi, @greiman .

Sorry. It didn't work. Followed your instructions precisely, but wasn't able to mount the card. The only difference I noticed from my code and the example you mentioned is that he did not use HSPI. What could I have done wrong?

If I adopt the usual SPI call (without using the "config" constructor), it works. So I also tried using HSPI for the LoRa instead, but wasn't succesfull.

So far, the only way I was able to get both LoRa and SD working, was using the SD lib, that allows for a different call, but as I mentioned, I need SDFat in order to also implement USB MSC.

I have seen problems with HSPI and SdFat. I don't know why. it may be the speed needs to be lower.

Edit:

Here is an example where the SD library seems limited to 16MHz.

I believe that pin reassignment causes the GPIO matrix to make the connection and lower SPI speeds are requires.

Hi,

I can't believe it...It really worked with lower frequency.

Thank you so, much, @greiman.

Excelent work with the library, by the way.