Add proposal for SPI

[soundanalogous]

via conversation with @ajfisher on gitter:

There was a pull-request for SPI opened a while back: firmata/arduino#135. However, it's going to be more complex than just wrapping the Arduino SPI api (since that would be inefficient). Lets start by drafting a protocol here: https://github.com/firmata/protocol.

My initial thoughts are that it should have a SPI_CONFIG message (like I2C and many other features in Firmata). The config message should let the user specify the following:

CS/SS pin (this is also how you'll keep track of multiple SPI devices) bit order (LSB or MSB first) data mode (see the 4 modes defined here) clock divider? (I'm not sure how this would work in Firmata or if it even applies... I think it depends on if a hardware enabled SS pin is selected on the board) We'll need a way to call SPI.begin and SPI.end. I'm not sure if it's better to call begin when the config message is received or if there is any advantage in having it be a separate command. SPI_END can be a simple message to call SPI.end, perhaps also specifying the CS pin to be compliant with Arduino Due's extended SPI protocol (for other boards this parameter would simply be ignored in the implementation).

We'll need a SPI_REQUEST message. It should contain the CS/SS pin number and the data to be sent via SPI.transfer(val) on the board. It will be more efficient to send multiple bytes rather than a single byte (this likely also depends on the particular device) - so in the implementation you'd call SPI.transfer(val) multiple times for a single SPI_REQUEST rather than sending a separate Request for each byte). We'll need a strategy around buffering as well (more for the implementation than the protocol). It will be helpful to look across a variety of SPI devices to see what what type of data is sent... what would constitute a "packet". Be aware however of the 64 byte serial data buffer on Arduino since this will likely come into play. It would also be helpful to support a READ_CONTINUOUSLY mode like I2C Firmata (this is especially useful for SPI-based sensors).

And we need a SPI_REPLY message to send data back the the Firmata client. This should include the CS pin, maybe the number of bytes in the reply and whether or not this is the full packet (or if it's broken up into multiple packets). Perhaps this is as simple as indicating if this is the LAST set of data.

Also try to support Arduino Due's extended SPI feature as an option. You'd add this to the protocol for SPI_END and SPI_REQUEST (although I'm not sure how beneficial this extended set would be in a Firmata client library... may add more complexity than it's worth). This is lower priority.

[soundanalogous]

The trick with SPI is the configuration parameters for each SPI device may differ. Therefore a single SPI_CONFIG message would not be sufficient if parameters are passed via CONFIG. Some options:

1. SPI_BEGIN for initialization of SPI hardware (only needs to be called once regardless of number of SPI devices).
2. SPI_CONFIG to be called for each device, passing along any required parameters: csPin, bitOrder, dataMode, clockDivider. The idea is to call CONFIG from the constructor in the client implementation. Then in SPI_REQUEST, pass the csPin and in the firmware implementation look to see if it has changed and if so, update any parameters with those passed with SPI_CONFIG.
3. Instead of SPI_CONFIG for each device, SPI_SELECT with the following parameters: csPin, bitOrder, dataMode, clockDivider. Would need to be called each time you want to transfer data to a different device. In this case SPI_REQUEST would not need to update configuration parameters if a different device was selected for data transfer.

[PaulStoffregen]

I haven't been active in Firmata development for a while, but I certainly have with SPI. :)

Arduino 1.6.0 was (finally) released 1 week ago, featuring a new SPI library with transaction support (contributed by me), and improved hardware-independent configuration (contributed by Matthijs Kooijman).

I know Firmata has historically supported older versions of Arduino. But for supporting SPI, I'd highly recommend using the new SPISettings() feature with SPI.beginTransaction(). If the settings will be delivered over the Firmata protocol, you can use them once to store a SPISettings object, so the overhead of converting the settings to SPISettings's efficient internal storage only needs to be done once.

This new SPISettings approach is much more efficient, and it's hardware neutral. There's no need to ever write code dealing with hardware dividers or clock speeds. You just give it the actual SPI settings, with clock speed as a plain 32 bit integer, and the SPISettings feature in the library automatically turns those into the hardware's closest possible configuration.

Arduino hasn't documented this stuff yet on their website. Hopefully they will soon. Today, the only docs are the code+comments, and this page on my site:

http://www.pjrc.com/teensy/td_libs_SPI.html

[soundanalogous]

It's great to see you chiming in here Paul! Your expertise is appreciated.

Thanks for the tip, that actually makes thing much easier for the Arduino implementation. I'm trying to keep the Firmata protocol documentation (this particular repo) from being Arduino-specific in case some day platforms other than Arduino adopt the Firmata protocol. However I think the Arduino SPI api is still generic enough to work on other platforms, especially now that the SPISettings allows specifying a general clockSpeed rather than a AVR-specific clockDivider parameter.

[soundanalogous]

Here's an initial pass at a generic Firmata SPI protocol.

For Arduino, SPI support would be for Arduino 1.6 and higher.

// SPI_BEGIN
// called once to initialize SPI hardware
// can optionally be set internally on first SPI_CONFIG_DEVICE message
0:  START_SYSEX
1:  SPI_DATA
2:  SPI_BEGIN
3:  END_SYSEX

// SPI_CONFIG_DEVICE
// sent once for each SPI device
0:  START_SYSEX 
1:  SPI_DATA
2:  SPI_CONFIG_DEVICE
3:  csPin (0-127)  // use csPin to identify individual devices
4:  bitOrder (bit 0)  | dataMode (bits 1-6) // also need to track csPin HIGH | LOW ?
5:  clockSpeed (bits 0 - 6)
6:  clockSpeed (bits 7 - 14)
7:  clockSpeed (bits 15 - 21)
8:  clockSpeed (bits 22 - 28)
9:  clockSpeed (bits 29 - 32)
10:  END_SYSEX

// implementation details:
// Upon receiving SPI_CONFIG_DEVICE, If SPI_BEGIN has not yet been called
// call it once internally
// for Arduino: store a SPISettings for each device (determined by csPin number)

// SPI_TRANSFER_REQUEST
// send a byte array of data to transfer to the SPI slave device
0:  START_SYSEX
1:  SPI_DATA
2:  SPI_TRANSFER_REQUEST
3:  csPin (0 - 127)
4:  numBytes (limit TBD) // needed because we'll reuse a single array for all transfers
5:  data 0 (LSB)
6:  data 1 (MSB)
... up to numBytes * 2
N: END_SYSEX

// implementation details:
// Track csPin and if the csPin on an incoming request differs from the last request,
// set the stored configuration parameters (if they differ from the last device) for the 
// device associated with the request.
// for Arduino: pass the stored SPISettings for this csPin to SPI.BeginTransaction, 
// then call digitalWrite on the last and current device to toggle the CS pin for each

// SPI_TRANSFER_REPLY
// a byte array of data received from the SPI slave device in response to the transfer
0:  START_SYSEX
1:  SPI_DATA
2:  SPI_TRANSFER_REPLY
3:  csPin (0 - 127)
4:  numBytes (limit TBD)
5:  data 0 (LSB)
6:  data 1 (MSB)
... up to numBytes * 2
N: END_SYSEX

// implementation details:
// Ensure that numBytes in reply is equal to numBytes in request.
// Zero pad array before sending reply.

// SPI_END
// called once to release SPI hardware
// send before quitting a Firmata client application
0:  START_SYSEX
1:  SPI_DATA
2:  SPI_END
3:  END_SYSEX

// implementation details
// for Arduino: call SPI.endTransaction()

[PaulStoffregen]

One thing I learned last year, while working on the Arduino SPI library, is the tremendous number of different ways SPI actually gets used. I really do not believe it's possible to design a Firmata protocol that will accommodate absolutely all SPI uses. The question is probably how large a fraction of uses should it cover?

For example, many of the SPI interface displays take 2 control signals, a chip select and an other CS-like signal which functions as an address bit, for which register within the chip will be accessed.

Some devices require an extra dummy byte after chip select is de-asserted. Some require chip select only on the first or last byte. There are lots of special cases.

A complex protocol that could cover a lot of cases might involve sending several extra bits with each data byte. Four or 5 bits could indicate which digital pins to assert for chip select, during each byte. Perhaps 1 bit could indicate if the received byte during that transfer needs to be returned, or simply discarded.

Or, you could implement the paradigm where a single chip select goes low before the first transfer, and returns high after the last byte. That certainly is enough for a lot of SPI chips. But there are also a good number of special cases, so at least know that other stuff exists.

[soundanalogous]

Paul do you remember any specific SPI devices you came across that exhibit some of the edge cases you encountered? I'd like to look over their data sheets.

I don't plan to support all possible cases. SPI devices requiring continuous high speed data transfer (like a mp3 decoder) would not be possible with Firmata and I'm fine with that. I think most users will want to use SPI for various sensors and perhaps some actuators. Those are the majority of requests for SPI support I've had so far are for sensors that use SPI and I usually point out the fact that the same sensors typically have an I2C interface which is already supported by Firmata.

The option of using the extra 6 bits from every other byte is interesting... I had not thought of that approach before. It would require a few extra cycles on the microcontroller to process but it would add a few options to expand the range of supported devices for sure so it may be worth it if that increases overall support by a significant factor. I'll look into this further.

My current thought is to return a byte for every byte transferred. If the transfer doesn't return data, I'd just send a value of 0 for that byte. Then it would be the responsibility of the Firmata client to properly parse the response data according to the particular SPI device in use. However, I may vary well be overlooking something here. It's been a while since I've done any extensive work with SPI.

I could also keep the CS pin out of the definition and require the user to manage that separately (along with any additional CS-like signal as you mentioned) but there is something convenient of supplying those pins in the initial CONFIG message rather than sending separate messages to toggle CS and transfer data.

[soundanalogous]

If a SPI device does not need to return data (such as an LCD screen or other write-only device) then my assumption about always returning data would be a waste of bandwidth. In that case, using one of the extra 6 bits to indicate whether or not to return a byte could work, or I could alternatively use a bit in the command byte of the SPI_TRANSFER_REQUEST message:

0: START_SYSEX
1: SPI_DATA
2: SPI_TRANSFER_REQUEST
    bits 0-2 = command
    if bit 6 is set, don't send a response because transfer does not return data for this device
    since this is sent by the Firmata client, it expects not to receive a response for this message
...

This would work if no bytes in the message (assuming a multi-byte message) return data. If at least 1 transferred byte in the message returns data, then an padded byte for each message would be returned since a response would be needed anyway.

[madeintheusb]

https://github.com/firmata/protocol/issues/26

For my USB device Nusbio, I implemented part of i2c and spi protocol.

Here some interresting chips, I tested

Classic SPI

MCP3008 - 8 ADC 10 bit (a good case to test regular SPI)
    - PINS Clock, Mosi, Miso, CS (SELECT)

MCP4231 - 2 digital potentiomenters
    - PINS Clock, Mosi, Miso, CS (SELECT)

8x8 LED Matrix, Seven Segment Display based MAX7219.
    - PINS Clock, Mosi, CS (SELECT)
    - No Miso, but support chaining up to 8 MAX7219. Because MAX7219 is a SPI shift register,
    the behavior of SELECT/UNSELECT act as a commit transacton. Good test case for supporting
    different SPI in chained shift register mode.

Not Classic SPI

Oled 128x64 Chip SH1106, SSD1306 (with this one there is  no Miso, but we need a DC and RESET pin)
    - PINS Clock, Mosi, CS (SELECT), DC (Data/Command), RESET
    - RESET pin is part of SPI, but not always used

APA 102 LED Strip - Not stricly speaking SPI, but fun. APA 102 should support a higher refresh rate than WS2812 and should work with SPI.
    - PINS Clock, Mosi

MCP4131 - Digital potetiometer - With this one MISO and MOSI are combined on the same pin. I could not get it to work with the regular arduino SPI library
- PINS Clock, Mosi/Miso, CS (SELECT)

[soundanalogous]

The latest iteration of the SPI proposal is here: https://github.com/firmata/protocol/pull/27

[raybuay]

but is there a firmata vertion with at least basic master to slave SPI support? what is the fastest way to be able to use digipots with Pure Data or Max/Msp? both digipots and digital to analog converters need the SPI protocol to operate and it is still not implemented in firmata!!!

i tried using RC low pass filters for DAC conversion but the quality is terrible. are there digipots that could operate with PWM instead of the SPI protocol? or a DAC chip that will convert PWM to an analog voltage without the SPI protocol? i found myself trapped with this problem.

when will firmata have SPI support?

regards r.y

[soundanalogous]

I need someone much more experienced with SPI than myself to contribute to lead this effort as I am not as familiar with all of the edge cases that Paul pointed out. So far no one has taken this on and any existing SPI implementations for Firmata are very basic and only cover one or two common cases. I don't want to lock into an implementation that will not scale well.

[raybuay]

but is there implementation for digipots like the AD5206 or the MCP41100?

[soundanalogous]

Not for Firmata

[soundanalogous]

Well it may work with some of the Firmata forks that add SPI. I haven't tried any of them. If you find one that works, encourage the author to submit a PR to add SPI support.

[madeintheusb]

Ray, Our device Nusbio may help you right away

• http://madeintheusb.net/TutorialSpi/Index
• http://madeintheusb.net/TutorialAnalogConverter/Index

[raybuay]

hello

the problem i have is that without at least some basic implementation of spi in firmata i cannot set up and control the digipots or DACs from WITHIN PD. firmata needs to receive the messages to enable the devices from the host program and then receive the 8 bit controller data to the available registers.

r.y

[soundanalogous]

Try ExtendedFirmata, it includes SPI support.

[raybuay]

tried to load extended firmata but i gives me errors i upgraded the arduino ide to 1.6.9 but it doesn't fix it

errors like serial1, 2, 3 are not declare on this scope and some conflict of pin modes redefinitions

how can i solve this thanks r.y

[soundanalogous]

I see now that ExtendedFirmata is only compatible with an Arduino Mega or Due.

[raybuay]

ok, i tried in a due and now i have a: fatal error SoftwareSerial

[raybuay]

i took the: #include SoftwareSerial.h out of the code and it did go through. i don't know if this will affect functionality in any way?

and where is the information need it to operate the spi functionality from the host program?

thanks so much for the help

regards r.y

[soundanalogous]

That's right... forgot Due doesn't support SoftwareSerial. It will only work with a Mega then.

[soundanalogous]

You will have to add SPI functionality to the host (sounds like PD in your case) unless you are using golang.

[raybuay]

in pd, you send a pinout definition message to enable the boards pins example, input,output,pwm,servo, spi??? . . . and then sending or receiving data based on this initial configuration

i don't know where are the instruction for driving a digipot from pd because the help files all have all the old implementation what is golang?

r.y

[soundanalogous]

You will likely need to create a new object for PD to implement the Firmata SPI interface as defined in ExtendedFirmata. I'm not a PD user so I can't help you there.

[raybuay]

thanks 4 all d help soundanalogous. the thing is that i cannot find anywhere the commands i need to enable the spi functionality. i tried finding some clues from the extendedfirmata file itself but all i have are errors. where can i find this messages or commands?

it could be nice to have this new implementation in the firmata test program so you get the idea of the structure order for enabling operation.

[soundanalogous]

You need to look at this code specifically: https://github.com/kraman/go-firmata/blob/master/contrib/ExtendedFirmata/ExtendedFirmata.ino#L626-L663.

However it is probably best to wait until SPI support is officially added to the Firmata protocol. At that point you can engage with the PD community for help on updating the PD Firmata client to add SPI support. It's not simply going to work just because Firmata gets SPI. Each firmata client also needs to implement that part of the protocol in order to get the new functionality.

[dprophet]

This is almost 2 years old. Whats the status of SPI + Firmata? I am in need of this feature as well because customized sensor libraries steal the 32k of space.

[soundanalogous]

I don't have a ton of experience with SPI so I don't feel the best suited person to take on this task, however no one else is volunteering which is why it has seen no progress in 2 years.

[soundanalogous]

@madeintheusb this isn't a place to advertise your services.