High-speed acquisition mode using the Ocean Binary Protocol on OceanFX

[aarpon]

spectrometer and system information

• model: OceanFX (FlameX)
• operating system: Windows 7 64bit
• python version: Python 3.6.6 :: Anaconda custom (64-bit)
• python-seabreeze version: 0.6.0
• installed-via: conda

current problem

Using the Ocean Binary Protocol (OBP) it should be possible to achieve a rate of 4500 spectra/s over USB. This requires the following sequence of operations (pseudocode from the OBP documentation for the OceanFX spectrometer):

Set trigger mode = 0 // software trigger (HW triggering can also be used)
Set buffering = enabled
Set number of back-to-back spectra per trigger = 50000
Set integration time = 10 // microseconds
Clear the buffer // optional
SpectrumData[] = new Spectrum[15] // Allocate a buffer to hold the spectra response data
RequestRawSpectrumWithMetadata(15) // Request up to 15 spectra
START_LOOP
    RequestRawSpectrumWithMetadata(15) // Request the next 15 spectra
    SpectrumData = ReadRawSpectrumWithMetadata(15) // Read (up to) the next 15 spectra
    // *** Do stuff with the returned spectra here ***
    // *** Note: any number of spectra from 0 – 15 may be returned ***
END_LOOP
SpectrumData = ReadRawSpectrumWithMetadata(15) // Read (up to) the last 15 spectra
// *** Do stuff with the returned spectra here ***
// *** Note: any number of spectra from 0 – 15 may be returned ***

Is there an API that already provides this buffered reading of spectra? If not, could you provide me with some hints on how to implement it? In particular, I would like to know if it is possible to perform the Set buffering = enabled and Set number of back-to-back spectra per trigger = 50000 using python-seabreeze with any of the backends, and how.

[ap--]

Hi @aarpon

I recommend you look for a datasheet of the OceanFX that has the full command set available. (I couldn't find one just now) An implementation of the OBP protocol is here https://github.com/ap--/python-seabreeze/blob/master/seabreeze/pyseabreeze/interfaces/communication.py#L100-L340 if you want to extend pyseabreeze you would just interface with the spectrometer via the .send_command and .query methods

Best, Andreas

[aarpon]

Hi,

great, thank you for the pointer! I will have a look.

a2

[tobiasium]

Hi aarpon, did you manage to have a look at this or get it to work? I also have two FX spectrometers that I would like to read out fast... Currently I get around 390 FPS with seabreeze, which is the same as a normal Flame spectrometer.

[aarpon]

Hi,

nope, not yet. But I plan to work on it soon (hopefully from next week).

Thanks, a2

[aarpon]

Hi,

the python backend of python-seabreeze does not recognize the OceanFX spectrometer (0x2001). I have been (quickly) looking at the interfaces\defines.py file. Any pointers on how and where I can collect all the relevant information (dark pixels, end points, trigger modes)? Also, is there some documentation on how I can add a complete new spectrometer to pyseabreeze?

Thanks, a2

[ap--]

There is one small thing missing that I didn't wrap in the cseabreeze backend for version 1.0.0 :disappointed: because it's only exposed via unformatted spectra...

https://github.com/ap--/python-seabreeze/blob/f0a08bb6b82856a8b0d2a87bbacaf7994cc335a2/src/seabreeze/cseabreeze/c_seabreeze_wrapper.pyx#L809-L815

There's two ways to do this:

• implement the cython interface for the function above, then you can access it via:

  spec = Spectrometer.from_first_available()
  spec.f.spectrometer.get_fast_buffer_spectrum()
  

  (the other fast_buffer feature functions are already wrapped)

• implement the functionality in pyseabreeze, which requires:

  • add support for the FlameX to pyseabreeze
  • implement the FastBufferFeature
  • implement the get_fast_buffer_spectrum() method in the SpectrometerFeature

With v1.0.0 I rewrote the whole pyseabreeze backend with v1.0.0 so that it's easier to extend:

https://python-seabreeze.readthedocs.io/en/latest/contributing.html Shows how to add a spectrometer. And the code should (hopefully :sweat_smile: ) be well documented :tada:

[Roy4777]

@aarpon I am using an Ocean FX-XR1. I am trying to add a new spectrometer using the documentation here

Can you please provide me a copy of inputs for the following variables for Ocean FX?

# communication config
transport = (USBTransport, )
usb_product_id = 0x4200
usb_endpoint_map = EndPointMap(ep_out=0x01, lowspeed_in=0x81)  # XXX: we'll ignore the alternative EPs
usb_protocol = OBPProtocol

# spectrometer config
dark_pixel_indices = DarkPixelIndices.from_ranges()
integration_time_min = 10
integration_time_max = 85000000
integration_time_base = 1
spectrum_num_pixel = 1024
spectrum_raw_length = (1024 * 2) + 64  # XXX: Metadata
spectrum_max_value = 16383
trigger_modes = TriggerMode.supported('OBP_NORMAL', 'OBP_EXTERNAL', 'OBP_INTERNAL')

[aarpon]

@ap-- Unfortunately, I won't have access to the Ocean FX for the next few weeks...

[Roy4777]

@aarpon Did you manage to implement the High-Speed Acquisition mode for Ocean FX on python-seabreeze? If yes, can you please share how you achieved it?

[aarpon]

@Roy4777 We ended up going down a completely different path (i.e. not using python-seabreeze). We should publish a paper about it soon, after which we will upload the code to either github or our gitlab instance.

[Roy4777]

@aarpon That's awesome! I'll be looking forward to it.

[ptapping]

I have just got hold of an Ocean FX VIS-NIR for a pre-purchase test, which I will have access to for a couple of weeks. We want to use it to capture at high speed, and integrate it into a python code base.

Just to clarify the current state of seabreeze at this time, it looks like seabreeze works with the FX, but it appears as a FlameX and doesn't support the short integration time and "throughput" or buffered mode of the FX required to reach the advertised 4500 spectra per second.

Initially I thought a basic ctypes interface into the omnidriver.so/omnidriver.dll would be quick and easy, but no, it doesn't work. So I'm looking at other options. Writing my own wrapper around the Java or C++ omnidriver libraries would work, but be horribly tedious and bloated. Ultimately, I think it would be nice if I could hack at an open source library like seabreeze instead...

Anyway, I have ssh access to the machine with the FX connected by USB, and will look at what it will take to implement the buffered acquisition mode. Any suggestions are welcome, and/or if you'd like me to test or report any data from the FX, let me know.

Also, @aarpon if you have published that paper or have other hints about how you achieved fast acquisition from the FX, I'd love to hear it!

[ap--]

Hi @ptapping

Thanks for offering to contribute! :heart: Implementing the fast buffer spectrum feature shouldn't take very long, as you have the hardware to test it. I don't have a developer data sheet for the OceanFX but since it uses the OceanBinaryProtocol for communication it should also be fairly standardized in its behavior. Still, if you find one it would be incredibly helpful.

As described in an older comment of mine above https://github.com/ap--/python-seabreeze/issues/59#issuecomment-530549301 you do have two options to get started: 1.) you can wrap the missing C++ functionality for the FlameX and expose the fast buffer spectrum functionality via the cseabreeze backend 2.) you could add the OceanFX to the pyseabreeze library and implement the feature there.

Ultimately it just depends in which codebase you feel more at home. (1) is probably faster to do if you're familiar with c++ and cython and you just need it to work. (2) will probably be easier to iterate and hack on.

I would probably recommend (2) and implementing it in the pyseabreeze backend. It's biggest problem is though, that it's quite a bit unpythonic, because I tried back then (bad decision) to make it mirror the abstraction layers of the c++ seabreeze library and over the years it became quite a bit ugly due to backwards-compatibility and hotfixes. But if I'll ever release python-seabreeze-v2 I would probably drop the cseabreeze backend and refactor pyseabreeze with only python3.6+ in mind...

So let me know if I can do anything to get you started, and feel free to ask questions if you get stuck. If you open a draft PR, I can help you with code review and point you in the right directions.

Cheers, Andreas :smiley:

[ptapping]

Thanks for the encouragement! I managed to have a bit of a look at this today.

So it turns out the Ocean FX detects as a FlameX using the cseabreeze API, and it reports that it supports the fastbuffer, databuffer etc features. So this is possible:

import seabreeze
seabreeze.use('cseabreeze')
from seabreeze.spectrometers import Spectrometer
spec = Spectrometer.from_first_available()
spec.f.fast_buffer.set_buffering_enable(True)
spec.f.fast_buffer.set_consecutive_sample_count(100)
# The FlameX only supports 1000 us, the FX should support 10 us
spec.integration_time_micros(1000)

wavelengths = spec.wavelengths()
intensities = spec.intensities()

print(spec.f.databuffer.get_number_of_elements())

# Remember to do this...
spec.close()

The databuffer reports the correct number of spectra acquired, but of course the call to intensities() only returns a single spectra. I have a feeling if I just implement the get_fast_buffer_spectrum() method on line 842 of c_seabreeze_wrapper.pyx then that should do it. The comment says it interfaces to the spectrometerGetFastBufferSpectrum call (in the FX OBP documentation).

So I'll start there and see how we go. If it works, then it should just need a new OceanFX device definition, which should be mostly a copy-paste from the FlameX one.

Ultimately though, yes, I think it would be nice to implement the pyseabreeze backend.

[aarpon]

Hi, guys. As I said in a previous comment, we "solved" the issue in a different way. We contacted Ocean Optics and were sent a DLL (written in C# by Oliver Lischtschenko) that allows us to unlock the complete set of functionalities in OceanFX. For us, this solution was fair enough, since we are implementing a whole analysis platform, and were fine relying on having the bidirectional communication between the photometer and the CPU set up by someone else. As for our discussions with OO, the DLL can be freely used but must be distributed in binary form. We are somehow being delayed with our publications, but the idea is to get the whole code on github soon(ish).

[ptapping]

Thanks for the update! I have access to the OmniDriver, but it's not pretty. I didn't get a ctypes interface into the .dll or .so working at all, and the C libraries just wrap the underlying Java library so would be a bloated mess of wrapped wrappers, hence looking at this library instead. It sounds like they gave you something different and not publicly released though.

I haven't worked on this in a week or so, but made some progress. I started implementing the FX support into pyseabreeze, but got stuck pretty quickly on what looks like a low-level communications issue. I can open the spectrometer (which reads the serial number), but after that communications will time out and the spectrometer will stop responding. I did find that doing a low-level read would seem to clear buffers or something and the spectrometer would respond again. My guess is that a read and/or write in the OBP or transport layer is sending/receiving the wrong size data chunks or unexpected data.

I found this issue #109 but not sure it's related.

It might also be to do with the acknowledgement request flag. For example:

import seabreeze.pyseabreeze as psb
api = psb.SeaBreezeAPI()
api.supported_models()          # FX is listed
api.list_devices()              # FX is detected
dev = api.list_devices()[0]     # USB read timeout
dev = api.list_devices()[0]     # Works this time...?
dev.serial_number               # Got serial number OK
# But now everything will timeout again
# If we just read raw bytes, it will come back to life
_ = dev._transport._device.pyusb_device.read(0x01, 2**16)

# Sending raw commands like this seems to work every time
dev._transport.protocol.send(0x00000100, request_ack=False) # Request serial number
dev._transport.protocol.receive()

# But times out if the acknowledgement reply is requested?
dev._transport.protocol.send(0x00000100, request_ack=True)
dev._transport.protocol.receive()

Anyway, have been busy with other tasks, so need to look into this more, but thought I'd report my progress. Any suggestions or ideas welcome of course.

I've also attached the output of lsusb -v just in case it's of some use to anyone. OceanFX-lsusb.txt

[ap--]

Hi @aarpon and @ptapping

We contacted Ocean Optics and were sent a DLL (written in C# by Oliver Lischtschenko) that allows us to unlock the complete set of functionalities in OceanFX. For us, this solution was fair enough, since we are implementing a whole analysis platform, and were fine relying on having the bidirectional communication between the photometer and the CPU set up by someone else.

I'd be happy to accept PRs for vendoring and integrating the DLL together with a python wrapper as another backend in python-seabreeze. It's too bad that it's win only.

It might also be to do with the acknowledgement request flag. For example:

The request_ack flag is used within the send command to make the spectrometer acknowledge it received the command. If the command does not produce a new reply by the spectrometer there won't be any data to receive afterwards.

It's best to use send to send commands to the spectrometer that don't cause a reply, and query to send commands that return data from the spectrometer. (query, is send with request_ack=False and receive afterwards.)

The command acknowledging mechanism doesn't really make sense for USB connected spectrometers. The OPBProtocol is also used on spectrometers that support RS232...

Let me know if that helps clearing things up, -Andreas 😃

[sharmila-velamur]

Hi, guys. As I said in a previous comment, we "solved" the issue in a different way. We contacted Ocean Optics and were sent a DLL (written in C# by Oliver Lischtschenko) that allows us to unlock the complete set of functionalities in OceanFX. For us, this solution was fair enough, since we are implementing a whole analysis platform, and were fine relying on having the bidirectional communication between the photometer and the CPU set up by someone else. As for our discussions with OO, the DLL can be freely used but must be distributed in binary form. We are somehow being delayed with our publications, but the idea is to get the whole code on github soon(ish).

@aarpon I am new to working with spectrometers and seabreeze, but for my purpose, I too need sustained burst mode (4500 Hz) data acquisition. I have been trying some solutions and then came across this discussion. Have you published your code/paper yet? Would really appreciate any information on how you achieved sustained high speed data acquisition. BTW, I am working with an OceanFX spectrometer. Since it is a DLL maybe it is meant only for Windows env, is there anything similar that might work for RPi?

Thank you.

[aarpon]

@sharmila-velamur I have to check with the original code by Oliver Lischtschenko: I think it could reach the maximum theoretical throughput; our solution saturates at around 2200 Hz, but in practice, for our purposes, we probably work at 800 Hz: to even get some signal from our fluorophores, we have to illuminate longer and therefore slow everything down. I am afraid for our code to be accessible online it will take some more time, but I can investigate whether we can share the original code from OO. As I mentioned in the past, it's C# and runs on Windows. I guess you could give mono a shot if you need to use it on other platforms and you (mostly) lose the UI. This code just gets spectra and plots them (a few times a second), but should have all the elements in place to get you started.

[sharmila-velamur]

@sharmila-velamur I have to check with the original code by Oliver Lischtschenko: I think it could reach the maximum theoretical throughput; our solution saturates at around 2200 Hz, but in practice, for our purposes, we probably work at 800 Hz: to even get some signal from our fluorophores, we have to illuminate longer and therefore slow everything down. I am afraid for our code to be accessible online it will take some more time, but I can investigate whether we can share the original code from OO. As I mentioned in the past, it's C# and runs on Windows. I guess you could give mono a shot if you need to use it on other platforms and you (mostly) lose the UI. This code just gets spectra and plots them (a few times a second), but should have all the elements in place to get you started.

Thank you so much for replying to me @aarpon. I do not need the UI at all. If I can get ~2000 Hz that would be pretty awesome (I am at 160 Hz now) to start with. What is mono? I have not come across that in the OceanFX documentation to the best of my recall.

[aarpon]

@sharmila-velamur. The code we got from Oliver Lischtschenko cannot be found anywhere officially, also I could never really find the documentation needed to replicate what it does. The problem is that Oliver's demo is heavily bound to a UI (it's just one big C# file). So, you'd need to extricate what you need from all the UI callbacks. Our tool is also a heavily graphical thing, and massively larger. Would you be able to work from a C# solution? Finally, mono is an opensource implementation of .NET that runs cross-platform (and has some support for Windows Forms).

[ap--]

Hi @sharmila-velamur and @aarpon

Hope you all had a good start into 2021. I can help with implementing the high-speed mode in python-seabreeze, but I lack the hardware to test it.

It would be incredibly helpful, if one of you could get a USB Packet dump using https://desowin.org/usbpcap/ of OceanView (or alternatively in aapron's case: Oliver's tool) requesting a spectrum in high speed mode. With that we should have everything needed to implement full support in python-seabreeze. (note: the packet dump will contain your spectrometer's serial number in case you share it publicly) (if we don't have it, it might require a bit more guesswork, and we might hit a roadblock)

I think I can reserve time this weekend to help test the code on your hardware and we could debug it together 😃

Let me know what you think or if I can be of further help in the meantime, Cheers, Andreas 😃

[aarpon]

Hi, @ap--. Happy new year to you, too! I should be able to get my hands on the hardware this week (can't tell you when, yet, though). And I can give it a shot.

[sharmila-velamur]

Hi @sharmila-velamur and @aarpon

Hope you all had a good start into 2021. I can help with implementing the high-speed mode in python-seabreeze, but I lack the hardware to test it.

It would be incredibly helpful, if one of you could get a USB Packet dump using https://desowin.org/usbpcap/ of OceanView (or alternatively in aapron's case: Oliver's tool) requesting a spectrum in high speed mode. With that we should have everything needed to implement full support in python-seabreeze. (note: the packet dump will contain your spectrometer's serial number in case you share it publicly) (if we don't have it, it might require a bit more guesswork, and we might hit a roadblock)

I think I can reserve time this weekend to help test the code on your hardware and we could debug it together 😃

Let me know what you think or if I can be of further help in the meantime, Cheers, Andreas 😃

Hi @ap-- Ocean Optics provided me an SDK called Ocean Direct. I've been working with it (using ctypes wrapper), but still have some issues. Setting 10 ms integration time throws Data Transfer Error and the API's get_raw_spectrum_with_meta_data which is supposed to return 15 rows at a time, maxes out at 50,000 (the max buffer capacity). I have access to the hardware and I will try capturing the USB packets, but it could take a few days into next week. It would be so awesome if sustained high speed data acquisition is integrated into the seabreeze library. Looking forward to working with you on that. Thank you.

[aarpon]

Cool! I should be able to get on the machine next week. I'll try to get a packet dump, too.

[ap--]

Ocean Optics provided me an SDK called Ocean Direct

Interesting! I assume this is closed source?

@sharmila-velamur wrote: have access to the hardware and I will try capturing the USB packets, but it could take a few days into next week. @aarpon wrote: Cool! I should be able to get on the machine next week. I'll try to get a packet dump, too.

Awesome. I have to see when I can free up a day to work on this. Though, it is unlikely that I will find time in the next two weeks. So no rush.

Have a great day :smiley: -Andreas

[sharmila-velamur]

Ocean Optics provided me an SDK called Ocean Direct

Interesting! I assume this is closed source?

@sharmila-velamur wrote: have access to the hardware and I will try capturing the USB packets, but it could take a few days into next week. @aarpon wrote: Cool! I should be able to get on the machine next week. I'll try to get a packet dump, too.

Awesome. I have to see when I can free up a day to work on this. Though, it is unlikely that I will find time in the next two weeks. So no rush.

Have a great day 😃 -Andreas

I am not sure. I will find out tomorrow and let you know if this is a source I can share or not.

Have a great one :)

[ptapping]

Hi all,

I had been meaning to post here again with my progress, and the recent activity has reminded me about this...

In the end, we did not end up purchasing the FX after our trial, so I didn't pursue this much further. We ended up choosing a more expensive option, but one which should hopefully also get us more spectra per second. Yes, Ocean Insight lost a customer because of their poor software and API.

I still think the way to go is implementing the pure python OBP methods for the buffer management and retrieval of spectra. Add the FX in pyseabreeze/devices.py with a class something like this:

class FX(SeaBreezeDevice):

    model_name = "FX"

    # communication config
    transport = (USBTransport,)
    usb_product_id = 0x2001
    usb_endpoint_map = EndPointMap(
        ep_out=0x01, lowspeed_in=0x81, highspeed_in=0x00, highspeed_in2=0x00
    )
    usb_protocol = OBPProtocol

    # spectrometer config
    dark_pixel_indices = DarkPixelIndices.from_ranges()
    integration_time_min = 10
    integration_time_max = 10000000
    integration_time_base = 1
    spectrum_num_pixel = 2068
    spectrum_raw_length = (2068 * 2) + 64  # XXX: Metadata
    spectrum_max_value = 65535
    trigger_modes = TriggerMode.supported(
        "OBP_NORMAL", "OBP_EDGE", "OBP_LEVEL", "DISABLED"
    )

    # features
    feature_classes = (
        sbf.spectrometer.SeaBreezeSpectrometerFeatureFX,
        sbf.rawusb.SeaBreezeRawUSBBusAccessFeature,
        sbf.nonlinearity.NonlinearityCoefficientsFeatureOBP,
    )

and to pyseabrease/features/spectrometer.py with something like:

class SeaBreezeSpectrometerFeatureFX(SeaBreezeSpectrometerFeatureOBP):
    def _get_spectrum_raw(self):
        timeout = int(
            self._integration_time_max * 1e-3
            + self.protocol.transport.default_timeout_ms
        )
        # the message type is different than the default defined in the protocol,
        # requires addition of a new message type in protocol to work
        datastring = self.protocol.query(0x00101000, timeout_ms=timeout)
        return numpy.fromstring(datastring, dtype=numpy.uint8)

(these are based off the HDX which is similar, but I'm not certain the spectrum_num_pixels is correct).

However, the initialisation will still not work because of the issue I described above --- I think the initialisation and/or initial query of device serial number fails because the spectrometer does not return the expected number of bytes. I didn't follow this further to see exactly why, but setting the request_ack to false in the OBPProtocol.send() method seemed to help I think? Anyway, once (sort of) initialised, I could empty the read buffers with something like _ = dev._transport._device.pyusb_device.read(0x81, 2**16), and then send and receive raw OBP commands with dev._transport.protocol.send(...) and dev._transport.protocol.receive() messages. So, assuming the above issue is fixed and the FX follows the rest of the OBP specification (here), it should just be the spectrometer buffer commands, particularly the GetRawSpectrumWithMetadata() method that needs to be implemented.

Good luck!

[ghost]

@ap-- I have an Ocean FX XR1 in my lab. Are you still interested in developing the high frame rate support? If so, I could get you the packet dump/other info you need. We are very interested in my group to get this working with SeaBreeze.

[ap--]

Hi @npeard

Time is a bit scarce on my side, but I'd be very happy to offer assistance in implementing high-speed acquisition mode. Having the USB packet dump would be a great resource for whoever implements it :)

I can sketch out the required functionality and maybe a savvy programmer from your group with more time can fill in the usb communication details?

Cheers, Andreas