ap--
Intensities incorrect for ST-VIS-25
spectrometer and system information
- model: ST, Serial: ST02185, ST-VIS-25
- operating system: Jetson Orin Ubuntu 22.04 - ARM processor
- python version: Python 3.10.12
- python-seabreeze version: current master
- installed-via: pip install seabreeze
current problem
When I try to use the script Seabreeze script I get a different array size of intensities array than when I do with the OceanDirect script. I'm not sure if the corrections are applied either? We want to run on an ARM processor
Seabreeze: First 10 values: [1505. 0. 1563. 0. 1570. 0. 1544. 0. 1522. 0.]
Size: 3032
OceanDirect: First 10 values: [14.654972076416016, 14.654972076416016, 13.45759105682373, -2.121690034866333, -0.9224371910095215, 4.272670745849609, 5.471170902252197, -2.521484851837158, 5.471170902252197, 3.8731637001037598]
Size: 1516
steps to reproduce
(venv) ~/seabreeze$ python3
Python 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import seabreeze
>>> seabreeze.use('pyseabreeze')
>>> from seabreeze.spectrometers import Spectrometer
>>> spec = Spectrometer.from_first_available()
>>> print(f"Connected to spectrometer: {spec.model}, Serial: {spec.serial_number}")
Connected to spectrometer: ST, Serial: ST02185
>>> spec.integration_time_micros(100000)
>>> spec.wavelengths()
array([348.31332397, 348.65684909, 349.00031106, ..., 811.36221513,
811.64102486, 811.91981221])
>>> print(f"Wavelengths shape: {spec.wavelengths().shape}")
Wavelengths shape: (1516,)
print(f"First 10 values: {spec.wavelengths()[:10]}")
First 10 values: [348.31332397 348.65684909 349.00031106 349.3437099 349.68704565
350.03031833 350.37352796 350.71667458 351.05975821 351.40277888]
>>> spec.intensities()
array([1515., 0., 1527., ..., 0., 1524., 0.])
>>> print(f"Intensities shape: {spec.intensities().shape}")
Intensities shape: (3032,)
>>> print(f"First 10 values: {spec.intensities()[:10]}")
First 10 values: [1505. 0. 1563. 0. 1570. 0. 1544. 0. 1522. 0.]
When I run a similar script using the the OceanDirect script I get these response from the spectrometer:
Connected to spectrometer: OBP2Device, Serial: ST02185
Dark Pixel Information:
Number of electric dark pixels: 13
Dark pixel indices: [1503, 1504, 1505, 1506, 1507, 1508, 1509, 1510, 1511, 1512, 1513, 1514, 1515]
Raw spectrum values at dark pixel locations: Index 1503: 33.334922790527344 Index 1504: 5.809284210205078 Index 1505: 5.809284210205078 Index 1506: 17.7861270904541 Index 1507: -18.186737060546875 Index 1508: -20.58944320678711 Index 1509: -3.78235125541687 Index 1510: -9.781708717346191 Index 1511: -50.67084884643555 Index 1512: -74.79936981201172 Index 1513: -27.80094337463379 Index 1514: 45.27930450439453 Index 1515: 96.47938537597656
OceanDirect Formatted Spectrum:
Length: 1516
First 10 values: [50.05310821533203, 50.05310821533203, 81.02778625488281, 45.27930450439453, -2.5829029083251953, -4.981940746307373, 45.27930450439453, -6.181671142578125, -20.58944320678711, -6.181671142578125]
INFO: Buffer size returned: 4
Calibration Coefficients:
Nonlinearity coefficients: [0.8339769840240479, 4.907249967800453e-05, 3.217550004208647e-09, -3.677040009281152e-12, 6.984339868505382e-16, -6.182079710757103e-20, 2.6929200754414477e-24, -4.671669886348679e-29]
Wavelength coefficients: [348.3133239746094, 0.3435567021369934, -3.158873732900247e-05, 4.492421545876368e-09]
Integration time set to 0.1 seconds.
Scans to average set to 3.
Wavelengths array details:
Size: 1516
First 10 values: [348.3133239746094, 348.6568603515625, 349.00030517578125, 349.3437194824219, 349.6870422363281, 350.0303039550781, 350.37353515625, 350.7166748046875, 351.05975341796875, 351.40277099609375]
Last 10 values: [809.409912109375, 809.6889038085938, 809.9678344726562, 810.2467651367188, 810.525634765625, 810.8045654296875, 811.0833740234375, 811.3621826171875, 811.6410522460938, 811.9197998046875]
Raw spectrum array details:
Size: 1516
First 10 values: [14.654972076416016, 14.654972076416016, 13.45759105682373, -2.121690034866333, -0.9224371910095215, 4.272670745849609, 5.471170902252197, -2.521484851837158, 5.471170902252197, 3.8731637001037598]
Last 10 values: [15.054130554199219, 14.654972076416016, 0.27667462825775146, -0.522741973400116, -10.120340347290039, -20.127416610717773, -0.9224371910095215, 13.45759105682373, 1.4756454229354858, -5.720295429229736]
minimal code example and error (very helpful if available)
import seabreeze
seabreeze.use("pyseabreeze") # Ensure the pyseabreeze backend is used
from seabreeze.spectrometers import Spectrometer, list_devices
from datetime import datetime
import csv
import matplotlib.pyplot as plt
import pytz
import numpy as np
# Configuration Variables
INTEGRATION_TIME_MICROS = 100000 # Integration time in microseconds (e.g., 100000 = 100 ms)
SCANS_TO_AVERAGE = 3 # Number of scans to average
TIMEZONE = "Australia/Brisbane" # Set your desired timezone or "UTC"
def capture_spectrum_and_plot():
try:
# Initialize the spectrometer
spec = Spectrometer.from_first_available()
print(f"Connected to spectrometer: {spec.model}, Serial: {spec.serial_number}")
# Set integration time
spec.integration_time_micros(INTEGRATION_TIME_MICROS)
print(f"Integration time set to {INTEGRATION_TIME_MICROS / 1e6} seconds.")
# Retrieve wavelengths and average multiple spectra
wavelengths = spec.wavelengths()
intensities_list = []
print(f"Wavelengths shape: {wavelengths.shape}")
print(spec.wavelengths())
for _ in range(SCANS_TO_AVERAGE):
intensities_list.append(spec.intensities(correct_dark_counts=True, correct_nonlinearity=True))
intensities = np.mean(intensities_list, axis=0)
print(f"Intensities shape: {intensities.shape}")
print(spec.intensities())
# Get timestamp with timezone
tz = pytz.timezone(TIMEZONE)
now = datetime.now(tz)
timestamp = now.isoformat()
filename_timestamp = now.strftime("%Y%m%d%H%M%S")
output_csv = f"spectrum_{filename_timestamp}.csv"
output_png = f"spectrum_{filename_timestamp}.png"
# Save data to CSV
with open(output_csv, mode="w", newline="") as file:
writer = csv.writer(file)
writer.writerow(["Timestamp", timestamp])
writer.writerow(["Model", spec.model])
writer.writerow(["Serial Number", spec.serial_number])
writer.writerow(["Integration Time (s)", INTEGRATION_TIME_MICROS / 1e6])
writer.writerow(["Scans Averaged", SCANS_TO_AVERAGE])
writer.writerow([])
writer.writerow(["Wavelength (nm)"] + list(wavelengths))
writer.writerow(["Intensity"] + list(intensities))
print(f"Spectrum data saved to {output_csv}")
# Create and save plot
plt.figure(figsize=(10, 6))
plt.plot(wavelengths, intensities, label="Spectrum", color="blue", linewidth=1.5)
plt.title(f"Spectrum - {timestamp}", fontsize=14)
plt.xlabel("Wavelength (nm)", fontsize=12)
plt.ylabel("Intensity", fontsize=12)
plt.grid(True, linestyle="--", alpha=0.6)
plt.legend(loc="upper right")
plt.tight_layout()
plt.savefig(output_png, dpi=300)
print(f"Spectrum plot saved to {output_png}")
# Close the spectrometer connection
spec.close()
except Exception as e:
print(f"Error: {e}")
# Run the function
if __name__ == "__main__":
capture_spectrum_and_plot()
Changing to a 32-bit signed integer and increasing the integration time from 100000 to 1000000 got a similar plot to OceanDirect:
class SeaBreezeSpectrometerFeatureOBP2(SeaBreezeSpectrometerFeatureOBP):
def get_intensities(self) -> NDArray[np.float_]:
tmp = self._get_spectrum_raw()
# Skip the 32-byte metadata header
ret = tmp[32:].view(numpy.dtype("<i4")).astype(numpy.double)
return ret * self._normalization_value
Seabreeze:
OceanDirect:
Thanks for reporting!
Would you be interested in contributing support for the ST-VIS-25?
The complicated part here is seemingly (and I am guessing here) that all the ST models share the same USB PID, so we need a similar mechanism as used for the USB2000+ and FlameS models where intitially the default model class is used but then replaced with the correct model dependent on the detected ST model.
If you'd like to work on a PR, I would write a step by step guide to help implement support.
Cheers, Andreas
Yeah, I would love to help.
Scans to average error
I think I may have identified what was causing the initial error. The ocean direct set_scans_to_average function changes the raw intensities output if set_scans_to_average is greater than 1:
For example: SCANS_TO_AVERAGE = 2 # Number of scans to average greater than 1 device.set_scans_to_average(SCANS_TO_AVERAGE)
Then run the pyseabreeze script, the intensities array appears to double with every second value as a zero, and the intensities appear to be a multiple of whatever I had SCANS_TO_AVERAGE set to. If I rerun the set_scans_to_average set to 1, the array goes back to the default length and no error.
For example when I set to 1, both scripts work as expected:
SCANS_TO_AVERAGE = 1 # In Ocean Direct script Ocean Direct output: Intensities array length: 1516 First 10 raw intensities values: [523.5, 523.5, 513.0, 510.5, 522.5, 512.5, 515.5, 510.0, 499.0, 522.0]
Pyseabreeze output: Intensities array length: 1516 First 10 raw intensities values: [504.0, 504.0, 528.0, 531.0, 534.0, 534.0, 527.0, 525.0, 510.0, 523.0] First 10 dark values: [-5.23077392578125, -5.23077392578125, -19.23077392578125, -29.23077392578125, -8.23077392578125, 19.76922607421875, 22.76922607421875, 11.76922607421875, -12.23077392578125, -3.23077392578125]
Setting the set_scans_to_average to a value greater than 1 causes the pyseabreeze error:
Ocean Direct output: SCANS_TO_AVERAGE = 2 # In Ocean Direct script Intensities array length: 1516 First 10 raw intensities values: [523.5, 523.5, 513.0, 510.5, 522.5, 512.5, 515.5, 510.0, 499.0, 522.0] First 10 dark values: [2.30767822265625, 2.30767822265625, 8.80767822265625, 6.80767822265625, 19.30767822265625, 18.30767822265625, -12.19232177734375, -20.69232177734375, -23.69232177734375, -10.69232177734375]
Pyseabreeze output: Intensities array length: 3032 First 10 raw intensities values: [1037. 0. 995. 0. 996. 0. 1032. 0. 1004. 0.] First 10 dark values: [ 401.15384615 -658.84615385 419.15384615 -658.84615385 405.15384615 -658.84615385 393.15384615 -658.84615385 373.15384615 -658.84615385]
Ocean Direct output: SCANS_TO_AVERAGE = 3 # In Ocean Direct script
Pyseabreeze output: Intensities array length: 3032 First 10 raw intensities values: [1579. 0. 1564. 0. 1548. 0. 1588. 0. 1589. 0.] First 10 dark values: [ 477.46153846 -1054.53846154 466.46153846 -1054.53846154 447.46153846 -1054.53846154 468.46153846 -1054.53846154 496.46153846 -1054.53846154]
This wouldn't be an issue in normal operation. I guess I only encountered this because I wanted to validate the pyseabreeze results where in line with ocean direct.
Integration time discrepancy
A second issue also seems to be the integration time when I compare the two outputs?
If I set the pyseabreeze integration time to the same time as Ocean Direct (100000usec):
If I up the pyseabreeze integration time to (1000000usec) the results look very similar:
PYSEABREEZE_INTEGRATION_TIME_MICROS = 1000000 OCEANDIRECT_INTEGRATION_TIME_MICROS = 100000
If I set both to (1000000usec):
PYSEABREEZE_INTEGRATION_TIME_MICROS = 1000000 OCEANDIRECT_INTEGRATION_TIME_MICROS = 1000000
I think the scale is wrong in pyseabreeze for the ST? Increasing the integration_time_max = 60000000 so I can increase the pyseabreeze integration time to 10000000 looks the same as 1000000 for the Ocean Directs script (note: axis scale is a little different in this image).
Non-linearity result difference
One other thing I noticed in the script above. Ocean Directs non-linearity correction appears to be corrected for dark pixels even when off as it's shifted?
Ocean Direct script:
Nonlinearity-corrected only
device.set_electric_dark_correction_usage(False)
device.set_nonlinearity_correction_usage(True)
non_linearity_corrected = device.get_formatted_spectrum()
Versus the pyseabreeze script:
non_linearity_corrected = spec.intensities(correct_dark_counts=False, correct_nonlinearity=True)