[libbladeRF] Cannot set some samplerates returned by bladerf_set_rational_rate()

[jynik]

There are "actual" sample rates returned by bladerf_set_sample_rate() that will result in BLADERF_ERR_INVAL or invalid values when attempting to set these rates via bladerf_set_rational_rate().

More investigation required. Example sample rate: 9142857 4712457141/32999999987

Setting samplerate = (8000000.0 * 8) / 7 from GRC:

[VERBOSE @ si5338.c:449] Requested sample rate: 9142857 + 1428 / 10000
[VERBOSE @ si5338.c:114] Reduced num=357, den=2500
[VERBOSE @ si5338.c:114] Reduced num=357, den=2500
[VERBOSE @ si5338.c:114] Reduced num=357, den=1250
[VERBOSE @ si5338.c:385] Found r value of: 1
[VERBOSE @ si5338.c:114] Reduced num=4571428578, den=7619047619
[VERBOSE @ si5338.c:395] MSx a + b/c: 138 + 4571428578/7619047619
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 4571428578/7619047619 to 2285714289/3809523809
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 2285714289/3809523809 to 1142857144/1904761904
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 1142857144/1904761904 to 571428572/952380952
[VERBOSE @ si5338.c:424] MSx a + b/c: 138 + 571428572/952380952
[VERBOSE @ si5338.c:213] MSx P1: 0x0000434c (17228) P2: 0x2d69bee0 (761904864) P3: 0x38c42e18 (952380952)
[VERBOSE @ si5338.c:114] Reduced num=4712457141, den=32999999987
[VERBOSE @ si5338.c:349] Calculated multisynth frequency: 9142857 + 4712457141/32999999987
[VERBOSE @ si5338.c:236] Writing MS1
[VERBOSE @ nios_access.c:385] nios_si5338_read: Read 0x01 from addr 0x25
[VERBOSE @ si5338.c:245] Wrote enable register: 0x01
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x01 to addr 0x25
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x4c to addr 0x40
[VERBOSE @ si5338.c:259] Wrote regs[0]: 0x4c
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x43 to addr 0x41
[VERBOSE @ si5338.c:259] Wrote regs[1]: 0x43
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x80 to addr 0x42
[VERBOSE @ si5338.c:259] Wrote regs[2]: 0x80
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0xfb to addr 0x43
[VERBOSE @ si5338.c:259] Wrote regs[3]: 0xfb
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0xa6 to addr 0x44
[VERBOSE @ si5338.c:259] Wrote regs[4]: 0xa6
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0xb5 to addr 0x45
[VERBOSE @ si5338.c:259] Wrote regs[5]: 0xb5
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x18 to addr 0x46
[VERBOSE @ si5338.c:259] Wrote regs[6]: 0x18
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x2e to addr 0x47
[VERBOSE @ si5338.c:259] Wrote regs[7]: 0x2e
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0xc4 to addr 0x48
[VERBOSE @ si5338.c:259] Wrote regs[8]: 0xc4
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x38 to addr 0x49
[VERBOSE @ si5338.c:259] Wrote regs[9]: 0x38
[VERBOSE @ si5338.c:274] Wrote r register: 0xc0
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0xc0 to addr 0x20

From bladeRF-cli:

[VERBOSE @ si5338.c:449] Requested sample rate: 9142857 + 4712457141 / 32999999987
[VERBOSE @ si5338.c:114] Reduced num=4712457141, den=32999999987
[VERBOSE @ si5338.c:114] Reduced num=4712457141, den=32999999987
[VERBOSE @ si5338.c:114] Reduced num=9424914282, den=32999999987
[VERBOSE @ si5338.c:385] Found r value of: 1
[VERBOSE @ si5338.c:114] Reduced num=23604313137133, den=73660714256250
[VERBOSE @ si5338.c:395] MSx a + b/c: 16 + 23604313137133/73660714256250
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 23604313137133/73660714256250 to 11802156568566/36830357128125
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 11802156568566/36830357128125 to 5901078284283/18415178564062
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 5901078284283/18415178564062 to 2950539142141/9207589282031
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 2950539142141/9207589282031 to 1475269571070/4603794641015
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 1475269571070/4603794641015 to 737634785535/2301897320507
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 737634785535/2301897320507 to 368817392767/1150948660253
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 368817392767/1150948660253 to 184408696383/575474330126
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 184408696383/575474330126 to 92204348191/287737165063
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 92204348191/287737165063 to 46102174095/143868582531
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 46102174095/143868582531 to 23051087047/71934291265
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 23051087047/71934291265 to 11525543523/35967145632
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 11525543523/35967145632 to 5762771761/17983572816
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 5762771761/17983572816 to 2881385880/8991786408
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 2881385880/8991786408 to 1440692940/4495893204
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 1440692940/4495893204 to 720346470/2247946602
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 720346470/2247946602 to 360173235/1123973301
[DEBUG @ si5338.c:418] Loss of precision in reducing fraction from 360173235/1123973301 to 180086617/561986650
[VERBOSE @ si5338.c:424] MSx a + b/c: 16 + 180086617/561986650
[VERBOSE @ si5338.c:213] MSx P1: 0x00000629 (1577) P2: 0x00930216 (9634326) P3: 0x217f3c5a (561986650)
[VERBOSE @ si5338.c:114] Reduced num=7818654122, den=9171873017
[VERBOSE @ si5338.c:349] Calculated multisynth frequency: 77644934 + 7818654122/9171873017
[VERBOSE @ si5338.c:236] Writing MS1
[VERBOSE @ nios_access.c:385] nios_si5338_read: Read 0x01 from addr 0x25
[VERBOSE @ si5338.c:245] Wrote enable register: 0x01
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x01 to addr 0x25
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x29 to addr 0x40
[VERBOSE @ si5338.c:259] Wrote regs[0]: 0x29
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x06 to addr 0x41
[VERBOSE @ si5338.c:259] Wrote regs[1]: 0x06
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x58 to addr 0x42
[VERBOSE @ si5338.c:259] Wrote regs[2]: 0x58
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x08 to addr 0x43
[VERBOSE @ si5338.c:259] Wrote regs[3]: 0x08
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x4c to addr 0x44
[VERBOSE @ si5338.c:259] Wrote regs[4]: 0x4c
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x02 to addr 0x45
[VERBOSE @ si5338.c:259] Wrote regs[5]: 0x02
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x5a to addr 0x46
[VERBOSE @ si5338.c:259] Wrote regs[6]: 0x5a
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x3c to addr 0x47
[VERBOSE @ si5338.c:259] Wrote regs[7]: 0x3c
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x7f to addr 0x48
[VERBOSE @ si5338.c:259] Wrote regs[8]: 0x7f
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0x21 to addr 0x49
[VERBOSE @ si5338.c:259] Wrote regs[9]: 0x21
[VERBOSE @ si5338.c:274] Wrote r register: 0xc0
[VERBOSE @ nios_access.c:397] nios_si5338_write: Wrote 0xc0 to addr 0x20
  Setting RX sample rate - req:   9142857 4712457141/32999999987Hz, actual:  77644934 7818654122/9171873017Hz

[jynik]

@bpadalino Just poking you on this since you wrote the multisynth calc stuff. I think problem resides in the multisynth calculation.

I pushed a dev-issue_427 branch that fixes the u32 vs u64 issue in the CLI. Attempting to: set samplerate rx 77644934 7818654122 9171873017

jon@nocontrol % bladeRF-cli -i -v verbose
[VERBOSE @ libusb.c:527] Using libusb version: 1.0.19.10903
[VERBOSE @ libusb.c:409] Found a bladeRF (idx=2)
[VERBOSE @ libusb.c:470] USB port reset succeeded for bladeRF 8bf831353d21aaa1dcf9124c0e66eead
[VERBOSE @ capabilities.c:41] Capability mask before FPGA load: 0x0000000300000000
[VERBOSE @ usb.c:112] Changing to USB alt setting 0
[VERBOSE @ usb.c:112] Changing to USB alt setting 1
[VERBOSE @ usb.c:169] Read FPGA version: 0.4.1
[VERBOSE @ capabilities.c:80] Capability mask after FPGA load: 0x00000003000000ff
[VERBOSE @ usb.c:180] Using current packet handler formats
[VERBOSE @ usb.c:112] Changing to USB alt setting 2
[VERBOSE @ usb.c:112] Changing to USB alt setting 1
[VERBOSE @ usb.c:112] Changing to USB alt setting 2
[VERBOSE @ usb.c:112] Changing to USB alt setting 1
[VERBOSE @ nios_access.c:664] Clearing RX retune queue.
[VERBOSE @ nios_access.c:687] RX operation duration: 0
[VERBOSE @ nios_access.c:664] Clearing TX retune queue.
[VERBOSE @ nios_access.c:687] TX operation duration: 0
[VERBOSE @ nios_access.c:298] nios_config_read: Read 0x00000057
[DEBUG @ tuning.c:82] Default tuning mode: FPGA
[VERBOSE @ nios_access.c:298] nios_config_read: Read 0x00000057
bladeRF> set samplerate rx 77644934 7818654122 9171873017

[VERBOSE @ si5338.c:383] Found r value of: 1
[VERBOSE @ si5338.c:393] MSx a + b/c: 3 + 64154541417877/173864619843750
[DEBUG @ si5338.c:404] Integer portion too small: 3


  Error: Invalid operation or parameter