christoph2
Using `pyxcp` as a library
Thank you very much, for bringing xcp to the python world!
We want to use this package as a library and we have currently two issues:
- There's a pretty harsh constraint on the dependencies: https://github.com/christoph2/pyxcp/blob/36843fd640b180dd9077123945b5e5be08a1b694/pyproject.toml#L86
- There's currently unconditionally pretty printing installed: https://github.com/christoph2/pyxcp/blob/36843fd640b180dd9077123945b5e5be08a1b694/pyxcp/init.py#L9-L16
- It is very difficult to make a
PyXCPobject programmatically as it expects to read from a file (can be worked around)
I would provide patches for both of these, if you're interested?
We're also interested in using this as a library.
We haven't spend much time on it yet, but the third point you are mentioning is problematic for us too.
We are still using 0.21 since Master can be configured using a simple dictionary.
@akermu did you manage to use pyxcp as a library? I am having hard times initializing the drivers without the commandline tool. I would like to have more control over what is going on under the hood to use it really as a library. All the initialization stuff is pretty "hidden". I am interested to the patches, can you paste them here?
@fross Indeed i am experiencing difficulties during the updete to 0.30. I had old versions working just fine. Now almost since a week trying to get 0.30 up and running but I am failing... I don't want to give up as I have spent now almost a week trying to make the new version work. I really like some of the updates @christoph2 has brought in....hope to find a solution soon.
Two main points I see critical currently and missing:
- Flexibility in controling protocol and layer: control of protocol (COMMANDS etc..) to be independent control of the transportation layer (e.g. when to initialize or shutdown the bus)
- Flexible way of initialization
@SaxElectronics I suspect it's only a matter of updating the configuration of Master from a dictionary to a Config object.
I gave it a shot last week, but did not have the time to complete the configuration (yet). I'll come back to it eventually. The legacy module is a good starting point to dig in this.
e.g.
from traitlets.config import Config
[...]
general_config = Config()
transport_config = Config()
transport_config.timeout = 5.0
can_config = Config()
can_config.bitrate = 500_000
can_config.can_id_master = 257
can_config.can_id_slave = 258
can_config.channel = "0"
can_config.daq_identifier = []
can_config.data_bitrate = 500_000
can_config.fd = False
can_config.interface = "Kvaser"
can_config.max_dlc_required = False
can_config.use_default_listener = True
config = Config()
config.general = general_config
config.transport = transport_config
config.transport.can = can_config
master = Master("CAN", config)
@SaxElectronics I suspect it's only a matter of updating the configuration of
Masterfrom a dictionary to aConfigobject.I gave it a shot last week, but did not have the time to complete the configuration (yet). I'll come back to it eventually. The legacy module is a good starting point to dig in this.
e.g.
from traitlets.config import Config [...] general_config = Config() transport_config = Config() transport_config.timeout = 5.0 can_config = Config() can_config.bitrate = 500_000 can_config.can_id_master = 257 can_config.can_id_slave = 258 can_config.channel = "0" can_config.daq_identifier = [] can_config.data_bitrate = 500_000 can_config.fd = False can_config.interface = "Kvaser" can_config.max_dlc_required = False can_config.use_default_listener = True config = Config() config.general = general_config config.transport = transport_config config.transport.can = can_config master = Master("CAN", config)
I tried your proposal, but seems not to work out of the box, as pyxcp seems to have some internal checks and failing to initialize:
WARNING PyXCP:can.py:358 XCPonCAN - 'slcan' has no support for parameter 'fd'.
WARNING PyXCP:can.py:358 XCPonCAN - 'slcan' has no support for parameter 'data_bitrate'.
WARNING PyXCP:can.py:358 XCPonCAN - 'slcan' has no support for parameter 'receive_own_messages'.
WARNING PyXCP:can.py:358 XCPonCAN - 'slcan' has no support for parameter 'timing'.
ERROR XCPDriver:xcp_driver.py:283 Error during CAN bus initialization: 'LazyConfigValue' object has no attribute 'btr'
even using directly the function convert_config(), seems not like every parameter in a configuration:
WARNING XCPDriver:xcp_driver.py:239 Starting CAN bus initialization (converted settings)
WARNING XCPDriver:legacy.py:92 Unknown keyword 'BAUDRATE_PRESET' in config file
WARNING XCPDriver:legacy.py:92 Unknown keyword 'BTL_CYCLES' in config file
WARNING XCPDriver:legacy.py:92 Unknown keyword 'SAMPLE_RATE' in config file
WARNING XCPDriver:legacy.py:92 Unknown keyword 'SAMPLE_POINT' in config file
WARNING XCPDriver:legacy.py:92 Unknown keyword 'DAQ_IDENTIFIER' in config file
also if you look into the legacy.py, the mappings here seem to be not bullet proof and tested for every driver, I doubt that it will work for every can interface because we see there hard coded mappings to certain hardware vendors, the underlaying hardware can be very different (Vector, SlCan, Kvaser, etc...)
"TSEG1": "Transport.Can.tseg1_abr",
"TSEG2": "Transport.Can.tseg2_abr",
"TTY_BAUDRATE": "Transport.Can.SlCan.ttyBaudrate",
"UNIQUE_HARDWARE_ID": "Transport.Can.Ixxat.unique_hardware_id",
"RX_FIFO_SIZE": "Transport.Can.Ixxat.rx_fifo_size",
"TX_FIFO_SIZE": "Transport.Can.Ixxat.tx_fifo_size",
"DRIVER_MODE": "Transport.Can.Kvaser.driver_mode",
"NO_SAMP": "Transport.Can.Kvaser.no_samp",
"SINGLE_HANDLE": "Transport.Can.Kvaser.single_handle",
"USE_SYSTEM_TIMESTAMP": "Transport.Can.Neovi.use_system_timestamp",
"OVERRIDE_LIBRARY_NAME": "Transport.Can.Neovi.override_library_name",
"BAUDRATE": "Transport.Can.Serial.baudrate",
"SLEEP_AFTER_OPEN": "Transport.Can.SlCan.sleep_after_open",
for example, this code:
can_config_hard_coded = {
"TRANSPORT": "CAN",
"ALIGNMENT": 8,
"TIMEOUT": 3.0,
"CREATE_DAQ_TIMESTAMPS": False,
"DISCONNECT_RESPONSE_OPTIONAL": False,
"LOGLEVEL": "DEBUG",
"CAN_DRIVER": "slcan",
"CAN_USE_DEFAULT_LISTENER": True,
"CHANNEL": "COM10",
"BITRATE": 500000,
"CAN_ID_MASTER": 0x03,
"CAN_ID_SLAVE": 0x04,
"CAN_ID_BROADCAST": 0xF4,
"MAX_DLC_REQUIRED": False,
"DAQ_IDENTIFIER": [0x5, 0x6, 0x7],
}
try:
# 1. Convert legacy dict to nested Config
converted_cfg = convert_config(can_config_hard_coded, logger=self.logger)
self.logger.debug("Converted legacy config to traitlets Config successfully")
# 2. Create XCP Master
self.master = Master("CAN", config=converted_cfg)
self.logger.warning("Created XCP Master with converted config")
leads to
ERROR XCPDriver:xcp_driver.py:276 Error during CAN bus initialization: 'LazyConfigValue' object has no attribute 'disable_error_handling'
The error message is not very informative, and seems to be even somewhere inside the traits library. 😫
The device I'm using has a dedicated port for XCP communication. It is not a problem if pyxcp manages the bus since it is not shared.
Note that the code of my previous message is not working. I basically created an empty configuration and gave it a shot. I added missing parameters based on errors returned by pxycp and/or python-can.
I still need to work on it, but it's not a priority at the moment. I'm still on 0.21. I"m confident I'll figure it out though.
Sure it is possible and I am confident that "I'll figure it out, too"... at some point. The problem is the time spent on it. I thought in 1-2 days the update will be over, now almost a week on it. This is a bit frustrating.
Update: It is not that simple, once you try to directly pass a traits configuration you get ton's of errors. It seems that "create_application()" function parses many things and fills up missing configurations (for example filling up certain parameters with None's in order for Master to accept it). Even all the AI out there is not that helpfull. 😉
If @akermu really made it work, I would really appreciate to apply directly a working solution. 🙏
Update: finally I made it work! 🥳🥳🥳 I am pasting here full working configuration for slcan without using "create_application" for anyone interested.
from pyxcp.config import PyXCP, General, Transport
def create_direct_pyxcp_can(
channel: str = "COM10",
bitrate: int = 500_000,
can_id_master: int = 0x03,
can_id_slave: int = 0x04,
can_id_broadcast: int= 0xF4,
daq_identifier: list = (0x5, 0x6, 0x7),
timeout: float = 3.0,
alignment: int = 8,
disconnect_response_optional: bool = False,
create_daq_timestamps: bool = False,
) -> PyXCP:
"""
Return a PyXCP app pre-configured for CAN-SLCAN.
You can then do:
app = create_direct_pyxcp_can(...)
master = Master(app.transport.layer, config=app)
"""
app = PyXCP()
# replicate what start() would do internally
app.general = General(config=app.config, parent=app)
app.transport = Transport(parent=app)
# General settings
app.general.disconnect_response_optional = disconnect_response_optional
# Transport settings
app.transport.layer = "CAN"
app.transport.timeout = timeout
app.transport.alignment = alignment
app.transport.create_daq_timestamps = create_daq_timestamps
# CAN-interface settings
cancfg = app.transport.can
cancfg.interface = "slcan"
cancfg.channel = channel
cancfg.bitrate = bitrate
cancfg.can_id_master = can_id_master
cancfg.can_id_slave = can_id_slave
cancfg.can_id_broadcast= can_id_broadcast
cancfg.daq_identifier = list(daq_identifier)
return app
Usage:
# use defaults
app = create_direct_pyxcp_can()
master = Master(app.transport.layer, config=app)
@SaxElectronics nice 👍 Your solution is actually cleaner, I just use a dummy config file:
empty_config_file = NamedTemporaryFile(delete=False, suffix=".py")
empty_config_file.close()
config = PyXCP()
config.config_file = empty_config_file.name
config._read_configuration(config.config_file)
Hi guys,
I have a small but important update for you, perhaps usefull for some of you. I have been testing my scripts with direct configuration and also observed an inconcistency while executing in different pyxcp versions. I managed to track this down to this very latest change causing it:
This is a key difference, if you are working without a context managet you need to call transport.connect() before calling master.connect()! Otherwise your code will fail with an error that the transport layer instance instance is not available:
INFO Connecting...
ERROR Test failed: 'PythonCanWrapper' object has no attribute 'can_interface'
INFO Test complete
@christoph2 introduced this fix to enable multiple connecting and disconnecting from the slave in the very same session (example: connect -> disconnect -> connect).
So correct order instantiate (currently) a connection would be:
master.transport.connect()
logger.info("Connected successfully to TRANSPORT LAYER")
response = master.connect()
logger.info(f"Connection successful to SLAVE! Response: {response}")
I think properly defined APIs in the master class with clear error handling would be very helpful for everyone. As the normal user (who is not that deep in to the library) has difficulties understanding the error messages.
- API for connecting and disconnecting to the BUS
- API for connecting and disconnecting to the slave: 1) is (of course) required for 2)
proper error handling can inform the user what he did wrong, for example in case he tries to connect to the slave without a transport layer connection established first.
@fross @akermu if you are interested in the examples there are added here in my fork: https://github.com/SaxElectronics/pyxcp/tree/master/pyxcp/examples
I can try to bring them in the upstream/master if @christoph2 wants them.