openconfig
gNOI for Optic Firmware Installation
https://github.com/openconfig/gnoi/issues/292
@robshakir @ahsaanyousaf @dplore FYI
Pull Request Test Coverage Report for Build 17222849893
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
- For more information on this, see Tracking coverage changes with pull request builds.
- To avoid this issue with future PRs, see these Recommended CI Configurations.
- For a quick fix, rebase this PR at GitHub. Your next report should be accurate.
Details
- 0 of 1151 (0.0%) changed or added relevant lines in 2 files are covered.
- 48 unchanged lines in 1 file lost coverage.
- Overall coverage remained the same at 0.0%
| Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
|---|---|---|---|
| transceiver_firmware_install/transceiver_firmware_install_grpc.pb.go | 0 | 104 | 0.0% |
| transceiver_firmware_install/transceiver_firmware_install.pb.go | 0 | 1047 | 0.0% |
| <!-- | Total: | 0 | 1151 |
| Files with Coverage Reduction | New Missed Lines | % |
|---|---|---|
| bgp/bgp.pb.go | 48 | 0.0% |
| <!-- | Total: | 48 |
| Totals | |
|---|---|
| Change from base Build 16924193294: | 0.0% |
| Covered Lines: | 0 |
| Relevant Lines: | 13593 |
đź’› - Coveralls
Can you include the compiled .pb.go files with this commit?
you'll need to update the regenerate-files.sh script to build make use of the bazel build file here.
Can you include the compiled
.pb.gofiles with this commit? you'll need to update theregenerate-files.shscript to build make use of the bazel build file here.
Sure, just updated.
@dplore based on our discussion about comparing to https://github.com/openconfig/gnoi/pull/160 I think these are different enough that they warrant separate gNOIs. Some initial takeaways/differences:
- Software bundle seems to be focused on software patches and therefore has terminology which might not align well with transceiver installs. For example, un-installing a software bundle doesn’t make much sense in the context of transceiver firmware. Similarly, I'm not sure concepts like correlating processes for affected_processes or processes_pending_restart makes sense for a transceiver.
- There is not a way to specify the list of transceivers needing to be installed.
- For transceivers, there is a concept of staging a firmware package on the Target and then in a second phase being able to install from the Target to a list of transceivers. That doesn't seem like something that could be done in the other PR.
- There is a transceiver banks concept in this PR that isn't supported in the other PR.
Please let me know your thoughts. These were just from an cursory first pass. I might have misinterpreted things, but an initial look feels like they are fairly different.
High level question: although this model is similar to what's being defined in the OIF, it has a number of places it diverges from the latest OIF proposals. I haven't done a full analysis here but here are some examples from a quick look.
Different terminology: OC service “TransceiverFirmwareInstall” vs OIF top level title "Firmware Update" OC RPC “Transfer” vs OIF “download” OC "Target" vs OIF "Host"
Different primitives/commands: OC combines transfers of remote->host and host->transceiver into a single primitive. The OIF has "transfer" vs "download". OC doesn't seem to have a separate "activate" primitive. OIF does.
Note that I'm not necessarily proposing that everything in the current OIF proposal is what we should converge on. Maybe there are changes/updates on both sides to come to alignment.
The latest OIF proposal is here: https://www.oiforum.com/bin/c5i?mid=4&rid=7&gid=0&k1=54575&k2=15&k3=11
@jsterne Thanks for the review. Some thoughts:
TransceiverFirmwareInstall vs Firmware Update.
- This was specifically changed to reflect that upgrade is as important as downgrade. Similar to os.proto where the RPC is called Install. Upgrade|update gives a connotation of going to a higher release, but that is not always the case.
Transfer vs download
- This is another case of aligning to elsewhere in OpenConfig for consistency. I'd probably also loosely argue since this is a streaming RPC, "transfer" is a better fit.
Target vs Host
- This is similar to os.proto where OpenConfig seems to prefer Target and trying to keep this consistency.
OC combines transfers of remote->host and host->transceiver into a single primitive. The OIF has "transfer" vs "download".
- Correct, but it differentiates still via a oneof.
OC doesn't seem to have a separate "activate" primitive. OIF does.
- This is part of the Install RPC. Could you clarify the separate actions you are referring to?
Reviewed in September 4th, 2025 OC Community meeting to increase visibility of this PR. One comment from a few participants is can we look to not duplicate file transfer RPC's? Maybe existing gNOI file transfer can be used and this PR could add RPCs to perform the transceiver firmware installations.
Reviewed in September 4th, 2025 OC Community meeting to increase visibility of this PR. One comment from a few participants is can we look to not duplicate file transfer RPC's? Maybe existing gNOI file transfer can be used and this PR could add RPCs to perform the transceiver firmware installations.
That was initially discussed, but the reason we went this way was:
- We wanted the device to understand that this was firmware package and store in it's own library of firmware versions. We didn't want operators to have to know where to transfer files to.
- OIF is proposing an embedded YAML file within the file that would contain interesting metadata. Having a separate RPC gives the device context to look for this YAML file so it can parse and present this data (i.e. version name, vendor, supported SKUs, etc.)
Some replies inline.
@jsterne Thanks for the review. Some thoughts:
TransceiverFirmwareInstall vs Firmware Update.
- This was specifically changed to reflect that upgrade is as important as downgrade. Similar to os.proto where the RPC is called Install. Upgrade|update gives a connotation of going to a higher release, but that is not always the case.
Here we're talking about the overall Service name. How about TransceiverFirmwareManagement since there are a number of different functions in this service (including the transfer, the activate, aborting, checking firmware versions, etc)?
Transfer vs download
- This is another case of aligning to elsewhere in OpenConfig for consistency. I'd probably also loosely argue since this is a streaming RPC, "transfer" is a better fit.
I haven't studied other transfer type RPCs in gRPC before but is this streaming because you intend to send a chunk of data, then get an ack, then send more, get an ack, etc?
Transfer seems OK as a term. So maybe we see if the OIF is happy to go back to that term (they had it before). At least for the host->module transfer. (I'd like to avoid re-creating yet another server->host transfer RPC when we already have something like that elsewhere for moving files around).
Target vs Host
- This is similar to os.proto where OpenConfig seems to prefer Target and trying to keep this consistency.
In that case there isn't really ambiguity about the "destination" of the binary file. The target is always the router/NE. But here it may be confusing where a host can be a target and so can a module. Target implies the destination of the transfer (not intuitive for it to be the source of the transfer). i.e. "The source of the transfer is the Target" (ughh).
OC combines transfers of remote->host and host->transceiver into a single primitive. The OIF has "transfer" vs "download".
- Correct, but it differentiates still via a oneof.
OC doesn't seem to have a separate "activate" primitive. OIF does.
- This is part of the Install RPC. Could you clarify the separate actions you are referring to? The whole description of the Install RPC is fairly different from the OIF "activate" command(s). The OC model has an option to install from device (device vs target vs host?). The OIF approach seems to be to transfer then switch banks. And the OIF model has a way to not necessarily commit the newly downloaded/active bank (that can be done as a separate step if desired, e.g. after the operator checks that things are working).
Reviewed in September 4th, 2025 OC Community meeting to increase visibility of this PR. One comment from a few participants is can we look to not duplicate file transfer RPC's? Maybe existing gNOI file transfer can be used and this PR could add RPCs to perform the transceiver firmware installations.
That was initially discussed, but the reason we went this way was:
- We wanted the device to understand that this was firmware package and store in it's own library of firmware versions. We didn't want operators to have to know where to transfer files to.
This concept of a magic hard-coded store (with pre-defined file system organization then I suppose?) isn't really in the OIF approach from what I can tell. Why not just let different devices and users decide where in a filesystem they may want to place these files? Also note that the latest OIF "download" command says "Note: the path must be included and result in a unique file". It seems a shame to reinvent file copy and would be nice for that part to be generic. Then other "firmware" commands can be special and know how to parse those files, transfer them to modules, etc.
- OIF is proposing an embedded YAML file within the file that would contain interesting metadata. Having a separate RPC gives the device context to look for this YAML file so it can parse and present this data (i.e. version name, vendor, supported SKUs, etc.)
Sure but that parsing doesn't have to be part of the transfer of a file from a server to the router.
Is there a way to abort a transfer host->module? That would be useful. The OIF defined an 'abort' command (although rather that be called abort-transfer to be specific which activity we're aborting.
Some replies inline.
@jsterne Thanks for the review. Some thoughts: TransceiverFirmwareInstall vs Firmware Update.
- This was specifically changed to reflect that upgrade is as important as downgrade. Similar to os.proto where the RPC is called Install. Upgrade|update gives a connotation of going to a higher release, but that is not always the case.
Here we're talking about the overall Service name. How about TransceiverFirmwareManagement since there are a number of different functions in this service (including the transfer, the activate, aborting, checking firmware versions, etc)?
I am good with "TransceiverFirmwareManagement". We've already changed the name a couple times, so I'll give this a few days unless anyone has objections to this new overall name. Although, any thoughts on just "TransceiverFirmware" for simplicity?
Transfer vs download
- This is another case of aligning to elsewhere in OpenConfig for consistency. I'd probably also loosely argue since this is a streaming RPC, "transfer" is a better fit.
I haven't studied other transfer type RPCs in gRPC before but is this streaming because you intend to send a chunk of data, then get an ack, then send more, get an ack, etc?
Transfer seems OK as a term. So maybe we see if the OIF is happy to go back to that term (they had it before). At least for the host->module transfer. (I'd like to avoid re-creating yet another server->host transfer RPC when we already have something like that elsewhere for moving files around).
Correct in that we'd stream bytes over the wire here. The preference in OpenConfig has been moving towards this and avoiding the many other permutations to transfer (i.e. FTP, SFTP, HTTP, SCP, etc.). AIUI this has been for security reasons (but also potentially performance).
I think it's fine if OIF wants to align terminology, but I really don't think that's a goal of OpenConfig to have identical naming formats with different standards bodies.
Target vs Host
- This is similar to os.proto where OpenConfig seems to prefer Target and trying to keep this consistency.
In that case there isn't really ambiguity about the "destination" of the binary file. The target is always the router/NE. But here it may be confusing where a host can be a target and so can a module. Target implies the destination of the transfer (not intuitive for it to be the source of the transfer). i.e. "The source of the transfer is the Target" (ughh).
OC combines transfers of remote->host and host->transceiver into a single primitive. The OIF has "transfer" vs "download".
- Correct, but it differentiates still via a oneof.
OC doesn't seem to have a separate "activate" primitive. OIF does.
- This is part of the Install RPC. Could you clarify the separate actions you are referring to? The whole description of the Install RPC is fairly different from the OIF "activate" command(s). The OC model has an option to install from device (device vs target vs host?). The OIF approach seems to be to transfer then switch banks. And the OIF model has a way to not necessarily commit the newly downloaded/active bank (that can be done as a separate step if desired, e.g. after the operator checks that things are working).
Reviewed in September 4th, 2025 OC Community meeting to increase visibility of this PR. One comment from a few participants is can we look to not duplicate file transfer RPC's? Maybe existing gNOI file transfer can be used and this PR could add RPCs to perform the transceiver firmware installations.
That was initially discussed, but the reason we went this way was:
- We wanted the device to understand that this was firmware package and store in it's own library of firmware versions. We didn't want operators to have to know where to transfer files to.
This concept of a magic hard-coded store (with pre-defined file system organization then I suppose?) isn't really in the OIF approach from what I can tell. Why not just let different devices and users decide where in a filesystem they may want to place these files? Also note that the latest OIF "download" command says "Note: the path must be included and result in a unique file". It seems a shame to reinvent file copy and would be nice for that part to be generic. Then other "firmware" commands can be special and know how to parse those files, transfer them to modules, etc.
There are a few reasons to this. As operators we don't want to know or care about the file management structure for every platform or if it changes in different software branches. We just want to provide the firmware package and let the device put it in the right place. Then, not in this first phase here, but we had a lot of discussion on the device having a readable library of available firmware packages. With that, the device could give us an inventory or available firmware packages, their versions, which SKU's they work with, etc. With something like that we can get to where we eventually hope to be; specifically being able to define default firmware packages for transceivers. So when a new transceiver is plugged in, it could automatically be determined what firmware package to install. This would put us in a plug-and-play approach that we'd like to eventually get to. We are not pushing all of this initially, but that's the vision we are hoping to move towards.
- OIF is proposing an embedded YAML file within the file that would contain interesting metadata. Having a separate RPC gives the device context to look for this YAML file so it can parse and present this data (i.e. version name, vendor, supported SKUs, etc.)
Sure but that parsing doesn't have to be part of the transfer of a file from a server to the router.
The preference is for this to just to be a single RPC. We'd expect under the hood vendors could re-use shared libraries for transferring files. However, standing up a new gRPC is trivial and I'd argue having this context specific RPC to do some extra context specific things around the file transfer would be valuable from a simplicity point of view for operators.
I am good with "TransceiverFirmwareManagement". We've already changed the name a couple times, so I'll give this a few days unless anyone has objections to this new overall name. Although, any thoughts on just "TransceiverFirmware" for simplicity?
I'd somewhat prefer TransceiverFirmwareManagement. I think that will just match how this would be referred to in other descriptions and in the OIF. I'm also ok with "TransceiverFirmware" but it feels a bit truncated (not sure if other services have similar truncated style names?).
I am good with "TransceiverFirmwareManagement". We've already changed the name a couple times, so I'll give this a few days unless anyone has objections to this new overall name. Although, any thoughts on just "TransceiverFirmware" for simplicity?
I'd somewhat prefer TransceiverFirmwareManagement. I think that will just match how this would be referred to in other descriptions and in the OIF. I'm also ok with "TransceiverFirmware" but it feels a bit truncated (not sure if other services have similar truncated style names?).
gNOI stle for service naming is migrating to a style with a single noun for the entity being "operated" on. This makes some sense in that gNOI is already telling us we are performing "operations" on something. We just need to know what the thing is we are "operating". So transceiverz or transceiver could be good.
I am good with "TransceiverFirmwareManagement". We've already changed the name a couple times, so I'll give this a few days unless anyone has objections to this new overall name. Although, any thoughts on just "TransceiverFirmware" for simplicity?
I'd somewhat prefer TransceiverFirmwareManagement. I think that will just match how this would be referred to in other descriptions and in the OIF. I'm also ok with "TransceiverFirmware" but it feels a bit truncated (not sure if other services have similar truncated style names?).
gNOI stle for service naming is migrating to a style with a single noun for the entity being "operated" on. This makes some sense in that gNOI is already telling us we are performing "operations" on something. We just need to know what the thing is we are "operating". So
transceiverzortransceivercould be good.
I see what you mean. So the "Management" part isn't needed. But I think it would be good to scope this service to firmware. So how about TransceiverFirmware that Eric suggested as a shorter option?
About 'transfer' vs 'download' for moving the file from the router->transceiver: At the moment: A) gnoi proposal = 'transfer' B) OIF proposal = 'download' C) CMIS spec = 'download'
I think the latest thinking is for OIF to change back to using 'transfer'. That's OK with me. It is nice that (A) and (B) use the same name.
I'm uncertain how to map the Install RPC to underlying CMIS primitives. CMIS has the following:
- Download (move binary into the bank of the transceiver (TC))
- Run Firmware Image (i.e. switch to the inactive bank where the new firmware was downloaded)
- Commit (mark the running bank as committed so this bank is used at any subsequent restart)
The Install RPC seems to have some hints of transfer aspects sprinkled around. e.g.:
- the entire InstallFromDeviceRequest
- TransferProgress transfer_progress = 3; as part of the response to Install
The OIF proposal has an 'activate' command that is clearly separated from any transfer/download.
OIF also combines both CMIS Run and Commit (where commit is an optional automatic step of the activate). But I think we should also have a separate Commit primitive in both OIF and OpenConfig. An operator may want to Run the new firmware on a transceiver and verify that everything is working well before doing a Commit.
I'd propose we have an 'Activate' RPC (instead of the Install RPC) that takes the following arguments:
- list of transceivers
- an 'action' that is one of these options (that match the actions available in the Run command in CMIS 5.4):
- Switch to Inactive Bank
- SwitchHitless to Inactive Bank
- Restart Active Bank
- RestartHitless Active Bank
- Factory Reset to Inactive Bank
- Factory Reset to Active Bank
- a no-commit option (otherwise commit the newly active bank by default)
I'm uncertain how to map the Install RPC to underlying CMIS primitives. CMIS has the following:
- Download (move binary into the bank of the transceiver (TC))
- Run Firmware Image (i.e. switch to the inactive bank where the new firmware was downloaded)
- Commit (mark the running bank as committed so this bank is used at any subsequent restart)
The Install RPC seems to have some hints of transfer aspects sprinkled around. e.g.:
- the entire InstallFromDeviceRequest
- TransferProgress transfer_progress = 3; as part of the response to Install
The OIF proposal has an 'activate' command that is clearly separated from any transfer/download.
OIF also combines both CMIS Run and Commit (where commit is an optional automatic step of the activate). But I think we should also have a separate Commit primitive in both OIF and OpenConfig. An operator may want to Run the new firmware on a transceiver and verify that everything is working well before doing a Commit.
I'd propose we have an 'Activate' RPC (instead of the Install RPC) that takes the following arguments:
list of transceivers
an 'action' that is one of these options (that match the actions available in the Run command in CMIS 5.4):
- Switch to Inactive Bank
- SwitchHitless to Inactive Bank
- Restart Active Bank
- RestartHitless Active Bank
- Factory Reset to Inactive Bank
- Factory Reset to Active Bank
a no-commit option (otherwise commit the newly active bank by default)
Made a few changes and had a few comments:
- I changed "Install" to "Activate" as I agree that seems to have better separation from other verbs.
- List of transceivers should already be supported at the layer above as the Activate request takes a repeated list of ActivateSingleRequest.
- We've discussed separating commit and want Activate to automatically commit. So this is probably not a primitive OpenConfig would support. Once a version is running on the transceiver that needs to be the committed version. If someone changes their mind that is just another Activation. However, leaving a device is a quasi-state is worrisome because we don't know the behavior and it's possible for the step to be forgotten. For example, if a transceiver's package is running and not-committed, then forgotten about, if the device has a power outage, what version comes back up? Often it can be the previous version which means something like a power outage on a device takes much much longer to recover since firmware upgrades are now needed.
- We originally called out Switchovers, but it was determined that it was redundant to an Activation, so we took it out for now.
- I'd be interested to learn about the factory reset options. Could the transceiver hold its factory image in a permanent place (not one of its banks)?
I am good with "TransceiverFirmwareManagement". We've already changed the name a couple times, so I'll give this a few days unless anyone has objections to this new overall name. Although, any thoughts on just "TransceiverFirmware" for simplicity?
I'd somewhat prefer TransceiverFirmwareManagement. I think that will just match how this would be referred to in other descriptions and in the OIF. I'm also ok with "TransceiverFirmware" but it feels a bit truncated (not sure if other services have similar truncated style names?).
gNOI stle for service naming is migrating to a style with a single noun for the entity being "operated" on. This makes some sense in that gNOI is already telling us we are performing "operations" on something. We just need to know what the thing is we are "operating". So
transceiverzortransceivercould be good.I see what you mean. So the "Management" part isn't needed. But I think it would be good to scope this service to firmware. So how about TransceiverFirmware that Eric suggested as a shorter option?
@dplore curious on your thoughts of using TransceiverFirmware? Please let us know.
I am good with "TransceiverFirmwareManagement". We've already changed the name a couple times, so I'll give this a few days unless anyone has objections to this new overall name. Although, any thoughts on just "TransceiverFirmware" for simplicity?
I'd somewhat prefer TransceiverFirmwareManagement. I think that will just match how this would be referred to in other descriptions and in the OIF. I'm also ok with "TransceiverFirmware" but it feels a bit truncated (not sure if other services have similar truncated style names?).
gNOI stle for service naming is migrating to a style with a single noun for the entity being "operated" on. This makes some sense in that gNOI is already telling us we are performing "operations" on something. We just need to know what the thing is we are "operating". So
transceiverzortransceivercould be good.I see what you mean. So the "Management" part isn't needed. But I think it would be good to scope this service to firmware. So how about TransceiverFirmware that Eric suggested as a shorter option?
@dplore curious on your thoughts of using TransceiverFirmware? Please let us know.
I do not object to that naming. Here's a question for you that might help finalize this name: Are there other transceiver operations that are on the horizon? Any RPC to force a transceiver restart, factory reset, etc? If yes, the structure could be:
gnoi.Transceiver.FirmwareTransfer, FirmwareActivate, FirmwareVerify, Restart, FactoryReset, etc
If that seems very unlikely or there is a reason not to have so many RPC's in this Service, then TransceiverFirmware as the service name seems good enough.
I see what you mean. So the "Management" part isn't needed. But I think it would be good to scope this service to firmware. So how about TransceiverFirmware that Eric suggested as a shorter option?
@dplore curious on your thoughts of using TransceiverFirmware? Please let us know.
I do not object to that naming. Here's a question for you that might help finalize this name: Are there other transceiver operations that are on the horizon? Any RPC to force a transceiver restart, factory reset, etc? If yes, the structure could be:
gnoi.Transceiver.FirmwareTransfer, FirmwareActivate, FirmwareVerify, Restart, FactoryReset, etc
If that seems very unlikely or there is a reason not to have so many RPC's in this Service, then
TransceiverFirmwareas the service name seems good enough.
I can imagine other non-firmware operations in the future. If we want that all to be part of a single service (and we probably would) because otherwise what's the name of the other transceiver service that doesn't include firmware?), then perhaps Daren's proposal here is good.
3. We've discussed separating commit and want Activate to automatically commit. So this is probably not a primitive OpenConfig would support. Once a version is running on the transceiver that needs to be the committed version. If someone changes their mind that is just another Activation. However, leaving a device is a quasi-state is worrisome because we don't know the behavior and it's possible for the step to be forgotten. For example, if a transceiver's package is running and not-committed, then forgotten about, if the device has a power outage, what version comes back up? Often it can be the previous version which means something like a power outage on a device takes much much longer to recover since firmware upgrades are now needed.
I think that's fine to have Activate automatically do a commit by default. But we should have the option to no-commit (as per the OIF proposal). Some operators will want to Activate (i.e. switch the inactive bank that was just updated using the transfer command), but then do some external testing before they use the CMIS "Commit Image" command to mark that new image as the one to use at next reboot. If they are in the middle of testing and there is a power outage, then they would indeed want to reboot using the previous/old image (that was still the committed image/bank). The inactive bank would still contain the new image and they could Activate it again and do their validation. Once that is done, they would issue a "Commit Image" command (which we should add to this service).
- We originally called out Switchovers, but it was determined that it was redundant to an Activation, so we took it out for now.
The CMIS command for "activation" is called "Run" and it has 6 options (the last 2 will be new in CMIS 5.4):
- Switch to Inactive Bank
- SwitchHitless to Inactive Bank
- Restart Active Bank
- RestartHitless Active Bank
- Factory Reset to Inactive Bank
- Factory Reset to Active Bank
We need to decide if our new service has:
- an RPC with the same scope as the CMIS "Run" command (i.e. could eventually support all 6 of those options), or
- an RPC with only a subset of the CMIS Run command (i.e. only the first 2 items).
This is related to the earlier discussion here: https://github.com/openconfig/gnoi/pull/293#discussion_r2279704625
If we want to eventually have RPCs that cover all the CMIS "Run" options do we want that all in a single RPC with the 6 options, or do we want a set of RPCs, each of which does a subset of the CMIS "Run" command, e.g.: RPC SwitchToInactiveBank (with some parameter about the Hitless vs non-hitless, and a no-commit option) RPC RestartActiveBank RPC FactoryReset