zephyr How not to break "out of tree" users

Provide some guarantees, guidelines and a process keeping out of tree users operational while the zephyr project code advances with new technologies, code cleanups and other major code and API changes.

Out of tree users are not limited to only drivers, we have users with their own subsystems, architectures, toolchains, SoCs, boards, drivers, driver subsystems etc. Any change in zephyr might break such users if changes are not following a deprecation process, announcements and a grace period given (deprecation period in many cases) to those users to adapt to the new interfaces or upstream code.

The process should find the sweet spot which allows the project to advance with its agenda and roadmap while allowing users to adapt to change.

Aug 10 '22 17:08 nashif

I think part of the question here is what is considered part of the Zephyr "interface" beyond APIs? Is the build system, Kconfig, devicetree/devicetree bindings? Each of these areas could break an 'out of tree' user due to a change.

Aug 17 '22 15:08 galak

I think part of the question here is what is considered part of the Zephyr "interface" beyond APIs? Is the build system, Kconfig, devicetree/devicetree bindings? Each of these areas could break an 'out of tree' user due to a change.

IMO, all of the above.

if we decide for whatever reason to drop, lets say a cmake macro (example, zephyr_library_sources_ifdef), out of tree code using this macro will break. Same with kconfig and devicetree. The level of severity if the breakage might vary and we will agree that not everything we support needs deprecation, however, we need to be aware and cautious about changes in general. The fact that removing any interface in a PR passes CI is not a green light it can be removed without any consequences. We probably need to introduce some categories of changes that needs more attention than others.

Aug 19 '22 19:08 nashif

Various thoughts/comments:

So for devicetree this means that any change to an existing binding would possibly be a breaking API change.
How do we track the stability of cmake, Kconfig, devicetree bindings?

Aug 22 '22 17:08 galak

Process WG:

Start the discussion by tackling APIs, with agreement that we need to consider other programming interfaces like Kconfig symbols and devicetree bindings, and keep them in the vision
Make include/zephyr contain only public, user-facing APIs, move internal code out of there (new use of the treewide process!) -- definition of API tbd
Make sure all APIs that remain have documentation and changes against them are checked, so no breaking changes to stable APIs, etc.
Continue discussion in the issue, revisit next week

@nashif this relates to treewide changes, but need to have some definition for what an API is and how not to break them. There are lots of interfaces that can break out of tree users.
@nashif e.g. if we have a board that gets dropped from upstream, but someone else was using a board with the same SoC, how to decide whether to delete the .dtsi?
@nashif or what about APIs (header files, function calls, defines)?
@mbolivar-nordic I would like to see a specific directory that contains the official APIs, with CI checks that make sure everything is documented, and changes being checked against the API stability level
@nashif agreed, I want to start with APIs first too
@gregshue at HP, everything was versioned. We would only make breaking changes with new versions that coexisted for a time in the tree until the removal of the deprecated API. Same with board definitions, cmake, etc. Downstream users were notified. A problem with Zephyr is we are not describing how downstream users should adapt to changes.
@nashif we did that with TCP/IP and logging in zephyr. We have the same proposal from @jfischer-no for USB.
@mbolivar-nordic this discussion should only apply to stable APIs IMO
@carlescufi the way we define "API" right now is "do we have to change your application's code?"
@keith-zephyr want to ask about the process for deprecating an API. Current process of deprecating a macro generates build warnings. In our downstream code base, warnings are promoted to errors, so that breaks our downstream / forces us to disable warning promotion
@carlescufi could you disable that particular warning instead? We have the same problem in our downstream; we have disabled particular warnings in the past
@keith-zephyr is that available for devicetree also?
@carlescufi There are a few sources of warnings: 1. compiler warnings (see above), 2. cmake warnings: we don't treat these as warnings; pretty sure Zephyr doesn't either. E.g. CMake warns on assert().
@keith-zephyr if you disable the warning, do you lose visiblity that something was deprecated?
@carlescufi I don't know; don't remember
@mmahadevan108 so cmake warnings don't break CI?
@carlescufi pretty sure no, because we build all the time with asserts enabled
@mbolivar-nordic is there some process change you want, @keith-zephyr ?
@keith-zephyr ideally a way to make deprecations print informational notices, not warnings. Becomes a source of tech debt to disable warnings
@mbolivar-nordic is there a way to make __DEPRECATED_MACRO 's behavior change with a Kconfig option to not emit a warning?
@carlescufi need to ask @tejlmand
@keith-zephyr if it's possible to make it informational, changing to a warning or error later on would make sense, but starting out informational would be a nicer start
@carlescufi there are also now kconfig warnings, handled in python since https://github.com/zephyrproject-rtos/zephyr/pull/47835/
@nashif the pip style of printing a message that doesn't prevent you from doing your work when a new version is available is nice too
@nashif a Kconfig option is a bad idea since you'd have to change the source code
@carlescufi west build ... -- -DCONFIG_WARNINGS_ARE_INFORMATIONAL=y is an alternative
@nashif an environment variable would only have to be set once, not for every build
@carlescufi we should create an issue about consistency in deprecation and get @tejlmand to come up with a proposal
@keith-zephyr sounds great; I can open the issue
@gregshue want to make sure we handle the question of what defines an API

Aug 24 '22 16:08 mbolivar-nordic

@carlescufi mentioned this, but does API mean application facing or broader?

Aug 24 '22 17:08 galak

broader, for example the arch_ interface which is not application facing needs to be in scope as well, this is being used by OOT architectures for example. There are a few other interfaces beside that to consider.

Aug 25 '22 13:08 nashif

My five cents: I'd limit this to stable public APIs. Other changes should just be written in the release notes. In any case, if a deprecation is cheap, just do it, your users will appreciate it.

Working out of tree is convenient, but it comes with a maintenance cost.

Good read: https://www.kernel.org/doc/html/latest/process/stable-api-nonsense.html#what-to-do

Aug 25 '22 13:08 gmarull

@mmahadevan108 so cmake warnings don't break CI? @carlescufi pretty sure no, because we build all the time with asserts enabled

FWIW I saw these two warnings every day for about a year and none ever stopped anything:

warning: the int symbol CORE_COUNT (defined at src/platform/Kconfig:299) has a
non-int default MP_NUM_CPUS (undefined)

CMake Warning (dev) at CMakeLists.txt:12 (zephyr_library_include_directories):
  uninitialized variable 'sof_module'
This warning is for project developers.  Use -Wno-dev to suppress it.

Aug 25 '22 17:08 marc-hb

broader, for example the arch_ interface which is not application facing needs to be in scope as well, this is being used by OOT architectures for example. There are a few other interfaces beside that to consider.

Agreed. Downstream users may need to add their own module with architectures, SOC definitions, drivers, boards, subsystems, tests, samples, etc. It may even need to contain alternate implementations of existing subsystems. Perhaps the collective set of "public APIs" needs to include whatever could be seen or replaced across modules.

So, what does it mean for something to be "application facing", especially if the product-unique source is only an empty main(){}?

Working out of tree is convenient, but it comes with a maintenance cost.

Working out of tree is strategic and supported. It is also essential for some of us and for an extensible platform.

Aug 25 '22 19:08 gregshue

Process WG:

@gregshue include/zephyr -- what is the point of the zephyr name?
@dleach02 I think we're only talking about what is directly inside of zephyr
@stephanosio we aren't going to change all modules
@gregshue my question is what policy are we recommending to avoid conflicts?
@keith-zephyr all the zephyr headers were moved under that directory to avoid that problem
@gregshue what wasn't clear is why the name zephyr was chosen
@fabiobaltieri what would be the alternative to zephyr?
@gregshue mcuboot has a native zephyr port; you're expecting the headers it defines in the include/zephyr namespace?
@mbolivar-nordic no; mcuboot's library interface (bootutil) is not a zephyr library, and its zephyr port is an application, not a library
@mbolivar-nordic I think we're saying that #include <zephyr/foo...> is something we'd like to reserve for APIs defined by the zephyr project
@mbolivar-nordic are we talking about module/module collisions or zephyr/module collisions?
@gregshue both
@mbolivar-nordic to some extent I think the first one is not our problem
@stephanosio we should make sure modules in zephyr/west.yml don't conflict. We can at least recommend that module-specific public headers are prefixed with module-name/. We can't enforce this.
@carlescufi I don't object to adding that recommendation to the modules documentation
@gregshue part of these recommendations should be to fill in the module name in the module.yml file
AI @gregshue to send a module documentation PR with those recommendations
@mbolivar-nordic my question is how/whether we can make sure that changes to stable APIs aren't breaking changes? Can we automate this somehow?
@keith-zephyr tagging the unit test files?
@stephanosio we have a label for "stable API change", we just don't automatically label it. Maybe we can trigger a label automation adding it to certain files. Could potentially add the test cases to a 'stable api tests' area in the maintainers file.
@dleach02 or we could put an SPDX tag in the file
@stephanosio in that case we'd have to scan everything. If we want to maintain a "stable" API set, it would be nice to have a database. If we can have a single file -- like MAINTAINERS -- with all the stable APIs, we have the database.
@MaureenHelm that's an odd indirection -- why not do that at the API level? We could do something at the doxygen tagging level. The declaration of whether an API is stable or not belongs with the API. We should be able to scrape the tree and get this table. The golden source of truth should be with the API itself.
@mbolivar-nordic I really agree; this also would make it a lot easier to maintain the "version changed" table
@gregshue how do we detect when APIs are changed
@mbolivar-nordic if we have the metadata in doxygen, with 'git log' and a script
@gregshue having trouble understanding the phrase 'we are testing the API'? My question is about 'testing', not 'modifying'. E.g. for safety, there are 4 ways to verify requirements: test, inspection, analysis, and one other. Testing I2C is always done against an implementation, not the API itself.
@dleach02 test cases are at least a starting point to make sure that changes don't break something. To the point of trying to figure out at each PR if stable APIs are not being changed in an incompatible way, we're trying to move towards automation. Inspection would imply eyes on it, so automation highlighting a PR would make the inspection easier.
@stephanosio so marking APIs for inspection, when they change
@gregshue the other possibility is writing test cases for the API. You end up with a mock of the API and a test of the API that fully uses all the definitions, without having to depend on any implementation. I'm not sure it would make sure every definition gets exercised.
@stephanosio I think here we should establish the process for how we test changes, but the actual testing strategy should be discussed in the testing WG.
@dleach02 question being asked here is since we've defined a stable API, how do we ensure it's still stable?
@MaureenHelm I was thinking about all the discussions we've had about "is-isr-safe". We had custom infrastructure to parse that stuff in our documentation generation.
@dleach02 so step 1 either way would be adding this metadata for PRs to flag
@gregshue what about whether a stable API is not the same version between different LTS versions? Semantic versioning?
@mbolivar-nordic can we try to get a resolution on whether @MaureenHelm 's approach is viable by me presenting at testing WG?
[consensus on that]
@mbolivar-nordic what are we looking for out of semantic versions of APIs? Don't think we can sell semantic versioning of APIs project wide
@fabiobaltieri I'm not really sure the 'version introduced' and 'version modified' columns in the api overview (https://docs.zephyrproject.org/latest/develop/api/overview.html) are really useful. Maybe we want something else.
@gregshue I like the question of 'scope'. I want to be able to write a module that works on multiple zephyr forks.
@carlescufi I personally don't think we should go there.
@gregshue if Zephyr is going to say we're not going to support the following use case, we should be explicit: As a downstream module owner, I need to be able to have one version of my module that builds and integrates with zephyrproject-rtos/zephyr's version of the ecosystem, nrfconnect's, and possibly other versions.
@nashif there shouldn't be an expectation that any upstream is going to support you integrating with other downstreams
@gregshue is the zephyr project going to put patterns in place to support this use case? I want to be able to identify what the fork is, what the version of the fork is, and get clues in each API about what the semantic version is.
@carlescufi I agree with @fabiobaltieri these columns don't add much value and should be removed

Aug 31 '22 17:08 mbolivar-nordic

My feedback offline as I was not able to join the major part of the meeting

@gregshue what wasn't clear is why the name zephyr was chosen

@fabiobaltieri what would be the alternative to zephyr?

@gregshue mcuboot has a native zephyr port; you're expecting the headers it defines in the include/zephyr namespace?

@mbolivar-nordic no; mcuboot's library interface (bootutil) is not a zephyr library, and its zephyr port is an application, not a library

@mbolivar-nordic I think we're saying that #include <zephyr/foo...> is something we'd like to reserve for APIs defined by the zephyr project

@mbolivar-nordic are we talking about module/module collisions or zephyr/module collisions?

@gregshue both

@gregshue It is really frustrsating to see this type of discussion always go into modules/mcuboot and things that you might be passionate about, derailing the discussion from the actual topic.

@MaureenHelm that's an odd indirection -- why not do that at the API level? We could do something at the doxygen tagging level. The declaration of whether an API is stable or not belongs with the API. We should be able to scrape the tree and get this table. The golden source of truth should be with the API itself.

+1

@mbolivar-nordic what are we looking for out of semantic versions of APIs? Don't think we can sell semantic versioning of APIs project wide

I think this would be a very good replacement to how we manage API stability right now using when some API was introduced/modified. Having a versioning scheme in place will help with making changes to stable APIs and marking those changes as non-breaking using semantic versioning. By just looking at the version it will be possible to see if you are still compatible without having to look at git logs or implementation changes in drivers. Tests would also need to continue working. IMO it is worth looking into and bringing it up as a proposal and get more feedback.

@fabiobaltieri I'm not really sure the 'version introduced' and 'version modified' columns in the api overview (docs.zephyrproject.org/latest/develop/api/overview.html) are really useful. Maybe we want something else.

yes, maybe drop this in favor of some versioning scheme maintained within the API using doxygen like @MaureenHelm suggested.

@gregshue I like the question of 'scope'. I want to be able to write a module that works on multiple zephyr forks.

the only thing of significance here is Zephyr project and its code base, any forks of zephyr are completely irrelevant to this discussion.

@gregshue if Zephyr is going to say we're not going to support the following use case, we should be explicit: As a downstream module owner, I need to be able to have one version of my module that builds and integrates with zephyrproject-rtos/zephyr's version of the ecosystem, nrfconnect's, and possibly other versions.

This is implicit and obvious and does not require any statements. We as the zephyr project are not responsible for content maintained in forks of Zephyr.

@gregshue is the zephyr project going to put patterns in place to support this use case? I want to be able to identify what the fork is, what the version of the fork is, and get clues in each API about what the semantic version is.

again, I am not sure why we are talking about forks. This is a distraction from the actual topic. Zephyr has 1000s of forks, why do we want this?

Sep 01 '22 13:09 nashif

It is really frustrsating to see this type of discussion always go into modules/mcuboot

@nashif It is also really frustrating to see the Zephyr Project not actually support the needs of users trying to complying with a development models it claims to support. module.yml provides an extension of the build system. As an integrator of multiple modules were I cannot control the consolidation or separation of the modules, I need everything done at a higher layer to be independent of which module the source exists in, unless it is scoped to apply to a specific directory subtree (e.g., .clang-format).

derailing the discussion from the actual topic.

As an "out-of-tree" user I am trying to rescope this discussion to meet my needs related to breakage. I'm sorry you think of it as derailing. Perhaps you need to clarify which users you are not trying to address.

Zephyr has 1000s of forks, why do we want this?

I'll assume most of the forks are tracking Zephyr main. I know some long term forks are not, and introducing incompatibilities. Not having a common way to identify the latter is the problem faced by end users. This is the same type of problem that led protocol specs to identify a field (or value range) for vendor-extension commands. If Zephyr Project defines a common mechanism for forks to be identified then end users can avoid conflicting identification solutions invented by each of the fork maintainers.

Sep 01 '22 19:09 gregshue

I believe this effort is trying to help people identify API and other incompatible changes between say upstream Zephyr version 42 and upstream Zephyr 45. If you spot one specific place/tool or identification technique that does not help with zephyr forks version 43-gregshue and 44-marc-hb, then offer a more flexible alternative there when discussing implementation details. If you can find such an alternative, chances are it will be better for upstream Zephyr branches too (cause branching and forking are the same thing). If you cannot find such an alternative, then the problem couldn't be solved anyway and no one wasted any time in abstract discussions.

All this without mentioning forks once! Magic :-)

We reject: kings, presidents, and voting. We believe in: rough consensus and running code.

Sep 01 '22 20:09 marc-hb

All this without mentioning forks once! Magic :-)

Almost ... "that does not help with zephyr forks" ... "cause branching and forking"

This is a present user need, not an abstract discussion. Other users independent of me have already indicated on Discord they are building one set of source reusable on both zephyrproject-rtos and nrfconnect ecosystems.

I am not concerned with the forks tracking Zephyr main. I am concerned about identifying interface changes introduced in the nrfConnect fork of Zephyr ecosystem. It would be less of a concern if nrfConnect maintained backwards compatibility at the SHA level, but it didn't.

One solution is to mix an identifier into the API semantic numbering indicating the organization defining the interface.

We reject: kings, presidents, and voting. We believe in: rough consensus and running code.

I hate to tell ya, but that ain't gonna cut it for certifiable code. ;-)

Sep 01 '22 20:09 gregshue

This is a present user need, not an abstract discussion

I was referring to the (lack of) solutions, not to the problem.

I am concerned about identifying interface changes introduced in the nrfConnect fork of Zephyr ecosystem.

What makes you think some (good and useful) API change(s) in nrfConnect won't be found in some future upstream Zephyr version? Remember: forking and branching are the same thing.

I hate to tell ya, but that ain't gonna cut it for certifiable code. ;-)

Off-topic again?

Sep 01 '22 20:09 marc-hb

What makes you think some (good and useful) API change(s) in nrfConnect won't be found in some future upstream Zephyr version?

I never thought that. Rather, I thought that an API change upstreamed into Zephyr would now be managed by Zephyr rather than nrfConnect. I hope nRFConnect would then deprecate/remove their implementation and align with the upstream (just like I do with my local patches to Zephyr when I integrate with a fixed version of Zephyr).

I also know that nrfConnect has rewritten Git history, so this fork isn't really the same as a branch.

Off-topic again?

Not really, and definitely on the topic of not breaking "out of tree" users. API definitions are specifications that will need to be traced back to requirements for certifiable executables. They not runnable code. (An inline implementation is distinct from the specification.)

Sep 01 '22 21:09 gregshue

I never thought that.

OK then why would API changes in nrfConnect not be manageable using the same processes and tools as API changes across upstream Zephyr branches? Considering these processes and tools don't exist yet, it sounds like you're complaining about a problem that does not exist yet.

I also know that nrfConnect has rewritten Git history, so this fork isn't really the same as a branch.

Pretty sure doxygen does not care about git history. If some other solution or tool ever relies on git history then it will be time to highlight this and discuss pros and cons.

The long story short is that Zephyr has a virtually infinite number of forks so a blanket and super vague request to "support forks" cannot possibly make sense. Only specific requests make sense; for instance: "Can this solution/tool be made compatible with rewritten git histories, pretty please?"

PS: making 1000s of random Zephyr forks "certifiable" sounds... fun! Whatever that means.

Sep 01 '22 22:09 marc-hb

OK then why would API changes in nrfConnect not be manageable using the same processes and tools as API changes across upstream Zephyr branches?

nrfConnect could use the same processes as API changes across upstream Zephyr branches - but it cannot assign different meanings to the same version identifiers unless some other mechanism exists to tell them apart. I look to the Zephyr Project to specify one mechanism for all forks to use.

In order for downstream developers to create a module that works with either upstream Zephyr or an incompatible nrfConnect interface must be able to know at build time which interface definition to call.

it sounds like you're complaining about a problem that does not exist yet.

It exists already. I've just had bigger issues to tackle.

Sep 01 '22 22:09 gregshue

Hi everyone, I'm also thinking about solutions to those problems you describe (but don't worry I won't be interfering much in your opinion exchange as I don't have much knowledge about processes in Zephyr). We should think about solutions to all tools/interfaces Zephyr has one by one. I'll start with Kconfig because it seems an easier problem that other ones.

Kconfig maintenance proposition

The problem:

Out of tree Zephyr application sets value to defined in Zephyr CONFIG_EXAMPLE_ZEPHYR_DRIVER=y.
Zephyr project renames CONFIG_EXAMPLE_ZEPHYR_DRIVER to CONFIG_DAI_EXAMPLE_DRIVER.
Out of tree Zephyr application can adjust itself during rebase to next zephyr revision but we also wish to somehow test compatibility from Zephyr perspective and introduce changes to end-users fluently.

Solution: Using Kconfig aliases and obsolete warnings generation using https://www.kernel.org/doc/html/latest/kbuild/kconfig-macro-language.html#built-in-functions $(warning-if,condition,text) function. Example:

$(warning-if,$(EXAMPLE_ZEPHYR_DRIVER ),Kocnfig option EXAMPLE_ZEPHYR_DRIVER is obsolete, please use DAI_EXAMPLE_DRIVER)
config EXAMPLE_ZEPHYR_DRIVER
    default n
    select DAI_EXAMPLE_DRIVER

Result: Out of tree application will receive a nice Kconfig warning that value that config value they set is obsolete... The only painful thing is monitoring on Zephyr side when to remove each of those obsolete variables (here we need robust process solution). The same goes for Devicetree, there is aliases feature (however I do not see any option of printing obsoletion messages here)...

Sep 01 '22 23:09 aborisovich

but it cannot assign different meanings to the same version identifiers unless some other mechanism exists to tell them apart

Right, different forks and branches must use different identifiers to signal that they are indeed different versions. Not breaking new ground.

I look to the Zephyr Project to specify one mechanism for all forks to use.

An upstream project cannot anticipate all the potentially crazy ways it will be forked and create a versioning scheme that will be compatible with everything and anything. You can offer and recommend a "fork-friendly" versioning scheme; that seems reasonable. Can't wait to see your research and proposition.

Sep 02 '22 00:09 marc-hb

@nashif It is also really frustrating to see the Zephyr Project not actually support the needs of users trying to complying with a development models it claims to support.

Funny, I wonder what this issue is about and why it was created in the first place. And what are those development models you are referring to exactly? Please be specific

module.yml provides an extension of the build system. As an integrator of multiple modules were I cannot control the consolidation or separation of the modules, I need everything done at a higher layer to be independent of which module the source exists in, unless it is scoped to apply to a specific directory subtree (e.g., .clang-format).

This is the most vague description of a problem I have seen in a while. I am not sure what are you asking for.

If your module has drivers, boards and anything that is supported out of tree in zephyr and you are interested in keeping those working with zephyr, then this issue is for you. If i understand the above correctly and you are asking us to make your module work with upstream zephyr and other forks the same way, then you are in the wrong place, this is not something we have ever promised, not something that we are interested in and to be honest a very strange request/expectation.

As an "out-of-tree" user I am trying to rescope this discussion to meet my needs related to breakage. I'm sorry you think of it as derailing. Perhaps you need to clarify which users you are not trying to address.

See above. If that is not clear, then I am not sure how else I would be able to clarify it.

Thats it from me. I have spent way too much time on this already.

Sep 02 '22 01:09 nashif

And what are those development models you are referring to exactly? Please be specific

In the T2: Star topology, application is the manifest repository, I am strategically reusing the repositories from zephyrproject-rtos, extending forks of other OSS projects to also be Zephyr modules, and putting licensed source into separate modules from my proprietary "applications"/boards/drivers/subsystems/tests/etc. All of my extended/proprietary repositories have the Zephyr glue in the module itself, kept in a Zephyr directory structure in the module-level zephyr/ subdirectory (next to module.yml. This relocation is necessary on some repositories due to name collisions with preexisting subdirectories. Other than the location of the Zephyr directory structure this follows the pattern in Zephyr Project's example-application module.

IIRC, the Zephyr documentation does not indicate any difference in support between the topologies or with developing as a Zephyr repository application, so I expect all the capabilities/tools/etc that work for a Zephyr repository application to also work with a Zephyr workspace application module. Maintaining this support requires all issues related to the Zephyr repository explicitly consider how the issue also may apply to any code in modules. Many (most?) issues will be independent of the module degree-of-freedom. Many issues will be impacted by the module degree-of-freedom.

It is really frustrsating to see this type of discussion always go into modules/mcuboot

I am not sure what are you asking for.

Fundamentally, I am asking the TSC Chair in particular, and voting members/maintainers/collaborators in general, to internalize that:

Modules are a supported mechanism for organizing Zephyr workspaces. Technical discussions MUST provide a solution that also applies to Zephyr-specific source in them.
The Zephyr Project already has created and owns an application module downstream of the Zephyr repository that is frequently recommended on the Discord channels as a pattern for adding proprietary content into a workspace.
Not explicitly considering how an issue or PR is impacted by the module degree-of-freedom is inconsistent with the support Zephyr Project documents.

I shouldn't have to repeatedly raise the unavoidable question about the impact of modules. But apparently I do because raising the question itself is causing frustrations rather than being accepted as necessary consideration.

Sep 02 '22 14:09 gregshue

Technical discussions MUST provide a solution that also applies to Zephyr-specific source in them

Then participate in these discussions and provide very specific, technical solutions that address your problems if/when possible. This is just getting started.

But apparently I do because raising the question itself is causing frustrations rather than being accepted as necessary consideration.

I suspect the frustration does not come from any particular topic but from the extend, vagueness and verbosity of the requests combined with the expectation level. e.g.: "MUST" support forks/modules without any technical detail. A fork and a module can be literally anything. Be specific: describe something that does not work and how it can be fixed. Except you can't yet because there's no solution yet.

Sep 02 '22 15:09 marc-hb

Be specific: describe something does not work and how it can be fixed.

Maintaining support for the topologies cannot depend on any single person being a watchdog on all issues. This is not my job full time. I have been engaged as I have time. (See Global Namespace Management? and Replacing zephyr driver/subsys implementations and Support module.yml in zephyr repo.)

If i understand the above correctly and you are asking us to make your module work with upstream zephyr and other forks the same way

I am not asking for you to make my module work with upstream Zephyr and other forks the same way.

Sep 02 '22 18:09 gregshue

Solution: Using Kconfig aliases and obsolete warnings generation using https://www.kernel.org/doc/html/latest/kbuild/kconfig-macro-language.html#built-in-functions $(warning-if,condition,text) function.

This works reasonably well for flagging deprecated symbols when users go from one release to the next. We also need a solution that works for users that go from one LTS to the next.

As Google members identified, I think a bigger need is to tell out-of-tree developers how to transform their code (and verify the transformation) when a rebase is attempted (which may be from one LTS to the next).

Sep 02 '22 18:09 gregshue

Be specific: describe something that does not work and how it can be fixed.

Maintaining support for the topologies cannot depend on any single person being a watchdog on all issues. This is not my job full time.

This is "only" an open-source project: as long as you're the only one who cares about some feature or request getting done then it is your job full time. Explaining and convincing others may help (exceptional communication skills required).

Sep 02 '22 19:09 marc-hb

Process WG: defer until next week since @nashif could not attend today

Sep 21 '22 16:09 mbolivar-nordic

I shouldn't have to repeatedly raise the unavoidable question about the impact of modules. But apparently I do because raising the question itself is causing frustrations rather than being accepted as necessary consideration.

@gregshue my opinion is that you are missing the point here.

Basically all of us work on out of tree modules and we do care about them; please accept that.

I think what people are trying to tell you is that your attempts to rescope this issue are unwelcome distractions from what we are trying to do first. As has previously been stated we are trying to tackle one thing at a time, and that's not modules or mcuboot. I'm going to try to refocus this discussion in the next meeting in half an hour.

Sep 28 '22 15:09 mbolivar-nordic

Provide some guarantees, guidelines and a process keeping out of tree users operational

Perhaps we need to clearly describe the range of out of tree users we are trying to keep operational. I assumed it included end users doing freestanding and workspace application modules as well as downstream module developers. Are they not part of the scope trying to be addressed?

Sep 28 '22 15:09 gregshue

Are they not part of the scope trying to be addressed?

I think it depends what code they write. Yes for users who follow the guidelines TO BE DEFINED (that's the entire purpose of this issue, read its description again). Others, probably not.

Types of users will be defined by which rules they follow (the rules do not exist yet)

Perhaps we need to clearly describe the range of out of tree users we are trying to keep operational.

A formal definition of this range will be the conclusion of this work, not the starting point.

So no one can tell yet where your personal, current use case(s) will be. Terrifying? That's what this issue wants to address in the future.

Sep 28 '22 16:09 marc-hb

zephyr zephyr copied to clipboard

How not to break "out of tree" users

Kconfig maintenance proposition

zephyr
zephyr copied to clipboard