Tech Stack Review Process

[chrispdriscoll]

Proposal

Vela Committers - This proposal intends to facilitate discussion around the creation of a new Issue template related to Tech Stack proposals. I'm looking for feedback (feedforward) to refine the categories and questions we think are necessary pre-work ahead of proposing changes to the technology used in Vela. Upon agreement, I'll create the new template for use in future proposals.

• Chris

Description

The goal of this template is to help guide decisions around the Vela tech stack. The original stack was implemented with quality discussions and considerations so future changes to the stack should also be made with similar due diligence

Process

The following steps describe the high-level categories of consideration as new tech is proposed into the Vela architecture. The questions can be refined and more can be added with the intent of providing as much information and context as possible ahead of a proposal.

1. Consensus – do we all agree from the start? If so, let’s document the reasoning across all the following factors but avoid the hassle of meeting to discuss something we all already agree
2. End of Life – do we need to migrate away from a technology due to an announced retirement or end-of-development/maintenance? a. If yes, the focus of this effort should be on identifying the best fit from all the new proposals b. If no, the focus of this effort should be to decide between staying on the existing stack vs. the new/proposed stack
3. Security a. Are vulnerabilities being found with no patches or long delays before they are patched? b. Does an underlying vulnerability exist that cannot/willnot be addressed and we deem it critical to address? c. Does this new tech reposition the product’s security posture in a dramatically manner? d. NOTE: Vulnerabilities should be considered as severity vs. exploitability
4. Maintainability / Stability / Scalability / Simplification a. Are we satisfied with the existing release / maintenance schedule provided by the vendor? b. Does the new/proposed solution reduce our infrastructure or streamline processes? If so, quantify c. Does the new/proposed solution improve performance and/or position us for greater capabilities in the future? Are there already plans on the roadmap to take advantage of those capabilities? (I’m trying to differentiate between changing the stack for perceived benefits as opposed to benefits we can take advantage with efforts already on our roadmap). Do your best to quantify d. What is the net-new effort required to maintain any change on an ongoing basis? Any increase should come with larger benefits in other areas
5. Migration Effort a. What is the time / effort involved in completing this migration? b. How many people would need to be involved? How long (duration) would they need to be engaged? How much effort (capacity) would be needed to be committed by these people? c. Will new testing or monitoring capabilities need to be developed in parallel to this change?
6. Team dynamics a. Does this change better align with corporate standards? This is a tricky question because, if no, we wouldn’t pursue the new tech – but, if yes, it doesn’t mean it would immediately be approved b. Do we have the existing talent / experience? Or would training be required? What is that effort? c. If we lost 2-3 engineers, could we easily hire individuals with the old/new talent?
7. Cost / Financial a. Is there a cost component to this change? If so, quantify b. Do we already have a relationship with the new vendor or would a new relationship need to be developed? c. NOTE: This should be focused on the technology cost (time and effort would be represented in a separate category)

Additional Considerations

Consider adding a SWOT analysis to better depict the current vs. new stack

Consensus

If we still cannot get consensus, then we may need to:

1. Keep things as-is until additional benefits are identified or
2. Go with the desires of the team maintaining the project (this seems like a very self-serving caveat because it will always benefit Target)

[KellyMerrick]

I wonder where the best place to have these process conversations is: community repo, discussions, or here as an issue?

3. c. Does this new tech reposition the product’s security posture in a dramatically manner?

• Was this meant to say "dramatically different manner," or something along those lines?
• Are all questions in this section intended to be "yes"/"no" in this section? Answering "yes" to this one could actually be a good thing, if it dramatically enhances security. Or does "yes" just lead to a discussion point?

5. Migration Effort

• Do we want to clarify "How would or could the work be distributed between Target and external?"

6. Team Dynamics

• Is this Target specific?

As a whole, I like it!

[wass3rw3rk]

6a. - define "corporate standard" 5. Migration - should we call out documentation separately? ie. who will create/maintain onboarding, contributing docs for new stack

Do we want to clarify "How would or could the work be distributed between Target and external?"

One additional modification is to change "external" to "proposal owner" maybe? I don't think we can have an expectation about any external folks taking up the work that's being proposed.

[wass3rw3rk]

6c. - "If we lost 2-3 engineers", without context these numbers don't have a whole lot of meaning, should we generalize this in some way?

[wass3rw3rk]

another thing to add - and it might require some additional work, but what if we added something along the lines of "does the proposal support the goals set out by the project". just for example - let's say one of the goals is to be less dependent on the NPM ecosystem. does the proposal support that goal? it would require us to formulate such goals with as much specificity as possible.