Issues for March Release

[bapivee]

Define BoW Basic for Laminate by March - work our way outwards

Issues, 28/14 - VDD - converge on 0.75 Issues 34/35 - wiring study - Ken Poulton from Keysight, Halil from Facebook Issues 22/23/10/6 - Initialization and calibration - Mark/Ramin Issues 26 - Clocking - Ken

[kuemerle]

Issues, 28/14 - VDD - converge on 0.75 - Continued concern about older technologies - How about requiring a supported range of 0.75 to 0.80 nominal for all applications? - No major strenuous objections - How does this look in relation to HBI discussions?
- Carrie to check with her people - Allow operation at lower voltages when both sides support it - Should there be a floor and what should it be? - Supporting 0.5, 0.4 V (HBI3) would give better voltage alignment with Open HBI (Open HBI2 is 1.2V) - Look for potential commonality, maybe 0.8 could be used for future interoperation - Maybe leave future versions Issues 34/35 - wiring study - Ken Poulton from Keysight, Halil from Facebook

• Key Concerns were preferred to non-preferred wiring, will the bump orderings specified cause any issue - Halil's SI person to meet Ken & keysight folks - align on what's needed and who's doing what
• What about # bits per clock - should we keep it 16, move it to being divisible by 10 for PIPE, should it be higher to improve bandwidth per wire?
  • 8 bits out of 10 of PIPE are used, maybe its not so good to have extra bits
  • Keep 2Gbps for interoperability, continue to support 5Gbps for max rate Issues 22/23/10/6 - Initialization and calibration - What other contributors would like to collaborate?
  • Volunteers greatly appreciated Issues 26 - Clocking - Ken - Thoughts on specification, other folks interested in collaborating? - Copy AIB specification for master/slave and improve name to leader/follower - Incorporate Ken's feedback on issue 37, Bapi to update the objectives

[kuemerle]

Should we also spec a maximum electrical length along with Ken's feedback on issue 37?

[kuemerle]

Feedback from Carrie on default required Voltage range of 0.75 - 0.8V Nominal voltage (some tolerance around each side) - Feedback says ok, clarify voltage/datarate is not connected. Desire to ensure new technologies can operate at these voltages -

should it be added to our goals?

Feedback from team on floor voltage of 0.4 V (in case someone wants to go there)

• maybe no benefit to determining floor?
• Planning for termination?
• electricals need to be aligned to allow for low voltage - lets drop it for now
• Document that lower or higher voltages are allowed, no limit set as long as 0.75 -0.8 nominal is supported.

Halil is working on staffing, Ken has some feedback on bump map

Ken's valuable feedback -

• lets put CNTL (if needed) and FEC in the regular databits.
• Use 1 bump map, do not alternate for stacked configs
• Ken to return with a proposed bump pattern combo of option 2 and 3
  • Ken to check and see what can be shared on routing study.

How about one common mode for a whole pile of pins?

[kuemerle]

Proposed revised bump order using MapA and MapB

18 bumps, 16 Data + FEC and CNTL Move Pwr/GND away from chipedge Want to be explicit that power and gnd should not be dictated, it should be left to the designer Greg's suggestion for separating off the CNTL bit - interest in breaking off the control bit more like AIB (use it as a bit of a side-band) - Mark to draw something for next week.

One Suggestion to add FEC in band

[kuemerle]

Ken presented followup charts for the discussion: Naming - Stack has been consistently used for a stack of small units, we'll call a small unit a slice Numbering - start with 0 like everything else in the world Power bumps in example - helpful with package routing, add a paragraph on power agnosticism in the specification - and show multiple examples

Lots of discussion on TX, RX patterns of slices and stacks - current interest is a checkerboard pattern, it would make asymmetric links horribly inefficient. Could we require checkerboard, let folks implement bidir for asymmetric communication.

Wiring study - great results from Ken.

Next week lets hash out the TX/RC pin map and control interface.

[kenpoulton]

Bapi, I give you permission to post the slides BoW_general_2020-02-04... to the ODSA or github sites. Thanks

[kuemerle]

Finished up with Ken's presentation - and a new question - wire vs. lane, dediced we'd call it a wire. General agreement that redundancy is not needed.

Question to OSATs: How much would adding redundancy improve yield?

Proposal to dictate either unidirectional blocks, checkerboards or bidirectional IO? Most developers would likely choose bidirectional, but it would not be forced.

Proposal to leave control block up to the user - another option is to have no embedded cntl, use JTAG to set up BoW and let it run. Can we nix it and config with JTAG using out of band handshakes to have a training pattern with a time zero indicator?

[bapivee]

Carrie - even if we allow Tx/Rx bumps, need to point out that you have to adhere to signal ordering.

[bapivee]

Suresh Subramaniam from Apex presented BoW studies.

Suresh proposal - minimize power/ground loops and impact on margin by having power/ground close to the signal bumps.

Channel length limited to 2-3 mm for basic designs.

Signal order out of PHY is more important than the actual bump map itself. Within that constraint, allow any bump maps.

Need to capture bump maps relative chip edge.

Will there be ESD requirements? Reduced, but exist. Impact capacitor.

[kuemerle]

Discussion on pin ordering - the group agreed that pin ordering is the only thing that needs to be maintained - update to spec would show pin ordering with multiple power and ground solutions, also Suresh's proposed pin ordering, and show that all options are valid.

Overshoot becomes a serious problem for long traces - Suresh's study @ 0.7V enforces the need for a channel limit for unterminated BoW.

Ken's analysis with a 10mm trace shows some marginality @ 5Gbps (1.0 V r2r) Adding 2:1 mismatched driver degrades the eye even more...

Ken's proposal is to spec the impedance of the driver and wire, maybe 8 mm makes sense...

Lots of discussion on extra bit - many folks interested in FEC, many others don't want to burn the beachfront, need to find a path - scheduled as second topic next week.

Writing the spec - including the updates - first topic for next week.

[kuemerle]

Discussion for today:

FEC Bit, spec update and BiDirectional Support (Ken's suggestion) Halil on FEC - don't necessarily need 2 clocks, need simple gearbox, want to see gearbox - proposal to over-provision datarate like ethernet, keep clock rate on user side For other purposes adding 2 bits could be a good fit. Ken proposes special pair one on the left and one on the right Optional FEC would need to be config'd statically Bapi added that moving to a non power of 2 will ensure we have to wait another cycle. Suggestions that we could just reduce chipside to add FEC. JSET participation next week along with Bidi from.

[kuemerle]

Great presentation from Nokibul from JCET

Discussion on Bidi - discussion on need for test and configurability.

Should we have a light internal loopback or full external loopback?

Try to run it to a decision next wed.

[bapivee]

VDD - Mark Bump map - Ken Loopback - Carrie (pending what to emulate) Initialization/Calibration - Suresh Clocking - need a volunteer Transmit/Receive - Halil

[kuemerle]

Discussion on Bidirectional links concerns: extra circuits, extra area and yield, capacitance. If we use loopback for test only, we need to build known good pattern to verify a link is working Most folks like loopback for test in the poll, close with David & Co next week Mark to write VDD for next week

[bapivee]

[bapivee]

Provide a functional at-speed (at bump wire rate) loopback capability for test for both receive and transmit covering as much of the complete datapath to/from the I/O bumps as is practically possible in the implementation.

Agenda: 4/8/20 Approve Mark's text for the Vdd

Ken: Bump map pictures and text Suresh: simulation results Suresh's initialization/calibration Shahab: Inter-chiplet loopback Bidi: reference suggestion or mandate - finalize text

[bapivee]

Limit requirement to:

Intra-die loopback Provide a functional at-speed (at bump wire rate) loopback capability for test for both receive and transmit covering as much of the complete datapath to/from the I/O bumps as is practically possible in the implementation.

Need to specify inter-chiplet loopback for interoperability.

Comments (not to be in the spec): How do we handle asymmetric links. Specifying where it happens can impact future spec growth for speed. Loopback does not also impact interoperability?

Agenda: 4/8

[kuemerle]

Review voltage text proposal - We should add additional voltages to electrical spec portion of the spec. Mark to update spec to specify it is supply voltage, Ken Poulton looking to see if 0.75 would work for older technologies.

Interoperability is defined around the minimum specifications (2Gbps @ 1 Ghz DDR)

Discussion that implementer is free to use any power supply range, as long as signal voltages are defined and supported.

Recommendation to maximize inter-operability - we recommend a voltage (0.75?) to support, but also to support a wide range of supplies.

Should we increase the required datarate? No, keep the spec easy, the market will drive to the maximum.

Ken to add pull request, team to review before next meeting.

[kuemerle]

Ken presented bump map proposals - action to rewrite slice limit to provide it as an example, not a limit

Good discussion to use redundancy for the AUX pin, need to define how it would be used

Ken to add more example bump patterns

Should we add some microbump options in example bump map?

Suresh to talk about initialization and calibration, simulation in next meeting

Leader/Follower - should we bring it back? different clock sources drive skipwords and flow control on protocol level...

[kuemerle]

Agreed on updated objectives for spec (Bapi)

Suresh on initialization

Lots of details to close - how do we want to do setup?

JTAG, SPI, I2C?

Desire to have one initialization block per 'link' vs per slice, designer gets to decide how many slices per link they want to implement.

Phase align clocking of each link, hit each slice at the same-ish time.

Should we use something like mac_rdy per link or should we have an open drain signal that says everyone is awake and alive and ready for data?

Open drain 'I'm ready' signal per link plus a serial interface per link is an interesting proposal.

could we reduce to just a 2 wire per link to include readiness.

Greg to present next week on serial initialization concept with or without open drain 'ready'

[kuemerle]

Discussion on initialization and calibration:

mac_rdy signal - should we look at an aux signal version of mac_rdy? Intel updating AIB to next generation - what will it look like? Will AIB make sense to talk to?

Lots of gravitation to a using SPI/JTAG/I2C, do we choose one? Would the system controller be on module or off module?

Should we have just one interface like SPI, or JTAG, or I2C?

Concern that we're forcing an initialization method for every chiplet.

For next week, lets discuss what initialization really looks like, and align that we'd have some external set up (JTAG, SPI, I2C).

[kuemerle]

Suresh presented initialization, proposed using a register map that aligns with register map for AIB, but use reserved bits for power on reset complete and pertner detect.

AIB defines stages of calibration between sideband interface, how do we do accomplish with BoW?

AIB - 'Calibration' is defined as just DLL locking, essentially.

actual Training essentially happens at link layer in AIB and is undefined

Do we want a sideband implementation like AIB?

Dedicated status per interface is desired as a config bit, single bit slow sideband may be interesting.

Next week everyone should bring their favorite register bit to add to config registers.

[kuemerle]

Discussion for 5/20

Lots of discussion about wire ordering for 180 degree rotation, do we require 2 slices with incrementing and decrementing bit ordering? Do we require the slice to handle re-ordering data-buses?

Also some good discussion on initialization, unidirectional and bi-directional. Next week Shahab agreed to pick some of his favorite PRBS.

[bapivee]

Halil: Any interest in differential signaling.

Still in flight. Number of bump map/physical orientation for simple wire ordering. Schemes need to consider link diversity in width and symmetry

Define a minimally capable receiver for unidirectional testing. PRBS needs to be a part of the spec. PRBS pattern constraints - package deltas may limit usefulness, eliminate correlation, consider intentional digital skewing, could be useful for link training.

source termination - can we achieve BoW basic w/o source termination to move from paper to spec, (a) need to discuss deltas and close on them; (b) choose whether this needs to be a requirement or an example. Each contributor evaluates this 6/3.

[kuemerle]

Ken presented approaches:

1. Define N S E W orientations with ordering Possible to do within slice re-ordering, but would need slice to slice re-ordering at link level
2. Define ascending TX, descending RX, assumption is that somewhere one of these would need to be swapped.

Next week we start with harvesting the paper for spec updates, then get back to looking at swapping and bit ordering

[kuemerle]

Following up on opposing ordering (clockwise and counterclockwise)

Pipe adapter meeting - some issues came up in tricking the controller that its using a d2d phy... want to emulate the usual behavior of a SERDES to trick it into using a phy.

1. Use out of band notification that the device is alive, if we implement a shutdown state
2. Implement an in-band (something like mac_rdy) that would do something similar Single bit could be used to operate as an interrupt to to external controller to notify out of band control system.

Suresh presented followon to physical arrangement, with Ken's additions on checkerboarding with polarity of TX, RX.

Lock down spec items as defined in Bapi's comment

[kuemerle]

1. Bump map - Suresh/Ken
2. Signal integrity - Suresh/Ken
3. Loop back test - Shahab
4. VDD - Mark
5. Virtual wire - Mark

[kuemerle]

Lots of good discussion on ordering - everyone to go back and make sure the polarity doesn't break your use case.

Targeting closing above issues for spec release co-incident with HotI.

[kuemerle]

Paper next week open spec items Dave on July 15 Suresh 22nd on Prototype