Purpose

The purpose of this issue to track overall progress on the storage system in the Carthage release.

┆Issue is synchronized with this Asana task by Unito

Jun 13 '22 12:06 bedeho

Meeting Date

Monday, June 14th, 2022.

Participants

Bedeho
Martin
Mokhtar
Ignazio
Luis
Zeeshan
Ondrej
Klaudiusz
Theo
Dmitry

Absent:

Leszek

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage and label:carthage-low-prio

Weekly Summaries

Runtime: Mokhtar progressing with Substrate v3 upgrades. Almost all pallets migrating. Issue tracked here:

#3875

Storage Node: no summary first week

Distributor Node: no summary first week

CLI: no summary first week

QN: Zeeshan completed Events mapping:

https://github.com/Joystream/joystream/issues/3888;

Ondrej completed checking if Hydra can work with v3 :

https://github.com/Joystream/joystream/issues/3883 and created a testing repo : link

Pioneer: Fees implementation scope agreed and tracked here:

https://github.com/Joystream/pioneer/issues/2531; and here - https://github.com/Joystream/pioneer/issues/2898;

Atlas: Fees implementation scope agreed and tracked here:

https://github.com/Joystream/atlas/issues/2815; and here:
https://github.com/Joystream/atlas/issues/2277

Agenda

The goal was to try to figure out the scope and critical path for Carthage release

Conclusions

Here is a brief summary of where we were:

Decision made not to focus on Audit comments in pursuit to de-risk the release.
Content Pallet Benchmarking is in progress with Luis. Timeline is to be refined
Bounties v2 module - supersedes v1 with Carthage release. For Carthage v2 bounties are only in scope for Runtime. FE apps and QN will be addressing v2 bounties after mainnet, meaning Bounties will be deprecated and unavailable in Pioneer.
Network Branches management (Merges to Master which has Rhodes now): Bounty Merge -> Ephesus to Audit3 merge -> Benchmarked Content Directory merge -> Substrate Merge
Playground Env: Node can be built ✅ ; QN currently is not compiling due to types lib errors ⚠️ **metadata differences create need to upd mappings.
Scripts to create test data not working 🔴 which would prevent FE apps from being tested with new runtime
Types lib is a pre-requisite for Atlas to test the changes, decision made to progress wo runtime playground and test when its available.
- https://github.com/Joystream/joystream/issues/3428 is done, but this also entails Creator Tokens extrinsic updates done in - https://github.com/Joystream/joystream/issues/3780 needs testing. Draft PR is opened pointing at Audit3 - testing this is a first step towards having Carthage Playground.
Disabled bounties Merged asap and proceed with Critical path (covered below)
Carthage scope CLI goes to Leszek
Channel Swapping removed out of scope
Minting limits - Remains in scope. Tasks to handle this on QN and Atlas need to be re-opened [❓ ⚠️ this point requires clarification from @ bedeho]
Ephesus only runtime in scope for Carthage. Any proposals not in Rhodes must be disabled by @ Ignazio
- https://github.com/Joystream/joystream/issues/3644 is closed [Minting limits to be moved out of this ticket ❓⚠️ this point requires clarification from @ bedeho]
Pioneer agreed to display fees on progressive modals on every step. Fees displayed in BDH, while staking and rewards in JOY. Design implications are consistent footer through all modals. Dev circa 3 days. Ticket link: - https://github.com/Joystream/pioneer/issues/2898. Denominations to be added making sure numbers are not used and no arithmetics is done - https://github.com/Joystream/pioneer/issues/2531
Question of balancing denomination decimals; weight to fees exchange rate; dependency on total supply and Fiat denomination of fees remains open. Decision was made to experiment with exchange rate and decimals based on testing an extreme range of transactions and evaluating the UX in FE apps. (Funds transfer / Runtime Upgrade/ HD video upload (atlas))
Address Format for Joystream - https://github.com/Joystream/joystream/issues/181 removed from scope. Carthage is removed and Urgent-post-mainnet label remains in all tickets originating from 181

Critical path

Runtime: Ignazio -> fixes types Bedeho -> Review and merge Zeeshan -> enable QN testing Mokhtar to merge disabled bounties asap (13th June) Ignazio fixes bounty types MVP *(after merge) Luis to take on Pull #3485 and aim to replace bounties module for Carthage branch Ignazio to review PR to go to remove blog #3872

QN Zeeshan -> ETA of mappings in 1-2 days; QN buildable; after deletion will be done [❓⚠️ this point requires clarification from @ bedeho] Ondrej -> Content category system - #3574 Theo -> quick help with mappings on external resource identifiers from Ondrej, but to take care of it otherwise [this is not in scope for Carthage] Screenshot 2022-06-14 at 12 41 15

CLI Leszek should take care of it

Atlas and Pioneer

Design and dev starts on fees introduction to all modals and JOY denomination Atlas -> In need of scripts to be working to test basic operations like content upload

Todo Now

@ Luis to inform the group on content pallet benchmarking duration - https://github.com/Joystream/joystream/issues/3221
@ Mokhtar to check if types lib is not preventing the build
@ Ignazio to take on task/ pr of checking and updating types lib (NB: this is already part of Ephesus- Audit3 merge but needs to be tested, so Ignazio needs to go back and make sure it works). ETA - 2 days.
@ Bedeho to review corresponding PR of https://github.com/Joystream/joystream/issues/3780
@ Leszek to test 3780 after PR reviewed by Bedeho
@ Iganzio Remove all proposals not in scope of Carthage

Next meeting

Recurring meetings Carthage all-hands held on Mondays and Wednesdays 10:30 CET.

Jun 14 '22 04:06 dmtrjsg

Notes

Exceptionally solid notes 🤯 , just phenomenal.
Lets do meeting Mondays+Wednesdays, same time.

Clarifications

Minting limits - Remains in scope. Tasks to handle this on QN and Atlas need to be re-opened [❓ ⚠️ this point requires clarification from @ bedeho] [Minting limits to be moved out of this ticket ❓⚠️ this point requires clarification from @ bedeho]

There are at least to three distinct parts to this question of NFT limits in the runtime:

can the council update global limits: UpdateGlobalNftLimitsProposal
can the lead update local limits on each channels: update_global_nft_limit
regardless of whether limits can be updated or not, should they be in place and have an effect constraining actual issuing: check_nft_limits

We should turn them all off, I have reflected this here: https://github.com/Joystream/joystream/issues/3882

Zeeshan -> ETA of mappings in 1-2 days; QN buildable; after deletion will be done [❓⚠️ this point requires clarification from @ bedeho]

Not sure what the question is here. QN buildability does not depend on supporting deletions per say, it is about updating mappings to reflect latest event signatures as far as I know. Obviously the types library has to be updated to reflect the runtime as well.

Jun 14 '22 18:06 bedeho

Meeting Date

Wednesday, June 15th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Luis
Zeeshan
Ondrej
Klaudiusz
Theo
Dmitry

Absent:

Leszek
Martin

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage and label:carthage-low-prio

⚠️ Some issues [potentially] were missing due to Asana-Github syncing outage; while some were marked as completed erroneously. Until next meeting pls refer to both Asana and Github for accurate representation of priorities.

Weekly Summaries

Scope Update Freeze all limits removed from scope. Cannot use, cannot purchase, issuing NFT not sensitive to limits QN don't care, Atals don't care for mainnet, so as a result of above removed from Carthage and Mainnet.

#3882 NFTDestroyed is in Rhodes - block this feature, no QN support reuquired. Label added post-mainnet

Runtime: Previously announced milestone for 16th Working local Node and QN -> unlikely but not impossible.

@ Mokhtar - continues with v3 substrate, nearing completion. ETA on new env is end of this week (optimistic), start of next week (realistic).

Ephesus -> Audit3 merge: complications of stale Audit3 branch caused overwhelming number of changes and conflicts. Decision was made to proceed with merge and handle the consequences as alternative scenarios do not seem to be feasible. [❓ ⚠️ Bedeho to confirm, as opening statement that we were "skipping it", not clear if for the meeting or as a task of focus for now]

@ Luis - pause on content pallet; already merged disabled Bounties v2;

[ ] @ Bedeho to approve/ review
#3485 Due date on content pallet benchmarking - 22nd June.

Next tasks for Luis: review cross-pallet calls:

https://github.com/Joystream/joystream/issues/3423; Founding members programme for membership pallet:
https://github.com/Joystream/joystream/issues/1737

@ Ignazio Fix bounty types ✅ done not on latest branch, need Bounty branch to be merged to Audit3.

Freezing runtime features : - https://github.com/Joystream/joystream/issues/3882 pushed back to next week ⏭️ in favour of new tasks (listed below)

Next focus on:

#3657 Decision made that Carthage label <> Carthage branch, so merge will be pointed at Ephesus branch. Ticket description remains valid.
and
https://github.com/Joystream/joystream/issues/3688. Both would be covered by single PR

@ Bedeho is focussed on planning and reviewing issues:

Storage Node: no summary first week

Distributor Node: no summary first week

CLI: Awaiting Leszek (next week)

QN: Zeeshan Working on deletion, ETA is 1 or 2 days

https://github.com/Joystream/joystream/issues/3870 Making sure QN builds - not started, due date is 18th june
https://github.com/Joystream/joystream/issues/3895

Ondrej Working on metaprotocol contend directory, ETA today - decision was made to pause ⏸️, Keep as draft PR. channel cats -> removed, never needed by Atlas; video cats -> removed from runtime. Andy to focus on trying to keep schema the same to minimise work for Atlas team 🤞

https://github.com/Joystream/joystream/issues/3574

Next: Channel agent permissions

https://github.com/Joystream/joystream/issues/3866

Pioneer: Fees implementation scope agreed and tracked here:

https://github.com/Joystream/pioneer/issues/3240
https://github.com/Joystream/pioneer/issues/2898;
https://github.com/Joystream/pioneer/issues/2531;

Bounties deprecation:

https://github.com/Joystream/pioneer/issues/3114

Atlas: Fees implementation scope agreed and tracked here (🆕 fees are to be displayed in JOY, with <0.1 JOY for small fees. Smaller denomination will be displayed in tooltip. TBC as designs are now available for review:

https://github.com/Joystream/atlas/issues/2815;
https://github.com/Joystream/atlas/issues/2277

Atlas designs will impact Pioneer designs. Scope of changes for existing pioneer component is TBC.

Conclusions

❗ Based on progress and remaining work, the Carthage release seem to be less risky and scope-heavy as initially anticipated, leaving some room to implement main-net focussed features, driven by FE apps demands. This point is however to be confirmed after Pioneer Regression testing is completed, subj to env availability.

This work will be undertaking by Builders WG.

Ticket to track:

https://github.com/Joystream/pioneer/issues/3217

Todo Now

[ ] @ All - review to-do from last meeting and critical path sections and mark respective boxes as completed.
[ ] @ Ignazio to elaborate on the points from this tasks fao @ Bedeho; that do not seem solvable (or nothing to be done there). 2 tasks are now done from this ticket: - https://github.com/Joystream/joystream/issues/3856
[ ] @ Bedeho to review #3485 Draft PR from Luis
[ ] @ Mokhtar - progress with runtime part of new environment, namely to Complete Substrate v3 and orchestrate sequential branches merging
[ ] @ Luis proceed with Content Pallet Benchmarking
[ ] @ Zeeshan finish deletion and make sure that QN builds
[ ] @ Ondrej proceed with Content Categories and Channel Agent Permissions
[ ] @ Klaudiusz [Atlas team] to review fees and proceed with implementation
[ ] @ Theo [Pioneer team] to wait till fees are signed off on Atlas to upd final designs, while already working on introducing fees as a separate storybook component.

To-do [last meetings]:

⚠️ superseded by section "To-do Now" above

[ ] @ Luis to inform the group on content pallet benchmarking duration - https://github.com/Joystream/joystream/issues/3221
[ ] @ Mokhtar to check if types lib is not preventing the build
[ ] @ Ignazio to take on task/ pr of checking and updating types lib (NB: this is already part of Ephesus- Audit3 merge but needs to be tested, so Ignazio needs to go back and make sure it works). ETA - 2 days.
[ ] @ Bedeho to review corresponding PR of https://github.com/Joystream/joystream/issues/3780
[ ] @ Leszek to test 3780 after PR reviewed by Bedeho
[ ] @ Iganzio Remove all proposals not in scope of Carthage

Critical path

Runtime:

[ ] Ignazio -> fixes types
[ ] Bedeho -> Review and merge
[ ] Zeeshan -> enable QN testing 🛠️
[ ] Mokhtar to merge disabled bounties asap (13th June) 🛠️
[ ] Ignazio fixes bounty types MVP *(after merge) 🛠️
[ ] Luis to take on Pull #3485 and aim to replace bounties module for Carthage branch
[ ] Ignazio to review PR to go to remove blog #3872

QN

[ ] Zeeshan -> ETA of mappings in 1-2 days; QN buildable; after deletion will be done
[ ] Ondrej -> Content category system - #3574
[ ] Theo -> quick help with mappings on external resource identifiers from Ondrej, but to take care of it otherwise [this is not in scope for Carthage]

CLI

[ ] Leszek should take care of it

Atlas

[ ] Introduce fees
[ ] Atlas -> In need of scripts to be working to test basic operations like content upload

Pioneer

[ ] Update fees component
[ ] Put placeholder screen on deprecated bounties
[ ] Perform Regression Testing

Next meeting

Recurring meetings Carthage all-hands held on Mondays and Wednesdays 10:30 CET.

Jun 15 '22 13:06 dmtrjsg

Meeting Date

Monday, June 20th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Zeeshan
Ondrej
Klaudiusz
Theo
Dmitry

Absent:

Martin

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

ℹ️ Asana-github syncing was fixed.

Weekly Summaries

Goal of getting runtime where Atlas, Pioneer, Argos and Colossus can be updated. ⚠️ New timeline: Node that has audit work done w new features disabled is going to be operational later this week. Original timeline of end of last week now extended.

#3875 - "all pallets migrated, except bounty which is commented out. Only tests left, tests can be migrated after, it would be ok"
⚠️ we may get Carthage w/o all content migrated till after Benchmarks are merged
Content Pallet is a challenge for refactoring due to file structure. Separate task created (for Mokhtar and Ignazio)

ℹ️ Carthage will be Substrate_v3, based on Audit3; now we keep merging to Audit3 ℹ️

Action: ⚠️ @ all - make sure all PRs are labelled correctly, assign "carthage", and correct pallet label (or project-tokens for Ephesus work)

Separate zoom call will be done on #3823

🆕 Milestone added - Carthage Merged, end of this week.

Scope Change

➖

#3920 - closed on the meeting, not needed
#3905 - closed, right on the meeting

➕

#3872 - carthage label added
#3833 - carthage label added
#3821 - carthage label added
#3924 - replacement for #3920

Runtime:

@ Mokhtar

#3875 - "all pallets migrated, except bounty which is commented out. Only tests left, tests can be migrated after, it would be ok" Focus on Content Directory conflicts with Iganzio

@ Ignazio

Runtime features freezing - done, in review now

Merge Master to Audit3 Focus on Content Directory conflicts with Mokhtar

#3924 - assigned to @ Ignazio, to be done before "pair programming"

@ Luis

2 PRs on CD video state bloat bond and channel state bload bond are ready. We need to merge before Mokhtar pulls it in. Can be merged now as code is ok (by Ignazio), only cosmetics left.

@ Leszek is back.

To review PRs from Luis and add to Audit3. To review the Merge that Ignazio picked. Merge to upgraded Master + Bounties v2 + Remove Blog pallet. To create issue on Withdrawal to Budget / Personal Acc to be done with same extrinsic

#3901

@ Bedeho

to add category to Asana for open PRs.

Runtime-PRs:

#3908 - done, targeted at Ephesus branch
#3905 - closed, right on the meeting
#3871 - base upd to Audit3, carthage label added
#3832 - already in code base
#3914 - should be merged
#3872 - carthage label added
#3833 - carthage label added, audit3 removed
#3920 - closed on the meeting, not needed
#3875 - "all pallets migrated, except bounty which is commented out. Only tests left, tests can be migrated after, it would be ok". Carthage will be Substrate_v3, based on Audit3; now we keep merging to Audit3. Audit3 is new master, need Audit3 merged to Master.
#3823 Separate zoom call will be done on this one
#3817 - targeted at Audit3, needs to be merged

discussed the following group of PRs separately

@Lezek123 to review this group before "pair programming"

#3809 - carthage and storage-pallet labels added
#3804 - carthage and storage-pallet and content-pallet labels added
#3671 - base updated to Audit3, can be merged
#3821 - carthage label added
#3833 - carthage label added, audit3 removed
#3485 - can be merged later
#3920 - closed, but needs to be repeated!
#3924 - Replacement ticket for 3920. Assigned to @ Ignazio, to be done before "pair programming"

QN

Can build for Audit3 branch, but recent changes listed above would extend this task Ondrej and Zeeshan to progress with tasks from last meeting

Atlas and Pioneer

Left undiscussed, awaiting working QN for testing while working on implementation of fees.

To-do

⚠️ @ all - make sure all PRs are labelled correctly, assign "carthage", and correct pallet label (or project-tokens for Ephesus work) @ Bedeho - to add new category to Asana - PRs to cater for merges dependencies. @ Leszek - Review of PRs list 3809-3924; Create task on using same extrinsic for two diff balance transfers; @ Mokhtar - to work on Content Directory with Ignazio @ Ignazio - to work on Content Directory with Mokhtar @ Luis - continues benchmarking

Jun 20 '22 10:06 dmtrjsg

Meeting Date

Wednesday, June 22th, 2022.

Participants

Bedeho
Mokhtar
Leszek
Luis
Zeeshan
Ondrej
Klaudiusz
Theo

Absent:

Martin
Dmitry
Ignazio

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Weekly Summaries

We discovered that Theo and Radek will both be away in period 24.06-11.07, hence we are cutting Pioneer scope in Carthage further to also not have any vesting releated treatment. This will have to be done for mainnet only. Testing should still try to verify manually that vesting works as expected, updated QA scope issue: https://github.com/Joystream/joystream/issues/3885
Availability of key people was added to the timeline to give greater visibility into such possible problems in the future.
Removed all QN issues relating to channel payouts, as they are no longer in scope, as channel payout proposal is being blocked: https://github.com/Joystream/joystream/issues/3935. Added ephesus label to issue for future work. The only remaining issue from this scope was relating to supporting plain old creator withdrawals from the channel account, as that is still needed: https://github.com/Joystream/joystream/issues/3615.
Luis will start work right away to pull in mokhtar carthage-substrate-v3, and retarget carthage branch, which has the partial work on substrate upgrade and audit3 work. He will be blocked later by the fact that tests there are still not working, but he can pull in those fixes later when mokhar is done.
We created many new issues relating to updating the version of Polkadot.js we are using, which is likely the case due to our update of Substrate. This has implications for all code bases, and Leszek is starting preliminary work to map out possible implications
- Types: https://github.com/Joystream/joystream/issues/3939
- QN+Hydra: https://github.com/Joystream/joystream/issues/3940
- CLI+Colossus+Argus: https://github.com/Joystream/joystream/issues/3941
- Pioneer: https://github.com/Joystream/pioneer/issues/3272
- Atlas: https://github.com/Joystream/atlas/issues/2898
Both Theo and Klaudiusz are still waiting on a working local node and QN.
Earliest time for a working local node and QN is likely Friday, as Leszek & Ignazio must provide review, and Mokhtar must must address it.

Jun 22 '22 11:06 bedeho

Meeting Date

Friday, June 24th, 2022.

Participants

Bedeho
Mokhtar
Leszek
Luis
Zeeshan
Ondrej
Klaudiusz
Theo

Absent:

Martin
Dmitry
Ignazio

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Scope Change

➕ New tasks:

Runtime: ix all clippy warnings for Substrate as a separate PR follow up to unblock other teams
QN: Curator Moderation actions

➖ Removed tasks:

Update types (in its original scope) due to the changes discovered by Leszek, described below:

🔴 Major complication

Weekly Summaries

Runtime

Node is buildable, but requires update wrt types library changes introduced with Substrate3. Types is a blocker for buildable QN, and this cascades to blocking upstream apps Atlas and Pioneer do not need the running node to start applying adjustments There are 2 components to it: Mappings and Hydra / Indexer, with the latter to be fixed before mappings Changes Required: 1 - Types naming changed 2 - Type Aliases are no longer part of metadata; Verdict - automated types library is concluded to be less work and more robust compared to changing custom joystream types lib (also implication on the API, which defaults to auto-generated).

⚠️ New immediate target

Next milestone - to get the changes of types introduced asap.

For this:

1️⃣ @ Leszek to introduce new types library to upstream codebase to make membership / channel creation work 2️⃣ @ Klaudiusz and @ Theo to do changes to Atlas and Pioneer Respectively 3️⃣ @ Leszek makes routines to separate library (to a new one) 4️⃣ @ Ondrej Hydra type gen picked up by (Leszek skips QN) 5️⃣ @ Leszek to fix all the mappings and all the imports

@ Mokhtar opicks whatever makes most sense while Leszek is working on new types lib @ Ignazio #3809 go through all prs and fix them @ Luis Continues with bloat bonds for channel and videos (ETA for channels is end of day 24th)

QN @ Zeeshan - continues with deletion of channels and making sure QN builds @ Ondrej - continues with channel agent permissions

Jun 24 '22 15:06 dmtrjsg

Meeting Date

Monday, June 27th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Zeeshan
Klaudiusz
Dmitry

Absent:

Martin
Theo
Ondrej

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Scope Change

➕ New tasks:

@ Leszek to update typegen in Hydra, he will create a respective issue for tracking

#3970 - opened today, to get benchmarks to build and generate weights

➖ Removed tasks:

Fees (merged with work on Audit3)
#3893 closed, not relevant
Rhodes upd2 - closed

⏭️ Postponed tasks

#3840 fix from Audit report -> not urgent pushed back
Faucet upd - moved to next week
Bounties v2

Weekly Summaries

Key Topic: Types Change

@ Leszek - completed the work to generate the types, there is a branch already. Needs to merge Mokhtar's branch to Carthage. The only thing that affects the Apps is that it does not have the FE constants, so some definitions are missing and need to be re-done on the apps side.

⚠️ When the apps can be updated wrt new types? - Once Leszek updates PR and do annotation, ETA today ⚠️

Discussed cross-pallet implications on weights @ Leszek shared video, @ Bedeho shared article from substrate.dev

Runtime

@ Mokhtar completed fixing content pallet tests @ Mokhtar is close to finish next iteration, pushed final changes -> final review pending by @ Leszek, ETA - today. @ Mokhtar - Fix build issues after upgrade, mainly benchmarks. ETA - tomorrow.

@ Luis - doing benchmarking, pushed. Fixed x2 state bloat bond tickets, merged with Audit3. To continue on benchmarking without changes till further notice

Runtime PRs

Ephesus revenue split - finished, but can't be updated
#3893 closed, not relevant

@ Ignazio to take care of the list below:

To review Luis's PR on state bloat bond
Audit freeze runtime, make sure it is workable
#3822
#3817 - base changed to Carthage
#3809 - base changed to Carthage
#3671 - base changed to Carthage
#3804 - CRITICAL ⚠️ 🔴 - base changed to Carthage
#3908 - base changed to Carthage
#3970 - opened today, to get benchmarks to build and generate weights
Drop blog pallet was merged to Audit3 - needs rebase to Carthage

QN, Hydra

@ Zeeshan to update QN wrt to new types #3940 @ Leszek to update typegen in Hydra, he will create a respective issue for tracking @ Ondrej meta protocol, channel agent permission status will be updated on Wednesday due to absense.

Colossus, Argus, CLI @ Zeeshan to update CLI wrt to new types (after 3940), but needs Hydra upd for it. @ Leszek to upd Colossus to work with Audit3 + Fees (tasks merged on the meeting) @ Leszek to upd Argus, Colossus wrt to new types + API scripts (to make an example)

** Atlas and Pioneer** @ Klaudiusz to update Atlas @ Theo to update Pioneer [work to commence after 11th July]

To-do Next

@ Mokhtar is close to finish next iteration, pushed final changes -> final review pending by @ Leszek, ETA - today. @ Leszek - completed the work to generate the types, there is a branch already. Needs to merge Mokhtar's branch to Carthage. @ Leszek to update typegen in Hydra, he will create a respective issue for tracking @ Zeeshan to update QN wrt to new types #3940 @ Zeeshan to update CLI wrt to new types (after 3940), but needs Hydra upd for it. @ Leszek to upd Colossus to work with Audit3 + Fees (tasks merged on the meeting) @ Leszek to upd Argus, Colossus wrt to new types + API scripts (to make an example) @ Klaudiusz to update Atlas

Jun 27 '22 14:06 dmtrjsg

Meeting Date

Wednesday, June 29th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Ondrej
Zeeshan
Klaudiusz
Dmitry

Absent:

Martin
Theo

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Scope and Focus Change

➕ New tasks: none

➖ Removed tasks:

#2969 closed as nothing severe discovered ✅
project token: test fixing - solved, merged ✅

⏭️ Postponed tasks

To next week to make room for timely PRs review

https://github.com/Joystream/joystream/issues/3902
https://github.com/Joystream/joystream/issues/3941
https://github.com/Joystream/joystream/issues/3837
Forum bug
Low10 and low 9
Atlas support fees
Colossus fees support

Weekly Summaries

Key Topic: Types Change Follow ups and Runtime Merges; Very big backlog of PRs, rather than issues prevents from accurate planning and became the priority overall.

Milestone of working local node is achieved 🦄 Community testing start date will be advised by @ Martin

Leszek and Zeeshan focussing on QN + Hydra types change implications.

@ Leszek - update Hydra Typegen - almost finished, tests already passing, what remains is to test with Joystream QN. @ Zeeshan - QN not testable yet, needs Hydra, will push the types to branch and Leszek will test.

#3940

Runtime

@ Leszek - updated QN to work with typegen and after that will review runtime PRs. Namely,

#3914

@ Mokhtar - fix referendum benchmarks - today; fix build issues post substratev3 upgrade; carthage bootstrapping script (only memebrships)

#3948

@ Luis - to finish benchmarking, remove council blog pallet - to fix conflicts; Rebase video and channel bloat bond to Carthage. ETA on both to be merged is today.

@ Ignazio - finished on Channel transfer nonses. Will continue to review

#3970 and Mokhtar's PRs (covered in this section above). Plus
low14 and low 15
Cross pallet extrinsic transaction
Unification of withdrawals to personal account

QN, Hydra

@ Leszek - update Hydra Typegen - almost finished, tests already passing, what remains is to test with Joystream QN.

@ Zeeshan - QN not testable yet, needs Hydra, will push the types to branch and Leszek will test.

#3940

@ Andy on channel agent premissions. Only 1PR reviewed = Curator moderation actions - To be merged after QN is buildable. Metaprotocol category system needs QN to test. Deletion of videos and channels

#3870 related to
#3937 will need to be updated.

Colossus, Argus, CLI

@ Zeeshan to take on - CLI to work in Carthage

Atlas and Pioneer

Theo is away Klaudiusz waiting for QN and infra but some progress can be done with just working QN

To-do Next

Highest prio = runtime and QN core final features to be merged asap, full focus on PRs reviews and merging.

Jun 29 '22 15:06 dmtrjsg

Meeting Date

Monday, July 4th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Ondrej
Zeeshan
Dmitry

Absent:

Martin
Theo
Klaudiusz

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Scope and Focus Change

➕ New tasks:

➖ Removed tasks:

Update size of call tests - we may not need to do it
#3948 closed as other PRs addressing problems
#3923 - fixed in #3970

⏭️ Postponed tasks

Drop multi address - not sure what the problem is. Enum, only one variant is the acc id, to do later, migration required
#3671 - Not started, to be done after content benchmark
#3840 - done, to be reviewed later, pushed back
Add privileged member invitation feature - first implementation of new extrinsic pushed back
Explore multi-sig support - next week

Weekly Summaries

Key Topic: Outstanding runtime PRs .

Milestone in focus - staging network. First community testing - ETA based on Martin, but need Runtime + QN + CLI updated. What do we need for it? - chain set up scripts to be fixed and QN merged.

Runtime

PRs

#3986 Done, we don't merge, @ Ignazio and @ Leszek to review
Drop multi address - postponed
Freeze Runtime - in review by @ Leszek
#3817 - already fixed, closed
#3809 - @ Ignazio done, @ Leszek to review
#3671 - Not started, to be done after content benchmark
#3840 - done, to be reviewed later, pushed back
Bounty perpetual working period - eta next week
Carthage: add transfer nonce - @ Leszek to review
Audit3 expose bucket selection and storage pallet - not merged needs to be upd by @ Ignazio
Base Call Filter - @ Ignazio
Update size of call tests - we may not need to do it
Add privileged member invitation feature - first implementation of new extrinsic pushed back
Fix linter/ dippy warnings - @ Ignazio
#3976 - ready, @ Mokhtar to review
#3975 - Blocked, @ Leszek left review, not fixed yet, conflict w metadata, to be fixed
#3984 - ready for review @ Ignazio

TASKS

#3948 closed as other PRs addressing problems
Everything before 4th July tasks are waiting
Unsafe form polls - end of this week ETA
#1737 - assigned to @ Leszek
#3953 - put this behind feature flag @ Mokhtar
"Sudo_dynamic_bag"- assigned to @ Mokhtar
#3951 assigned to @ Ignazio
Benchmarking ETA will be announced today by @ Luis
#3923 - fixed in #3970
Explore multi-sig support - next week

QN, Hydra

Focus: PRs dependant on buildable QN

Metaprotocol content category + Channel Agent Permissions @ondratra to merge to carthage, mismatch of versions to be fixed across working QN, Hydra, TypeORM. Propagate "not-fintering" to Carthage and Master + Test this with Atlas @drillprop can assist.

Colossus, Argus, CLI

Skipped

Atlas and Pioneer

Skipped

To-do Next

Milestone in focus - staging network.

Jul 05 '22 09:07 dmtrjsg

Meeting Date

Wednesday, July 6th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Ondrej
Zeeshan
Dmitry

Absent:

Martin
Theo
Klaudiusz

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Scope and Focus Change

➕ New tasks: QN and CLI ticket:

https://github.com/Joystream/joystream/issues/3993

➖ Removed tasks:

Tweak in-flight channel freezing policy - merged ✅
Freeze runtime - merged ✅

⏭️ Postponed tasks

Low13, Low15 and other Audit tasks - next week
QN support: founding member support - next week
CLI - duration extended

Weekly Summaries

Key Topic: Preparing for Carthage Staging Env

⭐ Carthage milestones ⭐ Staging network launch is nearing: node is ready for staging, QN almost ready with only tests left, ETA - end of this week. First community testing 18th July. Production release ETA is 1st August.

Runtime

PRs

update size of - @ Leszek
#3985 wip
#3984 update chain metadata required, can be merged after
Forum bug fixes - in review @ Ignazio
Info2: missing update_data - @ Leszek to do today
upd_category_membship.. - @ Leszek
#3911 - @ Leszek

TASKS

QN, Hydra

Focus: make QN buildable (only tests left) and merge all PRs in review.
Buildable QN - @ Zeeshan Migrating work on top of the merged branch and review Zeeshan's work - @ Ondrej ⚠️ Zeeshan is away early next week, will aim to finalize buildable qn and subsequent merges before he takes off

Colossus, Argus, CLI

@ Leszek to check if it compiles, and if has any issues + Colossus: update to work w Audit3

Atlas and Pioneer

Skipped

To-do Next

Milestone in focus - staging network.

Jul 07 '22 14:07 dmtrjsg

For the lack of a better place to put this planning note, I am putting it in this thread.

Content Directory Benchmarking Pivot

Background

@L3pereira noted that they believe it will take at least 3 weeks from today for them to complete remainder of benchmarking, but possibly much longer, and this is already some 6 weeks into it. This means we have to reorganize work on this, by parallelizing the effort, something which now should be possible as the core benchmarking infrastructure code has been completed.

TODO

We do not start any new content directory work until benchmarking is completed
Try to keep PRs small, ideally one extrinsic per PR if it makes sense.
Talk to each other proactively about identifying shared utility routines needed.
Use distinct files when possible.

@L3pereira

Completes review of his current PR, does not start any new work in this PR, and we do not merge this PR into carthage.
Make issues for groups of extrinsics, based on issue from @Lezek123, that should be tackled in a shared PR, and add carthage label.
Start executing in the order described in the issue.

@Lezek123

Make a new branch carthage_content_benchmark based on current carthage.
Pulls in initial PR from @L3pereira when all comments have been addressed.
Makes new issue which partitions content directory extrinsics into families and assigns each to a developer among @ignazio-bovo , @L3pereira and @Lezek123, and the ideal reviewer, with sensitivity to what parts of content pallet is most suitable for each person. Include ideas about utility routines that may be useful to share if it seems appropriate.
Rank extrinsics in terms of what order to do them, so that we can generate some useful early real weights if we want to.
Make issues for groups of extrinsics, based on issue from @Lezek123, that should be tackled in a shared PR, and add carthage label.
Start executing in the order described in the issue.

@ignazio-bovo

Make issues for groups of extrinsics, based on issue from @Lezek123, that should be tackled in a shared PR, and add carthage label.
Start executing in the order described in the issue.

Jul 12 '22 11:07 bedeho

Meeting Date

Tuesday, July 12th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Ondrej
Theo
Klaudiusz
Dmitry

Absent:

Zeeshan
Martin

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Scope and Focus Change

➕ Added tasks:

subkey 2.02 - fix for devops work @ Leszek ETA today
#4002 - for next week
#3979 and #3981
Integration test flow with CLI - ETA tomorrow

➖ Removed tasks:

#3827 - closed

⏭️ Postponed tasks

Benchmarking (see above)

Weekly Summaries

Key Topic: Carthage Staging Env and testing timelines

⭐ Milestone of 1st staging playground for Carthage (QN + Argus + Colossus) - ETA tomorrow, 13th July.

⭐ Community Testing CLI based, orchestrated by Martin - ETA next week QA-epic for Carthage will be moved to Community Project55 once shared environment is up with help from @ Mokhtar; and Updated Types+ Fees are merged- ETA next week.

⭐ New types will be published for Atlas by @ Mokhtar tomorrow

The tests above will complement each other.

Runtime

ℹ️ NB: runtime deep dive of the sync meeting is covered in the comment above

PRs

subkey 2.02 - fix for devops work @ Leszek and @ Zeeshan added as Reviewers
content pallet can_slash #4004 - Merged
Info14: #4000 - merged

TASKS

⚠️ startup the chain scripts to get fixed @ Mokhtar

#3979 and #3981 - test to allow up to 352 bytes -> open question to talk on the specifics of how big was teh call value, that caused problems with selected nodes outage, and produce the list of sizes for extrinsics across runtime - not blockers

QN, Hydra

Carthage branhc runs fine, need Leszek's PR to get merged Mappings: up to date, bounties not done, but getting deprecated so not an issue Missing features (video categories, agent permissions) - not merged but not a blocker - ETA for merge - today

Colossus, Argus, CLI

⭐ Milestone of 1st staging network QN + Argus + Colossus - ETA tomorrow

Pioneer

⭐ Pioneer "MVP" ready - end of next week

Started working on Types, ETA - > Tomorrow eod
Fees done, needs to be reviewed -> ETA

Atlas

⭐ Atlas "MVP" ready - week after next one Types -done, tested with local env Fees calc - in progress -> end of this week for ETA Storage details (manual buckets) - to test and implement follow ups Bloat bond fees -> to do

To-do Next

Deploy playground, fix startup scripts, make FE apps work with new env and test the fees with community.

Jul 12 '22 11:07 dmtrjsg

Meeting Date

Thursday, July 15th, 2022.

Participants

Bedeho
Mokhtar
Ignazio
Leszek
Luis
Ondrej
Theo
Zeeshan
Klaudiusz
Dmitry
Martin

Absent: None

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Weekly Summaries

Key Topic: Carthage Staging Env and testing timelines

⭐ Community Testing - moved to 25th July

⭐ Carthage Launch - moved to 12th Aug

⭐ Milestone of 1st staging playground for Carthage (QN + Argus + Colossus)

Pioneer MVP env -> Thursday next week QN: buildable Colossus Audit3 and Argus - some work remains to be done Faucet - small change to initial funds required, current amount (100) is not enough

⭐ Ephesus scope will be delivered after mainnet, as currently features are frozen on runtime.

Runtime

Staging Env: Locally already works. Playground will be up soon, scripts on starting it up remains, but manually can be done. Remedy for staging to allocate funds to workers who run in the startup script, and hire leads via sudo.

PRs

did not discuss this time

TASKS Benchmarking - 3 weeks to go. (Suggestion to bump up the weights to higher, this way some bugs can be better exposed)

🆕 @ Leszek to create an issue on Bload Bonds for Channel accounts

QN, Hydra

Content category system, CLI + tests @ Andy Channel agent permissions - before the end of week @ Andy

Colossus, Argus, CLI

CLI - @ Zeeshan Colossus Audit3 and Argus - some work remains to be done @ Zeeshan

Pioneer

Pioneer "MVP" ready - end of next week

Started working on Types, ETA - > Tomorrow eod
Fees done, needs to be reviewed -> ETA

Atlas

Atlas MVP - end of next week

Operations

Martin to create detailed testing plan

To-do Next

Deploy playground, fix startup scripts, make FE apps work with new env and test the fees with community.

Jul 15 '22 16:07 dmtrjsg

Meeting Date

Monday, July 25th, 2022.

Participants

Bedeho
Mokhtar
Leszek
Luis
Ondrej
Theo
Zeeshan
Klaudiusz
Dmitry

Absent:

Ignazio
Martin

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Weekly Summaries

Key Topic: FE apps readiness for testing

⭐ Pioneer - awaiting for denomination features merge tomorrow, and community testing to commence right after. ⭐ Atlas - everything will be working but not fully tested by Wednesday 27th expecting fully tested by end of this week.

Runtime

NB: comments here represent first part of the meeting, before deep dive into Runtime part of the meeting.

"Currency PR"- NFT limits, fee for membership, change to value denominated based on market value of tokens, so fees will go up, + state bloat bonds; @ Leszek to review; Int tests to be run locally while CI checks are not working. Community testing will happen after merge. Feature on forum Polls - added to runtime, not required for Pioneer testing. By Thursday will get new playground with created channel account; id + event + QN for it.

QN, Hydra

@ Andy - off from Wednesday till next Monday.

#4072 - merged #4065 - approved - fixes date created vs date updated #4048 - added to carthage #3931 - starts today, founding member ID "mainnet" + JSG members. #3436 - merged today #4075 - starts today on it as it breaks carthage rn #4071 - will update and get fixed today #3993 - next week #4057 - already on QN and Runtime; only need to be done on Pioneer.

@ Zeeshan - finalising tests and moving his focus to Youtube partnership programme. #3902 - moved to August 1st to commence.

Colossus, Argus, CLI

Did not discuss

Pioneer

Awaiting testable version tomorrow
https://github.com/Joystream/pioneer/issues/3387 - added to mainnet "Verified" status already on Runtime - not implemented on Pioneer

### Atlas

everything will be working but not fully tested by Wednesday 27th expecting fully tested by end of this week.

Jul 25 '22 17:07 dmtrjsg

Meeting Date

Monday, Aug 1st, 2022.

Participants

Bedeho
Mokhtar
Leszek
Luis
Theo
Zeeshan
Klaudiusz
Dmitry

Absent:

Ondrej

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Weekly Summaries

Key Topic: Carthage milestones update

⭐ Second Round of testing will be commenced on 18th Aug ⭐ Pioneer - QA commenced on Friday, some bugs reported, but nothing critical ⭐ Atlas - testing done, but to complete the suite, need staging env with multiple distributor nodes (to test allocation of storage buckets for channels creation and videos upl)

🆕 New Tasks #4103

https://github.com/Joystream/pioneer/issues/3452 to be tested later

🔜 Postponed

https://github.com/Joystream/pioneer/issues/3404 - genesis vesting to be tested separately. #4085 #4084 #4083

Runtime

NB: comments here represent first part of the meeting, before deep dive into Runtime part of the meeting.

QN, Hydra

#4086 - @kdembler will test and bring over the finish line

For week one after next one: #4085 #4084 #4083

Aug 02 '22 09:08 dmtrjsg

Meeting Date

Monday, Aug 8th, 2022.

Participants

Bedeho
Martin
Leszek
Theo
Zeeshan
Klaudiusz
Dmitry

Absent:

Mokhtar
Ignazio

Project Tracking

Tracked in Asana: Link to project timeline Tracked in Github: label:carthage

Weekly Summaries

Key Topic: Carthage milestones update

⭐ Second Round of testing will be commenced on 18th Aug - still on track ⭐ Pioneer - QA: In progress. Details in relevant section. ⭐ Atlas - testing completed for all tickets, apart from Invitation lock testing.

🆕 New Tasks

Test genesis vesting, blocked by Genesis block creation. @ Martin to create block, @ Mokhtar to deploy test env
#4111 - @ LEszek to Review PR
#3785 - @ Klaudiusz to Review
#4123 added for @ Leszek to take on
#4085 - to be removed from Forum Pallet

🔜 Postponed

#4107 Next week
#4105 No PR for this yet

Runtime

Only @ Leszek was present. Will address new issues described above and PR reviews earmarked for this week in Asana. @bedeho will speak to Luis on reviews status

QN, Hydra

#4086 - @ Klaudiusz will test and bring over the finish line #3574 - @ Klaudiusz will take on, ETA circa 3 days #3436 - @ Zeeshan will take on, ETA today

### Atlas

Collaborators adding signature will be added to YPP ob flow
#4071 Closed

### Pioneer QA status: Validators testing blocked by staging env, and Funding proposals by exposing max_limit. Members/ Council/ Forum/ Storage and Distributors are tested. Proposals; Invitation Lock; Vesting to be tested.

New task raised for post-mainnet raised

https://github.com/Joystream/pioneer/issues/3479

Aug 08 '22 19:08 dmtrjsg

TODO as of 22 aug: aim for full benchmarking to be done by Tuesday Aug 30th.

Enhancements

Video CUD: update benchmarking to use worst case number of paused features
Bloat bonds: not implemented yet
Adjustments post payments PR

Reviews

NFTs
Video CUD
Channel payouts & withdrawals
Moderation actions
NFTs
Remarks
CRT

Aug 23 '22 08:08 bedeho

Meeting on October 10th.

YPP is just about done, end-to-end, hence it we should include in mainnet scope, even if My Payments is not there.
My Payments will almost certainly not be included, since Atlas team has solid backlog of work on enhancements and Gleev adjustments. Hence My Payments related work should only start when other in-scope tasks are done.
Pioneer work outstanding largely relates to minor cleanup on forum and small tweaks, other work relating to tests and longer term objectives will be paused until this primary work is done.
@dmtrjsg will come back with
- new estimate on My payments
- new estimates on timeline for remaining mainnet Pioneer work.
- new estimates on timeline for remaining mainnet Atlas work.
- ensure all in Atlas is accounted for on Asan board, e.g. Gleev work is totally missing.
ETA for next tests seems to be around Thursday this week, largely due to significant review backlog. Work in runtime team will shift to mainnet from Wednesday on.
@zeeshanakram3 will look at Colossus/Argus enhancements for mainnet scope when he is done with last reviews and YPP work.

Oct 10 '22 11:10 bedeho

YPP is just about done, end-to-end

Well I wouldn't say that's 100% accurate. While most of the tickets we've drafted for Atlas are done, there definitely needs to be some testing involved before we can release. There's also the operational side of things - making sure whoever is responsible on the JSG side has all the tools needed to run the program. So can easily look up activities that should be rewarded, can make payments, etc.

Oct 10 '22 11:10 kdembler

there definitely needs to be some testing involved before we can release

Good observations, added issue for getting a plan defined to begin with: https://github.com/Joystream/joystream/issues/4357

Oct 10 '22 12:10 bedeho

Integration ticket for YPP was put on hold and awaiting the rest of the YPP gleev adaptation.

https://github.com/Joystream/atlas/issues/3154

There's no danger in including it to mainnet scope as can be launched in a disabled/ hidden mode. This ticket encapsulate the work to make it so:

https://github.com/Joystream/atlas/issues/3299

Let's take a view on this once the full scope is designed and estimated, which will be done end of day on October 17th, so we can discuss this on the meeting Oct 20th. The team will be busy with other mainnet labelled issues before then.

Oct 10 '22 13:10 dmtrjsg

joystream joystream copied to clipboard

Carthage Network Tracking Issue

Purpose

Meeting Date

Participants

Project Tracking

Weekly Summaries

Agenda

Conclusions

Critical path

Todo Now

Next meeting

Notes

Clarifications

Meeting Date

Participants

Project Tracking

Weekly Summaries

Conclusions

Todo Now

To-do [last meetings]:

Critical path

Next meeting

Meeting Date

Participants

Project Tracking

Weekly Summaries

Scope Change

To-do

Meeting Date

Participants

Project Tracking

Weekly Summaries

Meeting Date

Participants

Project Tracking

Scope Change

Weekly Summaries

Meeting Date

Participants

Project Tracking

Scope Change

Weekly Summaries

Meeting Date

Participants

Project Tracking

Scope and Focus Change

Weekly Summaries

Runtime

QN, Hydra

Colossus, Argus, CLI

Atlas and Pioneer

To-do Next

Meeting Date

Participants

Project Tracking

Scope and Focus Change

Weekly Summaries

Runtime

QN, Hydra

Colossus, Argus, CLI

Atlas and Pioneer

To-do Next

Meeting Date

Participants

Project Tracking

Scope and Focus Change

Weekly Summaries

Runtime

QN, Hydra

Colossus, Argus, CLI

Atlas and Pioneer

To-do Next

Content Directory Benchmarking Pivot

Background

TODO

Meeting Date

Participants

Project Tracking

Scope and Focus Change

joystream
joystream copied to clipboard