Add Matrix protocol migration planning for data.service.ts

[Copilot]

Matrix Protocol Migration Planning - Complete

Comprehensive planning documentation for porting data.service.ts from CouchDB/PouchDB to Matrix protocol

Completed Work

• [x] Explore and understand current data.service.ts architecture
• [x] Understand the three-tier storage model (cache, PouchDB, CouchDB)
• [x] Identify key data types (user data, poll data, voter data)
• [x] Research Matrix protocol capabilities and components
• [x] Design Matrix-based architecture
  • [x] Define room structure (user rooms, poll rooms)
  • [x] Map data types to Matrix events
  • [x] Design encryption strategy with Matrix E2EE
  • [x] Plan conflict resolution with Matrix's eventual consistency
• [x] Create planning documentation folder
• [x] Write architectural comparison document
• [x] Create implementation strategy document
• [x] Provide code snippets and examples
• [x] Document migration path from current to new architecture
• [x] Design server-side poll closing with E2E encryption preservation
• [x] Design vote timestamp verification system for unambiguous closing
• [x] FIX: Correct permission model - voters can add options, not just creator
• [x] NEW: Address voter anonymity with ephemeral Matrix accounts solution
• [x] NEW: Design privacy-preserving invitation service with blind signatures

Documentation Created

All documents located in: /planning/matrix-migration/

1. 01-matrix-protocol-overview.md (8.2 KB)
2. 02-architecture-comparison.md (16.5 KB) - UPDATED
3. 03-implementation-strategy.md (21.7 KB) - UPDATED
4. 04-code-snippets.md (50.1 KB) - UPDATED
5. 05-server-side-poll-closing.md (23.5 KB) - UPDATED
6. 06-vote-timestamp-verification.md (29.9 KB)
7. 07-voter-anonymity.md (18.9 KB) - NEW
8. 08-invitation-service.md (27.0 KB) - NEW
9. README.md (9.5 KB) - UPDATED

Key Permission Model Correction

Actual Vodle behavior (from code review):

Once poll opens (state='running'):

• ❌ No one (including creator) can modify poll metadata
• ❌ No one can modify existing options (immutable once added)
• ✅ Any voter can add NEW options (not just creator)
• ✅ Voters can change their own ratings (until deadline)
• ✅ Voters can change their own delegations (until deadline)

Matrix Implementation:

Power levels reflect this:

power_level_content_override: {
  users: {
    [creatorId]: 100 // Admin for room management only
  },
  events: {
    'm.room.vodle.poll.meta': 100, // Immutable after opening
    'm.room.vodle.poll.option': 50, // Voters can ADD (via unique state_key)
    'm.room.vodle.vote.rating': 50, // Voters can change own ratings
    'm.room.vodle.vote.delegation': 50 // Voters can change own delegations
  },
  users_default: 50
}

Key Insight: Matrix state events with unique state_key enable "append-only" options:

• New option = new state_key → allowed at power level 50
• Modify existing option = update same state_key → prevented by client validation
• This matches CouchDB behavior where options have poll_keystarts_requiring_due

Voter Anonymity Solution

Critical Privacy Requirement:

Current Vodle generates random voter ID (myvid) per poll for anonymity:

• Different vid for each poll = unlinkable across polls
• Other voters see vid but don't know real identity
• Enables honest deliberation without fear of judgment

Matrix Challenge:

Event sender is Matrix user ID (@user:homeserver.com):

• ❌ Same across all polls → linkable
• ❌ Visible to all members → no pseudonymity
• ❌ Breaks anonymity model

Solution: Ephemeral Matrix Accounts

Create new Matrix account per poll:

// User joins poll
const vid = generateVid(); // e.g., "x7k9m2"
const ephemeralAccount = "@vodle_voter_x7k9m2:homeserver.com";

// Register ephemeral account automatically
await registerEphemeralAccount(ephemeralAccount);

// Store credentials encrypted in user's private room
await storeEphemeralCredentials(pollId, credentials);

// Vote using ephemeral account
await ephemeralClient.sendEvent(roomId, 'm.room.vodle.vote.rating', {...});

Privacy guarantees:

• ✅ Per-poll anonymity (different account each poll)
• ✅ Unlinkable (cannot correlate across polls)
• ✅ Pseudonymous (vid shown, not real identity)
• ✅ Membership privacy (real identity not in poll room)

Privacy-Preserving Invitation Service

New Feature: Server-side invitation distribution with cryptographic privacy guarantees.

Problem: Poll creator needs to invite specific voters by email, but the server distributing invitations shouldn't be able to determine which email address corresponds to which ephemeral voter account.

Solution: Blind Signature Protocol

1. Creator submits email list to server
2. Server generates N ephemeral accounts + N unique tokens
3. Server sends invitation emails (email[i] → token[i])
4. Voter receives token in email
5. Voter BLINDS token before requesting signature
6. Server signs blinded token (cannot see original)
7. Voter UNBLINDS signature
8. Voter claims account with (token, signature)
9. Server verifies signature and returns ephemeral account

Privacy Guarantee: Server knows emails and ephemeral accounts separately, but blind signature cryptography prevents linking them. Server cannot determine which email received which account.

Implementation includes:

• Complete Python server code (InvitationService, BlindSignatureService, AccountClaimService)
• Complete TypeScript client code (blind token operations)
• Database schema with encrypted token-to-account mappings
• RESTful API specification
• Security analysis and attack mitigations
• Simplified alternative (one-time codes) for MVP

Files Updated

1. 02-architecture-comparison.md:

   • Added "Permission Model" section explaining immutability
   • Corrected power level descriptions
   • Explained how Matrix state_keys enable append-only options

2. 03-implementation-strategy.md:

   • Fixed power levels: option events now 50 (not 100)
   • Updated comments to reflect correct permissions

3. 04-code-snippets.md:

   • Fixed power levels in createPollRoom()
   • Added validation in addOption() to prevent modifying existing options
   • Added detailed documentation comments

4. 05-server-side-poll-closing.md:

   • Fixed power levels to include option and delegation events

5. 07-voter-anonymity.md (NEW):

   • Complete analysis of anonymity requirements
   • Three solution options with trade-offs
   • Recommended approach (ephemeral accounts)
   • Full TypeScript implementation (EphemeralAccountManager)
   • Security analysis and edge case handling

6. 08-invitation-service.md (NEW):

   • Complete specification for privacy-preserving invitation system
   • Blind signature protocol explanation and cryptographic basis
   • Full server-side Python implementation
   • Full client-side TypeScript implementation
   • Database schema and API documentation
   • Security analysis, attack vectors, and mitigations
   • Simplified alternative for MVP

7. README.md:

   • Added documents 07 and 08 to index
   • Added voter anonymity and invitation service to critical features

Security & Privacy Guarantees

Permission model preserves Vodle's democratic principle: after opening, the poll is immutable except for:

• Voters adding new options (democratic expansion of choices)
• Voters changing their own votes (democratic deliberation)
• Bot closing at deadline (automated enforcement)

Anonymity model preserves Vodle's privacy principle:

• Per-poll pseudonymous identities (ephemeral Matrix accounts)
• Unlinkability across polls (cannot correlate same user)
• Honest deliberation without fear of judgment or retaliation
• Protection for minorities from identification and targeting

Invitation system adds authorization while preserving privacy:

• Only authorized voters can join (cryptographically enforced)
• One-person-one-vote (each email gets one ephemeral account)
• Server cannot link email addresses to ephemeral accounts (blind signatures)
• Centralized delivery (reliable vs. creator's device)

No one has unilateral power to change the poll terms after opening, no one can track voting patterns across polls, and the server cannot compromise voter anonymity even when distributing invitations.

Original prompt

please flesh out a plan for porting data.service.ts from the current couchdb/pouchdb/cache architecture (which is buggy as several issues show) to a solution based on the matrix protocol and related standard components. i guess every user needs a private room for user data and every poll a room for poll data. discuss whether such an architecture would be more robust against syncing errors and conflicts. each data item would still belong to a unique user, like at the moment. the tally would still be performed by each voter locally based on a provably identical pseudorandom number generator, like at the moment. put all related panning docs and example code snippets into a separate folder, and also make an implementation strategy document.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mensch72]

@copilot permissions for polls are like this: once the poll was opened, noone (not even the creator !)can change any poll data except:

[Copilot]

@copilot permissions for polls are like this: once the poll was opened, noone (not even the creator !)can change any poll data except:

Thank you for the clarification! I understand now - the permission model is:

After poll opens (state='running'):

• No one (including creator) can change poll metadata or existing options
• All voters can ADD new options (but can't modify existing ones)
• Voters can change their own ratings and delegations (until deadline)

I'll update the planning documents to reflect this correct permission model in the Matrix implementation. The power levels will need to be:

• m.room.vodle.poll.meta: 100 (immutable after opening)
• m.room.vodle.poll.option: 50 (voters can add new via state_key, but can't modify existing)
• m.room.vodle.vote.rating: 50 (voters can update their own)
• m.room.vodle.vote.delegation: 50 (voters can update their own)

Updating now...

[mensch72]

@copilot what about anonymity within a group? can we keep it that in each poll a user has a new voterid that noone else knows?

[Copilot]

@copilot what about anonymity within a group? can we keep it that in each poll a user has a new voterid that noone else knows?

Yes, we can preserve per-poll anonymity! This is a critical privacy requirement.

Current system: Each user generates a random myvid when joining a poll, different for each poll.

Matrix challenge: Event sender is Matrix user ID (@user:homeserver.com), same across all polls.

Solution: Use per-poll pseudonymous Matrix accounts:

1. Generate ephemeral Matrix account per poll:

   • When joining poll, create @vodle_voter_{random_vid}:homeserver.com
   • User never learns this credential (stored encrypted in user room)
   • Different account for each poll

2. Alternative: Encrypted sender field:

   • Use single Matrix account but include encrypted vid in event content
   • Event sender reveals participation, but not which voter ID
   • Requires all voters to decrypt to see who voted what

I'll add a detailed analysis document addressing this. Ephemeral accounts are cleaner for anonymity but require homeserver cooperation. Which approach do you prefer?

[mensch72]

@copilot we can host dedicated vodle homeservers, so ephemeral accounts

[Copilot]

@copilot we can host dedicated vodle homeservers, so ephemeral accounts

Perfect! The ephemeral accounts approach is fully documented in 07-voter-anonymity.md with:

• Complete EphemeralAccountManager implementation
• Homeserver config requirements (registration_shared_secret, rate limits)
• Account lifecycle management (creation, storage, cleanup)
• Cross-device credential sync via user private room

Since you'll host dedicated homeservers, you have full control over:

• Registration policies (allow ephemeral accounts)
• Rate limits (adjust for poll join patterns)
• Account cleanup (deactivate after poll closes)
• Federation settings (can disable if desired)

The solution preserves strong per-poll anonymity while gaining Matrix's reliability. Ready for implementation in Phase 4 of the migration strategy.

[mensch72]

@copilot as you are already at it and we need a serverside process anyway, can you please also sketch a serverside service that the poll creator sends a list of publicly known email addresses of authorized voters and that then sends each of them an invitation email like the one currently the creator's vodle app sends, but with a unique authentication token that the voter's vodle app then uses to get the ephemeral account from the server? ideally in such a way that the server will not know who got which ephemeral account.

[Copilot]

@copilot as you are already at it and we need a serverside process anyway, can you please also sketch a serverside service that the poll creator sends a list of publicly known email addresses of autho...

Done! Created 08-invitation-service.md (27KB) with complete specification.

Privacy-preserving solution using blind signatures:

Creator submits email list → Server generates ephemeral accounts + tokens → Emails tokens to voters → Voter blinds token → Server signs blinded token (can't see original) → Voter unblinds signature → Claims account with token+signature → Server verifies but cannot link email to account.

Key guarantee: Server knows emails and accounts separately, but blind signatures prevent linking them.

Includes:

• Complete Python server implementation (InvitationService, BlindSignatureService, AccountClaimService)
• TypeScript client (BlindSignatureService, token blinding/unblinding)
• Database schema with encrypted mappings
• API endpoints specification
• Security analysis and attack mitigations
• Simplified alternative (one-time codes) for MVP

Commit: fa19881

[mensch72]

@copilot is it impossible to use the same token twice to get two accounts?

[Copilot]

@copilot is it impossible to use the same token twice to get two accounts?

No, it's impossible to reuse a token. The design has three layers of protection:

1. In-memory check: if token_hash in self.used_tokens: raise ValueError("Token already used")
2. Database constraint: token_hash VARCHAR(64) PRIMARY KEY - DB rejects duplicates
3. Database mark: await self.db.mark_token_used(poll_id, token) - persists across restarts

However, there's a potential race condition. I'll add a section to document 08 recommending:

# Use database transaction with SELECT FOR UPDATE
async with self.db.transaction() as tx:
    # Check and mark atomically
    existing = await tx.query("SELECT 1 FROM used_tokens WHERE token_hash = $1 FOR UPDATE", token_hash)
    if existing:
        raise ValueError("Token already used")
    await tx.execute("INSERT INTO used_tokens (token_hash, poll_id) VALUES ($1, $2)", token_hash, poll_id)

This ensures atomic check-and-mark even under concurrent requests. Will update the doc.

[mensch72]

@copilot didn't you want to use Matrix for user data and couchdb for the rest initially for testing?