layered-loader icon indicating copy to clipboard operation
layered-loader copied to clipboard

Add Valkey support with parametrized tests

Open kibertoad opened this issue 4 months ago β€’ 2 comments

Summary

This PR adds Valkey support to layered-loader with comprehensive parametrized testing against both Redis and Valkey servers.

Changes

Infrastructure

  • Added Valkey service to docker-compose.yml (port 6380)
  • Updated docker:start script to start both Redis and Valkey
  • Added @valkey/valkey-glide dependency for future native client integration

Test Improvements

  • Parametrized all Redis tests using Vitest's describe.each() pattern
  • Tests now run against both Redis (localhost:6379) and Valkey (localhost:6380)
  • No code duplication - single test suite validates both servers
  • Updated test files:
    • RedisCache.spec.ts
    • RedisGroupCache.spec.ts
    • RedisNotificationPublisher.spec.ts
    • RedisGroupNotificationPublisher.spec.ts

Configuration

  • Created testServerConfigs in TestRedisConfig.ts with both server configurations
  • Added redisOptions and valkeyOptions for easy server switching

Implementation Notes

Current Approach

  • Uses ioredis for all operations - ioredis is fully compatible with Valkey
  • All existing functionality works unchanged with Valkey
  • Tests validate compatibility across both servers

Future Work

The @valkey/valkey-glide dependency is included for future native client integration. An adapter pattern can be implemented to support both ioredis and valkey-glide clients when needed.

Test Results

βœ… All 321 tests pass against both Redis and Valkey

  • 13 test files
  • No regressions
  • Full compatibility validated

Running Tests

# Start both Redis and Valkey
npm run docker:start

# Run all tests (automatically tests both servers)
npm test

# Run specific test file
npm test -- RedisCache

Benefits

  1. Valkey compatibility validated - All operations work with Valkey
  2. Dual-server CI - Can run tests against both in CI/CD
  3. Zero code changes - Existing code works with Valkey via ioredis
  4. Future-proof - Foundation for native valkey-glide client migration
  5. Better test coverage - Parametrized tests ensure consistency

Related

  • Valkey: https://valkey.io/
  • valkey-glide: https://github.com/valkey-io/valkey-glide
  • Valkey compatibility: https://valkey.io/topics/compatibility/

Summary by CodeRabbit

  • New Features

    • Adds a local Valkey service and support for Valkey/Glide as an alternative cache/backend (exposed on a new port).
  • Tests

    • Test suites parameterized to run against both Redis and Valkey/Glide configurations.
  • Chores

    • Compose/startup updated to bring up Valkey alongside Redis; gitignore and dev-dependency ordering adjusted.
  • Refactor

    • Unified Redis client adapter and simplified subscription/lifecycle handling across caching and notification components.
  • Documentation

    • README, migration guide, parity review, and design notes for Valkey added.

kibertoad avatar Oct 11 '25 23:10 kibertoad

Walkthrough

Adds Valkey (valkey/valkey:8-alpine) as a new service and extends test harness and code to support both ioredis and @valkey/valkey-glide. Introduces a Redis client adapter, updates caches, group-cache, notification factories/publishers/consumers to use adapter types, parameterizes tests per-backend, and adds migration/docs material. (50 words)

Changes

Cohort / File(s) Summary
Docker orchestration
docker-compose.yml, package.json
Add valkey service (valkey/valkey:8-alpine, valkey-server --requirepass sOmE_sEcUrE_pAsS, ports 6380:6379, VALKEY_REPLICATION_MODE=master); wait_for_redis depends on valkey and WAIT_HOSTS includes valkey:6379. Start script updated to bring up valkey. Peer/dev dependency @valkey/valkey-glide added and devDeps ordering adjusted.
Test fakes / configs
test/fakes/TestRedisConfig.ts
New exports/types to support multiple backends: valkeyGlideConfig, PubSubPair, ServerConfig, and testServerConfigs with per-backend createClient/closeClient and createPubSubPair/closePubSubPair.
Parameterized tests
test/redis/*
test/redis/RedisCache.spec.ts, test/redis/RedisGroupCache.spec.ts, test/redis/RedisGroupNotificationPublisher.spec.ts, test/redis/RedisNotificationPublisher.spec.ts
Convert suites to describe.each(testServerConfigs) and wire per-config lifecycle. Use adapter-aware flushing, runtime guards for ioredis-only features, and RedisClientType typings; replace direct Redis construction/disconnect with config-provided hooks.
Redis client adapter layer
lib/redis/RedisClientAdapter.ts
New RedisClientInterface and adapters IoRedisClientAdapter / ValkeyGlideClientAdapter; runtime guards (isIoRedisClient, isGlideClient); factory createRedisAdapter; RedisClientType union. Normalizes core commands, scan, scripting, pub/sub, multi, and exposes underlying client.
Abstract & cache implementations
lib/redis/AbstractRedisCache.ts, lib/redis/RedisCache.ts, lib/redis/RedisGroupCache.ts, lib/redis/RedisGroupCache.ts.backup
Internal redis fields and constructors switch to RedisClientType/RedisClientInterface and use createRedisAdapter. Add serialization rules, adjust scan usage, add multi/batch paths for setMany, and route group operations via adapter scripting or multi depending on capabilities; backup preserves fuller group-cache variant.
Notification factories
lib/redis/RedisNotificationFactory.ts, lib/redis/RedisGroupNotificationFactory.ts
createNotificationPair and createGroupNotificationPair converted to async; configs accept `RedisClientType
Publishers & consumers
lib/redis/RedisNotificationPublisher.ts, lib/redis/RedisNotificationConsumer.ts, lib/redis/RedisGroupNotificationPublisher.ts, lib/redis/RedisGroupNotificationConsumer.ts
Replace direct ioredis usage with adapter-backed RedisClientInterface. Constructors accept RedisClientType and wrap with createRedisAdapter. Subscription flows use callback-based subscribe handlers. close() simplified to avoid quitting underlying client.
Docs & migration materials
README.md, VALKEY_FEATURE_PARITY.md, docs/VALKEY_MIGRATION.md, VALKEY_IMPLEMENTATION_REVIEW.md, MULTI_API_DESIGN.md
Add Valkey-Glide documentation, migration guide, parity analysis, implementation review, and multi-API design notes documenting dual-client support, differences, testing strategy, and migration recommendations.
Misc
.gitignore, package.json
Add .gitignore rule for core; package metadata updates (peer/dev deps), minor start-script adjustments, and import/type updates across lib/... and test/... to use adapter types.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Test as Test harness
  participant Factory as TestServerConfig
  participant Cache as Cache / Notifications
  participant AdapterFactory as createRedisAdapter
  participant Adapter as RedisClientAdapter
  participant IO as ioredis (Redis)
  participant VG as valkey-glide (GlideClient)

  Test->>Factory: request backend client (Redis or Glide config)
  Factory-->>Test: returns client instance
  Test->>Cache: construct Cache/Publisher/Consumer with client
  Cache->>AdapterFactory: createRedisAdapter(client)
  AdapterFactory-->>Adapter: instantiate IoRedisClientAdapter or ValkeyGlideClientAdapter
  alt ioredis path
    Adapter->>IO: map ops (GET/SET/MGET/MSET/MULTI/SCAN/PTTL/SUBSCRIBE/QUIT)
    IO-->>Adapter: responses/events
  else valkey-glide path
    Adapter->>VG: map ops (script/mget/mset, expiry, pub/sub)
    VG-->>Adapter: responses/events
  end
  Adapter-->>Cache: normalized responses/events
  Cache-->>Test: consistent behavior across backends

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

I munched on compose and found a key,
A valkey hop, ports opened for me.
Adapters sewed two clients into one seam,
Tests now dance through both β€” a single dream.
πŸ₯•πŸ‡

Pre-merge checks and finishing touches

βœ… Passed checks (3 passed)
Check name Status Explanation
Description Check βœ… Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check βœ… Passed The title succinctly conveys the primary enhancement of introducing Valkey compatibility along with the new parametrized test setup, accurately reflecting the key objectives and changes in the PR without unnecessary detail.
Docstring Coverage βœ… Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • [ ] πŸ“ Generate docstrings
πŸ§ͺ Generate unit tests (beta)
  • [ ] Create PR with unit tests
  • [ ] Post copyable unit tests in a comment
  • [ ] Commit unit tests in branch migrate-to-valkey-glide

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot] avatar Oct 11 '25 23:10 coderabbitai[bot]

Update: Added valkey-glide Support with Adapter Pattern

I've implemented the RedisClientAdapter to support both ioredis and @valkey/valkey-glide clients as requested:

What's Implemented

βœ… RedisClientAdapter - Abstraction layer supporting both clients
βœ… IoRedisClientAdapter - Wraps ioredis with unified interface
βœ… ValkeyGlideClientAdapter - Wraps valkey-glide with unified interface
βœ… Updated AbstractRedisCache to accept either client type
βœ… Updated RedisCache to use the adapter pattern
βœ… All 321 tests passing with ioredis against both Redis and Valkey

Current Scope

The adapter currently supports:

  • Basic cache operations (get, set, mget, mset, del)
  • TTL operations (pttl)
  • Scan operations with MATCH pattern
  • Pub/sub (ioredis only currently)

Known Limitations

RedisGroupCache still requires ioredis due to advanced operations:

  • for Lua scripts
  • for pipelined transactions
  • for counter operations

These would require significant architectural changes to support in valkey-glide due to API differences.

Next Steps

To complete full valkey-glide integration for RedisGroupCache would require:

  1. Implementing script execution adapter for Lua commands
  2. Handling transaction differences between clients
  3. Adding counter operation support to interface
  4. Separate pub/sub client architecture for valkey-glide

The current implementation provides a solid foundation and full Valkey compatibility via ioredis, with the adapter pattern enabling future migration when needed.

kibertoad avatar Oct 11 '25 23:10 kibertoad