DALI icon indicating copy to clipboard operation
DALI copied to clipboard

Refactor package naming: move from flavor-based names to version-based dev differentiation

Open JanuszL opened this issue 4 weeks ago • 7 comments

  • Previously, weekly/nightly builds were differentiated by flavor suffix in the package name (e.g., nvidia-dali-weekly-cuda*, nvidia-dali-nightly-cuda*). This change moves to a unified naming schema where:
    • Package names are consistent regardless of build type (nvidia-dali-cuda*)
    • Dev builds are differentiated by .dev{TIMESTAMP} suffix in version number
    • Stable and dev packages share the same package name, distinguished only by version
  • Removes DALI_FLAVOR_MINUS variable from CMakeLists.txt files
  • Updates package names to remove flavor suffix:
    • nvidia-dali-cuda* (was nvidia-dali-{flavor}-cuda*)
    • nvidia-dali-tf-plugin-cuda* (was nvidia-dali-tf-plugin-{flavor}-cuda*)
    • nvidia-dali-{plugin} (was nvidia-dali-{plugin}-{flavor})
  • Standardizes dev version format: use .dev{TIMESTAMP} instead of .{TIMESTAMP}
  • Updates install_requires in TF plugin to match new naming schema
  • Updates installation documentation to explain new distribution method:
    • Documents new unified package naming with .dev{TIMESTAMP} version suffix
    • Explains use of --pre flag for installing dev builds
    • Adds legacy distribution section for pre-1.53 builds
    • Clarifies nightly vs weekly build schedule and availability

Category:

Other (e.g. Documentation, Tests, Configuration)

Description:

  • Previously, weekly/nightly builds were differentiated by flavor suffix in the package name (e.g., nvidia-dali-weekly-cuda*, nvidia-dali-nightly-cuda*). This change moves to a unified naming schema where:
    • Package names are consistent regardless of build type (nvidia-dali-cuda*)
    • Dev builds are differentiated by .dev{TIMESTAMP} suffix in version number
    • Stable and dev packages share the same package name, distinguished only by version
  • Removes DALI_FLAVOR_MINUS variable from CMakeLists.txt files
  • Updates package names to remove flavor suffix:
    • nvidia-dali-cuda* (was nvidia-dali-{flavor}-cuda*)
    • nvidia-dali-tf-plugin-cuda* (was nvidia-dali-tf-plugin-{flavor}-cuda*)
    • nvidia-dali-{plugin} (was nvidia-dali-{plugin}-{flavor})
  • Standardizes dev version format: use .dev{TIMESTAMP} instead of .{TIMESTAMP}
  • Updates install_requires in TF plugin to match new naming schema
  • Updates installation documentation to explain new distribution method:
    • Documents new unified package naming with .dev{TIMESTAMP} version suffix
    • Explains use of --pre flag for installing dev builds
    • Adds legacy distribution section for pre-1.53 builds
    • Clarifies nightly vs weekly build schedule and availability

Additional information:

Affected modules and functionalities:

  • cmake files
  • setup.py.in files

Key points relevant for the review:

  • NA

Tests:

  • [x] Existing tests apply
    • nightly build runs
  • [ ] New tests added
    • [ ] Python tests
    • [ ] GTests
    • [ ] Benchmark
    • [ ] Other
  • [ ] N/A

Checklist

Documentation

  • [ ] Existing documentation applies
  • [x] Documentation updated
    • [ ] Docstring
    • [ ] Doxygen
    • [x] RST
    • [ ] Jupyter
    • [ ] Other
  • [ ] N/A

DALI team only

Requirements

  • [ ] Implements new requirements
  • [ ] Affects existing requirements
  • [x] N/A

REQ IDs: N/A

JIRA TASK: N/A

JanuszL avatar Dec 04 '25 06:12 JanuszL

Greptile Overview

Greptile Summary

Refactored DALI package naming from flavor-based suffixes to unified naming with version-based differentiation using PEP 440 dev releases.

  • Removed DALI_FLAVOR_MINUS variable from all CMakeLists.txt files
  • Unified package names: nvidia-dali-cuda* and nvidia-dali-tf-plugin-cuda* (removed -nightly/-weekly suffixes)
  • Changed version format from .{TIMESTAMP} to direct {TIMESTAMP} append (e.g., 1.53.0dev20250104)
  • Added CMake validation to ensure dev builds have 'dev' in VERSION file
  • Updated installation docs with new --pre flag instructions and legacy section for pre-1.53 builds
  • Fixed plugins/CMakeLists.txt to actually apply TIMESTAMP (was previously a no-op)

Confidence Score: 4/5

  • Safe to merge with minor documentation inconsistency in compilation.rst
  • Implementation is sound with proper validation and PEP 440-compliant versioning. The CMake changes correctly remove DALI_FLAVOR_MINUS and implement dev version validation. However, docs/compilation.rst was not updated to reflect the new naming convention, creating a minor inconsistency in documentation.
  • docs/compilation.rst should be updated to reflect new unified package naming (currently references outdated flavor-based names)

Important Files Changed

File Analysis

Filename Score Overview
dali/python/CMakeLists.txt 5/5 Removed DALI_FLAVOR_MINUS, changed version format from .{TIMESTAMP} to {TIMESTAMP}, added validation to ensure dev builds have 'dev' in VERSION
dali/python/setup.py.in 5/5 Updated package name from nvidia-dali@DALI_FLAVOR_MINUS@-cuda* to nvidia-dali-cuda*, removing flavor suffix from package name
dali_tf_plugin/setup.py.in 5/5 Updated TF plugin package name and dependency from flavor-suffixed to unified naming (nvidia-dali-tf-plugin-cuda* and nvidia-dali-cuda*)
docs/installation.rst 4/5 Updated installation docs with new unified package naming, added --pre flag instructions, reorganized nightly/weekly sections, added legacy section for pre-1.53 builds

Sequence Diagram

sequenceDiagram
    participant Build as Build System
    participant CMake as CMake
    participant Setup as setup.py
    participant PyPI as PyPI/Index
    participant User as End User
    
    Note over Build,CMake: Build Configuration Phase
    Build->>CMake: Set DALI_BUILD_FLAVOR (e.g., "nightly")
    Build->>CMake: Set TIMESTAMP (e.g., "20250104")
    CMake->>CMake: Read VERSION file (e.g., "1.53.0dev")
    alt Development Build (DALI_BUILD_FLAVOR set)
        CMake->>CMake: Validate VERSION contains "dev"
        CMake->>CMake: Append TIMESTAMP: "1.53.0dev20250104"
    else Stable Build
        CMake->>CMake: Keep VERSION as-is: "1.53.0"
    end
    
    Note over CMake,Setup: Package Configuration Phase
    CMake->>Setup: Configure setup.py with unified name
    Note right of Setup: OLD: nvidia-dali-nightly-cuda120<br/>NEW: nvidia-dali-cuda120
    Setup->>Setup: Set version to "1.53.0dev20250104"
    
    Note over Setup,PyPI: Publishing Phase
    Setup->>PyPI: Publish package
    Note right of PyPI: Package: nvidia-dali-cuda120<br/>Version: 1.53.0dev20250104<br/>PEP 440 dev release
    
    Note over User,PyPI: Installation Phase
    User->>PyPI: pip install --pre nvidia-dali-cuda120
    PyPI->>User: Return dev version (1.53.0dev20250104)
    User->>PyPI: pip install nvidia-dali-cuda120
    PyPI->>User: Return stable version only (1.53.0)

greptile-apps[bot] avatar Dec 04 '25 06:12 greptile-apps[bot]

!build

JanuszL avatar Dec 04 '25 08:12 JanuszL

CI MESSAGE: [39593786]: BUILD STARTED

dali-automaton avatar Dec 04 '25 08:12 dali-automaton

!build

JanuszL avatar Dec 04 '25 09:12 JanuszL

CI MESSAGE: [39597446]: BUILD STARTED

dali-automaton avatar Dec 04 '25 09:12 dali-automaton

CI MESSAGE: [39593786]: BUILD PASSED

dali-automaton avatar Dec 04 '25 16:12 dali-automaton

CI MESSAGE: [39597446]: BUILD PASSED

dali-automaton avatar Dec 04 '25 16:12 dali-automaton