node-zwave-js icon indicating copy to clipboard operation
node-zwave-js copied to clipboard

Published 20 hours ago • verified publisher icon zwave-js

[Tracking] Certification tests

Open AlCalzone opened this issue 2 years ago • 0 comments

✅ Category: Automatic

  • [x] ✅ Association Group Data
    • [x] ✅ AGD_AssociationGroupData_Rev01
  • [x] ✅Command Class Support Requirements
    • [x] ✅ CSR_AssocV3OrMCAV4IsSupported_Rev01
    • [x] ✅ CSR_LifelineAssociation_Rev02
  • [x] ✅ Common Device Requirements
    • [x] ✅ CDR_MultiChannelCCInNIF_Rev01
    • [x] ✅ CDR_ReceiveEncapCommands_Rev02
    • [x] ✅ CDR_ZWavePlusInfoCCInNIF_Rev01
    • [x] ✅ CDR_ZWPv2InfoReport_Rev01
    • [x] ✅ CDR_ZWPv2MandatoryCCs_Rev01
    • [x] ✅ CDR_ZWPv2NonOptionalCCs_Rev01
    • [x] ✅ CDR_ZWPv2OptionalBit_Rev01
  • [x] ✅ Controller Requirements
    • [x] ✅ CR_NoMultiChannelCCInNIF_Rev01
  • [x] ✅ Device Type Requirements
    • [x] ✅ DT_DeviceClasses_Rev01
    • [x] ✅ DT_GenericCtrlControlledActuatorCCs_Rev01
    • [x] ✅ DT_MandatorySupportedCCs_Rev01
  • [x] ✅ Management Command Classes
    • [x] ✅ CCM_VersionCmdClassCCVersions_Rev01
  • [x] ✅ Role Type Requirements
    • [x] ✅ RT_ListeningFlag_Rev02
  • [x] ✅ S2 Certification
    • [x] ✅ S2C_02070_DuplicateDetection_Rev01
    • [x] ✅ S2C_02566_MsgEncapCorrectResponse_Rev01
    • [x] ✅ S2C_02567_MsgEncapSequenceResAndExt_Rev01
    • [x] ✅ S2C_02568_MsgEncapExtensionObject_Rev01
    • [x] ✅ S2C_02570_MsgEncapEncyptedExtensions_Rev01
    • [x] ✅ S2C_02571_MsgEncapSPANExtension_Rev01
    • [x] ✅ S2C_02572_MsgEncapMPANExtension_Rev01
    • [x] ✅ S2C_02575_MsgEncapMGRPExtension_Rev01
    • [x] ✅ S2C_02576_MsgEncapMOSExtension_Rev01
    • [x] ✅ S2C_02578_SPANManagement_Rev01
    • [x] ✅ S2C_02580_S2NonceGet_Rev01

Category: Interactive

  • [ ] Command Class Control Requirements
    • [x] ✅ CCR_AssociationCC_Rev02
    • [x] ✅ CCR_BarrierOperatorCC_Rev02
    • [x] ✅ CCR_BasicCC_Rev02
    • [x] ✅ CCR_BatteryCC_Rev02
    • [x] ✅ CCR_BinarySwitchCC_Rev02
    • [x] ✅ CCR_CentralSceneCC_Rev03
    • [ ] ❌ CCR_ColorSwitchCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/51 [^6]
    • [ ] ❌ CCR_ConfigurationCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/12
    • [x] ✅ CCR_DeviceResetLocallyCC_Rev02
    • [x] ✅ CCR_DiscardLowerSecurityCmds_Rev02
    • [ ] ❌ CCR_DoorLockCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/21
    • [ ] ❌ CCR_EntryControlCC_Rev03 - https://github.com/zwave-js/certification-backlog/issues/14
    • [x] ✅ CCR_ImplementedCCVersions_Rev02
    • [x] ✅ CCR_IndicatorCC_Rev02
    • [x] ✅ CCR_InterviewCompletion_Rev02
    • [ ] ❌ CCR_MeterCC_Rev02 - https://github.com/home-assistant/core/issues/116222
    • [x] ✅ CCR_MultiChannelCC_Rev03
    • [ ] ❌ CCR_MultilevelSensorCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/26
    • [x] ✅ CCR_MultilevelSwitchCC_Rev02
    • [x] ✅ CCR_NotificationCC_Rev04 - [^5]
    • [ ] ❌ CCR_SoundSwitchCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/16
    • [x] ✅ CCR_ThermostatModeCC_Rev02 - Note: Needs service call, document this!
    • [ ] ❌ CCR_ThermostatSetbackCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/15
    • [ ] ❌ CCR_ThermostatSetpointCC_Rev03 - https://github.com/home-assistant/core/issues/92810
    • [ ] ❌ CCR_UserCodeCC_Rev04 - no user code support in HA
    • [x] ✅ CCR_VersionCC_Rev02
    • [x] ✅ CCR_WakeUpCC_Rev02
    • [ ] ❌ CCR_WindowCoveringCC_Rev02 - https://github.com/zwave-js/certification-backlog/issues/30
  • [x] ✅Command Class Support Requirements
    • [x] ✅ CSR_CommandClassIsSupported_Rev02
    • [x] ✅ CSR_LifelineMandatoryReports_Rev02
  • [x] ✅ Common Device Requirements
    • [x] ✅ CDR_ZWPv2IndicatorCCRequirements_Rev01 - Note: Document that this shows up in the notification sidebar
  • [x] ✅ Controller Requirements
    • [x] ✅ CR_NodeCommissionInterviewOnRequest_Rev01
    • [x] ✅ CR_NoInterviewOnConfigReport_Rev01
  • [x] ✅ Device and Role Type
    • [x] ✅ DR_DeviceAndRoleTypeSelection_Rev01
  • [x] Device Data
    • [x] ✅ DD_ManufacturerSpecificCCData_Rev01
    • [x] ⁉️ DD_VersionCCData_Rev01 - Passes! Form needs to be updated to actual versions prior to certification
    • [x] ✅ DD_ZWavePlusInfoCCData_Rev01
  • [x] ✅ Device Type Requirements
    • [x] ✅ DT_MandatoryControlledCCs_Rev02
  • [ ] Product Information
    • [ ] ⁉️ PI_CertificationData_Rev01 - The requested info does not exist on the certification page
  • [x] ✅ Role Type Requirements
    • [x] ✅ RT_PowerSupply_Rev02
  • [x] ✅ S2 Certification
    • [x] ✅ S2C_02010_S2NonceGetAndReport_Rev01
    • [x] ✅ S2C_02579_MPANManagement_Rev01

✅ Category: Scripts

  • [x] ✅ AssociationCmdClassV3_Rev04_Orig
  • [x] ✅ AssociationGrpInfoCmdClassV3_Rev02_Orig
  • [x] ✅ BasicMappingNotAllowed_Rev03_Orig
  • [x] ✅ IndicatorCmdClassV4_Rev05_Orig
  • [x] ✅ MultiChannelAssociationCmdClassV4_Rev06_Orig
  • [x] ✅ MultiCommandCmdClassV1_Rev04_Orig
  • [x] ✅ VersionCmdClassV3_Rev09_Orig
  • [x] ✅ ZWavePlusInfoCmdClassV2_Rev09_Orig

Category: Inclusion

  • [ ] Command Class Control Requirements
    • [ ] ❌ CCR_MultiChannelAssociationCC_Rev02 - https://github.com/zwave-js/node-zwave-js/issues/5874 https://github.com/zwave-js/node-zwave-js/issues/5875
  • [x] ✅ Command Class Support Requirements
    • [x] ✅ CSR_LifelineReportToMultipleNodes_Rev01
    • [x] ✅ CSR_LifelineReportTriggeredByZWave_Rev01
    • [x] ✅ CSR_ManufacturerS0Replies_Rev01
  • [x] ✅ Common Device Requirements
    • [x] ✅ CDR_ControlMark_Rev01
    • [x] ✅ CDR_DeviceResetLocallyReport_Rev01
    • [x] ✅ CDR_EncapOrder_Rev02
    • [x] ✅ CDR_MulticastNoResponse_Rev02
    • [x] ✅ CDR_NIFRequirements_Rev01
    • [x] ✅ CDR_ResetWithoutDelivery_Rev01
    • [x] ✅ CDR_ResponseEncap_Rev02
    • [x] ✅ CDR_WhenNodeReset_Rev01
  • [x] ✅ Controller Requirements
    • [x] ✅ CR_InteroperabilityRequirements_Rev01
  • [x] ✅ Device Type Requirements
    • [x] ✅ DT_SecurityClasses_Rev01
  • [x] ✅ Network-Protocol Command Classes
    • [x] ✅ CCN_PowerlevelCmdClassTestNode_Rev01
  • [ ] Role Type Requirements
    • [x] ✅ RT_CSCIsFirstNodeInNetwork_Rev02
    • [x] ✅ RT_CSCLifelineConfigIfSIS_Rev02
    • [x] ✅ RT_CSCMustAssumeSISRole_Rev02
    • [x] ✅ RT_CSCMustSupportS0AndS2_Rev01 - Note: 2nd inclusion must use "Legacy Secure"
    • [x] ✅ RT_CSCWakeUpConfigIfSIS_Rev02
    • [x] ✅ RT_CSCWakeUpConfigPENIfSIS_Rev01
    • [x] ✅ RT_SISReplaceFailingNode_Rev02 - Not supported by HA
    • [ ] ❌ RT_SUCAndSISFunctionality_Rev01 - No PIN entry shown for remote inclusion: https://github.com/zwave-js/node-zwave-js/issues/5888
    • [x] ✅ RT_WakeUpNoMoreToReportingNodes_Rev01
  • [ ] S2 Certification
    • [x] ✅ S2C_02008_SecureInclusion_Rev01
    • [x] ✅ S2C_02014_ManageFullSPANTable_Rev01
    • [ ] ❌ S2C_02487_InclusionControllerCC_Rev01 - https://github.com/zwave-js/node-zwave-js/issues/5888
    • [x] ✅ S2C_02561_PublicKeyReport_Rev01
    • [x] ✅ S2C_02562_KEXReport_Rev01
    • [x] ✅ S2C_02563_WrongKEXReport_Rev02
    • [x] ✅ S2C_02564_KEXGet_Rev01
    • [x] ✅ S2C_02573_MsgEncapMPANExtMoreToFollow_Rev01
    • [x] ✅ S2C_02574_MsgEncapMPANExtGroupID_Rev01
    • [x] ✅ S2C_02577_CompatibilityConsiderations_Rev01
    • [x] ✅ S2C_02581_S2NonceReport_Rev01
    • [x] ✅ S2C_02582_WrongPublicKeyReport_Rev01
    • [x] ✅ S2C_02583_NetworkKeyGet_Rev01
    • [x] ✅ S2C_02587_WrongTransferEnd_Rev01
    • [x] ✅ S2C_02588_S2CmdsSupportedGet_Rev01
    • [x] ✅ S2C_02590_S2CmdsSupportedReport_Rev01
    • [x] ✅ S2C_02593_KEXSet_Rev01
    • [x] ✅ S2C_02594_WrongKEXSet_Rev01
    • [x] ✅ S2C_02596_KeyExchange_Rev01
    • [ ] ⁉️ S2C_02599_FailingKeyExchange_Rev01 - Step 2.3: Granting no keys is not possible, because the user is prevented from doing so after choosing "secure if possible". Other steps pass
    • [x] ✅ S2C_02600_BootstrappingInterruptPoints_Rev01
    • [x] ✅ S2C_02601a_BootstrTimeoutsIncludingNode_Rev01
    • [x] ✅ S2C_02601b_BootstrTimeoutsJoiningNode_Rev01
  • [ ] Security Requirements
    • [x] ✅ S0_ControllerCanIncludeS0Devices_Rev01
    • [x] ✅ S0_ControllerPerformsS0Bootstrap_Rev01
    • [x] ✅ S0_DelayNetworkKeyVerify_Rev01
    • [x] ✅ S0_DelayNonceGetInclusionByDUT_Rev01
    • [x] ✅ S0_DelayNonceReportInclusionByDUT_Rev01
    • [x] ✅ S0_DelaySchemeReport_Rev01
    • [x] ✅ S0_DelaySchemeReportAfterSchemeInherit_Rev01
    • [x] ✅ S0_InvalidMACInNetworkKeyVerify_Rev01
    • [x] ✅ S0_InvalidNetworkKeyVerify_Rev01
    • [x] ✅ S0_InvalidNonceInNetworkKeyVerify_Rev01
    • [x] ✅ S0_SecurityS0IsImplemented_Rev01
    • [x] ✅ S0_SendInvalidSchemeReport_Rev01
    • [x] ✅ S2_AbortAfterEmptyKexReport_Rev01
    • [x] ✅ S2_BootstrTimeoutsIncludingNode_Rev01
    • [x] ✅ S2_BootstrTimeoutsJoiningNode_Rev01
    • [x] ✅ S2_CommandClasses_Rev01
    • [x] ✅ S2_ConfirmForAuthenticated_Rev01
    • [x] ✅ S2_ControllerMustAcceptExtendedCCs_Rev01
    • [x] ✅ S2_ControllerPerformsBootstraping_Rev01
    • [x] ✅ S2_GrantedS2Classes_Rev01
    • [x] ✅ S2_S0IncByNonSecureIncController_Rev01
    • [x] ✅ S2_SeveralSeparateECDHKeyPairs_Rev02
    • [ ] ❌ S2_SISInclusionCCSide_Rev02 - https://github.com/zwave-js/node-zwave-js/issues/5888
    • [x] ✅ S2_SISMustHaveS2ClassInputAndDisplay_Rev01
    • [x] ✅ S2_SISMustPerformS2Bootstrap_Rev01
    • [x] ✅ S2_SISMustSupportAnyS2ClassCombination_Rev01
    • [x] ✅ S2_StopParsingAtCommandClassMark_Rev01
    • [x] ✅ S2_SupportedCCRequirement_Rev02
    • [x] ✅ S2_WarningHighestKeyNotGranted_Rev01
    • [ ] ⁉️ S2_WarningS0BootstrapInS2Network_Rev01 - The 2nd test checks an impossible scenario: To use S0 over no security, the user has to manually opt in (except for certain device types). So the user is aware that the device is using S0 only.
  • [x] ✅ Smart Start Requirements [^3] [^4]
    • [x] ✅ SSR_CanPerformSmartStartInclusion_Rev01
    • [x] ✅ SSR_MustPerformS2BootstrapTest3_Rev02
    • [x] ✅ SSR_NodeProvListAddRemoveAndStatus_Rev01 [^2]
    • [x] ✅ SSR_NodeProvListAdvancedJoining_Rev01
    • [x] ✅ SSR_NodeProvListBootstrappingMode_Rev01
    • [x] ✅ SSR_NodeProvListInclusionSetting_Rev01
    • [x] ✅ SSR_PendingNodeProvisioningListEntry_Rev02
    • [x] ✅ SSR_RetryAfterFailedInclusion_Rev02
    • [x] ✅ SSR_S2AuthenticatedInclusionSupport_Rev01
    • [x] ✅ SSR_SmartStartControlNotAsInclCtrl_Rev01
  • [x] ✅ Transport-Encap Command Classes
    • [x] ✅ CCT_TransportServiceCmdClassV2_Rev02

Category: Manual

  • [ ] Command Class Control Requirements
    • [x] ✅ CCR_DocuRequirement_Rev01
    • [x] ✅ CCR_FirmwareUpdateMetaDataCC_Rev02
    • [ ] ❌ CCR_FullyOrPartiallyControlledCmdClasses_Rev01 - Need to decide which functionality to hide due to missing HA support, also document controlled CCs
    • [x] ✅ CCR_NoControlDefinition_Rev01
    • [x] ✅ CCR_Security0CC_Rev02
    • [x] ✅ CCR_Security2CC_Rev02
    • [x] ✅ CCR_SecurityClassLearning_Rev01 - No support for learn mode, DUT "learn" security classes during device inclusion
    • [x] ✅ CCR_SupervisionCC_Rev02
  • [x] ✅ Command Class Support Requirements
    • [x] ✅ CSR_ControlViaAssociationSecurityLearning_Rev01 - DUT does not send any controlling command via any association group, no support for learn mode, DUT "learn" security classes during device inclusion
    • [x] ✅ CSR_FirmwareUpdateCCConfig_Rev01 - No support for learn mode, DUT firmware cannot be OTA updated
    • [x] ✅ CSR_SupervisionEncapViaAssociation_Rev01 - No support for learn mode, DUT "learn" security classes during device inclusion
  • [x] ✅ Common Device Requirements
    • [x] ✅ CDR_ApplicationStatusCCSupport_Rev01
    • [x] ✅ CDR_DetermineResetCompleted_Rev01
    • [x] ✅ CDR_DynamicCapabilities_Rev01
    • [x] ✅ CDR_MulticastNoGet_Rev01
    • [x] ✅ CDR_PollingRequirements_Rev01 - Z-Wave JS does not poll automatically
    • [x] ✅ CDR_ResponseTimeouts_Rev01
    • [x] ✅ CDR_UnsolicitedFrames_Rev01 - Z-Wave JS does not send unsolicited frames
    • [x] ✅ CDR_UseOfMultiCommandEncap_Rev01 - Z-Wave JS does not send Multi Command frames, it only decodes them
  • [x] ✅ Controller Requirements
    • [x] ✅ CR_MinimumControlFunctionality_Rev02
    • [x] ✅ CR_NetworkManagementFunctionality_Rev01
  • [ ] Long Range Requirements
    • [ ] ❌ LR_ControllerBootstrappingModeTLV_Rev01 - Driver passes, HA has no support yet
    • [ ] ❌ LR_ControllerCanIncludeNodesIntoLRNetwork_Rev01 - Driver passes, HA has no support yet
    • [ ] ❌ LR_ControllerGrantsKeysRequiringAuthentication_Rev01 - Driver passes, HA has no support yet
    • [ ] LR_ControllerUsesDifferentKeysForClassicAndLR_Rev01
  • [x] ✅ Management Command Classes
    • [x] ✅ CCM_AssociationCmdClass_Rev01 - Note: Only Device Reset Locally is sent via Associations
    • [x] ✅ CCM_AssociationGrpInfoCmdClass_Rev01 - Note: Only Device Reset Locally is sent via Associations
  • [ ] Product Information
    • [x] ✅ PI_CertifiedZWaveStack_Rev01
    • [ ] PI_MarketAndFrequency_Rev02
    • [x] ✅ PI_ZWavePlusProductType_Rev01
  • [ ] Required Documentation
    • [ ] RD_AdditionalDocuRequirements_Rev01
    • [ ] RD_AssociationCC_Rev01
    • [ ] RD_FactoryDefaultReset_Rev01
    • [ ] RD_IndicatorCC_Rev01
    • [ ] RD_LongRange_Rev01
    • [ ] RD_MultipleManufacturers_Rev01
    • [ ] RD_SecurityS2CC_Rev01
    • [ ] RD_SmartStart_Rev02
    • [ ] RD_SmartStartControllers_Rev01
    • [ ] RD_SupervisionCC_Rev01
    • [ ] RD_ZWPv2ControllerDocu_Rev02
    • [ ] RD_ZWPv2RequiredTerminology_Rev01
  • [x] ✅ Role Type Requirements
    • [x] ✅ RT_SupportedSmartStartFunctionality_Rev01
  • [x] ✅ Security Requirements
    • [x] ✅ S2_GrantingSecurityClasses_Rev01
    • [x] ✅ S2_SecurityClassAdjustment_Rev01
    • [x] ✅ S2_SISMaySupportCSA_Rev01
    • [x] ✅ S2_UpgradeExistingDevice_Rev01
  • [x] ✅ Smart Start Requirements
    • [x] ✅ SSR_ControllerHasQRScanning_Rev01
    • [x] ✅ SSR_ControllerHasGen1QRScanning_Rev01
  • [ ] Transport-Encap Command Classes
    • [ ] ⁉️ CCT_CRC16EncapsulationCmdClassV1_Rev01 [^1]

[^1]: ⁉️ CRC-16_Command_Class_advertised_in_NIF - Should be covered by automatic NIF tests, learn mode not supported ⁉️ CRC-16_Command_Class_not_advertised_in_Security_Commands_Support - Learn mode not supported ✅ CRC-16_Command_Class_MUST_NOT_be_encapsulated_by_any_other_command_class - If more than one encapsulation command class is used for controlled command classes the test houses must receive sufficient information about how such encapsulated messages can be triggered: - Supervision: Control any device that supports Supervision - Security S0/S2: Control any securely included device - Multi Channel: Control an endpoint on any supporting device ✅ CRC16_Command_Class_must_answer_as_asked ✅ Receive_and_interpret_all_command_classes_in_the_NIF_also_with_CRC-16_encapsulation ✅ Ensure_destination_supports_CRC-16_Encapsulation_Command_Class - Z-Wave JS does not use CRC-16 to control on its own.

[^2]: Is the node reported as included in the network? - Yes, after closing and re-opening the provisioning list.

[^3]: To add a device using Gen1 QR code, press SUBMIT in the dialog despite any warnings. Add mode will be started anyways.

[^4]: To add a DSK to the provisioning list, use https://zwave-js.github.io/qr/ and copy the QR code string or scan the image.

[^5]: Some events appear at the top, some under Diagnostics at the bottom. Updating the notification list is done by updating the Z-Wave integration and/or addon
HA currently requires manually enabling some notifications: https://github.com/orgs/zwave-js/projects/1?pane=issue&itemId=63127528

[^6]: For viewing the current color, the device needs to be turned on

AlCalzone avatar Jun 02 '23 13:06 AlCalzone