tez icon indicating copy to clipboard operation
tez copied to clipboard

TEZ-4007: Introduce AmExtensions and Zookeeper-based FrameworkServices

Open abstractdog opened this issue 6 months ago • 30 comments

This PR contains a Zookeeper-based Framework service and a major refactor in this area.

Pieces of this PR:

  • include curator dependency
  • remove amUuid from DAGAppMaster
  • harmonized id/opaque id to externalId in terms of variable and env constant name
  • FrameworkService base interface: marker interface for framework-level services, implementations are framework-specific (currently yarn or zookeeper)
    • ClientFrameworkService: interface for client-side framework services; users of this class can acquire tez clients for the actual framework (yarn/zookeeper)
    • ServerFrameworkService: interface for server-side framework services; users of this class can acquire server-side logic for the actual framework (yarn/zookeeper)
  • AMExtensions: a class that encapsulates all the pluggable logic within DAGAppMaster
  • update unit tests to provide an AMExtensions to prevent NPE while mocking

abstractdog avatar Sep 06 '25 12:09 abstractdog