lighter
lighter copied to clipboard
exacaster
REST API for Apache Spark on K8S or YARN
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.12.11 to 20.12.12. Commits See full diff in compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps [react-router-dom](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router-dom) from 6.23.0 to 6.23.1. Release notes Sourced from react-router-dom's releases. [email protected] Patch Changes Updated dependencies [email protected] [email protected] [email protected] Patch Changes Updated dependencies [email protected] [email protected] [email protected] Patch Changes Updated...
currently session ID is created as be417708-645b-4302-8972-ebc9f6ead47c using below configuration. { "id": "session1", "submit-params": { "name": "session1", "numExecutors": 4, "executorCores": 2, "executorMemory": "2G", "driverCores": 2, "driverMemory": "1G", "conf": { "spark.pyspark.python":...
``` spec: containers: - env: - name: LIGHTER_KUBERNETES_ENABLED value: "true" - name: LIGHTER_MAX_RUNNING_JOBS value: "15" - name: LIGHTER_KUBERNETES_SERVICE_ACCOUNT value: lighter - name: LIGHTER_URL value: http://lighter-lighter-chart-lighter.sf.svc.cluster.local:8080 - name: KUBERNETES_CLUSTER_DOMAIN value: cluster.local...
Looking through the code of lighter i find that there is a check for "zombie". I imagine it is done to cover the case in which session is somehow killed/terminated...
Hi, It would be a nice feature if in addition to the REST endpoints there are new webhooks to notify on status change of : - session - batch -...
Hi, If the DB is hoster in a postgres instance isolated from Lighter, can Lighter be scaled horizontally ? For exemple : - 2 instances of Lighter for better HA...
Hi, The initial settings is maximum 5 batches running + 5 starting. There are no settings on the max number of sessions. Do you have some idea of the maximum...