Maxim Martynov

Allow to pass custom Cldr module

2

If one uses any language defined in Linguist.Cldr, it's fine, linguist will handle translations as usual. If one uses any language which is unknown for Linguist.Cldr, but does not use...

[DOP-14058] Improve Kafka documentation

1

## Change Summary * Move `parse_column` and `serialize_column` examples to docstrings. [Example](https://onetl--276.org.readthedocs.build/en/276/file_df/file_formats/avro.html#onetl.file.format.avro.Avro.parse_column) * Move example of Kafka DataFrame to separate page, and merge it with serialization/deserialization examples. [See](https://onetl--276.org.readthedocs.build/en/276/connection/db_connection/kafka/serde.html) ## Related...

Fix OpenLineage ingestor

1

### Describe your changes: OpenLineage ingestion wasn't updated to include changes from #15201 and #15130. Ingestion failed with errors like: ``` [2024-05-24, 15:14:18 UTC] {taskinstance.py:1937} ERROR - Task failed with...

I'm using PySpark a lot (which uses py4j under the hood), and quite often faced an issue with stopping Spark session. Calling `gateway.shutdown()` stops just Python threads and closes socket,...

[DOP-18743] Set default jobDescription

1

## Change Summary Set jobDescription in any method which interact with Spark in some way. Examples:   This does not work if Spark action was triggered outside...

Add Spark 4.0 support

1

## Change Summary * Update CI matrix to include Spark 4.x. By default, it is not used for tests, unless some integration has been changed (like 2.x). * Spark 4.x...

## Change Summary Python 3.13 does not allow inheritance like `A -> B, A -> C, B+C -> D`, this leads to raising error `TypeError: multiple bases have instance lay-out...

Bug Report: dependabot PRs cannot upload coverage

2

**Describe the bug** I have 2 pull requests created by bots: * pre-commit[bot]: https://github.com/MobileTeleSystems/data-rentgen/pull/74 * dependabot[bot]: https://github.com/MobileTeleSystems/data-rentgen/pull/73 First one successfully run tests, collected coverage and pushed it to Codecov: https://github.com/MobileTeleSystems/data-rentgen/actions/runs/11114600996/job/30881427873#check-step-9...

I have a REST API endpoint `GET /v1/runs` returning paginated list of `Run` objects, which has mandatory query params `since` (datetime) and `search_query` (text with minimal length). If I call...

Spark has a feature allowing to automatically download packages from MavenCentral/SonatypeOSS repo: ```python spark = SparkSession.builder.config("spark.jars.packages", "com.mygroup:artifact:version").getOrCreate() ``` It is very convenient to use, as there is no need to...