Daniel Gafni

Draft: [dagster-polars] Pandera integration for DataFrame schema validation :safety_vest:

1

## Summary & Motivation This PR: - :sparkles: adds Pandera support to dagster-polars (resolve #20584). The goal is to be able to load/save Pandera-validated DataFrames like this: ```python from dagster...

### What's the use case? Currently, Dagster's internal typing does not support generics. At least, it's not possible to automatically resolve `DagsterType` objects from type hints with generics. I am...

`path.iterdir()` yields `path` itself as the first item (with S3)

6

Given the following objects on S3: ``` s3://my-bucket/my-directory/0.txt s3://my-bucket/my-directory/1.txt ``` UPath("s3://my-bucket/my-directory").iterdir() yields: ``` s3://my-bucket/my-directory s3://my-bucket/my-directory/0.txt s3://my-bucket/my-directory/1.txt ``` The first item is wrong, right?

**Is your feature request related to a problem? Please describe.** `runpod` is built on top of `FastAPI`, yet it currently doesn't support one of FastAPI's main features: automatic serialization/deserialization with...

[feature request]: automatic markdown documentation generation

5

A pretty useful feature Typer [has](https://typer.tiangolo.com/tutorial/typer-command/#generate-docs) is automatic documentation generation. It's really helpful to have this as part of `pre-commit` hooks and in the CI pipeline. I wonder if `cyclopts`...

[dagster-aws] add `PipesEMRClient`

2

## Summary & Motivation This PR adds `PipesEMRCLient` to `dagster-aws`. As part of this PR, I decided to try standardizing the various AWS PIpes clients by making a common base...

[dagster-aws] ignore pyright errors with boto3-stubs

2

## Summary & Motivation Ignore pyright errors in `dagster-aws` which popped out after the introduction of `boto3-stubs-lite`. I suggest we slowly remove these ignore comments and fix the code for...

[dagster-aws] add `params` argument to `PipesGlueClient.run `and deprecate direct arguments

3

## Summary & Motivation This PR alignes `PipesGlueClient`'s API with `PIpesECSClient`. It adds a `params` dict argument for AWS API params. Old arguments are marked as deprecated. ## How I...

[dagster-aws] [docs] add documentation for ECS Pipes

3

This PR adds examples for using Dagster Pipes with AWS ECS. --- ## Summary & Motivation ## How I Tested These Changes

[dagster-aws] add Pipes ECS client

8

## Summary & Motivation Adds Dagster Pipes ECS client ## How I Tested These Changes - [x] add moto test for execution - [x] add moto test for interruption