Application examples?

[leplatrem]

Is rustus supposed to be exposed to the world or proxied by a "domain-specific" web service?

Did I understand well from this example:

https://github.com/s3rius/rustus/blob/320625059dd8ac362f29471952b596ad8348f911/docs/hooks.md?plain=1#L852-L855

That hooks could be used to authenticate file uploads?

I think it would be interesting as a narrative example to unroll a whole demo app that leverages rustus. Or give more context on why it was built? Or for example, answer questions like "if I want to built a WeTransfer clone, what is the intended way to integrate rustus?"

If you're willing to give me a few pointers, I'd be glad to contribute something :)

[s3rius]

HI! And thanks for you question. Rustus is supposed to be opened to the world. To make it more secure, you can use CORS.
If you have no hooks configured, then everyone can upload anything.

But if you have file hooks or http hooks, then on pre-create hook you can easily abort uploads by either exiting with non-zero status code (if you use file hooks) or return not 200 status code from your application (if you use http hooks) .

When upload is ready you can add post-finish hook to handle events when upload is complete.

I use rustus to upload renedered videos to one of my side-projects.

@router.post("/rustus_hooks")
async def rustus_hooks(
    hook: RustusHook,
    auth: AuthJWT = Depends(),
    hook_name: Optional[HookName] = Header(None),
) -> None:
    """
    Upload stream in database.

    :param hook: hook information.

    :param auth: service to validate authorization header.

    :param hook_name: hook name

    :raises HTTPException: if you dumb enough to simulate rustus.
    """
    auth.jwt_required()
    upload_id = hook.upload.id
    if hook_name == HookName.PRE_CREATE:
        logger.info("Stream upload started.")
        return
    elif hook_name == HookName.POST_TERMINATE:
        logger.info(f"Removing stream for upload_id={upload_id}")
        await Stream.filter(upload_id=hook.upload.id).delete()
    elif hook_name == HookName.POST_FINISH:
        logger.info(f"Stream upload_id={upload_id} successfully uploaded")
        user = await AdminTable.get(username=auth.get_jwt_subject())
        await Stream.create(
            upload_id=hook.upload.id,
            order=await Stream.all().count(),
            path=hook.upload.path,
            created_by=user,
        )
    else:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail="Are you drunk?",
        )

That's the whole code to handle uploads. I use rustus with these parameters:

    RUSTUS_MAX_BODY_SIZE: "1000000"
    RUSTUS_DIR_STRUCTURE: "{year}/{month}/{day}"
    RUSTUS_TUS_EXTENSIONS: "getting,creation,termination"
    RUSTUS_HOOKS_HTTP_PROXY_HEADERS: "Authorization"
    RUSTUS_HOOKS_HTTP_URLS: "http://{my_service}/rustus_hooks"
    RUSTUS_HOOKS: "pre-create,post-terminate,post-finish"

I hope it helped you.

[leplatrem]

That definitely helps! Thank you :)

[s3rius]

I'm really glad, you find this project useful.

If it works for you, please consider closing the issue.

[s3rius]

Or maybe before closing it would be nice to add examples section in docs.

[leplatrem]

Yes I think it would be useful to show a few recipes/patterns in the docs.

I'm thinking one of:

• minimal example with a tus client
• authenticated transfer (the above example)
• wetransfer-like (several files are added in something like a virtual folder)

[s3rius]

Good set of cases. I will try to implement suggested examples and will put them in folder inside this repo.

[tensorflowters]

Hi @s3rius since I discovered 2 years ago I used it at any right opportunity and I really like your tool 🔥

But I was starting to miss connecting rustus to kafka in order to directly to publish at specific topic... After having used rabbitMQ for a long time, I would like to stick with kafka for multiple reasons ⚡️

I'm currently working in a compose file like I always do and can't get working with kafka... I tried bitnami, ubuntu and confluent versions but I always and up with broker down or weird error I'm struggling too process.

Could do you me the favor to provide me a docker compose file using using rustus and kafka on the same internal network with an example of appropriate settings ? ⭐️

I can't get your's (in the repo) working either as it seems to me that you connect with rustus from an external access (PLAINTEXT/EXTERNAL) listeners on port 9094 instead of port 9092 PLAINTEXT:PLAINTEXT.

Thanks for your work and have a nice day ! 🚀

[s3rius]

Hello! Here's a small example of how you can declare kafka containers without exposing their ports from internal network.

https://gist.github.com/s3rius/d33ba2bf6a576e1272cc21ebedc8f1b5

They configure 3 listeners such as:

• PLAINTEXT on port 9092 that is used by clients
• INTERNAL on port 9093 for inter-node communication
• KRAFT on port 9094 for cluster metadata controller

As you can see, in advertised.listeners for PLAINTEXT listener i use internal docker hostname so clients connected to this kafka instance will be advertised with internal hostnames.

Also, I made a super simple python app that listens to some events and print them in stdout.

Hope it helps.

[tensorflowters]

Thanks a lot !!! You're so kind, it's a lot clearer now ! Again, great work 🏆

Thanks a lot, Have a nice day