Development Environment

[semohr]

Hello y'all,

what is the minimal approach to get a developer environment going? I'm mainly interested in the frontend for now.

The documentation points to the scicatlive or is broken. As far as I can see scicatlive does not include a straight forward way to get an editable frontend repo in dev mode. Am I missing something?

I would expect to be able to build the container myself and live mount the frontend repository. For most containerized FOSS projects this is the quasi standard.

Your compose setup seem quite complicated and I have not yet had an indepth look. In general I would expect something like this:

services:
    frontend:
        container_name: scicat_frontend
        build:
            context: ../
            dockerfile: docker/Dockerfile.frontend
        volumes:
            # Live mount for debuging
            - ./:/app
    [other services from registry]

I hope I'm not missing something here. Adding something like a standard dev setup would improve on-boarding significantly imo...

I'm happy to contribute, but I’d prefer a straightforward way to set up a usable dev environment without spending too much time on configuration.

Best, Sebastian

[nitrosx]

Dear Sebastian thank you for opening this issue and giving us the opportunity to work on the documentation and best practices. I will start sharing my dev environment... super easy and due more to laziness that anything else...

• a mongodb container with persistent volume
• a folder mirroring the backend repo with the correct branch. Local configuration and npm start
• a folder mirroring the frontend repo with the correct branch. Local configuration and npm start

That's it!!! No complicated docker containers, just code running on my bare-bone OS (which is a linux distro)

If you have a good idea and are able to implement it for an easy dev setup, I would love for you to share and contribute it back... even just a simple description.

[semohr]

Thanks for sharing your setup! I see the benefits of keeping things simple, but running everything directly on the host system can lead to inconsistencies, especially on more esoteric setups.

For me, a minimal Docker-based setup would be preferable because:

• It ensures a consistent environment across different machines and between dev and prod
• It avoids potential conflicts with system-wide dependencies
• It allows for quick cleanup and resets without affecting the host system.

Especially for multi service based applications fixing port bindings issues or environment based issues can be troublesome. I think in the ideal world, one command and at worst some centralized changes to a compose file or config file would be perfect for a dev setup. An initially seeding for the database would also be nice addition.

For me personally, it’s also a matter of privacy and trust -- I prefer running everything in a controlled, isolated environment rather than installing them directly on my system. The alternative is running a VM but this takes significantly more effort.

I started with a minimal docker based setup in this repo. This still has some issues tho. I'm not sure how to configure the frontend in dev mode. The (docs) don't really help as the dist folder is not generated in dev mode. How are configuration files loaded in the frontend? (Is that an angular feature or something selfmade?) I was looking for something like BACKEND_URL=... or a folder where I can place a config.overrides.json file.

[nitrosx]

@semohr I totally agree with you and I do appreciate your contribution. Myself or other contributors will check it and provide feedback.

Do you envision that it might be useful for others too? Is it a platform agnostic solution? Would you be open to make it an official project/repo? ...and maintain it?

[nitrosx]

Regarding incorrect or missing documentation, please open an issue in the documentation repo. Although we are working hard to update the documentation, we are struggling in keeping up with it. Any help is more than welcome.

Regarding SciCatLive, the maintainers are pretty responsive and active. I encourage you to reach out to them.

[semohr]

Well (if it fully works) this should be generally applicable to any environment that supports docker compose. We would have to test windows, mac and maybe arm but I don't see why this shouldn't work (I don't know about all your dependencies, just assuming they are mostly platform agnostic).

Let's keep it in my repo for now, we can still migrate it once it is a bit more mature I guess. As mentioned earlier this still has some issues and I haven't really spend much time setting it up and debugging it yet. Regarding this, some pointers on how to setup the proxy configuration for the frontend would be highly appreciated :) (Even if i change local.proxy.config.json the frontend still wants to make requests to localhost:3000).

[bpedersen2]

One of the things to keep in mind here is:

The production frontend is served from compiled sources (nginx serving static files, no node available inside the conatiner), but for development you want a dynamically served version via npm serve with live reloading).

So you really need a separate dev container for the frontend.

[semohr]

I'm normally using react and bundling with vite, which works very similar. Static files are transpiled for the production mode while one can run a node server for development. In dev mode I set a proxy for all backend requests. The dev setup should normally work without nginx.

The thing I'm searching for is a way to configure the backend urls. I found that using the angular proxyConfig partially works, but there are still some requests which point to localhost:3000.

I.e.

GET    http://localhost:3000/api/v3/datasets/metadataKeys?limits={"mode":{}}
GET    http://localhost:3000/api/v3/datasets/fullquery
GET    http://localhost:3000/api/v3/datasets/fullfacet

Not sure why and where this happens but it seem to me like the url might be hardcoded.