iris-web icon indicating copy to clipboard operation
iris-web copied to clipboard

[FR] Helm Chart

Open JuanPabloSGU opened this issue 2 years ago • 16 comments

Is your feature request related to a problem? Please describe. The feature I would like to request would be to have an helm chart available to deploy in conjunction with the docker-compose method of deployment.

Describe the solution you'd like Either the reason on why the helm chart has not been created (if there is any limitations, future plan, etc...) or an helm chart can be developed.

Describe alternatives you've considered This could be a later step for the project which I can understand but I would be more than willing to help out with the creation of the helm chart.

Additional context None

JuanPabloSGU avatar May 12 '23 03:05 JuanPabloSGU

Hello, JuanPablo

I hope this message finds you well. I wanted to express my interest and enthusiasm in writing a Helm chart for your application. I'm excited about the opportunity to contribute to your project.

I assure you that I will put in my best effort to ensure that the Helm chart is well-structured, scalable, and meets your expectations. I'm open to any preferences or ideas you may have regarding the organization or implementation of the Helm chart.

manjularajamani avatar May 25 '23 10:05 manjularajamani

Hi @manjularajamani!

Thanks a lot! You are more than welcome! 👍

whikernel avatar May 25 '23 11:05 whikernel

Thanks @manjularajamani for helping. I have a few notes for you to keep in mind for the helm chart. The first issue that I foresee happening is that the application runs on localhost. This means that the containers must be in the same deployment file to have each container be able to connect locally. The docker compose file gets around this by having a network to allow communication from the front-end to the Postgresql database for example. If the container is ran within the same pod, this means that there wouldn't be any support with scaling each pod independent from each other which is another issue. I would ask recommend the final helm chart to be deployed to Artifact Hub, which is popular location for helm charts to be pulled from. Good luck :) - let me know if you run into any issues

JuanPabloSGU avatar May 26 '23 01:05 JuanPabloSGU

The first issue that I foresee happening is that the application runs on localhost. This means that the containers must be in the same deployment file to have each container be able to connect locally.

  • It's not a big issue in the helm to expose the application on the browser. You have pointed localhost to expose applications on the browser using docker-compose.

The docker-compose file gets around this by having a network to allow communication from the front end to the Postgresql database for example.

  • Simply we can able to expose the application using the load balancer and also we can connect the application(microservices) using the service name like the docker-compose service. There is no need for a network

I would ask to recommend the final helm chart be deployed to Artifact Hub, which is a popular location for helm charts to be pulled from.

  • I will be pushing the helm chart to the itTrident GitHub repository and that helm chart will be pushed to itTrident ArtifactHub by GitHub actions which are CI/CD

    https://artifacthub.io/packages/search?page=1&org=ittrident

Good luck :) - let me know if you run into any issues

Thanks, @JuanPabloSGU! Sure thing! How would I reach out to you?

manjularajamani avatar May 26 '23 05:05 manjularajamani

I believe there was a misunderstanding about how iris communicates through this docker compose network. I am not concerned about it being able to be expose to a browser. I am concerned that the containers for each image must be in the same deployment file to allow for the images within the pod to able to talk on localhost. I could be wrong here but this is what I have noticed while looking through the repository. If this load balancer can solve this issue that would be a good solution. For the helm chart, I just want to clarify that this itTrident repository will be maintained for after the helm chart has been created. This helm chart is for DFIR and should be kept within the repository in my opinion but I would ask @whikernel since they are the primary maintainer of the repository. For reaching out I don't have any direct communication, I just mention me on the PR that you end up making.

JuanPabloSGU avatar May 27 '23 00:05 JuanPabloSGU

Hi @JuanPabloSGU

In IRIS the containers can talk even if they are not on the same host.

  • Nginx (reverse proxy) is communicating with the WebApp. The WebApp can be set as remote by setting the IRIS_UPSTREAM_SERVER and IRIS_UPSTREAM_PORT vars
  • WebApp is communicating with Postgres. This can be set with POSTGRES_SERVER POSTGRES_PORT.
  • WebApp is communicating with RabbitMq. This can be set with CELERY_HOST.

Below is a really basic schema of the default structure. https://docs.dfir-iris.org/_static/iris_structure.png

My k8s knowledge is very limited so maybe I'm over-sighting something here.

As for the Helm Chart, yes it would be nice to keep it in the repo.

whikernel avatar May 27 '23 06:05 whikernel

I believe there was a misunderstanding about how Iris communicates through this docker-compose network. I am not concerned about it being able to be exposed to a browser. I am concerned that the containers for each image must be in the same deployment file to allow for the images within the pod to able to talk on localhost. I could be wrong here but this is what I have noticed while looking through the repository. If this load balancer can solve this issue that would be a good solution. For the helm chart, I just want to clarify that this itTrident repository will be maintained after the helm chart has been created. This helm chart is for DFIR and should be kept within the repository in my opinion but I would ask @whikernel since they are the primary maintainer of the repository. For reaching out I don't have any direct communication, I just mention me on the PR that you end up making.

Hello, JuanPabloSGU

As @whikernel mentioned above I will fork this repository and create the helm charts for this app. Also, I will make this helm chart appear on the public artifact so everybody can pull the helm chart and can deploy the IRIS on the K8s platform.

manjularajamani avatar May 30 '23 08:05 manjularajamani

Hello, @whikernel May I know the purpose of nginx here? While testing the docker-compose file the iris application can't able expose throw the nginx container IP. I trying to expose throw DNS but can't able expose

manjularajamani avatar Jun 03 '23 15:06 manjularajamani

Hi @manjularajamani

The Nginx is the reverse proxy and offers SSL/TLS termination. It is configured to proxy pass the requests to "app", which is the DNS name of the WebApp within the private docker network of iris. This network is also defined in the docker-compose.

Nginx is the only service exposed to all interfaces. The WebApp is not.

Let me know if it helps

whikernel avatar Jun 03 '23 16:06 whikernel

Hi @JuanPabloSGU

In IRIS the containers can talk even if they are not on the same host.

* Nginx (reverse proxy) is communicating with the WebApp. The WebApp can be set as remote by setting the `IRIS_UPSTREAM_SERVER` and `IRIS_UPSTREAM_PORT` vars

* WebApp is communicating with Postgres. This can be set with `POSTGRES_SERVER`
  `POSTGRES_PORT`.

* WebApp is communicating with RabbitMq. This can be set with `CELERY_HOST`.

Below is a really basic schema of the default structure. https://docs.dfir-iris.org/_static/iris_structure.png

My k8s knowledge is very limited so maybe I'm over-sighting something here.

As for the Helm Chart, yes it would be nice to keep it in the repo.

I think the docker-compose network is messing me up. I don't have to much experience with docker networks but it seems after some initial reading. I am not to sure how these networks can be emulated in a k8s environment as there doesn't seem like a one to one feature built in.

JuanPabloSGU avatar Jun 03 '23 16:06 JuanPabloSGU

Hi @whikernel /@JuanPabloSGU

I'm here to provide you with an update on my progress regarding the Helm Chart for the project.

I have completed the Helm Chart and have drafted a pull request for your review. The pull request can be found at the following link: https://github.com/dfir-iris/iris-web/pull/263

I have made sure to thoroughly test the Helm Chart and ensure its compatibility with the project. It is ready for your review and feedback.

However, during testing, I encountered a minor issue related to the login functionality. Although the logs indicate a successful login, I am unable to log into the Iris web. I have thoroughly reviewed the logs and cannot pinpoint the exact cause.

I wanted to provide you with the logs of the Iris Web App for your reference and analysis.

logs

manjularajamani avatar Jun 06 '23 12:06 manjularajamani

Hello @manjularajamani.

This looks like a good start, I have reviewed the pr and it seems that there isn't any inclusion of the ldap ability that iris offers. Was there a complication on your end with integrating it into the helm chart?

These are my recommendations:

  • Add more documentation for the value files. An example of a good standard to strive for is
## @section PostgreSQL Configuration
##
postgresql:
  ## @param postgresql.enabled Enables the Postgresql deployment
  ##
  enabled: true
  ## PostgreSQL Primary configuration parameters
  ##
  primary:
    ## PostgreSQL Primary persistence configuration
    ##
    persistence:
      ## @param postgresql.primary.persistence.size PostgreSQL Persistent Volume size
      ##
      size: 8Gi
    ## PostgreSQL primary resource requests and limits
    ## @param postgresql.primary.resources.limits The resources limits for the PostgreSQL primary containers
    ## @param postgresql.primary.resources.requests The requested resources for the PostgreSQL primary containers
    ## ref: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits
    resources:
      ## Example:
      ## limits:
      ##    cpu: 500m
      ##    memory: 1Gi
      limits: {}
      ## Example:
      ## requests:
      ##    cpu: 500m
      ##    memory: 1Gi
      requests: {}
  • With helm there are features such as the NOTES.txt and the helper.tpl files that can clean up some of the redudant values that are currently present in the helm chart. For example once the NOTES.txt file will display after the application has been installed in a cluster. The helper.tpl file can assist with having global settings such as the deployed namespace, application name and version that is installed.
  • I would also recommend that there should not be any hard coded values in the helm chart. Having these values able to be modified will make the experience for end users much easier.

Thank you again for your contribution :)

JuanPabloSGU avatar Jun 06 '23 22:06 JuanPabloSGU

Hello @JuanPabloSGU

This looks like a good start, I have reviewed the pr and it seems that there isn't any inclusion of the LDAP ability that Iris offers. Was there a complication on your end with integrating it into the helm chart?

I saw it was mentioned as optional on .env.model so I skipped and you guys didn't give any configuration belonging to LDAP on the docker-compose file. Does LDAP comes under Dockerfile or Docker compose file? or else I need to setup the LDAP server because I'm not familiar with the LDAP setup

I would also recommend that there should not be any hard-coded values in the helm chart. Having these values able to be modified will make the experience for end users much easier.

  • Sure and I didn't hardcode anything everything is the environment. The values will be changed inside the values.yaml by the user and they can run
  • And I will add your recommendations

manjularajamani avatar Jun 07 '23 06:06 manjularajamani

Hi @JuanPabloSGU

I have updated your recommendation. The Updated pull request can be found at the following link: https://github.com/dfir-iris/iris-web/pull/263

manjularajamani avatar Jun 08 '23 08:06 manjularajamani

Hello @whikernel

I hope this message finds you well. I wanted to reach out regarding the pull request I submitted. I believe it's ready to be merged into the main branch. Could you please review it at your earliest convenience and merge it if everything looks good?

manjularajamani avatar Jun 08 '23 15:06 manjularajamani

Hi @manjularajamani!

Thank you for the pull request! We saw it but didn't had the time to entirely review it yet. We will try to do it in the next week(s). Thanks a lot for the work 👍

whikernel avatar Jun 08 '23 15:06 whikernel