cortx icon indicating copy to clipboard operation
cortx copied to clipboard

OVA cluster stops running when you setup AWS

Open hessio opened this issue 3 years ago • 1 comments

Problem

I was following the steps outlined in the IO setup guide associated with OVA PI-7 when following this guide I got to the testing section when I ran this command which just times out and when I run

[root@cortx-ova-rgw ~]# hctl status Cluster is not running command terminated with exit code 1

Expected behavior

You should be able to run s3 commands and they should works

hctl status command should not terminate with exit code

How to reproduce

Setup the OVA PI-7 and go through the s3 setup guide

Deployment information

VMware Workstation 16 Pro

Additional information

No response

hessio avatar Aug 11 '22 16:08 hessio

For the convenience of the Seagate development team, this issue has been mirrored in a private Seagate Jira Server: https://jts.seagate.com/browse/CORTX-33922. Note that community members will not be able to access that Jira server but that is not a problem since all activity in that Jira mirror will be copied into this GitHub issue.

cortx-admin avatar Aug 11 '22 16:08 cortx-admin

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

stale[bot] avatar Aug 16 '22 02:08 stale[bot]

@hessio, OVA is still not tested on AWS but has plans to test the same which is in the roadmap

mukul-seagate11 avatar Aug 16 '22 04:08 mukul-seagate11

I was running this on VMWare but when I did the IO setup, the OVA stopped working completely, all commands would fail

hessio avatar Aug 16 '22 09:08 hessio

The problem I faced was when I setup the OVA in VMWare Workstation hctl status command worked so cluster was running.

The next step in the process is to setup the s3 api endpoints and done this following the this guide once I tried to test that these steps had worked correctly I faced an error trying to create a bucket and no s3 commands would work. 

So I tried the hctl status command to make sure the cluster was still running but instead I got this error:

[root@cortx-ova-rgw ~]# hctl status

Cluster is not running command terminated with exit code 1

hessio avatar Aug 26 '22 14:08 hessio

I tried once again to verify issue - this time hctl status command completed but was showing services offline and s3 commands were not completing successfully.

[root@cortx-ova-rgw ~]# hctl status
Bytecount:
    critical : 0
    damaged : 0
    degraded : 0
    healthy : 0
Data pool:
    # fid name
    0x6f00000000000001:0x25 'storage-set-1__sns'
Profile:
    # fid name: pool(s)
    0x7000000000000001:0x3b 'Profile_the_pool': 'storage-set-1__sns' 'storage-set-1__dix' None
Services:
    cortx-server-headless-svc-cortx-ova-rgw
    [started]  hax                 0x7200000000000001:0x1d         inet:tcp:cortx-server-headless-svc-cortx-ova-rgw@22001
    [started]  rgw_s3              0x7200000000000001:0x20         inet:tcp:cortx-server-headless-svc-cortx-ova-rgw@21001
    cortx-data-headless-svc-cortx-ova-rgw  (RC)
    [offline]  hax                 0x7200000000000001:0x6          inet:tcp:cortx-data-headless-svc-cortx-ova-rgw@22001
    [offline]  ioservice           0x7200000000000001:0x9          inet:tcp:cortx-data-headless-svc-cortx-ova-rgw@21001
    [offline]  confd               0x7200000000000001:0x18         inet:tcp:cortx-data-headless-svc-cortx-ova-rgw@22002
[root@cortx-ova-rgw ~]#

Here is the output of aws s3 ls

[root@cortx-ova-rgw ~]# aws s3 ls

Could not connect to the endpoint URL: "http://10.100.43.117:80/"

hessio avatar Aug 29 '22 07:08 hessio

ova issues which are reported by community https://github.com/Seagate/cortx/issues?q=is%3Aopen+is%3Aissue+label%3Aova will be further raised to QA and QA will try to verify these issues in next sprint.

hessio avatar Aug 31 '22 14:08 hessio

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @mukul-seagate11 for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

stale[bot] avatar Sep 09 '22 00:09 stale[bot]

issue is still relevant for PI-7 but doesn't seem to appear in PI-8. cc: @mukul-seagate11

hessio avatar Sep 14 '22 15:09 hessio

issue is still relevant for PI-7 but doesn't seem to appear in PI-8. cc: @mukul-seagate11

@hessio, you can refer the exit criteria and validations logs of PI-7 OVA in https://seagate-systems.atlassian.net/wiki/spaces/PRIVATECOR/pages/1036517377/PI-7+Test+Entry+Criteria+Dev+RE+Validation

mukul-seagate11 avatar Sep 14 '22 15:09 mukul-seagate11

https://github.com/Seagate/cortx/pull/1657 is validated as per latest OVA image and no issue is observed

mukul-seagate11 avatar Sep 16 '22 07:09 mukul-seagate11