Volume creation failing for more than one brick under same mount point dir.

[rmadaka]

Observed behavior

 glustercli volume create rep2 --replica=3 10.70.35.219:/bricks/brick0/rep2 10.70.35.194:/bricks/brick0/rep2 10.70.35.33:/bricks/brick0/rep2 --create-brick-dir
Volume creation failed

Response headers:
X-Gluster-Cluster-Id: b3cb82b2-a247-40ac-ba83-d79b3846ca1b
X-Gluster-Peer-Id: 828fc89c-371c-4f8c-a2d2-dfb472151106
X-Request-Id: 2d6bd361-9515-4cd3-a5bc-cd0ec6601d0b

Response body:
Transaction step vol-create.ValidateBricks failed on peer 828fc89c-371c-4f8c-a2d2-dfb472151106 with error: xattr trusted.gfid already present on /bricks/brick0/rep2

Expected/desired behavior

Volume creation should be successful , if that particular mount point already having bricks as well

Details on how to reproduce (minimal and precise)

1. Create replica 3 volume 'rep1' brick path containing /bricks/brick0

rep1 Volume created successfully
Volume ID:  277dd59e-e6b7-430c-8bbf-d55265684975

2. Start the volume rep1

Volume rep1 started successfully

3. Try to create one more volume rep2, same mount point brick path containing .

glustercli volume create rep2 --replica=3 10.70.35.219:/bricks/brick0/rep2 10.70.35.194:/bricks/brick0/rep2 10.70.35.33:/bricks/brick0/rep2 --create-brick-dir
Volume creation failed

Response headers:
X-Gluster-Cluster-Id: b3cb82b2-a247-40ac-ba83-d79b3846ca1b
X-Gluster-Peer-Id: 828fc89c-371c-4f8c-a2d2-dfb472151106
X-Request-Id: 2d6bd361-9515-4cd3-a5bc-cd0ec6601d0b

Response body:
Transaction step vol-create.ValidateBricks failed on peer 828fc89c-371c-4f8c-a2d2-dfb472151106 with error: xattr trusted.gfid already present on /bricks/brick0/rep2

4. If i create one more sub directory under same mount point , then if i create one more volume, volume creation is successful.

rep3 Volume created successfully
Volume ID:  dd61c91a-7910-4acb-b5b9-df4eb31a7881```

### Information about the environment:

- Glusterd2 version used (e.g. v4.1.0 or master): v6.0-dev.47.gitaf5928b
- Operating system used: 
- Glusterd2 compiled from sources, as a package (rpm/deb), or container: 
- Using External ETCD: (yes/no, if yes ETCD version): yes
- If container, which container image: 
- Using kubernetes, openshift, or direct install: 
- If kubernetes/openshift, is gluster running inside kubernetes/openshift or outside: 


Logs:

```time="2018-11-30 13:06:59.351920" level=info msg="127.0.0.1 - - [30/Nov/2018:13:06:59 +0530] \"GET /v1/peers HTTP/1.1\" 200 821" reqid=30b8aec4-e9e1-4e6c-8702-1097be85fecb
time="2018-11-30 13:06:59.369602" level=error msg="Step failed on node." error="xattr trusted.gfid already present on /bricks/brick0/rep2" node=828fc89c-371c-4f8c-a2d2-dfb472151106 reqid=3af6dede-5cbf-439b-921d-0385f74188eb source="[step.go:119:transaction.runStepFuncOnNodes]" step=vol-create.ValidateBricks txnid=50be7826-e5de-43da-a71e-7ca64fd7c3d5
time="2018-11-30 13:06:59.374512" level=error msg="Transaction failed, rolling back changes" error="Step vol-create.ValidateBricks failed on 1 nodes" reqid=3af6dede-5cbf-439b-921d-0385f74188eb source="[transaction.go:143:transaction.(*Txn).Do]" txnid=50be7826-e5de-43da-a71e-7ca64fd7c3d5
time="2018-11-30 13:06:59.377497" level=info msg="127.0.0.1 - - [30/Nov/2018:13:06:59 +0530] \"POST /v1/volumes HTTP/1.1\" 500 217" reqid=3af6dede-5cbf-439b-921d-0385f74188eb

Other useful information

• glusterd2 config files from all nodes (default /etc/glusterd2/glusterd2.toml)
• glusterd2 log files from all nodes (default /var/log/glusterd2/glusterd2.log)
• ETCD configuration
• Contents of uuid.toml from all nodes (default /var/lib/glusterd2/uuid.toml)
• Output of statedump from any one of the node

Useful commands

• To get glusterd2 version

  glusterd2 --version
  

• To get ETCD version

  etcd --version
  

• To get output of statedump

  curl http://glusterd2-IP:glusterd2-Port/statedump