Watcher errors if GCP GCS bucket Get permissions are revoked

[shrikant0013]

trafficstars

Watcher currently errors out if certain GCP GCS buckets do not have GET/Read permissions Ideally, the account that is scanning various GCP project resources should have relevant permissions, but in certain cases these might be removed, changed, etc

Should we continue watcher for remaining GCS buckets in same projects and other projects?. Some kind of reporting/alerting can be put in place for failure cases

2017-04-26 00:42:07,866 INFO: Started thread pool with 25 core threads and 30 maximum threads [in build/bdist.linux-x86_64/egg/apscheduler/threadpool.py:52]
Traceback (most recent call last):
  File "/usr/local/src/security_monkey/venv/bin/monkey", line 11, in <module>
    load_entry_point('security-monkey', 'console_scripts', 'monkey')()
  File "/usr/local/src/security_monkey/security_monkey/manage.py", line 625, in main
    manager.run()
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/Flask_Script-0.6.3-py2.7.egg/flask_script/__init__.py", line 397, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/Flask_Script-0.6.3-py2.7.egg/flask_script/__init__.py", line 376, in handle
    return handle(app, *positional_args, **kwargs)
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/Flask_Script-0.6.3-py2.7.egg/flask_script/commands.py", line 145, in handle
    return self.run(*args, **kwargs)
  File "/usr/local/src/security_monkey/security_monkey/manage.py", line 77, in find_changes
    sm_find_changes(account_names, monitor_names)
  File "/usr/local/src/security_monkey/security_monkey/scheduler.py", line 52, in find_changes
    (items, exception_map) = cw.slurp()
  File "/usr/local/src/security_monkey/security_monkey/watchers/gcp/gcs/bucket.py", line 75, in slurp
    return slurp_items()
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/cloudaux-1.1.8-py2.7.egg/cloudaux/gcp/decorators.py", line 132, in decorated_function
    itm, exc = func(*args, **kwargs)
  File "/usr/local/src/security_monkey/security_monkey/watchers/gcp/gcs/bucket.py", line 65, in slurp_items
    bucket_name=bucket['name'], **kwargs)
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/cloudaux-1.1.8-py2.7.egg/cloudaux/orchestration/gcp/gcs/bucket.py", line 15, in get_bucket
    bucket = fetch_bucket(Bucket=bucket_name, **conn)
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/cloudaux-1.1.8-py2.7.egg/cloudaux/gcp/decorators.py", line 41, in decorated_function
    return f(*args, **kwargs)
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/cloudaux-1.1.8-py2.7.egg/cloudaux/gcp/gcs.py", line 35, in get_bucket
    bucket = client.lookup_bucket(kwargs['Bucket'])
  File "/usr/local/src/security_monkey/venv/local/lib/python2.7/site-packages/google/cloud/storage/client.py", line 186, in lookup_bucket
    return self.get_bucket(bucket_name)
  File "/usr/local/src/security_monkey/venv/local/lib/python2.7/site-packages/google/cloud/storage/client.py", line 166, in get_bucket
    bucket.reload(client=self)
  File "/usr/local/src/security_monkey/venv/local/lib/python2.7/site-packages/google/cloud/storage/_helpers.py", line 79, in reload
    _target_object=self)
  File "/usr/local/src/security_monkey/venv/local/lib/python2.7/site-packages/google/cloud/_http.py", line 335, in api_request
    error_info=method + ' ' + url)
google.cloud.exceptions.Forbidden: 403 Caller does not have storage.buckets.get access to bucket xxxxxx (GET https://www.googleapis.com/storage/v1/b/xxxxxx?projection=noAcl)

[scriptsrc]

The GCS watcher should use the @record_exception decorator on the slurp_items() method:

https://github.com/Netflix/security_monkey/blob/develop/security_monkey/decorators.py#L79

Similarly done in the IAM Role Watcher: https://github.com/Netflix/security_monkey/blob/develop/security_monkey/watchers/iam/iam_role.py#L50

@supertom - Do you need me to take this?

[supertom]

@monkeysecurity I apologize, I must have missed this. If you wouldn't mind, I'd appreciate it.