o365beat
o365beat copied to clipboard
counteractive
Question about possibility of monitoring multiple tenant domains
For my current project I am required to pull logs from multiple tenant domains and output each to a separate index. My current solution is running an instance of o365beat per domain that I am pulling logs from. Just curious if I can somehow configure the beat to pull from each domain and use conditionals to send the output to the different indices. In my experience with beats in the past this was done with multiple prospectors, but not sure if that is possible with o365beat. If it is not I will just continue running multiple instances.
I hadn't thought of this use-case, thanks for bringing it up, I can see why it'd be helpful. It's not currently supported, but I'll tag this as an enhancement request - until we work through the implications you can of course run multiple instances (you'll have to fiddle with the service registration(s), auto-starting, logging, etc., to make it all play nice, but it sounds like you've worked through that before).
To implement this I'll have to break out the configs for the various tenancies, but it's doable ... I'll target it for the 2.0 release, with any other breaking changes. Thanks for the suggestion!
@ipninichuck I also have a similar use case except my data can all go into the same index.
May I ask how you configured multiple instances of the beat to pull data from multiple tenants?
Essentially to run another instance of any beat all you have to do is provide a unique path.config, path.data and path.logs for systemd to use for arguments when starting the beat. Each instance basically needs an entry $tag_$beatname.service and then can be started and stopped separately. I created a bash script that does this for filebeat. With simple modifications it can be used for any beat including O365beat. https://github.com/ipninichuck/Filebeat-Utilities
On Tue, Dec 24, 2019 at 2:41 AM GenCr [email protected] wrote:
@ipninichuck https://github.com/ipninichuck I also have a similar use case except my data can all go into the same index.
May I ask how you configured multiple instances of the beat to pull data from multiple tenants?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/counteractive/o365beat/issues/26?email_source=notifications&email_token=AGI5DZ6PIDYCEO5QYAELAETQ2HRMNA5CNFSM4JYVT7XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHS7ZLA#issuecomment-568720556, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5DZ3CHSGIYD4QSJB3KFTQ2HRMNANCNFSM4JYVT7XA .
-- Ivan Paul Ninichuck 949-491-2908 [email protected]
I actually need to write a version for O365beat as well. Once I do I will link it on this thread, but I'm sure if you want it sooner my commenting makes it easier enough to see what I did.
On Tue, Dec 24, 2019 at 2:58 AM ivan ninichuck [email protected] wrote:
Essentially to run another instance of any beat all you have to do is provide a unique path.config, path.data and path.logs for systemd to use for arguments when starting the beat. Each instance basically needs an entry $tag_$beatname.service and then can be started and stopped separately. I created a bash script that does this for filebeat. With simple modifications it can be used for any beat including O365beat. https://github.com/ipninichuck/Filebeat-Utilities
On Tue, Dec 24, 2019 at 2:41 AM GenCr [email protected] wrote:
@ipninichuck https://github.com/ipninichuck I also have a similar use case except my data can all go into the same index.
May I ask how you configured multiple instances of the beat to pull data from multiple tenants?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/counteractive/o365beat/issues/26?email_source=notifications&email_token=AGI5DZ6PIDYCEO5QYAELAETQ2HRMNA5CNFSM4JYVT7XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHS7ZLA#issuecomment-568720556, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5DZ3CHSGIYD4QSJB3KFTQ2HRMNANCNFSM4JYVT7XA .
-- Ivan Paul Ninichuck 949-491-2908 [email protected]
-- Ivan Paul Ninichuck 949-491-2908 [email protected]
That's awesome, thanks! I've managed to copy the service and configure for the second tenant with the help of your script. The two instances run alongside each other successfully.
Looking forward to having this feature supported without having to run multiple instances.
Hi @ipninichuck ,
I follow the steps in your script manually and I run this:
o365beat -e -c /etc/itmx-o365beat/o365beat.yml -path.config /etc/itmx-o365beat -path.data /var/lib/itmx-o365beat -path.logs /var/log/itmx-o365beat
but I get this ERROR:
2020-10-08T16:04:56.431Z ERROR instance/beat.go:916 Exiting: non-200 status during api request.
newly enabled or newly subscribed feeds can take 12 hours or more to provide data.
confirm audit log searching is enabled for the target tenancy (https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off#turn-on-audit-log-search).
req: &{POST https://manage.office.com/api/v1.0/47d8db9b-4dd3-4ab1-8dea-4892453bf581/activity/feed/subscriptions/start?PublisherIdentifier=47d8db9b-4dd3-4ab1-8dea-4892453bf581&contentType=Audit.AzureActiveDirectory HTTP/1.1 1 1 map[Authorization:[Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCIsImtpZCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNDdkOGRiOWItNGRkMy00YWIxLThkZWEtNDg5MjQ1M2JmNTgxLyIsImlhdCI6MTYwMjE3Mjc5NSwibmJmIjoxNjAyMTcyNzk1LCJleHAiOjE2MDIxNzY2OTUsImFpbyI6IkUyUmdZTGl1TCtZajgyMzE3MEQrRXp2eTVRL3lBUUE9IiwiYXBwaWQiOiJmMjg4YzQ4Ni1hMjg3LTRkZTAtYWVkYy0yODk0M2EwZmI0NzAiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEvIiwib2lkIjoiZjhjYWMxODEtMmI4My00NTE2LWI0ODQtNjExYjY0MTYyNDNhIiwicmgiOiIwLkFBQUFtOXZZUjlOTnNVcU42a2lTUlR2MWdZYkVpUEtIb3VCTnJ0d29sRG9QdEhCSEFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCJdLCJzdWIiOiJmOGNhYzE4MS0yYjgzLTQ1MTYtYjQ4NC02MTFiNjQxNjI0M2EiLCJ0aWQiOiI0N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEiLCJ1dGkiOiJfRk9BNVUtSlJVMnBYNXdlQzdKeUFBIiwidmVyIjoiMS4wIn0.bWBoyHzIiLC_g-wdNABq9Y3VfZNWaajqNsiPKinq7tlGjDSFrJUtncN5FhF204gMJyLizww8kZ09URpH7fh2vhvQqbsJj7XYB-aqxSGnsHvcexfWYgH7ENP_w4B-UK2jB6whkg0jmE8XgPwHjZU061kGRFUJhTcAhd9jR9wJMapbISts-SzkTJzyzkfNXocoChHxp_z51q6HkK1zDt2JTMAHCgS-GkioBZWxUKKJUx_ZKIo99nqhdSuF295zV1QweSDxVjn6QATUXCINX8xTrXVCsPWCx2EXusDskshAJWTh9VsZfHArsz0vJBigFNyd5otN9EHNkZd-0vochZ3IQw]] {} 0x13fae20 0 [] false manage.office.com map[] map[]
I am not very expert at this, can you help me to understand where it stucks?
Thank you, Sara
Hello,
From the error message, it looks like o365 is replying that the application is not authorized. Did you complete the previous steps in the beat setup process of registering the app in Active Directory and giving it the needed permissions. I believe they are listed in the documentation on the Github page for the beat.
On Thu, Oct 8, 2020 at 9:06 AM scaruso [email protected] wrote:
Hi @ipninichuck https://github.com/ipninichuck ,
I follow the steps in your script manually and I run this:
o365beat -e -c /etc/itmx-o365beat/o365beat.yml -path.config /etc/itmx-o365beat -path.data /var/lib/itmx-o365beat -path.logs /var/log/itmx-o365beat
but I get this ERROR:
2020-10-08T16:04:56.431Z ERROR instance/beat.go:916 Exiting: non-200 status during api request. newly enabled or newly subscribed feeds can take 12 hours or more to provide data. confirm audit log searching is enabled for the target tenancy ( https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off#turn-on-audit-log-search ). req: &{POST https://manage.office.com/api/v1.0/47d8db9b-4dd3-4ab1-8dea-4892453bf581/activity/feed/subscriptions/start?PublisherIdentifier=47d8db9b-4dd3-4ab1-8dea-4892453bf581&contentType=Audit.AzureActiveDirectory HTTP/1.1 1 1 map[Authorization:[Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCIsImtpZCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNDdkOGRiOWItNGRkMy00YWIxLThkZWEtNDg5MjQ1M2JmNTgxLyIsImlhdCI6MTYwMjE3Mjc5NSwibmJmIjoxNjAyMTcyNzk1LCJleHAiOjE2MDIxNzY2OTUsImFpbyI6IkUyUmdZTGl1TCtZajgyMzE3MEQrRXp2eTVRL3lBUUE9IiwiYXBwaWQiOiJmMjg4YzQ4Ni1hMjg3LTRkZTAtYWVkYy0yODk0M2EwZmI0NzAiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEvIiwib2lkIjoiZjhjYWMxODEtMmI4My00NTE2LWI0ODQtNjExYjY0MTYyNDNhIiwicmgiOiIwLkFBQUFtOXZZUjlOTnNVcU42a2lTUlR2MWdZYkVpUEtIb3VCTnJ0d29sRG9QdEhCSEFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCJdLCJzdWIiOiJmOGNhYzE4MS0yYjgzLTQ1MTYtYjQ4NC02MTFiNjQxNjI0M2EiLCJ0aWQiOiI0N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEiLCJ1dGkiOiJfRk9BNVUtSlJVMnBYNXdlQzdKeUFBIiwidmVyIjoiMS4wIn0.bWBoyHzIiLC_g-wdNABq9Y3VfZNWaajqNsiPKinq7tlGjDSFrJUtncN5FhF204gMJyLizww8kZ09URpH7fh2vhvQqbsJj7XYB-aqxSGnsHvcexfWYgH7ENP_w4B-UK2jB6whkg0jmE8XgPwHjZU061kGRFUJhTcAhd9jR9wJMapbISts-SzkTJzyzkfNXocoChHxp_z51q6HkK1zDt2JTMAHCgS-GkioBZWxUKKJUx_ZKIo99nqhdSuF295zV1QweSDxVjn6QATUXCINX8xTrXVCsPWCx2EXusDskshAJWTh9VsZfHArsz0vJBigFNyd5otN9EHNkZd-0vochZ3IQw]] {} 0x13fae20 0 [] false manage.office.com map[] map[] map[] } res: &{401 Unauthorized 401 HTTP/2.0 2 0 map[Cache-Control:[no-cache] Content-Length:[124] Content-Type:[application/json; charset=utf-8] Date:[Thu, 08 Oct 2020 16:04:56 GMT] Expires:[-1] Pragma:[no-cache] Server:[Microsoft-IIS/10.0] Www-Authenticate:[Bearer] X-Aspnet-Version:[4.0.30319] X-Powered-By:[ASP.NET]] 0xc0001fe120 124 [] false false map[] 0xc0004ec700 0xc0000c28f0} {"error":{"code":"AF10001","message":"The permission set () sent in the request does not include the expected permission."}} Exiting: non-200 status during api request. newly enabled or newly subscribed feeds can take 12 hours or more to provide data. confirm audit log searching is enabled for the target tenancy ( https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off#turn-on-audit-log-search ). req: &{POST https://manage.office.com/api/v1.0/47d8db9b-4dd3-4ab1-8dea-4892453bf581/activity/feed/subscriptions/start?PublisherIdentifier=47d8db9b-4dd3-4ab1-8dea-4892453bf581&contentType=Audit.AzureActiveDirectory HTTP/1.1 1 1 map[Authorization:[Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCIsImtpZCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNDdkOGRiOWItNGRkMy00YWIxLThkZWEtNDg5MjQ1M2JmNTgxLyIsImlhdCI6MTYwMjE3Mjc5NSwibmJmIjoxNjAyMTcyNzk1LCJleHAiOjE2MDIxNzY2OTUsImFpbyI6IkUyUmdZTGl1TCtZajgyMzE3MEQrRXp2eTVRL3lBUUE9IiwiYXBwaWQiOiJmMjg4YzQ4Ni1hMjg3LTRkZTAtYWVkYy0yODk0M2EwZmI0NzAiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEvIiwib2lkIjoiZjhjYWMxODEtMmI4My00NTE2LWI0ODQtNjExYjY0MTYyNDNhIiwicmgiOiIwLkFBQUFtOXZZUjlOTnNVcU42a2lTUlR2MWdZYkVpUEtIb3VCTnJ0d29sRG9QdEhCSEFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCJdLCJzdWIiOiJmOGNhYzE4MS0yYjgzLTQ1MTYtYjQ4NC02MTFiNjQxNjI0M2EiLCJ0aWQiOiI0N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEiLCJ1dGkiOiJfRk9BNVUtSlJVMnBYNXdlQzdKeUFBIiwidmVyIjoiMS4wIn0.bWBoyHzIiLC_g-wdNABq9Y3VfZNWaajqNsiPKinq7tlGjDSFrJUtncN5FhF204gMJyLizww8kZ09URpH7fh2vhvQqbsJj7XYB-aqxSGnsHvcexfWYgH7ENP_w4B-UK2jB6whkg0jmE8XgPwHjZU061kGRFUJhTcAhd9jR9wJMapbISts-SzkTJzyzkfNXocoChHxp_z51q6HkK1zDt2JTMAHCgS-GkioBZWxUKKJUx_ZKIo99nqhdSuF295zV1QweSDxVjn6QATUXCINX8xTrXVCsPWCx2EXusDskshAJWTh9VsZfHArsz0vJBigFNyd5otN9EHNkZd-0vochZ3IQw]] {} 0x13fae20 0 [] false manage.office.com map[] map[] map[] } res: &{401 Unauthorized 401 HTTP/2.0 2 0 map[Cache-Control:[no-cache] Content-Length:[124] Content-Type:[application/json; charset=utf-8] Date:[Thu, 08 Oct 2020 16:04:56 GMT] Expires:[-1] Pragma:[no-cache] Server:[Microsoft-IIS/10.0] Www-Authenticate:[Bearer] X-Aspnet-Version:[4.0.30319] X-Powered-By:[ASP.NET]] 0xc0001fe120 124 [] false false map[] 0xc0004ec700 0xc0000c28f0} {"error":{"code":"AF10001","message":"The permission set () sent in the request does not include the expected permission."}}
I am not very expert at this, can you help me to understand where it stucks?
Thank you, Sara
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/counteractive/o365beat/issues/26#issuecomment-705670599, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5DZ5J36PY3KAQ4XQAIKDSJXPRXANCNFSM4JYVT7XA .
-- Ivan Paul Ninichuck 949-491-2908 [email protected]
Hello, From the error message, it looks like o365 is replying that the application is not authorized. Did you complete the previous steps in the beat setup process of registering the app in Active Directory and giving it the needed permissions. I believe they are listed in the documentation on the Github page for the beat.
Hi,
thank you for your reply.
I don't have direct access to o365 management. I have to configure a log collector to receive logs from o365. The customer gives me all required information that I put in o365beat.yml (tenant, client id, directory id, secret)
So do you think that this is not an error depending on running two instances of o365 beat?
Hi,
No, this error is not coming from the beats operation. It is being denied access to the o365 api because it needs to be given specific permissions as a registered app on Active Directory to work properly. In case you do have problems with this particular beat Elastic has created a filebeat module for this purpose now( https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-o365.html), but it is basically doing the same thing and when I used the o365 beat it worked fine. But the Elastic module still requires the app to be registered on Active Directory and given specific permissions.
My suggestion is to take a copy of the instructions from the o365 beat github page that give the permissions needed and provide it to your client. Explain that someone with admin access will need to register the app and give it the permissions. This is going to be true of any log collector for o365.
On Thu, Oct 8, 2020 at 2:22 PM scaruso [email protected] wrote:
Hello, From the error message, it looks like o365 is replying that the application is not authorized. Did you complete the previous steps in the beat setup process of registering the app in Active Directory and giving it the needed permissions. I believe they are listed in the documentation on the Github page for the beat. … <#m_940539386638148790_> On Thu, Oct 8, 2020 at 9:06 AM scaruso @.***> wrote: Hi @ipninichuck https://github.com/ipninichuck https://github.com/ipninichuck , I follow the steps in your script manually and I run this: o365beat -e -c /etc/itmx-o365beat/o365beat.yml -path.config /etc/itmx-o365beat -path.data /var/lib/itmx-o365beat -path.logs /var/log/itmx-o365beat but I get this ERROR: 2020-10-08T16:04:56.431Z ERROR instance/beat.go:916 Exiting: non-200 status during api request. newly enabled or newly subscribed feeds can take 12 hours or more to provide data. confirm audit log searching is enabled for the target tenancy ( https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off#turn-on-audit-log-search ). req: &{POST https://manage.office.com/api/v1.0/47d8db9b-4dd3-4ab1-8dea-4892453bf581/activity/feed/subscriptions/start?PublisherIdentifier=47d8db9b-4dd3-4ab1-8dea-4892453bf581&contentType=Audit.AzureActiveDirectory HTTP/1.1 1 1 map[Authorization:[Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCIsImtpZCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNDdkOGRiOWItNGRkMy00YWIxLThkZWEtNDg5MjQ1M2JmNTgxLyIsImlhdCI6MTYwMjE3Mjc5NSwibmJmIjoxNjAyMTcyNzk1LCJleHAiOjE2MDIxNzY2OTUsImFpbyI6IkUyUmdZTGl1TCtZajgyMzE3MEQrRXp2eTVRL3lBUUE9IiwiYXBwaWQiOiJmMjg4YzQ4Ni1hMjg3LTRkZTAtYWVkYy0yODk0M2EwZmI0NzAiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEvIiwib2lkIjoiZjhjYWMxODEtMmI4My00NTE2LWI0ODQtNjExYjY0MTYyNDNhIiwicmgiOiIwLkFBQUFtOXZZUjlOTnNVcU42a2lTUlR2MWdZYkVpUEtIb3VCTnJ0d29sRG9QdEhCSEFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCJdLCJzdWIiOiJmOGNhYzE4MS0yYjgzLTQ1MTYtYjQ4NC02MTFiNjQxNjI0M2EiLCJ0aWQiOiI0N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEiLCJ1dGkiOiJfRk9BNVUtSlJVMnBYNXdlQzdKeUFBIiwidmVyIjoiMS4wIn0.bWBoyHzIiLC_g-wdNABq9Y3VfZNWaajqNsiPKinq7tlGjDSFrJUtncN5FhF204gMJyLizww8kZ09URpH7fh2vhvQqbsJj7XYB-aqxSGnsHvcexfWYgH7ENP_w4B-UK2jB6whkg0jmE8XgPwHjZU061kGRFUJhTcAhd9jR9wJMapbISts-SzkTJzyzkfNXocoChHxp_z51q6HkK1zDt2JTMAHCgS-GkioBZWxUKKJUx_ZKIo99nqhdSuF295zV1QweSDxVjn6QATUXCINX8xTrXVCsPWCx2EXusDskshAJWTh9VsZfHArsz0vJBigFNyd5otN9EHNkZd-0vochZ3IQw]] {} 0x13fae20 0 [] false manage.office.com map[] map[] map[] } res: &{401 Unauthorized 401 HTTP/2.0 2 0 map[Cache-Control:[no-cache] Content-Length:[124] Content-Type:[application/json; charset=utf-8] Date:[Thu, 08 Oct 2020 16:04:56 GMT] Expires:[-1] Pragma:[no-cache] Server:[Microsoft-IIS/10.0] Www-Authenticate:[Bearer] X-Aspnet-Version:[4.0.30319] X-Powered-By:[ASP.NET]] 0xc0001fe120 124 [] false false map[] 0xc0004ec700 0xc0000c28f0} {"error":{"code":"AF10001","message":"The permission set () sent in the request does not include the expected permission."}} Exiting: non-200 status during api request. newly enabled or newly subscribed feeds can take 12 hours or more to provide data. confirm audit log searching is enabled for the target tenancy ( https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off#turn-on-audit-log-search ). req: &{POST https://manage.office.com/api/v1.0/47d8db9b-4dd3-4ab1-8dea-4892453bf581/activity/feed/subscriptions/start?PublisherIdentifier=47d8db9b-4dd3-4ab1-8dea-4892453bf581&contentType=Audit.AzureActiveDirectory HTTP/1.1 1 1 map[Authorization:[Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCIsImtpZCI6ImtnMkxZczJUMENUaklmajRydDZKSXluZW4zOCJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNDdkOGRiOWItNGRkMy00YWIxLThkZWEtNDg5MjQ1M2JmNTgxLyIsImlhdCI6MTYwMjE3Mjc5NSwibmJmIjoxNjAyMTcyNzk1LCJleHAiOjE2MDIxNzY2OTUsImFpbyI6IkUyUmdZTGl1TCtZajgyMzE3MEQrRXp2eTVRL3lBUUE9IiwiYXBwaWQiOiJmMjg4YzQ4Ni1hMjg3LTRkZTAtYWVkYy0yODk0M2EwZmI0NzAiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEvIiwib2lkIjoiZjhjYWMxODEtMmI4My00NTE2LWI0ODQtNjExYjY0MTYyNDNhIiwicmgiOiIwLkFBQUFtOXZZUjlOTnNVcU42a2lTUlR2MWdZYkVpUEtIb3VCTnJ0d29sRG9QdEhCSEFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCJdLCJzdWIiOiJmOGNhYzE4MS0yYjgzLTQ1MTYtYjQ4NC02MTFiNjQxNjI0M2EiLCJ0aWQiOiI0N2Q4ZGI5Yi00ZGQzLTRhYjEtOGRlYS00ODkyNDUzYmY1ODEiLCJ1dGkiOiJfRk9BNVUtSlJVMnBYNXdlQzdKeUFBIiwidmVyIjoiMS4wIn0.bWBoyHzIiLC_g-wdNABq9Y3VfZNWaajqNsiPKinq7tlGjDSFrJUtncN5FhF204gMJyLizww8kZ09URpH7fh2vhvQqbsJj7XYB-aqxSGnsHvcexfWYgH7ENP_w4B-UK2jB6whkg0jmE8XgPwHjZU061kGRFUJhTcAhd9jR9wJMapbISts-SzkTJzyzkfNXocoChHxp_z51q6HkK1zDt2JTMAHCgS-GkioBZWxUKKJUx_ZKIo99nqhdSuF295zV1QweSDxVjn6QATUXCINX8xTrXVCsPWCx2EXusDskshAJWTh9VsZfHArsz0vJBigFNyd5otN9EHNkZd-0vochZ3IQw]] {} 0x13fae20 0 [] false manage.office.com map[] map[] map[] } res: &{401 Unauthorized 401 HTTP/2.0 2 0 map[Cache-Control:[no-cache] Content-Length:[124] Content-Type:[application/json; charset=utf-8] Date:[Thu, 08 Oct 2020 16:04:56 GMT] Expires:[-1] Pragma:[no-cache] Server:[Microsoft-IIS/10.0] Www-Authenticate:[Bearer] X-Aspnet-Version:[4.0.30319] X-Powered-By:[ASP.NET]] 0xc0001fe120 124 [] false false map[] 0xc0004ec700 0xc0000c28f0} {"error":{"code":"AF10001","message":"The permission set () sent in the request does not include the expected permission."}} I am not very expert at this, can you help me to understand where it stucks? Thank you, Sara — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26 (comment) https://github.com/counteractive/o365beat/issues/26#issuecomment-705670599>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5DZ5J36PY3KAQ4XQAIKDSJXPRXANCNFSM4JYVT7XA . -- Ivan Paul Ninichuck 949-491-2908 [email protected]
Hi,
thank you for your reply.
I don't have direct access to o365 management. I have to configure a log collector to receive logs from o365. The customer gives me all required information that I put in o365beat.yml (tenant, client id, directory id, secret)
So do you think that this is not an error depending on running two instances of o365 beat?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/counteractive/o365beat/issues/26#issuecomment-705830641, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5DZ6SM2MSXIHHAM62RM3SJYUQVANCNFSM4JYVT7XA .
-- Ivan Paul Ninichuck 949-491-2908 [email protected]
Hi,
the client already provide us with tenant id, app id, directory id and secret. So I suppose that they already accomplished to your suggestion. Or am I wrong?
I ran
o365beat -e -c /etc/itmx-o365beat/o365beat.yml -path.config /etc/itmx-o365beat -path.data /var/lib/itmx-o365beat -path.logs /var/log/itmx-o365beat
but as output of
ps -ax | grep o365beat
I can't see two line as result but only one regarding the older istance.
Is that a good sign?
Hmm...yeah if they gave you the app ID and secret then they did register the app. I would double check the values you put in the config and have them double check it was given all the permissions that are needed.
On Fri, Oct 9, 2020, 6:56 AM scaruso [email protected] wrote:
Hi,
the client already provide us with tenant id, app id, directory id and secret. So I suppose that they already accomplished to your suggestion. Or am I wrong?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/counteractive/o365beat/issues/26#issuecomment-706195465, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGI5DZ5MVPLQHF7VVHGEB63SJ4I7NANCNFSM4JYVT7XA .
Hi @chris-counteractive,
Is the multi tenant support enhancement likely to be added in the near future?
We are currently using the o365 module within filebeat with multi tenant support but as with many others we are looking to move to AWS open distro and the o365 module is currently not included with the oss compatible version of filebeat.
o365beat will fill this gap for us but we do have quite a few tenancies that we currently monitor.
Thanks!
@Vetpeet thanks for the question! Short answer: we hadn't planned to add any features to o365beat since the "official" filebeat 365 module dropped in 7.7.0. Even though the o365 module is under x-pack, I don't know that there's any restriction in filebeat that requires a paid license for any specific modules. That is, I don't think there's any reason you wouldn't be able to ship to AWS-flavored elasticsearch, right? Does filebeat complain if you're trying to send to an oss-compatible ES instance? I've honestly not tried it.
And even if it did, it's might be a more reliable workaround to use the Elastic-licensed filebeat to dump to a jsonl file that you can re-ship with unencumbered filebeat. Certainly kludgy and a bit wasteful, but workable?
If there's an angle on this that I'm not seeing I'm definitely happy to re-assess and perhaps try to get back to feature-parity, definitely not opposed - it just didn't seem to make much sense when the elastic-sponsored filebeat gets most people where they need to go.
@chris-counteractive thanks for the reply, The AWS-forked elasticsearch is still under the Apache 2.0 license, the standard Filebeat is now under the new Elastic License, and as such the standard Filebeat will not work with the AWS-forked elasticsearch. There is a Filebeat version (Filebeat -oss) https://www.elastic.co/downloads/beats/filebeat-oss still under the Apache 2.0 license that will work with the AWS-fork, but not all the standard modules are included, I will add screenshot of included modules, hence why we had to turn to o365beat as an alternative in a nutshell the compatible filebeat distro for the AWS-forked elasticsearch don't include the o365 module.
Thanks.
