Gem for updated Logstash plugin needs to be updated at rubygems.org

[TheCloudScout]

Describe the bug Me and @pkhabazi created a PR recently because we developed a new feature for the Logstash output plugin for Log Analytics. This PR recently got approved and merged but now we need to make sure the respective Gem 💎 gets updated at rubygems.org as well. This ensures automatic updates for existing users and easier installation for new users.

But since Microsoft is the owner of this Gem, we need your help getting this done.

To Reproduce

• Check the latest version (1.1.1) of the Gem here at rubygems.org.
• You'll notice that user 'MS-HAIMN' is the owner.
• Since the recent merge of our PR the latest version is now 1.2.0.

Expected behavior Please upload the new Gem 💎 for us to rubygems.org 😊

Additional context I spoke with Haim Rubinstein about this. Apparantly a member of him team is able to perform these steps and he asked me to create this issue here.

If you have any question don't hesitate to reach out!

[v-muuppugund]

Hi @TheCloudScout ,Could you please share email id of team member will connect over teams meeting for getting this done

[TheCloudScout]

Hi @TheCloudScout ,Could you please share email id of team member will connect over teams meeting for getting this done

Please invite [email protected] and [email protected]

[v-muuppugund]

Hi @TheCloudScout ,Could you please share email id of team member will connect over teams meeting for getting this done

Please invite [email protected] and [email protected]

Noted,Will ask for convenient time slots for teams meeting for this issue.

[pkhabazi]

Hi @TheCloudScout ,Could you please share email id of team member will connect over teams meeting for getting this done

Please invite [email protected] and [email protected]

Noted,Will ask for convenient time slots for teams meeting for this issue.

We are both currently in Redmond for the MVP summit. So in PST time zone. Please let us now when you have time.

[v-muuppugund]

Hi @TheCloudScout ,Could you please share email id of team member will connect over teams meeting for getting this done

Please invite [email protected] and [email protected]

Noted,Will ask for convenient time slots for teams meeting for this issue.

We are both currently in Redmond for the MVP summit. So in PST time zone. Please let us now when you have time.

Could you please share time slots for teams meeting and asked the same over email

[v-muuppugund]

Hi @TheCloudScout ,Could you please share email id of team member will connect over teams meeting for getting this done

Please invite [email protected] and [email protected]

Noted,Will ask for convenient time slots for teams meeting for this issue.

We are both currently in Redmond for the MVP summit. So in PST time zone. Please let us now when you have time.

Could you please share time slots for teams meeting and asked the same over email

As discussed over email,will schedule meeting on 18Mar2024

[v-muuppugund]

Hi @pkhabazi / @TheCloudScout ,as discussed yesterday over call,waiting for credentials,once received ,will update you,so we can have a call

[v-muuppugund]

Hi @pkhabazi / @TheCloudScout got response from Haim that the commit will be reverted in PR,so there will be no action at rubygems.org ,so closing the issue,If you still need support for this issue(https://github.com/Azure/Azure-Sentinel/issues/10109), feel free to re-open at any time. Thank you for your co-operation!

[TheCloudScout]

Hi @v-muuppugund,

the commit will be reverted in PR

Are you saying that you're going to revert/undo the entire PR? What is the reason behind this? Does Haim not agree with the changes made by us?

Please me and @pkhabazi know what didn't work for him and need to be changed. We want to work together to get things right.

Thank you!

[v-muuppugund]

Hi @v-muuppugund,

the commit will be reverted in PR

Are you saying that you're going to revert/undo the entire PR? What is the reason behind this? Does Haim not agree with the changes made by us?

Please me and @pkhabazi know what didn't work for him and need to be changed. We want to work together to get things right.

Thank you!

Hi @TheCloudScout ,Ok,Sure,shared details over individual teams chat of email,please let me know there,so will proceed further steps.

[LuKePicci]

Where can we get updated about the status of this work? We would need a ballpark estimate of when we would get generally available managed identity support on this logstash output plugin. Thanks.

[pkhabazi]

@LuKePicci, after having multiple sessions with the MS PM team, they have decided that this feature will not be implemented in the current version that is published on RubyGems. This is mainly because they are in the process of planning a GA for the module in the current version without support for MI, and I can't share more details due to NDA constraints. The current code base will also be reverted to the earlier version without support for MI, as mentioned above.

Therefore, we have decided and agreed with the MS team to bring the current code base with support for MI to a separate GitHub repository and release it under a new Gem name. We are still in the process of updating the documentation, etc. However, the links are already online; feel free to test and share your feedback with us.

Link to the GitHub Repo: https://github.com/pkhabazi/microsoft-sentinel-logstash-output link to the new RubyGem: https://rubygems.org/gems/microsoft-sentinel-logstash-output

[LuKePicci]

Awesome, thanks for the update.

[nangirl]

Trying to test the new plugin, it installs correctly but logstash says: Unable to configure plugins: (PluginLoadingError) Couldn't find any output plugin named 'microsoft-sentinel-logstash-output'. Are you sure this is correct? Trying to load the microsoft-sentinel-logstash-output output plugin resulted in this error: Unable to load the requested plugin named microsoft-sentinel-logstash-output of type output. The plugin is not installed.

This is logstash configuration file:

output {
     microsoft-sentinel-logstash-output {
                      managed_identity => true
                      data_collection_endpoint => "${DATACOLLECTION_ENDPOINT}"
                      dcr_immutable_id => "${DCR_IMMUTABLE_ID}"
                      dcr_stream_name => "${DCR_STREAM_NAME}" 
    }
}

Here the plugin installation output:

Using bundled JDK: /usr/share/logstash/jdk
Validating microsoft-sentinel-logstash-output
Resolving mixin dependencies
Installing microsoft-sentinel-logstash-output
Installation successful

I'm using logstash 8.8.1

[TheCloudScout]

Hi @nangirl

I guess you were a bit too early. We hadn't finished sending the latest version to rubygems. Since the plug-in name had to change, we left a couple of instances in the code which we had to rename as well.

If you now run

sudo /usr/share/logstash/bin/logstash-plugin install microsoft-sentinel-logstash-output

You'll end up with version 1.2.3. And this is confirmed to work properly as detailed on Microsoft Github page.

This was the config I used:

output {
        microsoft-sentinel-logstash-output {
                managed_identity => true
                data_collection_endpoint => "https://<dceuri>.westeurope-1.ingest.monitor.azure.com"
                dcr_immutable_id => "dcr-<immutableid>"
                dcr_stream_name => "Custom-c<tablename>"
        }
}

It's a pity we had to renamed to be able to publish it this way. But I think my customers will appreciate the easier installation/update mechanism instead.

Please let me know if it worked out for you as well!

[pkhabazi]

@LuKePicci and @nangirl feels ree to test the latest version as mentioned bij Koos and share your feedback/issues with us in the new project: https://github.com/pkhabazi/microsoft-sentinel-logstash-output

Koos and myself will be monitoring this project for questions/issues.