[FEATURE] Allow custom base type

[cwuethrich]

ECS integration/library project(s) (e.g. Elastic.CommonSchema.Serilog): Elastic.CommonSchema.Serilog

Is your feature request related to a problem? Please describe. We are extending the ECS with our own fields. Let's use the Microsoft example company Contoso. We are extending the ECS with a new group named contoso. The current Serilog Converter doesn't support any Extension to the Base class.

The Method ConvertToEcs is implemented with the Base class. There is no other way to use the code in there with an other Type.

public static Base ConvertToEcs(LogEvent logEvent, IEcsTextFormatterConfiguration configuration)
{
	... 
	var ecsEvent = new Base();
	...
	return ecsEvent;
}

Describe the solution you'd like Without breaking changes we could at least change this to a method where it is possible to extend the Base class and add the own fields with casting in a other base type.

public static Base ConvertToEcs(LogEvent logEvent, IEcsTextFormatterConfiguration configuration)
	=> ConvertToEcs<Base>(logEvent, configuration);
public static Base ConvertToEcs<T>(LogEvent logEvent, IEcsTextFormatterConfiguration configuration)
	where T : Base, new()
{
	... 
	var ecsEvent = new T();
	...
	return ecsEvent;
}

With this change we can use a own base class and easily add new fields.

public class ContosoBase : Base
{
	[DataMember(Name = "contoso")]
	public Contoso Contoso { get; set; }
}
public class Contoso
{
	[DataMember(Name = "company_name")]
	public string CompanyName { get; set; }
}
public class ContosoEcsTextFormatter : EcsTextFormatter
{
	public override void Format(LogEvent logEvent, TextWriter output)
	{
		var ecsEvent = LogEventConverter.ConvertToEcs<ContosoBase>(logEvent, _configuration);
		if (ecsEvent is ContosoBase contosoEcsEvent)
		{
			contosoEcsEvent.Contoso = new Contoso
			{
				CompanyName = "Contoso",
			};
		}
		output.WriteLine(ecsEvent.Serialize());
	}
}

Describe alternatives you've considered A problem here is the property MapCustom in the configuration of the formatter.

public interface IEcsTextFormatterConfiguration
{
	Func<Base, LogEvent, Base> MapCustom { get; set; }
	...
}

If we accept breaking changes, we could even change this and return the custom base type in the converter.

public static T ConvertToEcs<T>(LogEvent logEvent, IEcsTextFormatterConfiguration configuration)
	where T : Base, new()
{
	... 
}

Additional context

What do you think about this change? Is it worth it to create a pull request?

[changemyminds]

Hi @cwuethrich, I have the same problem. I want to extend the Base class. What was your final solution?

Above code output.WriteLine(ecsEvent.Serialize()) have some bugs, beacuse the Serialize() use it self type. In this case, it won't write Contoso Property.

[cwuethrich]

Hi @changemyminds I used another work around. But this is for sure not a final solution. Just as long this repository is "dead".

I tried to make an example with Contoso again.

Custom Base class:

public class ContosoBase : Base
{
    [DataMember(Name = "contoso")]
    public Contoso Contoso { get; set; }

    protected override void WriteAdditionalProperties(Action<string, object> write)
    {
        base.WriteAdditionalProperties(write);

        write("contoso", Contoso);
    }
    protected override bool ReceiveProperty(string propertyName, object value)
    {
        if ("contoso".Equals(propertyName))
        {
            Contoso = value as Contoso;
            return true;
        }

        return base.ReceiveProperty(propertyName, value);
    }
    protected override bool TryRead(string propertyName, out Type type)
    {
        if ("contoso".Equals(propertyName))
        {
            type = typeof(Contoso);
            return true;
        }
        return base.TryRead(propertyName, out type);
    }
}

Base class extension:

public class Contoso
{
    [DataMember(Name = "company")]
    public string CompanyName { get; set; }
}

Custom EcsTextFormatter:

public class ContosoEcsTextFormatter : EcsTextFormatter
{
    public ContosoEcsTextFormatter() : base(GetConfiguration()) { }

    private static EcsTextFormatterConfiguration GetConfiguration()
    {
        var configuration = new EcsTextFormatterConfiguration();
        configuration.MapCustom(MapCustom);
        configuration.LogEventPropertiesToFilter(new HashSet<string>
        {
            "CompanyName",
        });
        return configuration;
    }

    private static Base MapCustom(Base ecsBase, LogEvent logEvent)
    {
        var contosoBase = CreateContosoBase(ecsBase);

        void Assign<T>(T obj, string key, Action<T, object> assign)
            where T : class, new()
        {
            if (!logEvent.TryGetScalarPropertyValue(key, out var value))
                return;

            assign(obj, value.Value);
        }

        contosoBase.Contoso = new Contoso();
        Assign(contosoBase.Contoso, "CompanyName", (o, v) => o.CompanyName = v?.ToString());

        return contosoBase;
    }

    private static ContosoBase CreateContosoBase(Base ecsBase)
    {
        return new ContosoBase
        {
            Metadata = ecsBase.Metadata,
            Agent = ecsBase.Agent,
            Client = ecsBase.Client,
            Cloud = ecsBase.Cloud,
            Container = ecsBase.Container,
            Destination = ecsBase.Destination,
            Dll = ecsBase.Dll,
            Dns = ecsBase.Dns,
            Ecs = ecsBase.Ecs,
            Error = ecsBase.Error,
            Event = ecsBase.Event,
            File = ecsBase.File,
            Group = ecsBase.Group,
            Host = ecsBase.Host,
            Http = ecsBase.Http,
            Log = ecsBase.Log,
            Observer = ecsBase.Observer,
            Organization = ecsBase.Organization,
            Package = ecsBase.Package,
            Process = ecsBase.Process,
            Registry = ecsBase.Registry,
            Related = ecsBase.Related,
            Rule = ecsBase.Rule,
            Server = ecsBase.Server,
            Service = ecsBase.Service,
            Source = ecsBase.Source,
            Threat = ecsBase.Threat,
            Tls = ecsBase.Tls,
            Url = ecsBase.Url,
            User = ecsBase.User,
            UserAgent = ecsBase.UserAgent,
            Vulnerability = ecsBase.Vulnerability,
            Timestamp = ecsBase.Timestamp,
            Labels = ecsBase.Labels,
            Message = ecsBase.Message,
            Tags = ecsBase.Tags,
            //Span = ecsBase.Span,
            Trace = ecsBase.Trace,
            Transaction = ecsBase.Transaction,
        };
    }
}

[Mpdreamz]

Thanks for raising this! Agreed this support for this usecase would be great.

I've opened: https://github.com/elastic/ecs-dotnet/pull/217 to implement this natively in the library.