ParamsSource attribute doesn't work as expected

[evilguest]

The properties marked with the ParamsSource attribute aren't set in the ExecutionValidatorBase. This causes issues when this validator calls the GlobalSetup. The property marked as this does work as expected (gets inited to the first listed value before calling the GlobalSetup method):

        [Params(
            typeof(BasicTestSet),
            typeof(UnsafeTestSet),
            typeof(SlowDeferredLinqTestSet),
            typeof(FastLinqTestSet),
            typeof(UnsafeLinqTestSet),
            typeof(UnsafeParallelTestSet)
        )]
        public Type FilterType { get; set; }

The property marked as this doesn't work as expected - it remains null upon the GlobalSetup method call:

        [ParamsSource(nameof(GetTestTypes))]
        public Type FilterType { get; set; }
        public static IEnumerable<Type> GetTestTypes() => new[] {typeof(BasicTestSet),
            typeof(UnsafeTestSet),
            typeof(SlowDeferredLinqTestSet),
            typeof(FastLinqTestSet),
            typeof(UnsafeLinqTestSet),
            typeof(UnsafeParallelTestSet)};

The root cause is in the ExecutionValidatorBase.TryToSetParamsProperties code. It just scans the properties for the Params attribute, ignoring the ParamsSource attribute:

var paramProperties = benchmarkTypeInstance
                .GetType()
                .GetAllProperties()
                .Where(propertyInfo => propertyInfo.GetCustomAttributes(false).OfType<ParamsAttribute>().Any())
.ToArray();

Instead it should rely on a common init code, like ReturnValueValidator does: InProcessRunner.FillMembers(benchmarkTypeInstance, benchmark);

[AndreyAkinshin]

@evilguest thanks for the report! @adamsitnik could you take a look?

[adamsitnik]

@AndreyAkinshin I don't have time ;( I am finishing the CoreFX port ;)

@evilguest would you like to send a PR? you did a great job finding the source of the problem and now you are one step away from becoming a BenchmarkDotNet contributor!