Unable to Reference Remote step assembly (app.config) from feature file

[sdekosky]

Not able to map feature file to step definition via remote assembly

SpecFlow Version:

• [ ] 3.0
• [X] 2.4.1
• [ ] 2.3
• [ ] 2.2
• [ ] 2.1
• [ ] 2.0
• [ ] 1.9

Used Test Runner

• [ ] SpecFlow+Runner
• [X] MSTest 1.4.0
• [ ] NUnit
• [ ] Xunit

Version number: 1.4.0

Visual Studio Version

• [X] VS 2017
• [ ] VS 2015
• [ ] VS 2013

Are the latest Visual Studio updates installed? 15.9.10

• [X] Yes
• [ ] No

.NET Framework:

• [X] >= .NET 4.5 4.7.1
• [ ] before .NET 4.5
• [ ] .NET Core 2.0
• [ ] .NET Core 2.1
• [ ] .NET Core 2.2
• [ ] .NET Core 3.0

Test Execution Method:

• [X] Visual Studio Test Explorer
• [ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
• [ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
  </configSections>
  <connectionStrings>
    <add name="DataInputContextDb" connectionString="Data Source=xxxxxxxx;Initial Catalog=xxxxxdfdfdf;Integrated Security=True" providerName="System.Data.SqlClient"/>
  </connectionStrings>
  <specFlow>
    <unitTestProvider name="MsTest"/>
    <stepAssemblies>
      <stepAssembly assembly="BSTGlobal.Testing.Rest"/>
    </stepAssemblies>
  </specFlow>
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- we are going to add a separate stepAssembly here to break up the acceptance tests from the generic libraries -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <appSettings>
    <add key="ApiServer" value="pv-devwebl02.bstsoftware.net"/>
    <add key="ApiEndpoint" value="https://xxxxxxxxxxxxxxxx/api"/>
    <add key="Protocol" value="http"/>
    <add key="Port" value="33333"/>
    <add key="ClientSettingsProvider.ServiceUri" value=""/>
  </appSettings>
  <system.web>
    <membership defaultProvider="ClientAuthenticationMembershipProvider">
      <providers>
        <add name="ClientAuthenticationMembershipProvider" type="System.Web.ClientServices.Providers.ClientFormsAuthenticationMembershipProvider, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" serviceUri=""/>
      </providers>
    </membership>
    <roleManager defaultProvider="ClientRoleProvider" enabled="true">
      <providers>
        <add name="ClientRoleProvider" type="System.Web.ClientServices.Providers.ClientRoleProvider, System.Web.Extensions, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" serviceUri="" cacheTimeout="86400"/>
      </providers>
    </roleManager>
  </system.web>
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.1"/></startup></configuration>

Repro Project

Issue Description

Unable to reference a remote assembly in app.config in order to access the step definitions in this assembly. This was working prior to Visual Studio 15.9.10 version (15.9.8) update with "SpecFlow for Visual Studio 2017 version 2019.0.4.X" with all other above versions.

Steps to Reproduce

• Create new solution
• Add feature file project to solution
• Add step definition project to solution
• build both
• reference step definition assembly in app.config
• reference step definition project in feature file project
• build feature file project.
• Check to see if existing feature file is linked to step definitions **Verify that it is not and is purple color
• remove code behind for all feature files -regenerate code behind for all feature files at project level -remove any SpecFlow step cache files from %TEMP% -Clean OBJ and BIN folders from Feature File project and step definition project -Rebuild solution
• Check to see if existing feature file is linked to step definitions **Verify that it is not and is purple color -Create new feature file and generate the steps in local step file to feature file project **Verify that the feature is correctly linked to the step definitions

Thoughts?

[SabotageAndi]

Sorry, but I don't get where the error is. At runtime? Does the Intellisense has a problem? Or the syntax highlighting?

[sdekosky]

At Runtime Syntax is not highlighting. When I attempt to go to definition the error is : "No matching step binding found". I have the correct reference in app.config which has always worked previously.

[sdekosky]

at runtime and when I attempt to go to definition

[sdekosky]

Just an FYI I would rather use the specflow.json configuration if I could instead of app.config. I could not get this to work in SpecFlow 2.4.1. Did the update to the "SpecFlow for VS 2017 extension" require use of specflow.json as the config for remote assembly step definitions?

[nuzolx]

What is the synthax to do that in json config? Because I have the same issue.

[sdekosky]

Hi @SabotageAndi: Have you had a chance to look at the issue i'm encountering?

[SabotageAndi]

No, I am at home in Vienna, Austria. It's 19:01 and I am in the weekend. I will have a look on Monday. A project to reproduce your issue would be great than.

[jimcomiskey]

@sdekosky I believe am having the same problem. have you tried enabling Specflow Tracing and looking for errors in the Specflow output? when I do so, I find this error:

[12:59:50.0554914] Tracing: Tracing enabled for all categories
[12:59:50.0554914] ProjectScopeFactory: subscribed to solution closed 1
[12:59:50.0674878] VsProjectScope: Initializing...
[12:59:50.1834850] VsProjectScope: Configuration loading error: System.InvalidCastException: Unable to cast object of type 'TechTalk.SpecFlow.Configuration.StepAssemblyConfigElement' to type 'TechTalk.SpecFlow.IdeIntegration.Configuration.AppConfig.StepAssemblyConfigElement'.
   at TechTalk.SpecFlow.IdeIntegration.Configuration.AppConfig.AppConfigConfigurationLoader.LoadAppConfig(SpecFlowConfiguration specFlowConfiguration, ConfigurationSectionHandler configSection)
   at TechTalk.SpecFlow.IdeIntegration.Configuration.ConfigurationLoader.Load(SpecFlowConfiguration specFlowConfiguration, IConfigurationHolder configurationHolder)
   at TechTalk.SpecFlow.VsIntegration.Implementation.LanguageService.VsProjectScope.LoadSpecFlowConfiguration(). Using default configuration instead.
[12:59:50.1834850] VsProjectScope: Initializing configuration file trackers ...
[12:59:50.1834850] VsProjectScope: Initializing file tracker for app.config ...

I am wondering if you are encountering the same problem.

[sdekosky]

Hi @jimcomiskey: Thanks for doing this!!! I enabled SpecFlow tracing and I am getting the same error as you. However, after re-installing SpecFlow for VS 2017 extension latest version (2019.0.8.3301), my tests do execute correctly during run time when referencing my step assembly through app.config. The issue now is that I can't "Go To Definition" from the Feature file with the remote reference assembly in app.config but can "Go To Definition" from a feature file connected to a locally generated step definition. This behavior definitely changed over the past couple days after upgrading VS 2017 to 15.9.10. During execution do you get any other errors than the configuation error?

[jimcomiskey]

No, that's the only error I'm seeing. I'm not having problems with running the tests, only the issue of the Feature files not finding step definitions in the external assembly. It was working fine for me until the Specflow extension was upgraded automatically in the past couple of days.

[sdekosky]

Looks like we definitely have the same issue.

[jimcomiskey]

@sdekosky I rolled back to an older version of the Specflow Visual Studio extension from here: https://github.com/techtalk/SpecFlow.VisualStudio/releases and turning off "Automatically update this extension", and things are back to normal for me now.

[jimcomiskey]

Submitted PR to fix this. https://github.com/techtalk/SpecFlow.VisualStudio/pull/55

[SabotageAndi]

PR is now merged. Thanks @jimcomiskey

In some minutes it will be on the preview extension gallery. See https://specflow.org/2018/specflow-3-public-preview-now-available/ - Installing the Preview Visual Studio Extension

[johns397]

Is it on the market place yet? If so, which version number should we be looking for?

[SabotageAndi]

@johns397 Did you read my last comment? It is in our preview extension gallery, not the market place. How you can set it up is written in the attached link.

Version number is 2019.0.9.46639

[sdekosky]

Hi @SabotageAndi, @jimcomiskey and all: Thanks for fixing the issue above. I just tested out the new extension and it works!

[johns397]

@johns397 Did you read my last comment? It is in our preview extension gallery, not the market place. How you can set it up is written in the attached link.

Version number is 2019.0.9.46639

Yes, I saw that but I don't want to upgrade to SpecFlow 3 if I don't have to (and don't have the bandwidth to test this) Will this fix be released on SpecFlow 2.4.0.x?

[SabotageAndi]

@johns397 this issue has nothing to do with SpecFlow 3 or 2.4. It is in the VS Extension. You will get the update automatically when it is published to the marketplace.

[StarWars999123]

When will we receive an update @SabotageAndi ? Do you have any plan for this yet?

Update: I found the bugfixed version. Will it get automatically updated with the latest release, when a release gets available?

[SabotageAndi]

@StarWars999123 Extension will get updated automatically. I am currently at a conference in London (and it's 7:00 pm here currently). I hope I find tomorrow morning some time to upload the extension to the market place.

[ds12will]

@SabotageAndi Thank you for quickly responding to this as I'm sure you'll get many of these

[StarWars999123]

I just wanted to know, whether it might be days, weeks or even month, until we get a release. Sounds perfect. I hope you are able to see something from the conference at least! Thank you very much!

[Cylox]

We make heavy re-use of step definitions through step assemblies and this hit us quite hard since we had to roll back the version manually on all dev and tester machines.

[johns397]

When will we receive an update @SabotageAndi ? Do you have any plan for this yet?

Update: I found the bugfixed version. Will it get automatically updated with the latest release, when a release gets available?

I have turned off auto-updates for this extension after being 'burned' by the one last week! I was just getting ready to use a SpecFlow test for a demo when the extension auto-updated and caused me problems. [Today I picked up the preview version and it does seem to fix the problem. I hope in the future you have a unit test for this so that this kind of problem doesn't get released to the user community.]

[SabotageAndi]

@johns397 this is an open source project. We are happy to receive PRs that increase our stability. The repo for the Visual Studio extension is here: https://github.com/techtalk/SpecFlow.visualStudio/

We had about more than 100 users of the preview extension and nobody filed an issue about it.

[SabotageAndi]

Fixed extension is uploaded to the marketplace. You should get it in the next minutes. You need to restart VS to update.

[fabiocardoso87]

@SabotageAndi, I think I'm facing same issue on Specflow 3 and MSTest. My Specflow extension on VS is 2019.0.42.36358. I configured the reference to another project on app.config exactly like described in this issue description, but it does't work at all. Test explorer can't find the Binding [BeforeScenario] and throw me: Could not load assembly file or assembly.

When I put the hooks class inside the project where the feature file is located, it works like a charm!

Is there anything else I could do?