Proper support for nuget packages

[sayedihashimi]

When a project template is added that uses NuGet packages there are two ways to get support for NuGet packages.

1. By manually changing the template to support NuGet packages
2. Rely on package restore to restore packages

The first approach is the best way to go, but its too difficult so it's not used here. Instead most rely on the second approach. This works well if users have the packages folder at ..\packages. If packages are stored in a different relative path then odds are the end user will get build errors. This is because in the .csproj file the Reference elements have a HintPath pointing to ..\packages.

To remedy this I'm proposing the following.

• [ ] Create a new custom wizard that can be used to change the HintPath value to the correct path to packages
• [ ] Update build process to inject this new custom wizard into all the .vstemplate files during processing

When the templates are installed the custom wizard will run and update the HintPath to the correct value for that project. We can also probably do this by updating the HintPath values in the .csproj on template creation to use a new template parameter, let's say $packageshintpath$. Then the custom wizard just populates that value.

[codewithtyler]

@sayedihashimi considering all the trouble I've been having lately with NuGet packages I'd like to suggest this as the next feature we add once we get the dynamic templates feature completed.

[RehanSaeed]

+1

I use option 2 for ASP.NET MVC Boilerplate and have been looking into option 1 myself. I don't currently use SideWaffle but am considering moving to it. I do have some questions:

1. Will you still have to download the nupkg package files and put them in the VSIX project (This seems a madness to me).
2. Option 1 causes the NuGet package custom tasks to run, causing web.config file edits, opening readme files and inserting files I don't want. Is there some way around this?
3. Does SideWaffle support putting ASP.NET MVC projects into the special 'New ASP.NET Project' dialog? I've been trying to work out how to do this and it seems a nightmare.
4. I have also read that Option 1 will force you to create a VSIX for VS 2012/13/15.

Overall I must say that creating project templates is far harder than it needs to be. I appreciate what you are trying to do with SideWaffle and indeed with SlowCheetah too.

[ghost]

If you inject a custom wizard, what will happen if the user already has a custom wizard specified in the template?

I'm using option 2 as "undoing" all changes made by NuGet packages prior to template building seems unrealistic.

Option 2 seems to work fine except that my template's packages directory is getting copied into the template and bloating the size of my VSIX. Since the template does not generate the packages directory when a new project is created, having the packages content inside my template serves no real purpose.

All I really need is a way to tell TemplateBuilder to ignore certain files or directories so that I can prevent the packages folder from my sub-solutions from being copied to obj and bundled into the template zip.

I'm using the How to create a multi project template instructions. I'm using a stub template with a custom wizard to add the projects one directory up due to an issue with VS creating multi-project templates with an extra directory. The relative location of the packages is the correct "..\packages" location in this case so there's no need for me change the HintPath.

I can manually delete my packages folder prior to building the VSIX project, but then I have to restore them when I build & test the sub-solution projects, so it would be preferable to retain the actual packages folder on the filesystem but prevent it from being copied into the temporary copy used to package the template.

[sayedihashimi]

@RehanSaeed

Will you still have to download the nupkg package files and put them in the VSIX project (This seems a madness to me).

No, it would still use option 2 to download nuget packages at design/build time. But the <HintPath> will be updated to the correct value, so it's a hybrid.

Option 1 causes the NuGet package custom tasks to run, causing web.config file edits, opening readme files and inserting files I don't want. Is there some way around this?

I don't think so, but not 100% sure.

Does SideWaffle support putting ASP.NET MVC projects into the special 'New ASP.NET Project' dialog? I've been trying to work out how to do this and it seems a nightmare.

No. While it's technically possible it's not supported at this time. I'm on the team that created the One ASP.NET dialog and the underlying support for it. If you would like us (Microsoft) to support that I'd recommend opening an item at http://aspnet.uservoice.com/ to see how much interest there is in that. With that said, for ASP.NET 5 we are planning some major updates to how templates are created in VS. Right now we are supporting creating projects in VS and by using yo aspnet for xplat scenarios. Since these are two different implementations it's a lot of extra work for us. We are investigating the possibility of implementing all templates in yo and then have the VS dialog simply invoke those. I don't think we will be able to even start that work for at least 6-8 months though because we have a lot of other more important work. For ASP.NET 4 I don't expect things to change much.

I have also read that Option 1 will force you to create a VSIX for VS 2012/13/15.

Not sure, you probably know more about that than I do because I've always avoided it.

Overall I must say that creating project templates is far harder than it needs to be.

I agree. I've spoke with the team that owns the VS template creation experience. I believe they would like to improve it but I'm not sure if they have any concrete plans. I showed them TemplateBuilder a while back, but never received any updates/thoughts on it. If you think MSFT should invest in & ship TemplateBuilder formally I'd suggest that you create an item at http://visualstudio.uservoice.com.

[sayedihashimi]

@webwizgordon

If you inject a custom wizard, what will happen if the user already has a custom wizard specified in the template?

VS templates can have several wizards declared in the .vstemplate file. So if we inject our own wizard it will likely be implemented such that it's the first wizard to run. Other wizards should run after that. You're wizard wouldn't even know that it ran.

All I really need is a way to tell TemplateBuilder to ignore certain files or directories so that I can prevent the packages folder from my sub-solutions from being copied to obj and bundled into the template zip.

Yes this would be good. Currently TemplateBuilder looks at the files/folders on disk to figure out what to include. In an ideal world we could add support for something like .templateignore so that you can drop the file in the root folder and then have TemplateBuilder exclude it. I'm really busy with other projects now so won't have time to implement that, but if someone else who is familiar with MSBuild wants to take a crack at it I can guide you.

~~As a workaround until we get such feature I think the easiest thing would be to remove the extra files from the template intermediate folder after the files are copied there, but before the VSIX is created. To do this you'll need to inject a new target into the build process for your VSIX/VSPkg project. In your .csproj file you'll need to add something like the following.~~

Ignore that, as I was looking at the targets file I noticed there is an Item that may be useful here ls-ProjectTemplateFilesToRemoveOnCopy. Try this, in the props file that template builder drops in Properties populate that item with the files to exclude. If that doesn't work then see the paragarph above that is ~~striked out~~.

Note: targets file is here https://github.com/ligershark/template-builder/blob/master/tools/ligershark.templates.targets

I'm using the How to create a multi project template instructions. I'm using a stub template with a custom wizard to add the projects one directory up due to an issue with VS creating multi-project templates with an extra directory. The relative location of the packages is the correct "..\packages" location in this case so there's no need for me change the HintPath.

Is this a generic wizard? If so would it make sense to add it to TemplateBuilder so that others can easily use it as well? I've heard a few others complain about this as well.

[RehanSaeed]

My Number 1 feature request from ASP.NET MVC Boilerplate is to add authentication support and the best way to do this would be using the One ASP.NET dialogue with it's authentication selector. I get contacted about it once or twice a week. I already raised the issue on UserVoice but nobody cares enough about this niche feature to up-vote it.

It looks like SideWaffle is definitely the easiest way to create a project template but with the above limitation. I'm working on an MVC 6 project template, SideWaffle seems to have a workaround to support it, presumably yeoman is the proper way to create a template? Is there a guide for creating a template using yoman?

[sayedihashimi]

@RehanSaeed Yeoman is best if you are targeting xplat scenarios. Typical VS users may not discover it. There are great tutorials on creating yo generators. Start at http://yeoman.io, I can help you if needed as well. That said I think you're trying to get to existing VS users. Have you considered creating a custom wizard? In your case you can create a WPF dialog that has auth options available which is launched during project create. Custom wizards work with SideWaffle just as they do standard templates. I have some rough notes at https://github.com/ligershark/side-waffle/wiki/How-to-use-a-custom-wizard. I can help you work through that as well. Honestly its much easier than hooking into the One ASP.NET dialog.

Let me know what direction you want to take and I'll help you achieve your goal.

[RehanSaeed]

Moved to here, so I don't hijack the thread.

[ghost]

@sayedihashimi

The wizard is fairly generic. It just uses a DTE object to programatically add the multi-project template one level up from the normal multi-project default path.

Here's a gist with the wizard. It's based on this Stack Overflow answer that I saw in #237 with changes to specify the target template as a custom parameter instead of a hardcoded path and to hold onto the DTE reference instead of using System.Runtime.InteropServices.Marshal.GetActiveObject("VisualStudio.DTE.10.0").

It's not ideal because it requires adding a second template for each multi-project template, but it works.

I have another wizard that simply lets VS create the projects at their default location then removes them from the solution, moves them up one directory, then re-adds them to the solution. When they get temporarily removed from the solution, any project references get removed though so I added logic to find the project references prior to removal and restore them after the move.

The advantage of this wizard is there's no need for a secondary stub project, but the disadvantage is that I don't know if there are any other ramifications caused by removing and re-adding the projects to the solution.

Both wizards try to delete the obsolete extra directory at the end but VS seems to like to create it after the wizard completes anyway.

[ghost]

@sayedihashimi

I couldn't get ls-ProjectTemplateFilesToRemoveOnCopy to work as desired. I added

  <ItemGroup>
    <ls-ProjectTemplateFilesToRemoveOnCopy Include="**\packages\**\*" />
  </ItemGroup>

to template-builder.props and I could see the files listed from GetFilesToExlucudeAsList() while debugging CreateTemplateTask.cs.

It doesn't seem to have any effect though., It looks like the exclusion items are compared to the project items, and the packages folder and its contents aren't included as items. I also see that ls-ProjectTemplateFilesToRemoveOnCopy by default includes "_preprocess.xml" (among others) but _preprocess.xml ends up being included the template, I assume because of the same reason of it not belonging to the sub-solution's project.

My current template build process is like this:

1. Make changes to sub-solution containing projects to be exported as templates. Ensure solution compiles, test, etc.
2. Clean sub-solution to free up obj/bin folder contents.
3. Close sub-solution and delete its packages folder.
4. Open template-building solution and build.
5. Re-download packages next time changes are made to the sub-solution.

I can probably add some build tasks to help with the deleting (though my MSBuild skill level is novice) prior to building templates but that still leaves me with restoring packages after. Ideally I'd like to configure TemplateBuilder to copy over only the project items instead of everything on disk.

In addition to packages, it also appears that the sub-project's obj & bin folder contents are copied into the template zips as well (though I don't believe VS actually uses them when creating a project from the template).

My VSIX is 30 MB if I don't delete the packages and obj/bin content first. If I delete the packages, it drops down to 11 MB. If I also clean the sub-solution first, it drops down to a much more reasonable 800 KB.

[sayedihashimi]

@webwizgordon did you delete the bin/obj folder before build? The files may have been left over from a previous build.

[ghost]

I do a clean before build but I don't manually delete anything that may be left over in bin/obj (which seems to be 0-byte TemporaryGeneratedFile_*.cs files and a DesignTimeResolveAssemblyReferencesInput.cache file).

[sayedihashimi]

@webwizgordon ok that's fine. Do this: create an issue in your repo that mentions me and lists everything you want deleted. I'll author the changes and send you a PR.

[ghost]

My project isn't on GitHub (we use TFS).

I did figure out that I can prevent the copying of my sub-projects' packages and bin/obj content to obj/$configuration$/ls-Templates by changing ligershark.templates.targets line 387 (line 386 in the NuGet v1.1.3 release) from:

<_ls-ItemsToZip Include="%(_ls-ItemTemplatesOutputs.ItemTemplateFolder)**\*"  />

to

<_ls-ItemsToZip Include="%(_ls-ItemTemplatesOutputs.ItemTemplateFolder)**\*" Exclude="%(_ls-ItemTemplatesOutputs.ItemTemplateFolder)packages\**\*;%(_ls-ItemTemplatesOutputs.ItemTemplateFolder)**\obj\**\*;%(_ls-ItemTemplatesOutputs.ItemTemplateFolder)**\bin\**\*" />

_Item_Templates confused me at first as I didn't realize it also affected _Project_Templates.

That way rather than copying and then deleting, the copy is avoided in the first place.

Building is so much faster now.

Is there a way I can do this from template-builder.props? I'd prefer to not make changes to the NuGet-provided .targets file in the packages folder, since I typically exclude the packages folder from source control and I assume it would cause issues updating the package.

[sayedihashimi]

@webwizgordon instead of modifying that create a target that executes after the item is populated and the use the Remove attribute with the same expression you have in Exclude. Yes, the item is poorly named. After implementing item templates I realized I could use the same process for project templates.

[ghost]

<_ls-ItemsToZip> is populated and consumed within the same ls-CoreCopyTemplateFilesToIntermediateFolder Target so I don't know how to get a reference to it at the time needed to use the Remove attribute effectively (before it is consumed by a Copy task).

The closest I can get is to delete the files from the destination folder after they are copied but before they are processed:

<PropertyGroup>
  <ls-PreprocessTemplatesDependsOn>
    $(ls-PreprocessTemplatesDependsOn);
    RemoveUnwantedItemsFromTemplate;
  </ls-PreprocessTemplatesDependsOn>
</PropertyGroup>

<Target Name="RemoveUnwantedItemsFromTemplate"
        Outputs="%(_ls-ItemTemplatesOutputs.Identity);All">
  <PropertyGroup>
    <_ls-IntermediateItemTemplateFolder>$(ls-ItemTemplateOutputRoot)%(_ls-ItemTemplatesOutputs.OutputRoot)%(_ls-ItemTemplatesOutputs.ItemTemplateName)\</_ls-IntermediateItemTemplateFolder>
  </PropertyGroup>

  <Message Text="RemoveUnwantedItemsFromTemplate for template: $(_ls-IntermediateItemTemplateFolder)" Importance="normal" />

  <ItemGroup>
    <ItemsToRemove Remove="@(ItemsToRemove)" />
    <ItemsToRemove Include="$(_ls-IntermediateItemTemplateFolder)packages\**\*;$(_ls-IntermediateItemTemplateFolder)**\obj\**\*;$(_ls-IntermediateItemTemplateFolder)**\bin\**\*" />      
  </ItemGroup>

  <Delete Files="@(ItemsToRemove)" />
</Target>

This seems to work (though with some unnecessary disk usage copying files only to immediately delete them). It also leaves the folders behind. I have a longer version that tries to clean up any empty folders too.

I'm not that experienced with MS Build so it would not surprise me to learn there's a better way to accomplish this than what I've come up with.