Allow embedded projects to suppress generating members that reference types from specified namespaces

[StephenMLucas]

Proposal: Allow embedded projects to suppress generating members that reference types from specified namespaces

Summary

The current <CsWinRTIncludes> section allows namespaces to be included in the code generation, a second <CSWinRTExcludes> section could be used to list namespaces and types that are explicitly not wanted for the embedded projection. Any members that would produce a reference to an excluded type or namespace would be omitted from the projection.

Rationale

• Embedded mode in 1.4.1 works well for referencing low level components of the Windows SDK, but higher level components may have a very large dependency graph that must be fully satisfied to use a single member of the type. This results in a slow compilation and large amounts of embedded code that are not wanted.
• Ideally the embedded generator would only produce the projections used in the embedded library, but this would be A Difficult Problem to make work well with Intellisense. Being able to manually exclude the unwanted dependencies is a much simpler compromise.
• Having an explicit exclusion list extends the current workflow of receiving CS0234 errors that list the namespaces that need to be either included or excluded.

Important Notes

• As a sample case, consider Windows.ApplicationModel.AppService. This has a relatively small set of dependencies, however there are a small number of rarely used methods that reference the types Windows.System.RemoteSystems.RemoteSystemConnectionRequest and Windows.System.User. Projecting these unused methods results in a very large dependency tree that eventually pulls in a significant fraction of the total Windows SDK surface area.

Open Questions

• Is there already a better approach planned to otherwise manage and prune the dependency tree?

[StephenMLucas]

Note that although it's not the root cause this feature would also provide a mitigation for issues like https://github.com/microsoft/CsWinRT/issues/1073 where a problematic dependency is not actually required.

[maxkatz6]

YES! I definetly agree this is option is a must have.

[manodasanW]

Thanks @StephenMLucas for this feedback. If I am understanding your proposal, are you saying we should expand our current CsWinRTExcludes mechanism to exclude methods / members of included types which references types that are excluded? So for instance, if a type has 5 methods and one of those methods had a reference to a type that was in the CsWinRTExcludes, then the type would be projected with just 4 methods rather than 5.

This does make me wonder whether we should expand the CsWinRTIncludes / CsWinRTExcludes mechanism to support operating on methods and not just namespaces and types. Something we will need to think about and consider, but I do see how it can be helpful with embedded support.

[StephenMLucas]

Hi @manodasanW - yes that's correct.

I agree that extending the includes mechanism down to method level granularity would also have the same functional outcome. From a workflow perspective I think on balance it's usually going to be more convenient to think at the type level and then exclude problematic referenced types rather than choosing the individual methods to include or exclude. But they are certainly complementary approaches and if one is significantly easier to implement then having either would help with these scenarios.

[dotMorten]

Here's my scenario: I want to use the BluetoothLEAdvertisementWatcher, but before you know it, you're spending forever trying to figure out which namespaces you need to include to generate valid code that'll compile, and your list of APIs getting included just grows and grows and grows. What I'd like to do is just specify the class Windows.Devices.Bluetooth.Advertisement.BluetoothLEAdvertisementWatcher and all the required types to generate this will be included - nothing less, nothing more. The code generator today generates invalid non-compiling code because it doesn't walk the dependency graph and figure out what is actually needed. So perhaps we could just specific not the class, but the members we intend to use, and only their dependencies would be included.

I ended up giving up and just using reflection to the APIs I need instead of using this generator :-(

[tthiery]

Like @dotMorten I also fail on Bluetooth CsWinRTIncludes. In my case, I discover a Bluetooth LE device by a known service id, wirte/read from a characteristics. No fanciness here. To only other type outside of the Bluetooth namespace I need is Windows.Storage.Streams.DataReader.

Honestly, the only reason I access WinRT APIs are for device access. And I think most of these namespaces are cluttered with references to other namespaces for edge cases. I think the embedded concept is doomed until this is not resolved.

I am down to the following (without even knowing whether it will work in the end) ...

  <PropertyGroup>
    <TargetFrameworks>net8.0-windows</TargetFrameworks>
    <CsWinRTEmbedded>true</CsWinRTEmbedded>
    <CsWinRTWindowsMetadata>10.0.19041.0</CsWinRTWindowsMetadata>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Windows.CsWinRT" Version="2.0.4" />
  </ItemGroup>

  <PropertyGroup>
    <CsWinRTIncludes>
      Windows.Foundation;
      Windows.Devices.Bluetooth.Advertisement.BluetoothLEAdvertisement
      Windows.Devices.Bluetooth.Advertisement.BluetoothLEAdvertisementWatcher;
      Windows.Devices.Bluetooth.Advertisement.BluetoothLEScanningMode;
      Windows.Devices.Bluetooth.Advertisement.BluetoothLEAdvertisementReceivedEventArgs;
      Windows.Devices.Bluetooth.Advertisement.IBluetoothLEAdvertisement
      Windows.Devices.Bluetooth.Advertisement.IBluetoothLEAdvertisementBytePattern;
      Windows.Devices.Bluetooth.Advertisement.IBluetoothLEAdvertisementReceivedEventArgs;
      Windows.Devices.Bluetooth.Advertisement.IBluetoothLEAdvertisementPublisher;
      Windows.Devices.Bluetooth.Advertisement.IBluetoothLEAdvertisementWatcher;
      Windows.Devices.Bluetooth.Advertisement.BluetoothLEManufacturerData;
      Windows.Devices.Bluetooth.Advertisement.IBluetoothLEManufacturerData;
      Windows.Devices.Bluetooth.BluetoothLEAppearance;
      Windows.Devices.Bluetooth.BluetoothSignalStrengthFilter;
      Windows.Devices.Bluetooth.BluetoothDeviceId;
      Windows.Devices.Bluetooth.BluetoothLEDevice;
      Windows.Devices.Bluetooth.GenericAttributeProfile;
      Windows.Devices.Bluetooth.BluetoothAddressType;
      Windows.Devices.Bluetooth.BluetoothCacheMode;
      Windows.Devices.Bluetooth.BluetoothConnectionStatus;
      Windows.Devices.Bluetooth.BluetoothError;
      Windows.Devices.Bluetooth.IBluetoothDeviceId;
      Windows.Devices.Bluetooth.IBluetoothLEAppearance;
      Windows.Devices.Bluetooth.IBluetoothSignalStrengthFilter;
      Windows.Devices.Bluetooth.IBluetoothLEDevice;
      Windows.Storage.Streams.IBuffer;
      Windows.Storage.Streams.Buffer;
      Windows.Storage.Streams.IInputStream;
      Windows.Storage.Streams.IOutputStream;
      Windows.Storage.Streams.InputStreamOptions;
      Windows.Storage.Streams.IRandomAccessStream;
      
      Windows.Storage.Streams.IContentTypeProvider;
      Windows.Devices.Enumeration.IDeviceAccessInformation;
      Windows.Devices.Enumeration.DeviceAccessChangedEventArgs;
      Windows.Devices.Enumeration.IDeviceAccessChangedEventArgs;
      Windows.Devices.Enumeration.DeviceAccessStatus;
      Windows.Devices.Enumeration.DeviceAccessInformation;
      Windows.Devices.Enumeration.DeviceClass;
    </CsWinRTIncludes>
    <CsWinRTExcludes>
      Windows.Foundation.Diagnostic;
      Windows.Devices.Bluetooth.IBluetoothLEDevice2;
      Windows.Devices.Bluetooth.IBluetoothLEDevice3;
      Windows.Devices.Bluetooth.IBluetoothLEDevice4;
      Windows.Storage.Streams.IRandomAccessStreamReferenceStatics;
      Windows.Storage.Streams.RandomAccessStreamReference;
      Windows.Foundation.PropertyType;
    </CsWinRTExcludes>
  </PropertyGroup>

with the following remaining errors (on the core BluetoothLEDevice type)

CsWinRT\Windows.Devices.Bluetooth.cs(593,52): error CS0234: [..] "DeviceInformation" [..] "Windows.Devices.Enumeration" [..]

As soon as I add Windows.Devices.Enumeration.DeviceInformation hell breaks loose and I am up to 39 errors and with the next DeviceWatcher I am up to over 100 errors.

I will be happy with any way of breaking this dependency graph issue: Auto-analyzing the used WinRT objects, enumerating namespaces with automatic trimming, manual enumeration of excluded members.

@manodasanW any updates here?

[Marv51]

I would like to given example of why this issue is important to me.

In my app I'm using only this line of WinRT related code. The app runs in an MSIX package and this just gets the version number of the package:

var version = Windows.ApplicationModel.Package.Current.Id.Version;

An extremly limited use of WinRT like this, should be a great use case for the embedded support in CsWinRT (at least that's what I thought).

It seems to be working nicely, however the way to get there is pretty painful (all artisanally handcrafted):

Very long CsWinRTIncludes list (click to expand)

<PropertyGroup>
<CsWinRTEmbedded>true</CsWinRTEmbedded>
<CsWinRTWindowsMetadata>sdk</CsWinRTWindowsMetadata>
<CsWinRTGenerateProjection>true</CsWinRTGenerateProjection>
<CsWinRTIncludes>
	Windows.ApplicationModel.Package;
	Windows.ApplicationModel.AppInstallerPolicySource;
	Windows.ApplicationModel.IFindRelatedPackagesOptions;
	Windows.ApplicationModel.IAppInstallerInfo;
	Windows.ApplicationModel.AddResourcePackageOptions;
	Windows.ApplicationModel.Core.AppListEntry;
	Windows.ApplicationModel.Core.IAppListEntry;
	Windows.ApplicationModel.AppInfo;
	Windows.ApplicationModel.IAppInfo;
	Windows.ApplicationModel.AppExecutionContext;
	Windows.ApplicationModel.AppDisplayInfo;
	Windows.ApplicationModel.IAppDisplayInfo;
	Windows.Foundation.Metadata.ContractVersion;
	Windows.Foundation.UniversalApiContract;
	Windows.ApplicationModel.AppInstallerInfo;
	Windows.ApplicationModel.IPackage;
	Windows.Foundation.TypedEventHandler;
	Windows.Foundation.IAsyncOperation;
	Windows.Foundation.IAsyncInfo;
	Windows.Foundation.AsyncOperationWithProgressCompletedHandler;
	Windows.Foundation.AsyncStatus;
	Windows.Foundation.FoundationContract;
	Windows.Foundation.AsyncOperationProgressHandler;
	Windows.Foundation.AsyncOperationCompletedHandler;
	Windows.Foundation.IAsyncAction;
	Windows.Foundation.AsyncActionCompletedHandler;
	Windows.Foundation.AsyncActionProgressHandler;
	Windows.Foundation.AsyncActionWithProgressCompletedHandler;
	Windows.Foundation.Metadata.Overload;
	Windows.System.ProcessorArchitecture;
	Windows.ApplicationModel.FindRelatedPackagesOptions;
	Windows.Storage.StorageFolder;
	Windows.Storage.Streams.RandomAccessStreamReference;
	Windows.System.User;
	Windows.Storage.IStorageFolder;
	Windows.Storage.IStorageItem;
	Windows.Storage.CreationCollisionOption;
	Windows.Storage.StorageFile;
	Windows.Storage.StorageLibraryChangeTracker;
	Windows.Storage.NameCollisionOption;
	Windows.Storage.StorageDeleteOption;
	Windows.Storage.FileProperties.BasicProperties;
	Windows.Storage.StorageItemTypes;
	Windows.Storage.FileProperties.StorageItemThumbnail;
	Windows.Storage.FileProperties.StorageItemContentProperties;
	Windows.Storage.FileProperties.ThumbnailMode;
	Windows.Storage.FileProperties.ThumbnailOptions;
	Windows.Storage.IStorageFile;
	Windows.Storage.StorageProvider;
	Windows.Storage.Streams.IInputStreamReference;
	Windows.Storage.Streams.IRandomAccessStreamReference;
	Windows.Storage.StreamedFileDataRequestedHandler;
	Windows.Storage.Streams.IRandomAccessStream;
	Windows.Storage.FileAccessMode;
	Windows.Storage.StorageStreamTransaction;
	Windows.Storage.Streams.IInputStream;
	Windows.Storage.StorageOpenOptions;
	Windows.Storage.Search.IStorageFolderQueryOperations;
	Windows.Storage.Search.IndexedState;
	Windows.Storage.Search.StorageFileQueryResult;
	Windows.Storage.Search.StorageFolderQueryResult;
	Windows.Storage.Search.QueryOptions;
	Windows.Storage.Search.CommonFileQuery;
	Windows.Storage.Search.StorageItemQueryResult;
	Windows.Storage.Search.CommonFolderQuery;
	Windows.Storage.IStorageLibraryChangeTracker;
	Windows.Storage.StorageLibraryChangeReader;
	Windows.Storage.StorageLibraryChange;
	Windows.Storage.IStorageLibraryChange;
	Windows.Storage.IStorageProvider;
	Windows.Storage.IStorageStreamTransaction;
	Windows.Storage.StreamedFileDataRequest;
	Windows.Storage.Streams.IOutputStream;
	Windows.Storage.IStreamedFileDataRequest;
	Windows.Storage.StreamedFileFailureMode;
	Windows.Storage.Streams.IBuffer;
	Windows.Storage.FileAttributes;
	Windows.Foundation.MemoryBuffer;
	Windows.Foundation.IMemoryBuffer;
	Windows.Storage.Streams.InputStreamOptions;
	Windows.Storage.Streams.IContentTypeProvider;
	Windows.Storage.Streams.Buffer;
	Windows.Storage.FileProperties.IBasicProperties;
	Windows.Storage.FileProperties.IStorageItemExtraProperties;
	Windows.Storage.FileProperties.IStorageItemContentProperties;
	Windows.Storage.FileProperties.MusicProperties;
	Windows.Storage.FileProperties.IMusicProperties;
	Windows.Storage.FileProperties.VideoProperties;
	Windows.Storage.FileProperties.IVideoProperties;
	Windows.Storage.FileProperties.ImageProperties;
	Windows.Storage.FileProperties.IImageProperties;
	Windows.Storage.FileProperties.DocumentProperties;
	Windows.Storage.FileProperties.IDocumentProperties;
	Windows.Storage.FileProperties.PhotoOrientation;
	Windows.Storage.FileProperties.IThumbnailProperties;
	Windows.Storage.FileProperties.ThumbnailType;
	Windows.Storage.FileProperties.VideoOrientation;
	Windows.Storage.FileProperties.PropertyPrefetchOptions
	Windows.Globalization.DayOfWeek;
	Windows.Storage.Search.SortEntry;
	Windows.Storage.Search.IQueryOptions;
	Windows.Storage.Search.DateStackOption;
	Windows.Storage.Search.FolderDepth;
	Windows.Storage.Search.IndexerOption;
	Windows.Storage.Search.IStorageFileQueryResult;
	Windows.Storage.Search.IStorageQueryResultBase;
	Windows.Storage.Search.IStorageFolderQueryResult;
	Windows.Storage.Search.IStorageItemQueryResult;
	Windows.Foundation.Collections.MapChangedEventHandler;
	Windows.Data.Text.TextSegment;
	Windows.System.IUser;
	Windows.Foundation.Metadata.DefaultOverload;
	Windows.Foundation.Collections.IPropertySet;
	Windows.Foundation.Collections.IObservableMap
	Windows.Foundation.Collections.IMapChangedEventArgs;
	Windows.Foundation.Collections.CollectionChange;
</CsWinRTIncludes>
</PropertyGroup>