dotnet
Ideas for .NET Data Community Standup guests, products, demos, etc.
Let us know which projects, people, demos, or other ideas you have for the show! This is the community standup, so we want to feature you, your ideas, and your projects! Comment on this issue and we'll update the list here with suggestions.
Guest/theme suggestions
- AutoMapper
- Provider writers! Get all the provider writers around a round table, and talk about what it's like to write a provider etc.
- Invite SqlClient team to show Microsoft.Data.SqlClient 2.0 and discuss roadmap and open sourcing experience.
- SQLite day featuring
- System.Data.SQLite (mistachkin)
- SQLite-net (praeclarum)
- SQLitePCLRaw (ericsink)
- SQLite Toolbox (ErikEJ)
- Dive deep into spatial data with NetTopologySuite (airbreather & FObermaier)
- Scheduled! See Sept 30th 2020 - Geographic Data with NetTopologySuite
- ArcGIS (dotMorten)
- @Mimetis - Show us DotMim.Sync
- @tonysneed - Trackable Entities and Handlebars
Demos you would like to see
- Many to many demo
- TPT
- ToQueryString
- Index Fill Factor
- Partition Keys (Cosmos)
- How to configure single query VS multi query (when available) to load related entities
- EF value conversions (for enums and timestamp), and other gRPC best practices
- How to manage context lifetime and disposal
Original threads from discussion on guests:
@JeremyLikness There is another thread for specific demos see this. This thread is about highlighting community. Let us know what project, people, or other ideas you have for the show. It is the community standup so we want to feature you, your ideas, and your projects.
Thank you!
- @ErikEJ - @tonysneed Trackable Entities and Handlebars
- @tonysneed - Thank you for the recommendation! Happy to share about Handlebars templates. There have been a number of contributions from the community that have helped it grow over the past few years.
- @bricelam - We could have a SQLite day featuring System.Data.SQLite (mistachkin), SQLite-net (praeclarum), SQLitePCLRaw (ericsink), and SQLite Toolbox (ErikEJ)
- @bricelam - Dive deep into spatial data with NetTopologySuite (airbreather & FObermaier) and ArcGIS (dotMorten)
- @bricelam - I'd love to have @Mimetis show us DotMim.Sync
- @Mimetis - Thanks @bricelam :) I would love to present Dotmim.Sync ;) Just out of curiosity, is it interesting for the community to show this framework, even if it's not using EF ? (at least I'm using your Microsoft.Data.Sqlite provider ) Let me know ! (Be careful, you will have to listen a french guy trying to speak english ! )
- @ErikEJ - It would be VERY interesting!
- @bricelam - Despite the name, our goal is to cover all things related to data on .NET. 👌
Original threads from discussion on demos:
@ajcvickers
Starting a discussion on what kinds of things people would like to see demoed in the EF Community Standup. Please try to be specific about what you want to see.
@ErikEJ - Many to many demo (when available) @ralmsdeveloper - TPT - (Progress); ToQueryString - (Demo); Index Fill Factor - (Demo); Partition Keys (Cosmos) - (Demo) @ilmax - How to configure single query VS multi query (when available) to load related entities #18022 @JeepNL - First of all: thank you for creating the EF Community Standup I really enjoy watching it. I'm using gRPC protobuf files with EF (Core) 5 code first for generating classes & SQLite (relational) tables (for my Blazor WASM project) and it's amazing imo. I'm just starting with it and would love to see some experts talking about the possibilities and best practices. Pretty soon after I started with it I needed to learn more about EF Value Conversion (for enums and timestamp) so that would be a great combo.
- @githubfanster - i'm interested in learning more about gRPC + EF Core possibilities and best practices too
- @ErikEJ - Invite SqlClient team to show version 2 and discuss roadmap and open sourcing experience.
- @roji - Provider writers! Get all the provider writers around a round table, and talk about what it's like to write a provider etc.
- @ErikEJ - And SQLite Toolbox 😎
- @janschreier - Thanks for the shows so far. I really enjoy listening to you guys explaining stuff 👍 on to my suggestion. I still don't know if/when I should/have to call dispose on a dbContext if that is long-lived. E. g. if I use a dbContext in an WPF MVVM app to work on a database record (using the context for changetracking) and thus not being able to wrap it in a using()-block. Here MS gets quoted that unless you do weird stuff on dbContext.Database you don't habe to call Dispose()
https://blog.jongallant.com/2012/10/do-i-have-to-call-dispose-on-dbcontext/
is this still true for EF Core? are there any patterns recommended atm by the ef core team? Properly not enough to talk about for an entire show but if you do a Q&A session, this hopefully might fit in =)
- @JeremyLikness - It's a good practice to ensure your context is disposed when you are done with it. We have updated docs for WPF that should go live today, but in essence a common pattern is for the context to share the same lifecycle as the immediate parent control that hosts it. You can create the context as part of the constructor and dispose of it in OnClosing. You can also choose to create on per operation. If you're not using EF Core to track an object graph or handle concurrency, so for example you're using the disconnected pattern where you attach entities on save/delete/etc., then you can simply enclose the operation in a using block.
- @JonPSmith - I watched the August 5th EF Core community standup and @ajcvickers asks "why people don't use the nightly builds?" I just wanted to say that I use the EF Core 5 previews in my example code because as EF Core progresses it uses different versions of other, non EF Core, NuGet packages that I use in other features such as ASP.NET Core. This means I sometimes (maybe 3 times in the last 7 previews??) I get into "NuGet hell" where you can't easily do a "update all" and I have to find what non-EF Core NuGet package version are you using now and fix that before it will accept the new EF Core 5 preview. Also, my repo is available to readers of my book and they clone it as any time. Therefore they might get a new version of the nightly build and something could break. I hope this feedback is useful.
- @roji - Thanks @JonPSmith. In many (most?) cases, transitive dependencies of EF Core itself shouldn't be a problem - updating to a new EF Core preview/daily should just bring those in. However, if your program takes a dependency on those as well, regardless of EF Core, then yeah, some nuget dependency hell may occur (though in many cases these are harmless warnings). (I'm mentioning this because some people are unaware of how transitive dependencies work and think they should be explicitly taking dependencies on nugets that EF Core depends on, which isn't the case...) I also agree that daily builds are probably not a great idea in sample/documentation code... We also avoid it in our documentation samples, although we do take dependencies on previews where a new feature is being shown.
- @JonPSmith - I updated my comment to refer to "... other, non EF Core, NuGet packages that I use in other features such as ASP.NET Core."
- @ajcvickers - @JonPSmith Some questions. EF Core progresses it uses different versions of other, non EF Core, NuGet packages. Which packages are you referring to? Therefore they might get a new version of the nightly build and something could break. If your repo references a specific daily build, then why would people get a different daily build when they clone it?
- @JonPSmith - Hi @ajcvickers. I had problems with the following non-EF Core packages used by ASP.NET Core app. <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="5.0.0-preview.7.20364.11" /> PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="5.0.0-preview.7.20364.11" /> These changed at least twice. Yep, my bad - I forgot that each nightly build has its own signature. The other thing I didn't mention (and might only applicable to me) is that the previews tell me whether a feature is finished or not and often provider a short intro on how to use the feature. Using the nightly build would make that much more difficult to understand if a feature is fully finished or it might change. For instance I am waiting for the release on table UDF - I think the code is there but you haven't said its finished, so I might change.
- @ajcvickers - @JonPSmith You should not need new versions of those packages unless you're also trying to run ASP.NET Core 5.0 or .NET 5 daily builds. Just a clarification on feature changes--even when we release and announce a feature in a preview doesn't mean we won't change it again before release. You are correct is saying that we don't announce until the feature is relatively usable, but that's somewhat different from being stable.
- @JonPSmith - You are right again Obi-Wan - my training continues. I removed the two NuGet packages and it still compiled, so they weren't needed. I did get into a "NuGet hell" at one point early on and tried a few things including the DependencyInjection and Logging abstractions NuGet. Once there were there I had to update them when you updated in EF Core 5. I will see what issues I have when I update next time. Got it on feature changes. For my very specific instance, i.e. writing a book about EF Core, the releases are good review points but I can see that the daily builds should work well for other people.
- @DickBaker - great sessions so far, thanks. Given the widespread use of AutoMapper to shuffle data shapes around, I suggest that team invite authors of AutoMapper.Collection.EFCore to explain why/when/how best to use such useful package (+dependencies).
Hi @ajcvickers,
I wondered if you would like me to come on a community standup and talk about some of the ways to performance tuning EF Core? I have an demo ASP.NET Core application (open-source) which implements a book selling site. I start with 700 books, then goes to 100,000 books, and then to 1/2 million books. I can show what I did at each stage, with the performance figures to see the differences.
What do you think?
PS. Just watched the Entity Framework Community Standup - EF Core 6.0 Survey Results. Really great info!
@ajcvickers,
I would also be happy to come on the community standup to talk about EFCore.Triggered if you have me. It essentially brings database triggers down a notch to the level of EF (All made possible by the awesome ChangeTracker). I have a simple demo application that I can showcase with various uses of triggers to enforce various business rules. I can furthermore talk about implementation details and how EF triggers vary from domain events and database triggers.
@koenbeuk are you on Twitter? If so, DM me and I'll share my email so we can connect and schedule it.
@JonPSmith @ajcvickers @roji I raised Issue #2991 and a PR but this has now been junked by EF Team https://github.com/dotnet/EntityFramework.Docs/pull/2992#issuecomment-766878278 the code enacted many [real-world experience] ways to use EF (and even ADO) in good/bad/ugly ways and demoed Benchmark.net results to compare, soI would like ppl to dust off and revisit. I tried to see what Jon had done but the MEAP didn't expose it as still a WIP. Anyway, hope helps progress to perf nirvana. Best wishes for more StandUps incl perf in future to fend off Dappr competition
If there is interest I would be happy to talk about LINQPad.QueryPlanVisualizer It is a custom visualizer for LINQPad that shows query plan directly in LINQPad. It supports LINQ-to-SQL, Entity Framework Core and can show query plan for SQL Server and PostgreSQL. For SQL Server it can extract missing indexes from the plan xml.
As the EF team is interested in exploring ideas around GraphQL, I think it would be interesting to invite authors of different GraphQL libraries such as GraphQL.EntityFramework, GraphQlClientGenerator, hotchocolate and my own GraphQLinq :blush:
@simon-reynolds could probably teach us all a thing or two about using EF in F#.
I would love to join the standup to talk about EFCore Projectables. It fits in with a bunch of libraries the likes of LinqKit, Expressionify and as hinted to me by @Giorgi DelegateDecompiler. All thrive for something similar which is to bridge the gap between expressing intend in C# and having that translated to an actual database query.
I can talk about the current limitations in EF and how these different libraries try to tackle these limitations. (As an example, with raw EF, we can't have a computed property on a model be directly translated to SQL or the likes as EF has no insights into its implementation). I can furthermore talk about how EF Core Projectables uniquely plugs in into the EF Core infrastructure to not only translate queries including computed properties and methods- but even increase EF Core's performance internally by plugging in at the right point in the pipeline.
Hi,
I'm just updating my EfCore.TestSupport library which provides useful helpers for xUnit-type testing, and I don't think you have had a EF Core Community standup on testing. I wondered if you would like a talk from me called "Testing application that use EF Core". I haven't worked out exactly what I would say but here are some suggestions form me, but you have a LOT more tests than I have :smile: !
- Talk about the different types of testing: Unit testing. Integration testing. System testing, Acceptance testing
- What you need to automate testing your EF Core code - xUnit, database or facade, and test code
- Three ways to simulate an EF Core database: InMemory (SQLite), actual database, facade
- Anatomy of a xUnit test: Setup, Attempt, Verify (or Arrange, Act, Assert)
- Things to watch out for - Model changes, getting empty database, disconnect states
- Examples of InMemory (SQLite), SQL Server, PostgreSQL, Cosmos
- Using logging to help you debug your tests
What you think?
@JonPSmith I just happen to be in the middle of doing some substantial work on our testing docs, so this is very much on my mind! I definitely agree we should do a session on this topic...
I am not sure if here is the right place to share ideas about EF 7 or not, but this is what I am thinking about.
It would fantastic if we can have something like a TAG for the Commands that we run against or DB by EF and then be able to commit or roll back whatever has the same TAG.
By this, we can change the DB from a different place and not with the same Command, but we would have something like a Transaction at the end.
@AliCharper this is a thread about ideas for the .NET Data community standup, not for general feature ideas. You can open a new issue on this repo with a detailed proposal.
Maybe a discussion about the dev cycle. This is not enough to fill an entire episode, so could probably be squeezed into the short "State of the Unicorn" at the beginning of an episode. Or wait until the EF 8 Plan video next year and be a part of that.
- What is the MQ milestone. I know this is not an EFCore specific thing, and that other devdiv teams also do MQ. But what does it mean for EFCore. Is it more of a label to apply to quality related issues? Or is more of a milestone, like version x.x?
- The type of the work that occurs during the which part of the cycle. For example, here is my naive assumption: Right after X.0 branches, then the most risky and breaking changes get merged to main. Since this is the farthest point from the next release, doing so would give these changes the most time to bake and stabilize. However, this does not seem to be the case? For example, with EF7.0, the first few previews had mostly bug fixes rather than big changes. So perhaps the MQ is what comes first after a major release?
How about talking about this https://github.com/dotnet/datalab/issues/6 ...if this is still a thing and not cancelled.
@DaveSenn it's definitely still a thing and not cancelled, but it might take a bit more time before there's something out there for us to talk about...
What about a talk about https://github.com/koenbeuk/EntityFrameworkCore.Projectables that was proposed in one of the comments above?
I would like to hear about more maybe basic DB stuff I had hard time searching on google.
Tips: Transactions + lock + deadlocks
- In our project we do not use EF core, but Dapper. We have internal rule to access tables in transactions always in the same order to prevent deadlocks. It helps in most cases but not always. Now EF uses transaction by default in every SaveChangesAsync. But with EF core I do not have to access my tables in the same order right ? So how does EF internally prevent deadlocks ? With EF core when would deadlocks possibly occur and is it possible to make sure they will never occur ? What about lock and isolation levels, are they even relevant in EF core ?
Indexing
- In my code I have queries that do filtering by a lot of arguments (10 and more). But users can choose to filter just by some of them as well. What can I do to make quering effective. Should I add single multi-column index for all 10 columns, just some of them or mixture ?
@Martinn2 thanks for these suggestions... Yeah, standups focusing on general database topics could be useful, I agree.
Very quick responses to your questions:
- When you call SaveChanges, EF internally always sorts its modification statements in a deterministic order, precisely in order to avoid deadlocks between concurrent updates (and if you're doing SQL updates yourself with Dapper you have to do this yourself as well). The order of the EF change-tracking calls in your code (Add, Remove...) is completely irrelevant to the ordering of the SQL statements that EF actually sends.
- Indexing is indeed tricky, and scenarios where users can choose to filter by "everything/anything" are hard to get right. A multi-column index isn't necessarily useful, since a composite index on columns A,B cannot be used if the user only filters on B. Such an index can be used when filtering on A, or on A and B only.
+1 for indexes. I ran into this earlier in the week and ended up creating an index that made my overall performance (when also considering updates) worse. I'm also not sure when to use a multi-column index, or when the direction of the index matters. I also still find it fascinating that indexes on discriminators are not something we create by default, but indexes on FKs are. Lots for me to learn here!
Oh, and how I create an index and then write a query that misses it!
Ideas:
- Custom migrations?
- Differences between DB providers
- Reverting migrations
- changing primary keys on live data (yes!) and what to expect from it -> yes. another segway to migrations
Too many customers complain about Ef taking too much memory. Can we have a segment on what to expect when we load a lot of data and how to fine tune the memory usage in those cases, best approaches, debugging tips etc.
Minimum entity mapping in DbContext
When you don't use the EF Core migrations features, what is the minimum entity mapping needed in DbContext when built-in default mappings are used? The background is when using Visual Studio's Database project for SQL-Server, you can handcraft the database in detail AND you get a very good database migration feature. The documentation is not very clear on this topic, so some clarification, maybe demonstrating an example in a stand-up session?
If you reverse engineer from a .dacpac only needed configuration is added, and conventions respected.