core icon indicating copy to clipboard operation
core copied to clipboard

Library Loader

Open mkolar opened this issue 6 years ago • 21 comments

We have and interesting request that we'll be looking into imminently.

A Studio has a large library of assets that are used regularly in all the projects they work on. We need to provide a way to open a loader with this library and populate the scene with subsets from there.

Proposed solution

We're thinking of adding another loader button to the avalon menu, that wouldn't be loading from the context of current project, but rather let user choose from a list of projects that are tagged as libraries and then present the assets from that projects.

Say we have projects CityAssets and NatureAssets, to publish into the library artist treats it as any other project.

When loading to coolProject. Artist clicks on avalon -> load from library -> chooses a library from the dropdown -> loads the subset he wants.

To prevent cross contamination of projects by loading assets from one to another, we'd only allow importing and during that process we'd strip all the references to the library project from the subset (ids for example). This would force the artist to re-publish it within the project he's working on making it fully contained.

mkolar avatar Jan 17 '19 11:01 mkolar

Great idea!

I have thought about something like this too, and if you go down this route I'd like to see this as abstract as possible. We were thinking about setting up reusable "light rigs" and possibly a "shader library" across different projects - so in a sense that would just be a "global library" that happens to have content that is also tagged for the project where they originated from. In that sense, the library isn't necessarily per project but is one universal with tags, etc.

However, describing that, I must admit it's hard to find out how well this "fits" into Avalon itself since it behaves so differently from the rest. Also, what you're describing at the end means it has its own type of loaders since they would behave differently? Again, sounds like quite a separation from what the Loader currently does. My first thought makes me think this is more of a standalone tool functionality (even in how far the database might be related), but I'd love to see it be part of what Avalon has to offer.

I think it's an interesting thing to look into and for now I can only raise my concerns. I want to stress I'm not trying to shoot this down but rather give some food for thought to get this started the right way.

BigRoy avatar Jan 17 '19 13:01 BigRoy

However, describing that, I must admit it's hard to find out how well this "fits" into Avalon itself since it behaves so differently from the rest

precisely. I'd say that for a full robust solution it's way more work than it seems, which is exactly why we'd like to do a simple usable version within avalon for now.

The loader should be absolutely identical to the one we already have, no changes at all. The only addition would be the option to choose which project in the db it should connect to. And that could be implemented as a popup with options before we open the loader, so wouldn't need to touch the loader itself. Limiting the loader option to only import should also be a question of simple filter in the existing loaders. and extra property in loader would do it

families = ["mayaAscii"]
representations = ["ma"]
library = True

The point is that I think we can get to a usable version of this with absolutely minimal effort and no disturbance to current workflows.

mkolar avatar Jan 17 '19 13:01 mkolar

Could "silo" be used for this purpose? E.g. a "global" or "library" silo.

mottosso avatar Jan 17 '19 18:01 mottosso

Possibly a "project" used as a global library; are we able to load assets from other shows at the moment?

Edit: Reading this a bit more carefully, it sounds like we're on the same page.

The only addition would be the option to choose which project in the db it should connect to

With something like that in place, that sounds like as robust a solution as can be. But it sounds like you had something else in mind for a "full" solution; what do you feel is missing with this in place?

mottosso avatar Jan 17 '19 18:01 mottosso

Functionally it would all be there. But eventually artists would like to see thumbnails. Shader assignments to selected objects rather than id matching as we have it now (using colorbleed look assigner). Publishing to library from any projects..etc

We haven't started putting together a comprehensive list of features yet, I expect we'll do that after we use the basic version for a while.

mkolar avatar Jan 18 '19 09:01 mkolar

I have few ideas in head: 1.) on action trigger in Avalon menu, a widget with projects list will pop up. The widget where user choose project from which he wants to import subsets, loader will be opened with project's context he choose 2.) add button 'Change Project' directly to loader that triggers the widget mentioned in 1.) 3.) add availibility of 'Public silo', this silo will appear in SiloTab for each project - issue is how to mark and store these assets, maybe as new Table like project's table but it won't store any 'type': 'project' inside so it won't appear in projects instead will store 'type': 'public' entity

iLLiCiTiT avatar Jan 18 '19 10:01 iLLiCiTiT

I'd go with option 1. no changes to loader required and very clear for user.

Library silo would introduce huge amount of noise. Imagine a studio has 4-5 different asset libraries. You add 5 tabs which is just annoying.

mkolar avatar Jan 18 '19 10:01 mkolar

Hi, after some discussion with Milan about the relevance of my proposal as his idea is something more strait forward I had decided to post it here anyway. Perhaps it is more complex and I appologise for more reading but I hope I had explained it understandably enough.

In this concept we are treating studio library just as a storage of already published subset/assets without workfiles. Only published subset inside particular project could be propagated into this target destination by Loader action. If any need for working on a library asset would be required then a project studio_library_wip would be created and tasks assigned. After the final look or functionality of subset version would be approved we could propagate it into studio_lib for sharing within the studio workspace.

  • Instead of having two different Loaders as Milan proposed, could be used only one with use of silo tab Studio Lib (lib)
  • Each project would have folder [silo] Library [{projects_root}/studio_lib/{project_name}_{asset}<_{subset}>/{representation}] (<> optional)
  • Avalon Loader would have available silo-tab Studio Lib in which all projects would share the content organized optionally by project|tag|assetType

Propagation asset to the studio library

  1. Some subsets would have available context action. Defined by Create plugin with attribute library such as Milan is proposing:
 families = ["mayaAscii"]
 representations = ["ma"]
 library = True
  1. Loader action Propagate to library will popup input dialogue with available tags and add new button. And bool as subset
  2. If the subset is already propagated into the lib it will popup confirmation dialogue asking for overriding
  3. All available representations would be coppied into studio_lib folder into human readable asset name {project_name}_{asset}<_{subset}>

Import lib asset into a workfile

  1. filter and select asset/subset in Studio Lib tab and available context action import [representation]
  2. Loader plugin will popup input dialogue for name of subset
  3. Instead of importing it as version container it will pass it to a create plugin and create publishable subset with correct family
  4. User will publish it within the context of asset/task he is working on

jakubjezek001 avatar Jan 18 '19 10:01 jakubjezek001

Structure wise have you guys considered just copying the existing loader as is, and continue the development as a Library tool?

Seem like you might be introducing unnecessary feature to the loader without knowing the full extent of the feature set.

tokejepsen avatar Jan 19 '19 03:01 tokejepsen

Structure wise have you guys considered just copying the existing loader as is, and continue the development as a Library tool?

good point. Besides it wouldn't make sense to add "propagate to library" in Loader anyway..

jakubjezek001 avatar Jan 21 '19 13:01 jakubjezek001

@tokejepsen That's exactly what I meant in the first place and I agree with that. Functionally I thought that ideally we wouldn't copy it just yet and only add another button that would load standard loader, but change session just before that. so that we load from a different project. "forking" the loader is of course an option, but I'm a bit worried that it will lead to double maintenance work when new features get added to either of the. Considering that their functionality is I would say 99% the same. I'd probably try to start with the question...

Can we force the launcher to open in a different project than the one in the current session. From our first investigation it doesn't seem particularly trivial.

mkolar avatar Jan 21 '19 13:01 mkolar

Can we force the launcher to open in a different project than the one in the current session. From our first investigation it doesn't seem particularly trivial.

The Launcher or the Loader?

  1. The launcher can already switch between projects, right?

  2. I believe the loader will always try to rely on the active api.Session and will never act on a copy of it. Similarly I believe the avalon.io will always currently operate on a single live project - to reconnect you will need to uninstall and install I believe?

BigRoy avatar Jan 21 '19 16:01 BigRoy

Ah sorry my bad I mean Loader of course. And yes that is the key problem currently we think. If we fudge the session to make loader think it's in different project, it won't fly smoothly I'm afraid

mkolar avatar Jan 21 '19 16:01 mkolar

@iLLiCiTiT made a prototype of how this could work and it seem very promising. We'll have to see if it breaks something and we'll create a PR for discussion over the code.

jan 23 2019 11_43 pm

mkolar avatar Jan 23 '19 22:01 mkolar

Has anyone idea why GUI raises

TypeError: 'AssetWidget' is not callable

in Maya and Nuke?

iLLiCiTiT avatar Jan 24 '19 20:01 iLLiCiTiT

@iLLiCiTiT Are you still seeing that error? Could you include a few more lines of the traceback? It looks like a widget is being called after having been instantiated.

button = QtWidgets.QPushButton("MyButton")
button()
# TypeError

mottosso avatar Feb 22 '19 22:02 mottosso

It was because I used attribute "parent" to store parent widget but one widget had also method called "parent" so it crashed when this method was called. Already fixed :). I think it's actually ready to show @mkolar ?

iLLiCiTiT avatar Feb 23 '19 10:02 iLLiCiTiT

Ah, yes. All widgets actually have .parent() so better not use that as an instance variable.

  • https://doc.qt.io/qt-5/qobject.html#parent

mottosso avatar Feb 23 '19 10:02 mottosso

I think it's actually ready to show

certainly. it's in production with a few clients now, so we can make PR and discuss there.

mkolar avatar Feb 28 '19 00:02 mkolar

How is this one coming along in production @mkolar ? :) Smooth? Any issues? Can we try and get a PR set up?

BigRoy avatar Dec 01 '19 19:12 BigRoy

It's been running smoothly for ages actually. We were slowly working on getting on par with avalon-core because of our custom code. Let me check with @iLLiCiTiT how close we are with PRs for these tools

mkolar avatar Dec 03 '19 16:12 mkolar