wekan
Feature Request: Provide different ways to upload and store attachments, and drag attachment to card to upload it
Large databases are not easy to handle and to backup. It would be useful to store attachments as file in the local filesystem (or maybe on a Cloud Storage provider like Google Cloud Storage or Amazon S3?).
Local filesystem would definitely be nice. Not sure someone who wants a self-hosted board is going to really want to store data on Google or Amazon clouds though, as that largely defeats the point.
Users may or may not find value in self hosted attachments. I'd prefer if i could like to my google drive account. Any ideas of how we might achieve non-gridfs storage?
supporting something like owncloud or next cloud would be great, also supporting access to davros on sandstorm would also be pretty cool! I doubt that using google, or AS would help much, this defeats the point point of own hosting !
Storing attachments in a different manner would certainly help organisations using Wekan self-hosted. The last thing they want is to have data stored in many places. In my mind AWS and GDrive would be the most applicable places to store and they're both supported by the "files" package
Moved to here from #2274
From @rossmansfield
Link Card to Object Storage Bucket or SMB Directory
Hello all, first time posting here, so be gentle!
In lieu of attaching files into the actual WEKAN cards, it would be fantastic if we could somehow place a thumbnail image of an object in a network location or object-storage bucket. Even better would be if the Card could show the contents of a Bucket or file directory and allow the user to open said files from the Card. This is very similar to how Trello can embed Google Drive.
I've been experimenting with (SMB "Samba", OpenstackSwift, S3, etc. This would really help with team collaboration. A custom field could work to send users to the Bucket via an HTML link, but it doesn't allow for links to SMB type directories.
By having integration with a Swift or S3 type object storage system, redundancy can be handled at the infrastructure level vice the container level.
I have some experience with Openstack and Minio and could support on that end, I'm just not very strong with code development (yet).
Any thoughts on this?
@xet7 it could be highly useful to use https://uppy.io
Very nice documentation, great features, and supports multiple backends. Can be integrated with Meteor server through the WebApp connect.
@justinr1234
How it's different from https://atmospherejs.com/malibun23/files ?
@justinr1234
I don't find at uppy website any support for GridFS, where currently all Wekan attachments are saved. So it seems I can't use uppy. malibun23/files seems to have GridFS.
@xet7 I thought the idea was to move away from GridFS? Either way, if the other works then no need to use Uppy
@justinr1234
There needs to be ability to read existing Wekan attachments from GridFS to be able to move them to other storage. If I remember correctly, somewhere at malibun23/files docs is some storage migration guide.
I will add S3, Google Drive support etc. I'll add this to my todo list.
@xet7 this hasn't been completed yet, has it? is there any reason, other than a lack of time/priority?
What would be a fair bounty to prioritize this for 1Q 2020?
Ross Mansfield, PE
O: (281) 888-0808
M: (713) 510-5000
KWSH
From: azban [mailto:[email protected]] Sent: Thursday, November 14, 2019 7:41 PM To: wekan/wekan Cc: rossmansfield; Mention Subject: Re: [wekan/wekan] Feature Request: Provide different ways to upload and store attachments, and drag attachment to card to upload it (#142)
I will add S3, Google Drive support etc. I'll add this to my todo list. @xet7 https://github.com/xet7 this hasn't been completed yet, has it? is there any reason, other than a lack of time/priority?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wekan/wekan/issues/142?email_source=notifications&email_token=AHMVLCNRVGHFWE6QHCTQKXDQTX43JA5CNFSM4A4UYZW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEED7QCI#issuecomment-554170377 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHMVLCI5LSP5MMR6OHT6P23QTX43JANCNFSM4A4UYZWQ . https://github.com/notifications/beacon/AHMVLCNIE2L543JHJI6ZMOTQTX43JA5CNFSM4A4UYZW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEED7QCI.gif
My 1Q 2020 is full.
Does some other Wekan contributor have time to do this at 1Q ?
@ocdtrekkie @ahashibon please, note, s3 does not "defeat the point" because it's a protocol and there's https://github.com/minio/minio self-hosted alternative with the same api (@rossmansfield mentioned it as well)
@xet7 @blaggacao did you have time to get back on this btw? could you maybe add hacktoberfest topic to the repo?
p.s oh, it seems like tech discussion continues here https://github.com/wekan/wekan/issues/3272
@rossmansfield feel free to plegde anything https://www.bountysource.com/ or in crypto https://gitcoin.co/ do you still need this?
The current implementation on #3273 is just a special implementation of a storage backend towards GridFS MongoDB Buckets.
With the same strategy, those can be stored anywhere given a connector is provided. See for example here. I'd assume, storing to file system is as simple as not deleting the uploaded files and not storing them into a specific backend.
@blaggacao thanks for such a quick reply, btw, what do you think about another idea: given there's already a self-hosted minio and some images inside - can we just render direct img src= to the direct location, i.e. not uploading twice, but specifying the link?
Yes the download location of ostrio:files is configurable. Upload goes through meteor, though. With upload comes the proper initialization of the image reference in the mongo database. So
given there's already a self-hosted minio and some images inside
would be rather awkward to support. Though nothing speaks against using a self hosted minion and populate it through wekan uploads.
already a self-hosted minio and some images inside
would be rather awkward to support
oh, sorry, didn't realised input can contain just html https://github.com/wekan/wekan/issues/2635#issuecomment-694781543
Here is where a hook intercepts the upload and stores it into the back-end (and deletes the uploaded files):
https://github.com/wekan/wekan/pull/3273/files#diff-4a858d7364de26a72d82ca64882d740c4ecf2f5efe5eab5bd481a3a909d49518R26
From @razum2um
feel free to plegde anything https://www.bountysource.com/ or in crypto https://gitcoin.co/ do you still need this?
For Wekan, current payment options are at https://wekan.team/commercial-support/ . Those BountySource and Gitcoin or anything else are not in use at Wekan, it was already a lot of work to move away from BountySource #3177 .
@mfilser @NotTheEvilOne @blaggacao
Any ideas how to fix my first try of AWS S3 code https://github.com/wekan/wekan/commit/21e2eabd607cc7fccbce8ed8562d886ab54fee68 so that after upload and optionally checking file with ClamAV etc, it would upload file to S3 ? And show attachments at cards, be able to move files to S3, etc? Beginnings of those changes are at feature-s3 branch https://github.com/wekan/wekan/tree/feature-s3
Related https://github.com/veliovgroup/Meteor-Files/blob/master/docs/aws-s3-integration.md
Hi,
Fortunately from the validation part no changes are required at all. Validation and storage movement are separated steps by now and can easily work with your changes. I'll try to find some time to look more detailed into the S3 branch over the holidays.
@NotTheEvilOne
I fixed some typos and added some more code so that WeKan starts properly with those S3 related changes, but I have not yet tested does it work. Have you made any changes?
https://github.com/wekan/wekan/commit/028633b00ab25bcd5c7ce6f78368aa6e33102a0c
@mfilser @NotTheEvilOne
I listed related S3 commits at https://github.com/wekan/wekan/blob/master/CHANGELOG.md#upcoming-wekan--release
I merged feature-s3 branch to master, and deleted feature-s3 branch, because it seems WeKan starts anyway with these changes.
Currently I'm getting this error, when trying to click button to move attachment to S3. Trying to figure it out.
I20221226-06:37:48.337(2)? Exception while invoking method 'moveAttachmentToStorage' TypeError: this.classFileStoreStrategyS3 is not a constructor
I20221226-06:37:48.338(2)? at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:56:15)
I20221226-06:37:48.338(2)? at models/lib/fileStoreStrategy.js:458:52
I20221226-06:37:48.338(2)? at Array.forEach (<anonymous>)
I20221226-06:37:48.338(2)? at moveToStorage (models/lib/fileStoreStrategy.js:456:33)
I20221226-06:37:48.338(2)? at MethodInvocation.moveAttachmentToStorage (models/attachments.js:147:7)
I20221226-06:37:48.339(2)? at packages/check/match.js:118:15
I20221226-06:37:48.339(2)? at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1329:12)
I20221226-06:37:48.339(2)? at Object._failIfArgumentsAreNotAllChecked (packages/check/match.js:116:43)
I20221226-06:37:48.339(2)? at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1899:18)
I20221226-06:37:48.339(2)? at getCurrentMethodInvocationResult (packages/ddp-server/livedata_server.js:772:38)
I20221226-06:37:48.339(2)? at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1329:12)
I20221226-06:37:48.339(2)? at packages/ddp-server/livedata_server.js:791:46
I20221226-06:37:48.339(2)? at new Promise (<anonymous>)
I20221226-06:37:48.340(2)? at Session.method (packages/ddp-server/livedata_server.js:739:23)
I20221226-06:37:48.340(2)? at packages/ddp-server/livedata_server.js:603:43