Ejabberd mod_http_fileserver and mod_http_upload support for AWS RDS and S3

[armor009]

trafficstars

Ejabberd mod_http_fileserver and mod_http_upload support for AWS RDS and S3 will greatly enhance usability of ejabberd.

[badlop]

They don't work? What are the error messages?

[armor009]

They host data on server. I am planning to use AWS EC2 for running the server. The cost of EBS is high. I am considering using RDS and AWS S3. Can there be support for sending files across http using RDS (Postgres) and S3 as DB.

[armor009]

@badlop hoping to hear back from you.

[Neustradamus]

@armor009: There is now a PR:

• https://github.com/processone/ejabberd/pull/3893

cc: @RomanHargrave, @ArtemisPseudo, @licaon-kter.

[armor009]

Thank you

[RomanHargrave]

Hey! Wow, I've been sucked into work a lot lately.

I am amenable to adding this to contrib, but I still feel strongly about getting into ejabberd directly. I'll be attending to holiday festivities this coming week, but would be interested in seeing what needs to be done to get this into contrib if it is desired.

[badlop]

I was curious, and did it following https://docs.ejabberd.im/developer/extending-ejabberd/modules/#add-your-module

It just required placing your files in new subdirectories and adding mod_s3_upload.spec. For completeness, I also added README, COPYING and an empty disabled config file. It's here: https://github.com/badlop/ejabberd-contrib/commits/s3_upload_draft

How to try it ?

1. Apply this simple patch to ejabberd source code, to use this branch from my fork:

diff --git a/src/ext_mod.erl b/src/ext_mod.erl
index 1d8cb685e..b0adc7908 100644
--- a/src/ext_mod.erl
+++ b/src/ext_mod.erl
@@ -48,7 +48,7 @@
 -include("translate.hrl").
 -include_lib("xmpp/include/xmpp.hrl").
 
--define(REPOS, "https://github.com/processone/ejabberd-contrib").
+-define(REPOS, "https://github.com/badlop/ejabberd-contrib").
 
 -record(state, {}).
 
@@ -387,10 +387,10 @@ extract_github_master(Repos, DestDir) ->
         [$t,$i,$g,$.|T2] -> lists:reverse(T2);
         _ -> Base
     end,
-    case extract(zip, geturl(Url++"/archive/master.zip"), DestDir) of
+    case extract(zip, geturl(Url++"/archive/s3_upload_draft.zip"), DestDir) of
         ok ->
             RepDir = filename:join(DestDir, module_name(Repos)),
-            file:rename(RepDir++"-master", RepDir),
+            file:rename(RepDir++"-s3_upload_draft", RepDir),
             maybe_write_commit_json(Url, RepDir);
         Error ->
             Error
@@ -746,7 +746,7 @@ maybe_write_commit_json(Url, RepDir) ->
 
 write_commit_json(Url, RepDir) ->
     Url2 = string_replace(Url, "https://github.com", "https://api.github.com/repos"),
-    BranchUrl = lists:flatten(Url2 ++ "/branches/master"),
+    BranchUrl = lists:flatten(Url2 ++ "/branches/s3_upload_draft"),
     {ok, _Headers, Body} = geturl(BranchUrl),
     {ok, F} = file:open(filename:join(RepDir, "COMMIT.json"), [raw, write]),
     file:write(F, Body),
@@ -877,7 +877,7 @@ get_page(Node, Query, Lang) ->
 
 get_module_home(Module, Attrs) ->
     case element(2, lists:keyfind(home, 1, Attrs)) of
-        "https://github.com/processone/ejabberd-contrib/tree/master/" = P1 ->
+        "https://github.com/processone/ejabberd-contrib/tree/s3_upload_draft/" = P1 ->
             P1 ++ atom_to_list(Module);
         Other ->
             Other

2. Compile and install ejabberd

3. Get the updated ejabberd-contrib source code

ejabberdctl modules_update_specs

4. Compile and install the module

ejabberdctl module_install mod_s3_upload

5. Now it's time to configure it in ejabberd.yml or in the provided mod_s3_upload.yml

6. Finally, restart ejabberd or simply reload the configuration

ejabberdctl reload_config

Right now the tests are not used in the ejabberd-contrib action; I'll check that later.

I was able to compile and install the module. But I didn't test it works correctly.

Once you are able to test it, if it works correctly, you can use the content from this draft to prepare your PR.

[RomanHargrave]

@Neustradamus this is now in contrib

[Neustradamus]

@RomanHargrave: Good job, thanks, your PR has been merged!

Linked to:

• https://github.com/processone/ejabberd-contrib/issues/305
• https://github.com/processone/ejabberd/pull/3893
• https://github.com/processone/ejabberd/issues/3625
• https://github.com/processone/ejabberd/issues/3624

[Neustradamus]

@RomanHargrave: Your PR in ejabberd has not been closed, it is time, it is now in ejabberd-contrib since 2 months:

• https://github.com/processone/ejabberd/pull/3893

@badlop: Several tickets too?

[Cronoburn]

I couldn't get this to work.. has something changed, or more likely I am doing it wrong? I'm not getting any errors, nor are the uploads showing up in my bucket. I've since tried using a python script to grab the upload from mod_http_upload, but that's throwing all sorts of errors.

[licaon-kter]

@Cronoburn better open a new issue with exact OS/ejabberd/config/etc.