post_publish_recording_ready_callback failed

[cod3r0k]

I configure my BBB server. When I rebuild my video to have a mp4 format of each session, an error was occured.

I can't find any log file which focus to my problem. I just have this comment in bbb-record --watch command.

bbb-rap[99962]: Success?: false
bbb-rap[99962]: Process exited? true
bbb-rap[99962]: Exit status: 1
bbb-rap[99962]: Post publish script /usr/local/bigbluebutton/core/scripts/post_publish/post_publish_recording_ready_callback.rb/post_publish_recording_ready_callback failed
bbb-rap[99962]: Task: Getting external meeting id
bbb-rap[99962]: Task: Getting meeting metadata

[cod3r0k]

Dear @tilmanmoser, Do you have any idea about this error?

I can run manually successfully with the following command. But It never run automatically in post publish process

cd /opt/bbb-video-download
sudo -u bigbluebutton docker-compose run --rm --user 998:998 app node index.js -i /var/bigbluebutton/published/presentation/9a9b6536a10b10017f7e849d30a026809852d01f-1597816023148 -o /var/bigbluebutton/published/presentation/9a9b6536a10b10017f7e849d30a026809852d01f-1597816023148/video.mp4

[tilmanmoser]

Did you install the post publish hook as described in the readme?

# (as root or with sudo)
cd /opt/bbb-video-download
export BBB_VIDEO_DOWNLOAD_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
export BBB_UID="$(cat /etc/passwd | grep bigbluebutton | cut -d: -f3)"
export BBB_GID="$(cat /etc/passwd | grep bigbluebutton | cut -d: -f4)"

## bbb v2.2:
envsubst < ./snippets/post_publish_bbb_video_download.rb.template > /usr/local/bigbluebutton/core/scripts/post_publish/a0_post_publish_bbb_video_download.rb
## bbb v2.3:
envsubst < ./snippets/bbb23_post_publish_bbb_video_download.rb.template > /usr/local/bigbluebutton/core/scripts/post_publish/a0_post_publish_bbb_video_download.rb

[tilmanmoser]

Btw: The recording ready callback is not from bbb-video-download. So, I can't comment on that.

[cod3r0k]

Did you install the post publish hook as described in the readme?

# (as root or with sudo)
cd /opt/bbb-video-download
export BBB_VIDEO_DOWNLOAD_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
export BBB_UID="$(cat /etc/passwd | grep bigbluebutton | cut -d: -f3)"
export BBB_GID="$(cat /etc/passwd | grep bigbluebutton | cut -d: -f4)"

## bbb v2.2:
envsubst < ./snippets/post_publish_bbb_video_download.rb.template > /usr/local/bigbluebutton/core/scripts/post_publish/a0_post_publish_bbb_video_download.rb
## bbb v2.3:
envsubst < ./snippets/bbb23_post_publish_bbb_video_download.rb.template > /usr/local/bigbluebutton/core/scripts/post_publish/a0_post_publish_bbb_video_download.rb

Dear @tilmanmoser , Of course, yes. But, I don't know how to debug this issue.

It is working when I run it manually (in BBB2.3-beta5) but it has that problem when It run automatically with post_publish process and I never find any logs from that.

[tilmanmoser]

Just to clarify on that: You did install the v2.3 post publish hook which sets the "-f" flag that is now required to run the script?

If so, you can find the logs at /var/log/bigbluebutton/post_publish.log

If this is empty|not existent, the post publish hook was not executed.

[cod3r0k]

Just to clarify on that: You did install the v2.3 post publish hook which sets the "-f" flag that is now required to run the script?

If so, you can find the logs at /var/log/bigbluebutton/post_publish.log

If this is empty|not existent, the post publish hook was not executed.

OK, I will check it again and send you a feedback. Thanks

[cod3r0k]

Dear @tilmanmoser , I install another 2.3.0 BBB. I clearly installed bbb-video-download. It runs properly in manual mode. But it isn't run automatically. BTW, I show the log of post_publish.log in the following.

It seems that it runs clearly but it never produce the video mp4 format.

I, [2021-04-30T16:38:24.527409 #14614]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800461061] end
I, [2021-04-30T16:39:20.882593 #15842]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:39:20.883986 #15842]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800709382] start
I, [2021-04-30T16:39:21.563145 #15842]  INFO -- :
I, [2021-04-30T16:39:21.563278 #15842]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800709382] end
I, [2021-04-30T16:39:21.870061 #15855]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:39:21.871488 #15855]  INFO -- : Recording Ready Notify for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800709382] starts
I, [2021-04-30T16:39:21.871565 #15855]  INFO -- : Recording Ready notify ends
I, [2021-04-30T16:51:39.984067 #21298]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:51:39.985439 #21298]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786] start
I, [2021-04-30T16:51:40.658129 #21298]  INFO -- :
I, [2021-04-30T16:51:40.658276 #21298]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786] end
I, [2021-04-30T16:51:40.964001 #21312]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:51:40.965246 #21312]  INFO -- : Recording Ready Notify for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786] starts
I, [2021-04-30T16:51:40.965307 #21312]  INFO -- : Recording Ready notify ends
I, [2021-04-30T16:54:08.418691 #22728]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:54:08.420331 #22728]  INFO -- : Create downloadable video for [c44c3927fc3ed9c15caa4af9b0e31e5127d4ce30-1619799774108] start
I, [2021-04-30T16:54:09.113472 #22728]  INFO -- :
I, [2021-04-30T16:54:09.113596 #22728]  INFO -- : Create downloadable video for [c44c3927fc3ed9c15caa4af9b0e31e5127d4ce30-1619799774108] end
I, [2021-04-30T16:54:09.427451 #22741]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:54:09.429132 #22741]  INFO -- : Recording Ready Notify for [c44c3927fc3ed9c15caa4af9b0e31e5127d4ce30-1619799774108] starts
I, [2021-04-30T16:54:09.429188 #22741]  INFO -- : Recording Ready notify ends
I, [2021-04-30T16:54:30.094871 #23065]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:54:30.099299 #23065]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800461061] start
I, [2021-04-30T16:54:30.802365 #23065]  INFO -- :
I, [2021-04-30T16:54:30.802560 #23065]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800461061] end
I, [2021-04-30T16:54:31.130691 #23078]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:54:31.135383 #23078]  INFO -- : Recording Ready Notify for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800461061] starts
I, [2021-04-30T16:54:31.135446 #23078]  INFO -- : Recording Ready notify ends
I, [2021-04-30T16:54:52.127053 #23420]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:54:52.128418 #23420]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800709382] start
I, [2021-04-30T16:54:52.791728 #23420]  INFO -- :
I, [2021-04-30T16:54:52.791883 #23420]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800709382] end
I, [2021-04-30T16:54:53.107497 #23433]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:54:53.108927 #23433]  INFO -- : Recording Ready Notify for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800709382] starts
I, [2021-04-30T16:54:53.108982 #23433]  INFO -- : Recording Ready notify ends
I, [2021-04-30T16:55:49.443393 #24124]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:55:49.444685 #24124]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786] start
I, [2021-04-30T16:55:50.108550 #24124]  INFO -- :
I, [2021-04-30T16:55:50.108692 #24124]  INFO -- : Create downloadable video for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786] end
I, [2021-04-30T16:55:50.416142 #24137]  INFO -- : Task: Getting meeting metadata
I, [2021-04-30T16:55:50.417467 #24137]  INFO -- : Recording Ready Notify for [d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786] starts

[tilmanmoser]

Have a look at /var/bigbluebutton/published/presentation/d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786 on your server - since the log doesn't show any error, it should have a video.mp4 file inside.

[cod3r0k]

Have a look at /var/bigbluebutton/published/presentation/d1abf9c6b6edc0cd4d0daa15f64ecb7cbaab48f3-1619800960786 on your server - since the log doesn't show any error, it should have a video.mp4 file inside.

Thanks @tilmanmoser , Unfortunately I can't see it.

[tilmanmoser]

can you show me your post publish hook you'd installed?

[tilmanmoser]

I just double checked - the script is not executed by your post publish hook. It would have a START and END message with the time it needed to process the video. So the post publish hook is not executing correctly.

[cod3r0k]

I put both of them @tilmanmoser , Is eveything OK?

post_publish_recording_ready_callback.rb

require "trollop"
require 'net/http'
require "jwt"
require "java_properties"
require File.expand_path('../../../lib/recordandplayback', __FILE__)

logger = Logger.new("/var/log/bigbluebutton/post_publish.log", 'weekly' )
logger.level = Logger::INFO
BigBlueButton.logger = logger

opts = Trollop::options do
  opt :meeting_id, "Meeting id to archive", :type => String
  opt :format, "Playback format name", :type => String
end
meeting_id = opts[:meeting_id]

processed_files = "/var/bigbluebutton/recording/process/presentation/#{meeting_id}"
meeting_metadata = BigBlueButton::Events.get_meeting_metadata("/var/bigbluebutton/recording/raw/#{meeting_id}/events.xml")

#
# Main code
#
BigBlueButton.logger.info("Recording Ready Notify for [#{meeting_id}] starts")

begin
  callback_url = meeting_metadata.key?("bbb-recording-ready-url") ? meeting_metadata["bbb-recording-ready-url"].value : nil
  # For compatibility with some 3rd party implementations, look up for bn-recording-ready-url or canvas-recording-ready, when bbb-recording-ready is not included.
  callback_url ||= meeting_metadata.key?("bn-recording-ready-url") ? meeting_metadata["bn-recording-ready-url"].value : nil
  callback_url ||= meeting_metadata.key?("canvas-recording-ready-url") ? meeting_metadata["canvas-recording-ready-url"].value : nil


  unless callback_url.nil?
    BigBlueButton.logger.info("Making callback for recording ready notification")

    props = JavaProperties::Properties.new("/usr/share/bbb-web/WEB-INF/classes/bigbluebutton.properties")
    secret = props[:securitySalt]
    external_meeting_id = meeting_metadata["meetingId"].value

    payload = { meeting_id: external_meeting_id, record_id: meeting_id }
    payload_encoded = JWT.encode(payload, secret)

    uri = URI.parse(callback_url)
    http = Net::HTTP.new(uri.host, uri.port)
    http.use_ssl = (uri.scheme == 'https')

    BigBlueButton.logger.info("Sending request to #{uri.scheme}://#{uri.host}#{uri.request_uri}")
    request = Net::HTTP::Post.new(uri.request_uri)
    request.set_form_data({ signed_parameters: payload_encoded })

    response = http.request(request)
    code = response.code.to_i

    if code == 410
      BigBlueButton.logger.info("Notified for deleted meeting: #{meeting_id}")
      # TODO: should we automatically delete the recording here?
    elsif code == 404
      BigBlueButton.logger.info("404 error when notifying for recording: #{meeting_id}, ignoring")
    elsif code < 200 || code >= 300
      BigBlueButton.logger.info("Callback HTTP request failed: #{response.code} #{response.message} (code #{code})")
    else
      BigBlueButton.logger.info("Recording notifier successful: #{meeting_id} (code #{code})")
    end
  end

rescue => e
  BigBlueButton.logger.info("Rescued")
  BigBlueButton.logger.info(e.to_s)
end

BigBlueButton.logger.info("Recording Ready notify ends")

exit 0

============================================ a0_post_publish_bbb_video_download.rb

require "optparse"
require File.expand_path('../../../lib/recordandplayback', __FILE__)

meeting_id = nil
OptionParser.new do |opts|
  opts.on('-m', '--meeting-id MEETING_ID', 'Internal Meeting ID') do |v|
    meeting_id = v
  end
  opts.on('-f', '--format FORMAT', 'Recording Format') do |v|
  end
end.parse!

logger = Logger.new("/var/log/bigbluebutton/post_publish.log", 'weekly' )
logger.level = Logger::INFO
BigBlueButton.logger = logger

published_files = "/var/bigbluebutton/published/presentation/#{meeting_id}"
meeting_metadata = BigBlueButton::Events.get_meeting_metadata("/var/bigbluebutton/recording/raw/#{meeting_id}/events.xml")

#
### Main Code
#

require 'shellwords'
input_dir = Shellwords.escape(published_files)
output_file = Shellwords.escape("#{published_files}/video.mp4")
base_dir = '/opt/bbb-video-download'

BigBlueButton.logger.info("Create downloadable video for [#{meeting_id}] start")
rs = `cd #{base_dir} && docker-compose run --rm --user 997:996 app node index.js -i #{input_dir} -o #{output_file}`
BigBlueButton.logger.info(rs)
BigBlueButton.logger.info("Create downloadable video for [#{meeting_id}] end")

exit 0

[tilmanmoser]

Is this correct that bigbluebutton has uid 997 and did 996? What happens when you execute this line directly? cd #{base_dir} && docker-compose run --rm --user 997:996 app node index.js -i #{input_dir} -o #{output_file} Please replace the variables with a proper presentation directory and outfile

[cod3r0k]

Thanks for your attention @tilmanmoser.

I installed BBB 2.3.0 today from the bbb_install.sh shell script correctly.

My output of $(cat /etc/passwd | grep bigbluebutton | cut -d: -f3) and $(cat /etc/passwd | grep bigbluebutton | cut -d: -f4) are 997 and 996 respectively.

When I run the below code, the video is properly created in the meeting id published directory at /var/bigbluebutton/published/presentation/meetingid/meetingid.mp4

sudo -u bigbluebutton docker-compose run --rm --user "$(cat /etc/passwd | grep bigbluebutton | cut -d: -f3)":"$(cat /etc/passwd | grep bigbluebutton | cut -d: -f4)" app node index.js -i /var/bigbluebutton/published/presentation/c44c3927fc3ed9c15caa4af9b0e31e5127d4ce30-1619799774108 -o /var/bigbluebutton/published/presentation/c44c3927fc3ed9c15caa4af9b0e31e5127d4ce30-1619799774108/c44c3927fc3ed9c15caa4af9b0e31e5127d4ce30-1619799774108.mp4

[tilmanmoser]

Strange. Unfortunately I have currently no bbb2.3 running; the hook for v2.3 was provided by users of the script. I might upgrade soon, but will definitely not do so this weekend. I'm very sorry, I can't help here. But since the script is working when called manually, it's definitely the way the hook is executed.

[cod3r0k]

Strange. Unfortunately I have currently no bbb2.3 running; the hook for v2.3 was provided by users of the script. I might upgrade soon, but will definitely not do so this weekend. I'm very sorry, I can't help here. But since the script is working when called manually, it's definitely the way the hook is executed.

I think that, If I change your script code, I might get the result. I will send you a feed back ASAP.

[tilmanmoser]

I'd appreciate it. If you solved it, feel free to make a merge request.

[cod3r0k]

I'd appreciate it. If you solved it, feel free to make a merge request.

Dear @tilmanmoser , I couldn't success in this task yet!

[cod3r0k]

Dear @tilmanmoser is there any update?

[edukimod]

Hi, I ran into same issue with BBB 2.3. It works fine when running manually but fails when in the post_publish hook. I've found this error in the syslog: Jul 6 11:21:34 bbb bbb-rap[13632]: Executing: ruby /usr/local/bigbluebutton/core/scripts/post_publish/a0_post_publish_bbb_video_download.rb -m <meeting_id> -f presentation Jul 6 11:21:34 bbb bbb-rap[13632]: Couldn't connect to Docker daemon at http+docker://localhost - is it running? Jul 6 11:21:34 bbb bbb-rap[13632]: Jul 6 11:21:34 bbb bbb-rap[13632]: If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable. Jul 6 11:21:34 bbb bbb-rap[13632]: Success?: true I've been looking around and that seems to point to a permission issue, but I've added the bigbluebutton user to the docker group and even it runs ok with sudo -u bigbluebutton ruby /usr/local/bigbluebutton/core/scripts/post_publish/a0_post_publish_bbb_video_download.rb -m <meeting_id> -f presentation

Any help would be really appreciated!

[edukimod]

Finally managed to solve this issue, just restarted the bigbluebutton server (although probably restarting only the service would have been enough) and everything works fine now!