HandBrake icon indicating copy to clipboard operation
HandBrake copied to clipboard

Common Metadata Passhtru (Date Handling)

Open sr55 opened this issue 4 years ago • 15 comments

Since we now have exposed the common metadata passthru we can potentially extend the behaviour for the date field to handle a common request to copy the source filesystem dates over.

New global preferences under "Output Files" section:

"When passthru common metadata is enabled, set the date field in the container metadata to:" [DROPDOWN]

  • Source Container Date
  • Source File System Creation Date
  • Source File System Modification Date
  • Destination (Todays Date)
  • None

[X] Only when source container metadata field is empty. (Uses source container date)

For clarity: This change will have no impact on the system metadata creation/modification times of the final file. Only the containers metadata is set.

sr55 avatar May 12 '21 19:05 sr55

We could promote the first part of the label to be a subheading for the group. Per our discussion on IRC, I've added title metadata and agree we should limit this feature to title & date, so those can be prefilled using filesystem data in the absence of source container metadata.

When passthru common metadata is enabled...

Set title metadata to:

    [  special field entry based on filename  ]

    [×] Only when Source date metadata is blank or missing

Set date metadata to:

  • Source File Creation Date    ← default
  • Source File Modification Date
  • Destination File Creation Date (Encode Date)
  • None

    [×] Only when Source date metadata is blank or missing

bradleysepos avatar May 12 '21 20:05 bradleysepos

Anyone coming here from #51, please note we are still against setting the destination's filesystem creation/modification dates to the source's. Rather, we are considering providing the option to promote the source's filesystem info to be the destination's container metadata. This way, it is possible to preserve this information for sources with correct filesystem dates but lacking proper metadata (an unfortunately common scenario), without relying on or abusing the filesystem info going forward.

Of course, there is the possibility that source's filesystem dates are incorrect (e.g., set to the epoch or copy date after copying across filesystems or downloading on the internet), which the proposed options cannot correct in the absence of proper date metadata. This only serves to illustrate why utilizing filesystem information for such information is unreliable and discouraged.

bradleysepos avatar May 12 '21 20:05 bradleysepos

For everyone looking for a solution to convert videos and preserve the original file's creation date I can recommend ffWorks It works like a charm and now I like it even more than Handbrake!

barty82pl avatar Jun 12 '21 19:06 barty82pl

Any updates here? Waiting for the feature. Thanks

MudassarAli avatar Aug 16 '21 21:08 MudassarAli

No updates. It's not being worked on at present.

Ultimately it's a hobby project for a very small number of people, with very little time. Ergo, patience is required. If anyone is interested in building this functionality, they can get in touch.

sr55 avatar Aug 16 '21 21:08 sr55

Re-targeting for 1.6.0.

bradleysepos avatar Jan 06 '22 03:01 bradleysepos

For anyone interested in this functionality, i made a bash script that can pass through the date using ffmpeg. it works on windows through WSL. ill drop it here in case it is useful for anyone familiar with the handbrake codebase

https://gist.github.com/domportera/d23b02695874c207dede0e7c5d8ecfe3

domportera avatar Nov 04 '22 17:11 domportera

just a suggestion, how about using/integrating FFmpeg library?

how about we modify the HandBrake transcoding process to first use FFmpeg to copy the metadata from the input file to a temporary file, and then use HandBrake to transcode the video and audio streams from the temporary file to the final output file.

Lastly, we can again use the FFmpeg library to copy the metadata from the temporary file to the final output file.

this could be simplified by telling people that they would need to download and install FFmpeg and then just put the program path of FFmpeg in Handbrake.

the below code is made with Python and is only for the common metadata

import subprocess

input_file = 'input.mp4'
temp_file = 'temp.mp4'
output_file = 'output.mp4'

# Use FFmpeg to copy metadata from input file to temp file
subprocess.run(['ffmpeg', '-i', input_file, '-c', 'copy', '-map_metadata', '0', temp_file])

# Use FFmpeg to copy metadata from temp file to output file
subprocess.run(['ffmpeg', '-i', temp_file, '-c', 'copy', '-map_metadata', '0', output_file])

For the Extended attributes, which i believe is what we need for promoting the source's filesystem info to be the destination's container metadata, would differ from OS to OS.

I think the below would work for MacOS:

import subprocess

source_file = "/path/to/source.mp4"
destination_file = "/path/to/destination.mp4"

# Use FFmpeg to copy metadata from source to destination
subprocess.call(["ffmpeg", "-i", source_file, "-map_metadata", "0", "-c", "copy", destination_file])

# Extract extended attributes of the source file
extended_attributes = subprocess.check_output(["xattr", "-l", source_file]).decode()

# Use HandbrakeCLI to convert the source file to destination file and pass the extended attributes
subprocess.call(["HandBrakeCLI", "-i", source_file, "-o", destination_file, "--all-metadata="+extended_attributes])

For Windows, this might work:

#include <windows.h>
#include <tchar.h>

int main()
{
    // Open the source file
    HANDLE hSource = CreateFile(_T("source.txt"), GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    if (hSource == INVALID_HANDLE_VALUE)
    {
        _tprintf(_T("CreateFile failed (%d)\n"), GetLastError());
        return 1;
    }

    // Get the extended metadata of the source file
    FILE_BASIC_INFO sourceInfo;
    if (!GetFileInformationByHandleEx(hSource, FileBasicInfo, &sourceInfo, sizeof(sourceInfo)))
    {
        _tprintf(_T("GetFileInformationByHandleEx failed (%d)\n"), GetLastError());
        CloseHandle(hSource);
        return 1;
    }

    // Close the handle to the source file
    CloseHandle(hSource);

    // Open the output file
    HANDLE hOutput = CreateFile(_T("output.txt"), GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
    if (hOutput == INVALID_HANDLE_VALUE)
    {
        _tprintf(_T("CreateFile failed (%d)\n"), GetLastError());
        return 1;
    }

    // Set the extended metadata of the output file
    if (!SetFileInformationByHandle(hOutput, FileBasicInfo, &sourceInfo, sizeof(sourceInfo)))
    {
        _tprintf(_T("SetFileInformationByHandle failed (%d)\n"), GetLastError());
        CloseHandle(hOutput);
        return 1;
    }

    // Close the handle to the output file
    CloseHandle(hOutput);

    return 0;
}

Then for Linux (also not sure):

ffmpeg -i sourcefile.mp4 -map_metadata 0 -c copy -y -map_metadata:s:v 0:s:v -map_metadata:s:a 0:s:a destinationfile.mp4

If you have time, please check if the code i provided could be used to add the metadata passthru functionality.

Thanks for reading

DarinStephen avatar Jan 28 '23 08:01 DarinStephen

HandBrake uses libav for a variety of things including muxing already. Changing the creation_time tag is relatively easy. You can pass through the original files creation_time tag simply by commenting out a few lines of code in libhb/muxavformat.c. The problem is that there are competing alternatives to what that tag should be set to and therefor UI issues surrounding how to allow the various alternatives to be selected. Since this is a pretty low priority item (i.e. none of the small number of HandBrake developers uses this feature) it simply hasn't received much attention.

jstebbins avatar Jan 28 '23 15:01 jstebbins

Maybe not the developers but a lot of end users - at least if I browse throught the topics in the forum and related requests here. This is relevant for everyone using cloud photo services. The compression on most smartphones is really bad, and you can easily reduce the file size to around 30% of the original size with no significant loss in quality. Saving you a lot of space in the cloud if you regularly make videos. Up to now I change the EXIF creation date manually after encoding with the exif-tool (which is used by Google Photos and OneDrive for sorting). This really costs a lot of time. I fully understand that no one likes to mess around with the metadata in an encoding tool. However, would be great if you can consider this as an exception. Just check your own cloud space usage :-)

a82kd avatar Apr 02 '23 18:04 a82kd

HandBrake is opensource, and everyone can contribute to it, patches are very welcomed.

galad87 avatar Sep 05 '23 10:09 galad87

Ticket name has a typo, should be 'Passthru'

luzpaz avatar Nov 17 '23 11:11 luzpaz

just a suggestion, how about using/integrating FFmpeg library?

how about we modify the HandBrake transcoding process to first use FFmpeg to copy the metadata from the input file to a temporary file, and then use HandBrake to transcode the video and audio streams from the temporary file to the final output file.

Lastly, we can again use the FFmpeg library to copy the metadata from the temporary file to the final output file.

this could be simplified by telling people that they would need to download and install FFmpeg and then just put the program path of FFmpeg in Handbrake.

the below code is made with Python and is only for the common metadata

import subprocess

input_file = 'input.mp4'
temp_file = 'temp.mp4'
output_file = 'output.mp4'

# Use FFmpeg to copy metadata from input file to temp file
subprocess.run(['ffmpeg', '-i', input_file, '-c', 'copy', '-map_metadata', '0', temp_file])

# Use FFmpeg to copy metadata from temp file to output file
subprocess.run(['ffmpeg', '-i', temp_file, '-c', 'copy', '-map_metadata', '0', output_file])

For the Extended attributes, which i believe is what we need for promoting the source's filesystem info to be the destination's container metadata, would differ from OS to OS.

I think the below would work for MacOS:

import subprocess

source_file = "/path/to/source.mp4"
destination_file = "/path/to/destination.mp4"

# Use FFmpeg to copy metadata from source to destination
subprocess.call(["ffmpeg", "-i", source_file, "-map_metadata", "0", "-c", "copy", destination_file])

# Extract extended attributes of the source file
extended_attributes = subprocess.check_output(["xattr", "-l", source_file]).decode()

# Use HandbrakeCLI to convert the source file to destination file and pass the extended attributes
subprocess.call(["HandBrakeCLI", "-i", source_file, "-o", destination_file, "--all-metadata="+extended_attributes])

For Windows, this might work:

#include <windows.h>
#include <tchar.h>

int main()
{
    // Open the source file
    HANDLE hSource = CreateFile(_T("source.txt"), GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    if (hSource == INVALID_HANDLE_VALUE)
    {
        _tprintf(_T("CreateFile failed (%d)\n"), GetLastError());
        return 1;
    }

    // Get the extended metadata of the source file
    FILE_BASIC_INFO sourceInfo;
    if (!GetFileInformationByHandleEx(hSource, FileBasicInfo, &sourceInfo, sizeof(sourceInfo)))
    {
        _tprintf(_T("GetFileInformationByHandleEx failed (%d)\n"), GetLastError());
        CloseHandle(hSource);
        return 1;
    }

    // Close the handle to the source file
    CloseHandle(hSource);

    // Open the output file
    HANDLE hOutput = CreateFile(_T("output.txt"), GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
    if (hOutput == INVALID_HANDLE_VALUE)
    {
        _tprintf(_T("CreateFile failed (%d)\n"), GetLastError());
        return 1;
    }

    // Set the extended metadata of the output file
    if (!SetFileInformationByHandle(hOutput, FileBasicInfo, &sourceInfo, sizeof(sourceInfo)))
    {
        _tprintf(_T("SetFileInformationByHandle failed (%d)\n"), GetLastError());
        CloseHandle(hOutput);
        return 1;
    }

    // Close the handle to the output file
    CloseHandle(hOutput);

    return 0;
}

Then for Linux (also not sure):

ffmpeg -i sourcefile.mp4 -map_metadata 0 -c copy -y -map_metadata:s:v 0:s:v -map_metadata:s:a 0:s:a destinationfile.mp4

If you have time, please check if the code i provided could be used to add the metadata passthru functionality.

Thanks for reading

I think this is a good idea, but if you want to perform batch operations, it would be too cumbersome to change the file names one by one. Can we improve it so that all files in the target path with the same names as the files in the source path can have their matching EXIF source data copied over?

Tinyu-Zhao avatar Dec 11 '23 04:12 Tinyu-Zhao