udftools
udftools copied to clipboard
pali
The 2.50 version of UDF image for --media-type HD can not be created.
Hello： I use the udftools version 2.0 to make udf image can not mount
mkudffs --lvid="20180606-1" --media-type=hd --udfrev=2.50 b.udf
mmount -o loop,rw b.udf disc/ mount: /dev/loop4 readonly
i will read write mode mount
mkudffs --lvid="20180606-1" --media-type=hd --udfrev=2.50 b.udf
mkudffs: Error: UDF revision above 2.01 is not currently supported for specified media type
And also UDF driver in Linux kernel does not support write operation for UDF 2.50+ filesystems (it is read-only).
UDF 2.50 for non-VAT disks requires metadata partition which mkudffs is not able to generate (yet).
Also patches for generating metadata partition which is needed for UDF 2.50 support on HD are welcome!
If the kernel doesn't support mounting udf 2.50+ rw.. how do you add files?
Maybe through a fuse driver?
There is 'mkisofs': this app can create hybrid iso9660/udf1.0 "isos".. maybe some day mkudffs will be able to create blu-ray data isos from a directory ..
There is (userspace) project udfclient which can add or delete files on UDF disk, including UDF 2.50.
But adding files via mkudffs when formatting disk is not bad idea. Patches for such functionality are welcome.
Hmm i must be honest.. udfclient sounds like a udf tool.
It could be absorbed by this project.. but i see it's a BSD project, probably with the BSD license.... what does this mean? Those people don't like the GPL and its viral nature, so they wouldn't contribute any code
Or at least this project could have a "mirror" with no code changes, only to provide the udfclient so that people can edit udf 2.50 isos or something.
I've never used that app.. the bsd apps are unknown to most mortals, nobody knows about them, but if they are included in udftools, then something bigger can happen..
Even now that i know that there's a way to edit udf 2.50 isos.. i haven't compiled udfclient yet.
It's really sad the linux kernel didn't add rw support for udf 2.50, but that just shows that something is needed to fill that gap.
But there's the licensing issues i guess..
It could be absorbed by this project..
I do not see reason why. udfclient is separated project and works without udftools. Basically udfclient and udftools provides different utilities.
I've never used that app
It is working fine on Linux. You can really try it. E.g. Debian already has package for it.
It's really sad the linux kernel didn't add rw support for udf 2.50
That is truth. As always patches are welcome!
digging into old feature requests via google I found link to this project: https://sourceforge.net/p/fudf/news/2008/07/fudf-v20-released/
description says: "version 2.0 supports creation, reading and editing of UDF2.50 compliant filesystems." But.. not sure if anything from this project can be reused..?
Well, anything which is GPLv2+ license compatible can be reused. But it means that somebody has to take foreign code and adopt it. As I wrote, patches and contributions are welcome. (Sometimes reusing foreign code for another projects means lot of work, more than rewriting it from scratch...)
so, I compiled udfclient 0.8.11 (with small edit in uio. h due to conflicting iovec define.. on termux/android, arm 32 bit system).
next I booted netbsd 9.2 boot iso (200mb) on qemu-system-i386 (pentium2 cpu, 320mb ram). I append fileimage to qemu as '-hda file', but this resulted in 512 byte/sector virtual hdd as seen by guest netbsd
next I formatted it with newfs_udf with '-v' and '-V' switches both set to 2.50. (on /dev/rwd0d!) then mounted with 'mount - t udf /dev/wd0d /mnt' and copied some stuff and unmounted and 'shutdown - h now' my virtual machine.
then I tried udfclient -W .. only to discover.. *** marked read-only due to read-only support for Metadata partition ***
newfs_udf from udfclient software on linux makes strange udf with version (both of them) set to 0.00. mkudffs - m bdr also makes image not yet writable by udfclient.
A lot of learning happened, but no solution was found ( (not counting qemu vm with netbsd, probably going via vndconfig on file located on virtual hdd! (for 512 vs 2048 byte/sector mismatch!). I have no such amount of free space on my tablet..
FUDF v2, looks like it was written for Windows, RtlAnsiStringToUnicodeString()
a lot of .cpp files, but I think it doesn't use C++ specific stuff, looks like C for MSVCRT.......... but some classes are implemented CDataBaseCore
I don't think it will compile on UNIX(-like) systems, but it's interesting. There's no project to build the lib, so with some effort an autotools project for MinGW might make the thing compile
It would take a long time to convert it to a pure C project and port it to Linux, but as an open source project it's interesting ...
Ah... there's no LICENSE anywhere, the author forgot to add a license, so it must be treated as public domain ...
also there was mkudfiso from 2010 on sourceforge, but it only makes udf 1.02?
from my download manager: https://phoenixnap.dl.sourceforge.net/project/blurayauthor/mkudfiso/mkudfiso-svn-checkout/mkudfiso-svn-checkout-20100208.tar.gz
also there is 'our' bdwrite, but it probably does not do real 2.50 level stuff (not tested its output yet) https://git.cinelerra-gg.org/git/?p=goodguy/cinelerra.git;a=blob;f=cinelerra-5.1/cinelerra/bdwrite.C;h=6a2b4ca3cb4b9bcec54d7b8620a74b51cafd7395;hb=HEAD
and there is netbsd kernel commit "First part of shrinking/growing metadata partition support:
- extending the metadata partition" netbsd
If you want to add support for UDF 2.50+ into mkudffs for non-VAT images, then I think it is required to implement just these two things:
- create empty metadata partition during formatting (similarly like is created sparable or vat partition)
- in function udf_alloc_blocks() insert blocks also into metadata partition
And if you want to implement inserting files into filesystem during formatting by mkudffs, then it is just needed to implement ICBTAG_FLAG_AD_SHORT and ICBTAG_FLAG_AD_LONG cases in insert_data() function (currently there are errors with exit(1) calls). As support for creating files and directories is there, just missing support for putting data (ADs) to (empty) files.
I think that for both things it would be much harder to copy this functionality from external 3rd UDF implementation into udftools as writing it from scratch for udftools.
When you want to add files into UDF filesystems during formatting, there is no need to implement shrinking/growing metadata partition. Just calculate size which is enough for all data you want to put.
On Tuesday, November 23, 2021, pali @.***> wrote:
If you want to add support for UDF 2.50+ into mkudffs for non-VAT images, then I think it is required to implement just these two things:
- create empty metadata partition during formatting (similarly like is created sparable or vat partition)
- in function udf_alloc_blocks() insert blocks also into metadata partition
just enough blocks for storing metadata file? looking at http://web.archive.org/web/20120314004528/http://homepage.mac.com/wenguangwang/myhome/udf.html
so metadata file itself will consume as many File Entries (each block-sized, 2048 for optical media?) as we have files and directories in our source tree? Thus without Mirror Metadata and free space we can just allocate Metadata partition that big?
And if you want to implement inserting files into filesystem during formatting by mkudffs, then it is just needed to implement ICBTAG_FLAG_AD_SHORT and ICBTAG_FLAG_AD_LONG cases in insert_data() function (currently there are errors with exit(1) calls). As support for creating files and directories is there, just missing support for putting data (ADs) to (empty) files.
I tried to read more on how udf structured.. https://lists.gnu.org/archive/html/libcdio-devel/2010-10/msg00011.html
it seems big files (1-4 gb and more) must be broken into multiple extents?
but short/long extents themselves are same in udf 1.02 and 2.50?
I think that for both things it would be much harder to copy this
functionality from external 3rd UDF implementation into udftools as writing it from scratch for udftools.
When you want to add files into UDF filesystems during formatting, there is no need to implement shrinking/growing metadata partition. Just calculate size which is enough for all data you want to put.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pali/udftools/issues/20#issuecomment-976454447, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJSS7TNHG5V3YXAWCG3SQW3UNOAWBANCNFSM4FDQZJMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
some more links
[https://www.yumpu.com/en/document/view/26257714/all-approved-udf-260-dcns](udf 2.60 errata? ) - talks about BD-R requrements and how (dcn-5102) Metadata Bitmap file shall not be recorded for read-only media..
[https://stackoverflow.com/questions/8832031/udf-filesystem-reading-bluray-metadata-partition-libdvdread-with-iso-images](stackowerflow question dealing with analysis of some udf 2.5+ images)
[https://github.com/firecore/libudfread/blob/master/src/udfread.c](libudfread source) - so, this metadata partition is by itself virtual, and main three (two?) files within it really live on real partition as files with special filetype?
But how to construct those metadata files?
But how to construct those metadata files?
@Randrianasulu What about to read official specification :D and not looking at random/incomplete suggestions? Nothing wrong, just I want to point that that all important details are in official specifications, including DCNs for BD-R requirements.
Just to note that udfinfo has already support for reading/parsing metadata partition and metadata file.
On Thursday, November 25, 2021, pali @.***> wrote:
Just to note that udfinfo has already support for reading/parsing metadata partition and metadata file.
ah, thanks.
Sorry, official spec is... not small. I just trying to wrap my head around all those concepts....
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pali/udftools/issues/20#issuecomment-979442882, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJSS7TJEPSSCZZCAJOWZKCDUN2HGBANCNFSM4FDQZJMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
put mkudfiso on github:
https://github.com/Randrianasulu/mkudfiso/commits/main
right now I am not even sure if my simple fprintf addition actually compileable - working purely via web-interface from tablet...
On Thursday, November 25, 2021, Andrew Randrianasulu < @.***> wrote:
On Thursday, November 25, 2021, pali @.***> wrote:
Just to note that udfinfo has already support for reading/parsing metadata partition and metadata file.
ah, thanks.
Sorry, official spec is... not small. I just trying to wrap my head around all those concepts....
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pali/udftools/issues/20#issuecomment-979442882, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJSS7TJEPSSCZZCAJOWZKCDUN2HGBANCNFSM4FDQZJMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
mkudfiso project (from sourceforge.net) produce invalid and broken UDF images. I was using it during testing of udfinfo to ensure that such broken/invalid images do not cause segfaults of udfinfo and instead print error messages.
On Friday, November 26, 2021, pali @.***> wrote:
mkudfiso project (from sourceforge.net) produce invalid and broken UDF images. I was using it during testing of udfinfo to ensure that such broken/invalid images do not cause segfaults of udfinfo and instead print error messages.
yeah.. now udfinfo (2.3) produces nice set of warnings if used on output of this tool. thing is, hopefully this project contain some basics about getting filesizes etc. At least some starting point.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pali/udftools/issues/20#issuecomment-980136204, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJSS7TJEXE6MG6OTOVQSNZDUN63N5ANCNFSM4FDQZJMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
@pali, I found patent apparently describing udf 2.50 write process
sony patent
Examples of meta-information recorded in the metadata partition include an FSDS, an ICB (Information Control Block) and an FID (File Identifier Descriptor) created for each directory and file.
so... it seems metadata partition creation part in itself not very complex, but managing metadata files even for create-only-once case is not trivial (for outsiders like me).
On the related news udf verifier compiles on arm32/termux and reports no errors for Netbsd-created udf 2.50 filesystem image (with blocksize 512)
giant hack, but appears to work on image files: https://github.com/Randrianasulu/newfs_udf-linux/tree/main
now, thing is I am not sure if mkudffs itself should hold logic for creating random dirs and extending files and working with directory trees like mkiso.. May be separate script (using dd for per-sector writing!) can be hacked together just for putting some data in.
But yes, it seems extending at least metadata files should be implemented for formatting 2.50+ udfs...
Yea, generating metadata partition should not be too hard. Just take UDF 2.60 spec, fill required structures with correct data, place them into list of extends and finally write to the disk. Just somebody needs to do it.
What's the current state on this issue? Is it possible to create a non-empty UDF 2.50 (or 60) filesystem in 2023 on Linux? I tried newfs_udf and it's not 2.50, whereas mkudffs does seem to create a 2.60 filesystem but I found no way to fill it with data. udfclient is not able to even create an empty directory on filesystems as created by mkudffs. Any known hack/workaround to make this work?
