Add support for dynamic and unknown UTI types

[llamafilm]

I get a lot of plain text files with various extensions such as .1 or .inc and currently they only show the icon in QuickLook. Looking at one such file with mdls I see it has a dynamic type. But the content type tree includes public.data. Would it be possible to extend support to ALL filetypes?

$ mdls wget-log.2 
...
kMDItemContentType                 = "dyn.ah62d4rv4ge8xe"
kMDItemContentTypeTree             = (
    "dyn.ah62d4rv4ge8xe",
    "public.data",
    "public.item"
)
...

$ mdls function.inc 
...
kMDItemContentType                 = "dyn.ah62d4rv4ge80w5xd"
kMDItemContentTypeTree             = (
    "dyn.ah62d4rv4ge80w5xd",
    "public.item",
    "dyn.ah62d4rv4ge80w5xd",
    "public.data"
)
...

[tsdorsey]

Please move all discussion of file types and extensions in here. #30 #32 #81 #82 #89

Apple's UTI reference documentation.

From what I can tell, it's all the same issue. The kMDItemContentType and kMDItemContentTypeTree data can either be missing or dynamic for some files and this is causing issues.

[tsdorsey]

To add to the set of tested files, here is what happens when you use the LICENSE file in this repo. I also copied it and added .1 to the end of it.

$ mdls LICENSE
...
kMDItemContentType                     = "public.data"
kMDItemContentTypeTree                 = (
    "public.data",
    "public.item"
)
...

$ mdls LICENSE.1
...
kMDItemContentType                     = "dyn.ah62d4rv4ge8xc"
kMDItemContentTypeTree                 = (
    "public.item",
    "dyn.ah62d4rv4ge8xc",
    "public.data"
)
...

[tsdorsey]

Fun with UTIs Found this to be very helpful in understanding what is happening.

Right now the plugin registers itself at the highest point that makes sense in both the physical and functional hierarchies. public.data and public.content

There is a good case to be made for changing public.content to public.text but that's not a discussion we need to have here. Let's figure out how we can get this plugin working for more files.

[knaaptime]

Hi, thanks for your work on this. I want to add +1 for yaml support. Fwiw, that has been implemented here but hasn't been updated in quite awhile

[tsdorsey]

@relikd I found this: PrivateLogs I have to use sudo when enabling/disabling private logs but it worked.

Here is a log of running qlmanage -p LICENSE.1 This is a copy of the LICENSE file in this repo with a .1 extension added.

This confirms my suspicion that this is due to macOS setting the kMDItemContentType to a "dyn.*" value.

[relikd]

Thanks for the PrivateLogs link.

Yep, I encountered this issue too. Even pre Catalina, in fact even pre Mojave. Just recently I was working on QLOPML on a 10.13 machine. Had the same issue; macOS would identify files only by the dyn.* type. In my main application I had registered UTIs and file types. But they were not identified as such. What I did is add all dynamic types to my plist.

<key>LSItemContentTypes</key>
<array>
  <string>org.opml.opml</string>
  <string>dyn.ah62d4rv4ge8086drru</string>
</array>

Additionally, I found this blog post very helpful: Analysis of UTIs (had to search a while to find this post again).

[tsdorsey]

If anyone has any details on what the file type list looks like for the "Generic" QuickLook plugin that ships with macOS it might lend a clue. Or possibly, it's just hardcoded as the final fallback. I hope that's not the case.

[tsdorsey]

tl;dr on the dyn.* namespace: the values are base32 encoded query strings. So the suggestion to add them info.plist is not really viable as it's whack-a-mole on an infinite grid... 😞

[relikd]

Do you happen to know which of them might be the generic QL plugin? The best I got is /System/Library/QuickLook/Text.qlgenerator, which conforms to:

public.plain-text
public.rtf
com.apple.rtfd
org.oasis-open.opendocument.text
com.apple.property-list
public.xml
public.json

But then I'm not sure why .m files are previewed:

kMDItemContentTypeTree = (
    "public.objective-c-source",
    "public.source-code",
    "public.plain-text",
    "public.text",
    "public.data",
    "public.item",
    "public.content"
)

And why QLstephen fails for e.g., the .in extension:

kMDItemContentTypeTree = (
    "public.item",
    "public.data",
    "dyn.ah62d4rv4ge80w5u"
)

[relikd]

Also, I tried creating a custom dyn.* adopting the public.data UTI without any extension. But that didn't work either. Setting the extension to * or nothing at all, also didn't work.

Somehow the UTI is not recognized as being inherited from public.data although the type tree clearly shows that it is. It also correctly displays the preview if the generator is specified directly:

qlmanage -g ~/Library/QuickLook/QLStephen.qlgenerator -c "_" -p MANIFEST.in

So the problem really is, that macOS can't seem to map dyn.ah62d4rv4ge80w5u (*.in) to public.data. And therefore skips QLStephen because it does not seem to apply to the extension. Which is funny, because it clearly works for *.m files and the Text.qlgenerator.

[philfreo]

I was sent here from #81. I used to have a working qlstephen setup but haven't since Catalina.

I have rebooted my machine, brew cask reinstall qlstephen (v1.5.1), and followed the instructions in the README, and I still can't QuickLook on some files like LICENSE, CHANGELOG, or Makefile.

[relikd]

Can you uninstall the brew and try again with https://github.com/whomwah/qlstephen/releases/latest (following the xattr -cr procedure). Maybe something was messed up in the brew installation. I have a working† qlstephen on 10.15.3

† at least working on files w/o extension. Unknown extensions are known to be broken (see discussion above).

[philfreo]

Just did this:

$ brew cask uninstall qlstephen
==> Uninstalling Cask qlstephen
==> Backing QuickLook Plugin 'QLStephen.qlgenerator' up to '/usr/local/Caskroom/qlstephen/1.5.1/QLStephen.qlgenerator'.
==> Removing QuickLook Plugin '/Users/philfreo/Library/QuickLook/QLStephen.qlgenerator'.
==> Purging files for version 1.5.1 of Cask qlstephen

Then downloaded, unzipped, copied to ~/Library/QuickLook/.

$ xattr -cr ~/Library/QuickLook/QLStephen.qlgenerator 
$ qlmanage -r
qlmanage: resetting quicklookd
$ qlmanage -r cache
qlmanage: call reset on cache

Then relaunched finder via right-click method.

LICENSE etc still not working. macOS 10.15.3

More debug info:

$ qlmanage -m | grep public.data
  public.data -> /Users/philfreo/Library/QuickLook/QLStephen.qlgenerator (1.5.1 - loaded)

$ ls -l /Users/philfreo/Library/QuickLook 
total 0
drwxr-xr-x  3 philfreo  staff  96 Oct 24  2018 DropboxQL.qlgenerator
drwxr-xr-x  3 philfreo  staff  96 Feb  4 12:35 QLStephen.qlgenerator
drwxr-xr-x@ 3 philfreo  staff  96 Oct 21  2014 WebP.qlgenerator
drwx------  3 philfreo  staff  96 Nov 10  2018 qlImageSize.qlgenerator

$ ls -l /Library/QuickLook 
total 0
drwxr-xr-x  3 root  wheel  96 Nov 10  2018 Video.qlgenerator
drwxr-xr-x  3 root  wheel  96 Sep 15  2018 iWork.qlgenerator

[philfreo]

Okay perhaps this is helpful:

The files I were trying are reporting public.unix-executable when I pass it to mdls

mdls [...]/LICENSE
...
kMDItemContentType                 = "public.unix-executable"
kMDItemContentTypeTree             = (
    "public.item",
    "public.executable",
    "public.data",
    "public.unix-executable"
)
...

I've also got some public.yaml files that aren't working that would be great if they could.

[relikd]

seems very strange for a plain text file to be executable. Can you create a new empty file with just "hello world" and save that as LICENSE. Also, try chmod 644 on the file (although I think mds shouldnt read the permissions...)

.yaml should be currently broken. Dont know if this works, but you can follow the instructions in the readme on adding custom types to the config. You can get the dyn.* type like you did with the License file.

[philfreo]

The LICENSE I happened to be trying did indeed have executable permissions (654) for some reason. Changing it to 644 solved the issue with that file. Thanks.

But I do have some actual unix executables (e.g. little extension-less shell scripts in a bin/ directory) that I'd like to see working.

[relikd]

I think executable files were never supported by QLStephen. .sh will display, though that might be an apple ql plugin in action, haven't verified. The problem with executable files is to know beforehand if it is human readable. But maybe we could add executables to the list of types and only check the first 4 bytes if they are readable? What do you think @tsdorsey?

[relikd]

I've created PR #94 just in case. If we dont want it or there are unforeseen side effects, just say so and I'll delete my fork ;-)

[blackketter]

This might be crazy talk, but I'd be happy if qlstephen showed me a text view of any file for which it can't otherwise distinguish. (At least the first few kilobytes of larger files.) Not sure if this is possible.

[relikd]

My current understanding of the issue is that:

1. QuickLook does not fallback to other UTI in kMDItemContentTypeTree
2. Instead the kMDItemContentType is used to assign the corresponding QL plugin.
3. Every file extension has its own dyn.* UTI

Which means, we would need a list of extensions that should be supported by QLStephen. That implies that QLStephen can potentially override another QL plugin that does a better job in representing a known extension. E.g., there are specialised markdown plugins and I would be pissed if QLStephen would break them for me. Not sure how QuickLook handles plugin precedence.

Some plugins will define "Imported Type UTIs". That basically replaces the "unkown" dyn.* UTIs with a developer set UTI. For example, in my extension I register org.opml.opml for *.opml files. On my system, for opml files, mdls returns that UTI instead of dyn.ah62d4rv4ge8086drru.

kMDItemContentType = "org.opml.opml"

[jchannon]

So we still don't have a solution ? For now I'm just adding the dyn.foobar entries into the plist

[nabastak]

So we still don't have a solution ? For now I'm just adding the dyn.foobar entries into the plist

Sorry people, I've read this tread and still didn't understand what you're doing with those "dyn entries"... For example if I want to preview text of files with "sql" extensions - what should I do ? I open the info.plist file in the qlstephen.qlgenerator(temporary remove extension "qlgenerator" to manage it as a folder and open that file in text editor). There some lines -

<key>LSItemContentTypes</key>
			<array>
				<string>public.data</string>
				<string>public.content</string>
			</array>
<key>LSTypeIsPackage</key>

Do I need to change something here ?

[relikd]

Ok, so according to the source I references above, I would do the following:

1. Generate the dyn content, in this case I guess its ?0=6:1=sql.
   Though I am not sure if the 6 is correct or if it should be 7. Where numbers are substituted as follows:

0: UTTypeConformsTo
1: public.filename-extension
2: com.apple.ostype
3: public.mime-type
4: com.apple.nspboard-type
5: public.url-scheme
6: public.data
7: public.text
8: public.plain-text
9: public.utf16-plain-text
A: com.apple.traditional-mac-plain-text
B: public.image
C: public.video
D: public.audio
E: public.directory
F: public.folder

2. Next you put this string into a custom base32 converter. E.g. this website

   Input: ?0=6:1=sql
   Variant: Custom
   Alphabet: abcdefghkmnpqrstuvwxyz0123456789
   Padding: – Delete if there is any –

3. The output should be h62d4rv4ge81g6pq. If you have any trailing = delete it, thats the padding.

4. Prepend dyn.a and that is your final string.

5. What you should insert in the Info.plist is dyn.ah62d4rv4ge81g6pq

<key>LSItemContentTypes</key>
<array>
	<string>public.data</string>
	<string>public.content</string>
	<string>public.unix-executable</string>
	<string>dyn.ah62d4rv4ge81g6pq</string>
</array>

PS. You dont need to remove the extension, you can also right-click and 'Show Package Contents'

[nabastak]

Wow, thanks so much for such detailed answer !! It surely helped.

Meanwhile I also tried another method(from internet) for quicklook to work (guess it works also without the qlstephen plugin)

Register a new file extension via the dummy app :

1. Create completely empty application in AppleScript , save it somewhere as executable application

2. in its Info.plist add these lines to the end (before last 2 closing tags)

<key>UTImportedTypeDeclarations</key>
<array>
    <dict>
        <key>UTTypeIdentifier</key>
        <string>com.idsoftware.wad</string>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>public.filename-extension</key>
            <array>
                <string>wad</string>
            </array>
        </dict>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.data</string>
        </array>
        <key>UTTypeDescription</key>
        <string>Doom WAD file</string>
        <key>UTTypeIconFile</key>
        <string>DoomWAD.icns</string>
        <key>UTTypeReferenceURL</key>
        <string>http://en.wikipedia.org/wiki/Doom_WAD</string>
    </dict>
</array>

For the experiment I changed the <string>wad</string> to <string>sql</string> and <string>public.data</string> to <string>public.source-code</string> (I tried public.text etc but with public.source-code it finally worked)

3. Register dummy app via terminal lsregister ~/Desktop/MyDummyApp.app

[alvarotrigo]

I understand the problem hasn't been fixed yet?

“QLStephen.qlgenerator” cannot be opened because the developer cannot be verified.

[jchannon]

Execute mdls -name kMDItemContentType ~/path/to/file.ext to get the dyn value, add it to the plist

[JoeStrout]

1. Register dummy app via  terminal
   lsregister ~/Desktop/MyDummyApp.app

I am trying this solution on macOS 10.15.7, but I get "zsh: command not found: lsregister". Any suggestions?

[relikd]

I am trying this solution on macOS 10.15.7, but I get "zsh: command not found: lsregister". Any suggestions?

I guess its in a non-PATH. You can created a symlink or use the full path instead:

lsregister@ -> /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister

[NSGod]

So, I've come up with what I think might be a workable solution, though I'm looking for thoughts and feedback on it.

Rather than having to worry about generating dyn.* UTIs, etc., all we really need to do is simply have a way to tell the system that an unknown filename extension (e.g. .map) conforms to public.plain-text. By doing so, we then allow the system’s built-in text quicklook plugin to be used to preview the contents.

The solution I came up with is based on the assumption that only the end-user is going to know what filename extensions they come across and whether those files are text-based or not. It uses 2 different apps:

1. QLStephen.app: the primary app you interact with. It holds a list of the filename extensions that you've mapped to being treated as text. To map a particular filename extension to be treated as text, we declare that it conforms to “public.plain-text”. By doing so, we then allow the system’s built-in text quicklook plugin to be used to preview the contents. To implement these file mappings on the system, QLStephen.app uses a second application, “QLStephen Dummy.app”, in which the mappings are declared in the UTExportedTypeDeclarations of its Info.plist file. QLStephen.app also adds a Service so that you can right-click/control-click on any file in the Finder, and choose “Quick Look with QLStephen ✅” (yes, this is a bit misleading, I guess). This automatically opens it in QLStephen.app, and if it’s found to be text-based, is added to the list of file extension mappings. Then, after updating the dummy app, you can deselect and then reselect the file in the Finder to preview its contents as text.

2. QLStephen Dummy.app: This app is created in the ~/Applications/ folder. As mentioned previously, this is a dummy app that declares the file extension mappings in its ‘UTExportedTypeDeclarations` key in its Info.plist file. After modifying the Info.plist of the installed app, it is codesigned to run locally.

To be clear, these 2 apps exist separately from the QuickLook plugin which is still installed into ~/Library/QuickLook.

Some notes/known issues: • Despite having a “Kind” column for you to fill out a file extension’s description, it appears the Finder will always refer to the kind as “Plain Text Document”. (I think declaring the documents in the CFBundleDocumentTypes as well might convince Launch Services to use their descriptions). • There are some filename extensions that still won’t preview even though they are text-based: .css files and .strings files are 2 examples I’ve encountered.

I have a qlstephen.app branch where I pushed this to in my fork: https://github.com/NSGod/qlstephen/tree/qlstephen.app.

A link to a compiled and signed binary: https://markdouma.com/developer/QLStephen.zip I welcome any feedback.

[xupefei]

As another solution, I wrote a Fish script that automatically detect the UTI type of a given file and append the dyn.* type to QLStephen:

function ql
	set type (mdls -name kMDItemContentType $argv[1] | sed -n 's/^kMDItemContentType = \"\(.*\)\"$/\1/p')
	echo $type
	plutil -insert CFBundleDocumentTypes.0.LSItemContentTypes.0 -string $type ~/Library/QuickLook/QLStephen.qlgenerator/Contents/Info.plist
	qlmanage -r
end

Usage for Fish shell:

$ ql <drag a file to the terminal window>

[plpxsk]

To summarize the current workaround, below are instructions to quick look a given file or file type (*.Rmd in the below example).

This was tested on macOS Big Sur 11.3 Terminal (zsh).

1. Run the following command on an existing file of interest to find a string that begins with dyn.a...:

   $ mdls -name kMDItemContentType ~/path/file.Rmd
   kMDItemContentType = "dyn.a..."
   

2. Open Info.plist to edit it, at ~/Library/QuickLook/QLStephen.qlgenerator/Contents.

   To navigate there, you can right click QLstephen.qlgenerator and "Show Package Contents" or use a code editor to get there

3. Add the relevant dyn.a... to Info.plist like in the following:

   ...
   <key>LSItemContentTypes</key>
   	<array>
   		<string>public.data</string>
   		<string>public.content</string>
   		<string>public.unix-executable</string>
   		<string>dyn.a...</string>
   	</array>
   ...
   

4. Reset quick look with a) qlmanage -r and b) Relaunch Finder: Apple menu - Force Quit ... - Finder - Relaunch.

[muyinliu]

To summarize the current workaround, below are instructions to quick look a given file or file type (*.Rmd in the below example).

This was tested on macOS Big Sur 11.3 Terminal (zsh).

1. Run the following command on an existing file of interest to find a string that begins with dyn.a...:

   $ mdls -name kMDItemContentType ~/path/file.Rmd
   kMDItemContentType = "dyn.a..."
   

2. Open Info.plist to edit it, at ~/Library/QuickLook/QLStephen.qlgenerator/Contents. To navigate there, you can right click QLstephen.qlgenerator and "Show Package Contents" or use a code editor to get there
3. Add the relevant dyn.a... to Info.plist like in the following:

   ...
   <key>LSItemContentTypes</key>
   	<array>
   		<string>public.data</string>
   		<string>public.content</string>
   		<string>public.unix-executable</string>
   		<string>dyn.a...</string>
   	</array>
   ...
   

4. Reset quick look with a) qlmanage -r and b) Relaunch Finder: Apple menu - Force Quit ... - Finder - Relaunch.

Works fine, thanks!

Note: command killall Finder equal to Relaunch Finder.

[alexchandel]

@NSGod Should be possible to merge those into one app right?

[NSGod]

@NSGod Should be possible to merge those into one app right? @alexchandel: Possibly, though I didn't really like the idea of having an app modify its own Info.plist while it's running (which would probably invalidate its code signature), and then have it re-codesign itself while it's running. I thought it'd be easier to use a dummy stand-in app.

Granted, I can sometimes come up with solutions that tend to be overly complex.

[toy]

After several evenings I've stumbled upon a fix and would appreciate a confirmation.

First to showcase the problem: https://github.com/whomwah/qlstephen/compare/master...toy:extensions-problem-showcase?expand=1

1. Make QLStephen also handle UTI toy.explicit
2. Add application extensions.app with the goal to define UTIs:
   • toy.explicit (for .toy-explicit extension) conforming to public.plain-text
   • toy.a0 (for .toy-a0 extension) conforming to public.plain-text
     • toy.a1 (for .toy-a1 extension) conforming to toy.a0
       • toy.a2 (for .toy-a2 extension) conforming to toy.a1
   • toy.l0 (for .toy-l0 extension) conforming to public.data
     • toy.l1 (for .toy-l1 extension) conforming to toy.l0
       • toy.l2 (for .toy-l2 extension) conforming to toy.l1
3. Add files for each UTI + an unknown one with extension .toy-unknown

To test checkout extensions-problem-showcase and run make && make install, in this state only .toy-a0, .toy-a1, .toy-a2 and .toy-explicit should have preview/thumbnail generated (first 3 by Text.qlgenerator, last by QLStephen because it is explicitly in its list of UTIs).

Fix: https://github.com/toy/qlstephen/compare/extensions-problem-showcase...toy:extensions-showcase-fix?expand=1 Just change the bundle id so that it starts with com.apple. and suddenly all test files get the preview/tumbnail. To test checkout extensions-showcase-fix and run make && make install (wait for qlmanage -m to show the list, sometimes repeating make && make install is needed).

If confirmed #135 is the fix unless someone has a better idea