Missing 'marp' binary resource in the repo

[seclorum]

Trying to build this locally using my own Apple dev account, I get as far as this target failing:

"Copy marp" Showing All Errors Only /Users/seclorum/Desktop/Lab/Sidekick/Sidekick/Logic/View Controllers/Tools/Slide Studio/Resources/bin/marp: No such file or directory

Is this marp file a binary that is supposed to be produced by one of the targets in the Sidekick project, or is this a missing hand-copy step?

[johnbean393]

The marp binary is too large to upload to GitHub, so it has to be added manually. Here are the steps:

1. Get the binary.
2. Sign the binary and enable hardened runtime with custom entitlements (Allow execution of JIT code).

Command: codesign --force --options runtime --entitlements entitlements.plist --sign "myteam" ./marp

Entitlements:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>com.apple.security.cs.allow-jit</key>
		<true/>
		<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
		<true/>
		<key>com.apple.security.cs.disable-library-validation</key>
		<true/>
		<key>com.apple.security.cs.disable-executable-page-protection</key>
		<true/>
	</dict>
</plist>

3. Add the binary in Xcode

This is a tedious process, and it should probably replaced with a CI workflow at some point. I'll do this if the number of contributors continues to scale.

[runxiyu]

This should be written into documentation (a special section for dev documentation would help) rather than just leaving it here in an issue

[seclorum]

Okay, understood - but why can't we just use the marp bin that is included in the homebrew marp-cli package, instead? This would be better than including a foreign binary in the worktree/repository, and also give us a way of side-stepping all the signing and entitlements by using homebrew instead ..

[runxiyu]

I'm not familiar with the Swift ecosystem and how Xcode's build process works, but I don't think introducing brew as a runtime dependency is appropriate due to the target audience

I'd personally just clone the marp repo (or perhaps use it as a submodule) and build it into a static binary during the build process, if possible

[johnbean393]

Given Xcode's build process, it looks to me that integrating it as a build dependency is the easiest to add and maintain. Let me look into this.

[seclorum]

If there are no modifications being done to the marp sources specifically for Sidekick, it makes not much sense to include the marp project as a submodule, just to get the binary 'marp' built, when the following is a lot better developer experience:

$ brew install marp-cli
$ which marp  # <-- use that path to the marp tool instead of a custom internal path nobody knows about

For those developers who don't want to use homebrew, instructions to manually locate the marp binary can be added to the README - but honestly, why not just use homebrew for its intended purpose and simplify the issue entirely .. ?

[runxiyu]

@seclorum Correct me if I'm wrong, but end-users would also need to have the marp binary available. And it doesn't make sense to require end-users to install homebrew.

Not sure if copying the marp binary from /opt/homebrew and embedding it would work though.

[runxiyu]

Homebrew's marp-cli doesn't seem to produce a standalone binary

[runxiyu]

Anyways, I personally don't like marp as a dependency here anyway. Perhaps a native Swift one could be found or sth

[seclorum]

Ah, sorry - I didn't realize that marp was an end-user tooling requirement - thought it was only for building Sidekick itself. That sorta puts things into context a bit .. its a pity, though. Maybe it'd be better to have Sidekick check for homebrew and, if its installed, use it to install the marp-cli .. but then again, if marp is a dependency, it should probably be a submodule in this repo, be built as part of the process, and then be packaged in the .app bundle - which is close to what is happening already, I guess, its just real fiddly in the current configuration.

[runxiyu]

Mhm. It should probably be bundled because the target userbase likely doesn't home homebrew installed.

I think for the purposes of Sidekick, the most appropriate method is to download the binary as part of the build process, but then again I'm not sure how feasible this is with Xcode's build system. (What I do know, though, that it's absolutely possible to achieve this with a Makefile (curl the binary down then call xcode build), but I'd have a hard time convincing macOS developers to go this route 😃.)

I don't like Marp as a dependency (although I guess it's not that bad as a JS runtime is already bundled for the code interpreter). Unfortunately, I couldn't find anything popular other than Marp that supports converting Markdown to PowerPoint, although it should be relatively trivial (citation needed) to write a pandoc output format (also nice as it could generate Beamer presentations and reveal.js and stuff).

[johnbean393]

@runxiyu

@seclorum was correct, Marp is a binary bundled with the app to negate the need for homebrew and other developer tooling on the user's end.

What I'm trying to figure out right now is how to make the configuration (i.e. getting and signing a copy of the Marp binary) for devs easier as it is needed in the build. Currently the plan is to put together a script that would be run before a build to obtain and sign the binary.

[seclorum]

As a dev - to build this, I'd be more content to use the official docker image:

https://hub.docker.com/r/marpteam/marp-cli/

As a user, I'd prefer to have it installed in my .app bundle, but from official sources - i.e. actually built from source as part of the main Sidekick build, as a submodule dependency. This way, I can inspect the build products for marp, if needed - and/or sources for the actual binary.

Its the disconnect between some random .tar.gz file and the marp sources/official community/resources which kinda bugs me... that download URL would be a pretty good place to put a MitM attack, or an exfiltrator, or some such thing .. whereas the 'official' sources and subsequent builds, can be more easily triaged, should that ever be necessary.

[mikedg1]

Added a PR that makes it a tiny bit easier for devs to get the marp stuff setup. https://github.com/johnbean393/Sidekick/pull/23

Just adds a few scripts that download, sign marp, and while I was in there and needing you to specify a team, another script that sets the team in the Xcode project file. Also added a blurb in the readme

for devs with concerns about the marp binary, I think you can just stick anything there and it'll build fine so like touch Sidekick/Logic/View\ Controllers/Tools/Slide\ Studio/Resources/bin/marp you just won't be able to generate the presentation visuals

that's an official binary from the marp team though right? so like it's really a matter of them being compromised or their machine being compromised. for users, I'm not sure if johnbean building marp from scratch is really necessary, as there's already a million points of intrusion on a developer machine. if you consider marp's build's being compromised, then there's just as much chance as the sidekick builds being compromised.

[johnbean393]

Thanks to @mikedg1 , this issue has been resolved as of commit 412ba56.