llvm icon indicating copy to clipboard operation
llvm copied to clipboard

Ease support of passing quoted arguments to host compiler

Open dvrogozh opened this issue 1 year ago • 1 comments

Some projects are using dpc++ compiler only for the sycl code and build the rest of c++ sources with regular clang or gcc compilers. For that such projects are using -fsycl-host-compiler={cxx} to specify host compiler and -fsycl-host-compiler-options="{cflags} to pass arguments to host compiler. Unfortunately you need to apply extensive escaping to be able to pass quoted arguments to dpc++ compiler. Please, help to improve this. Expectation is that single escaping should be enough. Such as:

-fsycl-host-compiler-options="<args...>  -DMY_DEF=\"xxx\"    <args...>"

Pytorch is one of such projects which passes quoted arguments to host compiler. This resulted into the following escape sequence:

# escaping quoted arguments to pass them thru icpx
host_cflags = [item.replace('"', '\\\\"') for item in host_cflags]
host_cflags = ' '.join(host_cflags)
# note that -fsycl-host-compiler-options will be quoted as needed
# with shlex.quote below
sycl_cflags += [f'-fsycl-host-compiler={host_cxx}', f'-fsycl-host-compiler-options={host_cflags}']

See the following PR where we've met with this problem on pytorch side:

  • https://github.com/pytorch/pytorch/pull/132945

CC: @mdtoguchi

dvrogozh avatar Aug 08 '24 17:08 dvrogozh

There is another place where I noticed different handling of quoted arguments between dpc++ and intel/llvm. Here: https://github.com/intel/torch-xpu-ops/blob/e0408741414624dc266849236b6bab1ea2720b96/cmake/Modules/FindSYCL.cmake#L412

I applied this to be able to build with intel/llvm:

-      -Xs "\"${SYCL_OFFLINE_COMPILER_FLAGS}\""
+      -Xs "${SYCL_OFFLINE_COMPILER_FLAGS}"

dvrogozh avatar Aug 22 '24 19:08 dvrogozh

Is there any update on this issue? We've noticed backward incompatible change in dpc++ compiler version 2025.2 on handling quoted arguments. It was reported in:

  • https://github.com/intel/torch-xpu-ops/issues/1938

We are currently handling that by checking compiler version and treating them differently depending on the version:

  • https://github.com/pytorch/pytorch/pull/161012

dvrogozh avatar Aug 22 '25 21:08 dvrogozh

@dvrogozh, the changes done for 2025.2 to address the differences between the intel/llvm and the oneAPI compiler when processing the quoted arguments. If you are seeing any additional differences that need to be addressed, we can address those moving forward. If the changes are in line with expectations from this issue, we can close accordingly.

mdtoguchi avatar Aug 22 '25 23:08 mdtoguchi

Thank you for the confirmation on the change. I believe we can close now this issue as we have a note on the change.

dvrogozh avatar Aug 23 '25 00:08 dvrogozh