hadoop-lzo icon indicating copy to clipboard operation
hadoop-lzo copied to clipboard

Full build instructions for windows 10

Open C-h-e-r-r-y opened this issue 4 years ago • 1 comments

I am trying to buil lzo with maven and have some issues. They may be obvious for C++ developers but not for others (like me). Here list of issues:

On Windows, you can build lzo2.dll with this command: B\win64\vc_dll.bat B\win64\vc_dll.bat command should be invoked from "Visual C++ Command Prompt" not simple cmd.exe Now let's build hadoop-lzo.

  1. OS.PLATFORM variable is not resolved. E.g, I got target/native/Windows_NT-${env.PLATFORM} folder. From experiments I figured out that default value is Win32. The value Win64 as I expected does not work.
  2. Line <exec executable="sh" in mycase have be modified to <exec executable="C:\Program Files\Git\git-bash.exe" because windows do not have sh by default. I think sh should be extracted to some maven property - so every body can set sh implementation.
  3. MSBuild should be isntalled and added to PATH system variable before run mvn cllean install
  4. After all of that I got error warning MSB4078: project file "gplcompression.vcxproj" is not supported by programm MSBuild (Error message translated) The full error log is:
C:\work\IdeaProjects\git\hadoop-lzo>mvn clean install -f pom.xml
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------< com.hadoop.gplcompression:hadoop-lzo >----------------
[INFO] Building hadoop-lzo 0.4.21-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-lzo ---
[INFO] Deleting C:\work\IdeaProjects\git\hadoop-lzo\target
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (check-platform) @ hadoop-lzo ---
[INFO] Executing tasks

check-platform:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (set-props-non-win) @ hadoop-lzo ---
[INFO] Executing tasks

set-props-non-win:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (set-props-win) @ hadoop-lzo ---
[INFO] Executing tasks

set-props-win:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-lzo ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\work\IdeaProjects\git\hadoop-lzo\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ hadoop-lzo ---
[INFO] Compiling 25 source files to C:\work\IdeaProjects\git\hadoop-lzo\target\classes
[WARNING] bootstrap class path not set in conjunction with -source 1.6
C:\work\IdeaProjects\git\hadoop-lzo\src\main\java\com\hadoop\compression\lzo\LzoIndexer.java:[82,18] [deprecation] isDir() in FileStatus has been deprecated
[WARNING] C:\work\IdeaProjects\git\hadoop-lzo\src\main\java\com\hadoop\compression\lzo\DistributedLzoIndexer.java:[52,20] [deprecation] isDir() in FileStatus has been deprecated
[WARNING] C:\work\IdeaProjects\git\hadoop-lzo\src\main\java\com\hadoop\compression\lzo\DistributedLzoIndexer.java:[112,14] [deprecation] Job(Configuration) in Job has been deprecated
[WARNING] C:\work\IdeaProjects\git\hadoop-lzo\src\main\java\com\hadoop\mapreduce\LzoIndexOutputFormat.java:[31,28] [deprecation] cleanupJob(JobContext) in OutputCommitter has been deprecated
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (build-info-non-win) @ hadoop-lzo ---
[INFO] Executing tasks

build-info-non-win:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (build-info-win) @ hadoop-lzo ---
[INFO] Executing tasks

build-info-win:
[propertyfile] Creating new property file: C:\work\IdeaProjects\git\hadoop-lzo\target\classes\hadoop-lzo-build.properties
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (check-native-uptodate-non-win) @ hadoop-lzo ---
[INFO] Executing tasks

check-native-uptodate-non-win:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (check-native-uptodate-win) @ hadoop-lzo ---
[INFO] Executing tasks

check-native-uptodate-win:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (build-native-non-win) @ hadoop-lzo ---
[INFO] Executing tasks

build-native-non-win:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (build-native-win) @ hadoop-lzo ---
[INFO] Executing tasks

build-native-win:
    [mkdir] Created dir: C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\lib
    [mkdir] Created dir: C:\work\IdeaProjects\git\hadoop-lzo\target\classes\native\Windows_NT-Win32\lib
    [mkdir] Created dir: C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\src\com\hadoop\compression\lzo
    [javah] [Forcefully writing file RegularFileObject[C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\src\com\hadoop\compression\lzo\com_hadoop_compression_lzo_LzoCompressor.h]]
    [javah] [Forcefully writing file RegularFileObject[C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\src\com\hadoop\compression\lzo\com_hadoop_compression_lzo_LzoCompressor_CompressionStrategy.h]]
    [javah] [Forcefully writing file RegularFileObject[C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\src\com\hadoop\compression\lzo\com_hadoop_compression_lzo_LzoDecompressor.h]]
    [javah] [Forcefully writing file RegularFileObject[C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\src\com\hadoop\compression\lzo\com_hadoop_compression_lzo_LzoDecompressor_CompressionStrategy.h]]
     [exec] Выполняется последовательная сборка проектов в этом решении. Чтобы включить параллельную сборку, добавьте параметр "-m".
     [exec] Сборка начата 29.04.2020 23:53:55.
     [exec] Проект "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.sln" в узле 1 (целевые объекты по умолчанию).
     [exec] ValidateSolutionConfiguration:
     [exec]   Создание конфигурации решения "Release|Win32".
     [exec] Проект "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.sln" (1) выполняет сборку "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj" (2) в узле 1 (целевые объекты по умолчанию).
     [exec] C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj : warning MSB4078: файл проекта "gplcompression.vcxproj" не поддерживается программой MSBuild. Построение невозможно.
     [exec] Сборка проекта "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj" завершена (целевые объекты по умолчанию).
     [exec] Сборка проекта "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.sln" завершена (целевые объекты по умолчанию).
     [exec]
     [exec] Сборка успешно завершена.
     [exec]
     [exec] "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.sln" (целевой объект по умолчанию) (1) ->
     [exec] "C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj" (целевой объект по умолчанию) (2) ->
     [exec] (Целевой объект Build) ->
     [exec]   C:\work\IdeaProjects\git\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj : warning MSB4078: файл проекта "gplcompression.vcxproj" не поддерживается программой MSBuild. Построение невозможно.
     [exec]
     [exec]     Предупреждений: 1
     [exec]     Ошибок: 0
     [exec]
     [exec] Прошло времени 00:00:00.20
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.150 s
[INFO] Finished at: 2020-04-29T23:53:55+07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (build-native-win) on project hadoop-lzo: An Ant BuildException has occured: Warning: Could not find file C:\work\IdeaProjects\git\hadoop-lzo\target\native\Windows_NT-Win32\gplcompression.dll to copy.
[ERROR] around Ant part ...<copy file="${build.native}/gplcompression.dll" todir="${build.native}/lib"/>... @ 19:80 in C:\work\IdeaProjects\git\hadoop-lzo\target\antrun\build-build-native-win.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Could you please help? P.S. I can create pull request with required changes to readme.md but it is needed to make it buildable from start to end first.

C-h-e-r-r-y avatar Apr 29 '20 17:04 C-h-e-r-r-y