data-access icon indicating copy to clipboard operation
data-access copied to clipboard

Published 20 hours ago • verified publisher icon pentaho

[BACKLOG-6461] Oracle repository: Impossible to create CSV datasource…

Open ivanpogodin opened this issue 8 years ago • 32 comments

… where column names in csv contain - and / which get converted to text, HYPHEN and DIVIDED_BY, causing column name in table to be too long [ORA-00972 - Identifier too long].

ivanpogodin avatar Apr 14 '16 09:04 ivanpogodin

@mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin avatar Apr 14 '16 09:04 ivanpogodin

Build Commands:

Build:

  1. ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

  • Complexity Change : + 1.64%
  • Instruction Change : + .70%
  • Line Change : + .83%
  • Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 30.30%
  • Complexity Change : + 7.78%
  • Instruction Change : + 21.32%
  • Line Change : + 22.26%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : -.18%
  • Instruction Change : -.03%
  • Line Change : -.05%
  • Method Change : -.91%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : -50.51%
  • Complexity Change : -41.11%
  • Instruction Change : -42.84%
  • Line Change : -37.95%
  • Method Change : -13.89%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : + .53%
  • Instruction Change : + .09%
  • Line Change : + .11%
  • Method Change : + .60%

brosander avatar Apr 14 '16 10:04 brosander

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it? The code coverage is improved

ivanpogodin avatar Apr 15 '16 14:04 ivanpogodin

@mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin avatar Apr 19 '16 12:04 ivanpogodin

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin avatar Apr 20 '16 12:04 ivanpogodin

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin avatar Apr 21 '16 12:04 ivanpogodin

@brosander , @mdamour1976 , @pentaho-nbaker , could you review it?

ivanpogodin avatar Apr 22 '16 12:04 ivanpogodin

Build Commands:

Build:

  1. ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

  • Complexity Change : + 1.64%
  • Instruction Change : + .70%
  • Line Change : + .83%
  • Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 31.25%
  • Complexity Change : + 10.61%
  • Instruction Change : + 21.11%
  • Line Change : + 21.91%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : -.18%
  • Instruction Change : -.03%
  • Line Change : -.05%
  • Method Change : -.91%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : -2.78%
  • Complexity Change : -8.59%
  • Instruction Change : -3.39%
  • Line Change : -.17%
  • Method Change : + 2.78%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : + .53%
  • Instruction Change : + .09%
  • Line Change : + .11%
  • Method Change : + .60%

brosander avatar Apr 26 '16 09:04 brosander

CsvTransformGenerator Integration test coverage has dropped significantly. Can you review please, @ivanpogodin ?

pamval avatar Apr 26 '16 09:04 pamval

@pamval , could you review it, please?

ivanpogodin avatar Apr 28 '16 13:04 ivanpogodin

Build Commands:

Build:

  1. ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

  • Complexity Change : + 1.64%
  • Instruction Change : + .70%
  • Line Change : + .83%
  • Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 32.81%
  • Complexity Change : + 12.88%
  • Instruction Change : + 21.11%
  • Line Change : + 21.91%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Branch Change : + 4.00%
  • Complexity Change : + 3.48%
  • Instruction Change : + 1.38%
  • Line Change : + 1.74%
  • Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + .35%
  • Complexity Change : -4.04%
  • Instruction Change : -1.61%
  • Line Change : + .91%
  • Method Change : + 2.78%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : + .53%
  • Instruction Change : + .09%
  • Line Change : + .11%
  • Method Change : + .60%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

  • Branch Change : + 50.00%
  • Complexity Change : + 3.45%
  • Instruction Change : + 9.03%
  • Line Change : + 5.36%
  • Method Change : + 3.70%

wingman-pentaho avatar Apr 28 '16 14:04 wingman-pentaho

@mdamour1976 @e-cuellar, can you take a look at this one ? thanks!

pamval avatar Apr 28 '16 14:04 pamval

@pmalves The pull request failed some checks at Wingman...

e-cuellar avatar Apr 28 '16 16:04 e-cuellar

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin avatar Apr 29 '16 11:04 ivanpogodin

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin avatar May 04 '16 06:05 ivanpogodin

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin avatar May 05 '16 12:05 ivanpogodin

Build Commands:

Build:

  1. ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Newly Broken Tests:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGeneratorTest.testGetMaxColumnNameLength:

java.lang.NullPointerException

java.lang.NullPointerException
    at org.pentaho.di.core.database.Database.getObjectName(Database.java:4507)
    at org.pentaho.di.core.logging.LoggingObject.grabLoggingObjectInformation(LoggingObject.java:144)
    at org.pentaho.di.core.logging.LoggingObject.<init>(LoggingObject.java:56)
    at org.pentaho.di.core.logging.LoggingRegistry.registerLoggingSource(LoggingRegistry.java:65)
    at org.pentaho.di.core.logging.LogChannel.<init>(LogChannel.java:58)
    at org.pentaho.di.core.database.Database.<init>(Data

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

  • Complexity Change : + 1.64%
  • Instruction Change : + .70%
  • Line Change : + .83%
  • Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 40.62%
  • Complexity Change : + 17.42%
  • Instruction Change : + 26.32%
  • Line Change : + 28.39%
  • Method Change : + 8.33%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Branch Change : + 4.00%
  • Complexity Change : + 3.48%
  • Instruction Change : + 1.38%
  • Line Change : + 1.74%
  • Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 3.47%
  • Complexity Change : + 2.78%
  • Instruction Change : + .16%
  • Line Change : + 4.69%
  • Method Change : + 11.11%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : + 1.75%
  • Instruction Change : + 1.03%
  • Line Change : + 1.01%
  • Method Change : + 3.73%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

  • Branch Change : + 50.00%
  • Complexity Change : + 3.45%
  • Instruction Change : + 9.03%
  • Line Change : + 5.36%
  • Method Change : + 3.70%

wingman-pentaho avatar May 05 '16 12:05 wingman-pentaho

@ivanpogodin, broken test in wingman. Can you take a look ?

pamval avatar May 05 '16 12:05 pamval

Build Commands:

Build:

  1. ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Newly Broken Tests:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGeneratorTest.testGetMaxColumnNameLength:

java.lang.NullPointerException

java.lang.NullPointerException
    at org.pentaho.di.core.database.Database.getObjectName(Database.java:4507)
    at org.pentaho.di.core.logging.LoggingObject.grabLoggingObjectInformation(LoggingObject.java:144)
    at org.pentaho.di.core.logging.LoggingObject.<init>(LoggingObject.java:56)
    at org.pentaho.di.core.logging.LoggingRegistry.registerLoggingSource(LoggingRegistry.java:65)
    at org.pentaho.di.core.logging.LogChannel.<init>(LogChannel.java:58)
    at org.pentaho.di.core.database.Database.<init>(Data

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

  • Complexity Change : + 1.64%
  • Instruction Change : + .70%
  • Line Change : + .83%
  • Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 40.62%
  • Complexity Change : + 17.42%
  • Instruction Change : + 26.32%
  • Line Change : + 28.39%
  • Method Change : + 8.33%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Branch Change : + 4.00%
  • Complexity Change : + 3.48%
  • Instruction Change : + 1.38%
  • Line Change : + 1.74%
  • Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 3.47%
  • Complexity Change : + 2.78%
  • Instruction Change : + .16%
  • Line Change : + 4.69%
  • Method Change : + 11.11%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : + 1.75%
  • Instruction Change : + 1.03%
  • Line Change : + 1.01%
  • Method Change : + 3.73%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

  • Branch Change : + 50.00%
  • Complexity Change : + 3.45%
  • Instruction Change : + 9.03%
  • Line Change : + 5.36%
  • Method Change : + 3.70%

wingman-pentaho avatar May 06 '16 11:05 wingman-pentaho

Build Commands:

Build:

  1. ant -Dtestreports.dir=bin/reports/unit-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml clean-all resolve jacoco && ant -Dpentaho.coding.standards.version=1.0.4 -Dtestreports.dir=bin/reports/integration-test -Djunit.jvmargs=-XX:PermSize=256m -f build.xml jacoco-integration checkstyle

Cleanup:

rm -r ~/.ivy2/local || echo no publish local to remove

Changed files:

src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGenerator.java src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGenerator.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorIT.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/CsvTransformGeneratorTest.java test-src/org/pentaho/platform/dataaccess/datasource/wizard/service/agile/StagingTransformGeneratorTest.java

Unit test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.models.ModelInfo:

  • Complexity Change : + 1.64%
  • Instruction Change : + .70%
  • Line Change : + .83%
  • Method Change : + 4.00%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 45.31%
  • Complexity Change : + 19.70%
  • Instruction Change : + 31.05%
  • Line Change : + 31.64%
  • Method Change : + 8.33%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Branch Change : + 4.00%
  • Complexity Change : + 3.48%
  • Instruction Change : + 1.38%
  • Line Change : + 1.74%
  • Method Change : + 2.22%

Integration test coverage change:

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.CsvTransformGenerator:

  • Branch Change : + 3.47%
  • Complexity Change : + 2.78%
  • Instruction Change : + .16%
  • Line Change : + 4.69%
  • Method Change : + 11.11%

org.pentaho.platform.dataaccess.datasource.wizard.service.agile.StagingTransformGenerator:

  • Complexity Change : + 1.75%
  • Instruction Change : + 1.03%
  • Line Change : + 1.01%
  • Method Change : + 3.73%

org.pentaho.platform.dataaccess.datasource.wizard.sources.csv.FileTransformStats:

  • Branch Change : + 50.00%
  • Complexity Change : + 3.45%
  • Instruction Change : + 9.03%
  • Line Change : + 5.36%
  • Method Change : + 3.70%

wingman-pentaho avatar May 06 '16 13:05 wingman-pentaho

@pamval , @mdamour1976 , @e-cuellar , could you review this, please?

ivanpogodin avatar May 06 '16 13:05 ivanpogodin

@pamval , @mdamour1976 , @e-cuellar , could you review this, please? All checks have passed

ivanpogodin avatar May 10 '16 11:05 ivanpogodin

@e-cuellar can you take a look ?

pamval avatar May 11 '16 10:05 pamval

:x: Build failed in 11s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

Pull request 727 : Not mergeable at 5631d0972de814b0b5e26712ffc32650dd4482c2+b38d779adab7f3d0b40842f599ed44d7bf922633 (NOT_MERGEABLE)

:information_source: This is an automatic message

buildguy avatar Nov 06 '19 13:11 buildguy

:x: Build failed in 10s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

/build/wingman/workspace/maintenance/wingman/builds/pentaho.data-access.PR-727/pom.xml does not exist.

:information_source: This is an automatic message

buildguy avatar Jan 04 '20 17:01 buildguy

:x: Build failed in 10s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

/build/wingman/workspace/maintenance/wingman/builds/pentaho.data-access.PR-727/pom.xml does not exist.

:information_source: This is an automatic message

buildguy avatar Jan 08 '20 21:01 buildguy

:x: Build failed in 9s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

/build/wingman/workspace/maintenance/wingman/builds/pentaho.data-access.PR-727/pom.xml does not exist.

:information_source: This is an automatic message

buildguy avatar Jan 17 '20 05:01 buildguy

:x: Build failed in 9s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

Pull request 727 : Not mergeable at 5631d0972de814b0b5e26712ffc32650dd4482c2+b38d779adab7f3d0b40842f599ed44d7bf922633 (NOT_MERGEABLE)

:information_source: This is an automatic message

buildguy avatar Jan 23 '20 17:01 buildguy

:x: Build failed in 14s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

/build/wingman/workspace/maintenance/wingman/builds/pentaho.data-access.PR-727/pom.xml does not exist.

:information_source: This is an automatic message

buildguy avatar Feb 04 '20 17:02 buildguy

:x: Build failed in 1m 27s

Build command:

mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!

Errors:

Filtered log (click to expand) 

/build/wingman/workspace/maintenance/wingman/builds/pentaho.data-access.PR-727/pom.xml does not exist.

:information_source: This is an automatic message

buildguy avatar Mar 31 '20 12:03 buildguy