HikariCP icon indicating copy to clipboard operation
HikariCP copied to clipboard

Hibernate generated value (SEQUENCE) violates non-null constraint when saving using a statement.

Open Voyen opened this issue 2 years ago • 0 comments

To start, I am trying to follow this tutorial: https://medium.com/@wahyaumau/boost-jpa-bulk-insert-performance-by-90-3a6232d9068d

When I try to persist 50,000 records of my own using this method, I get the following stack trace:

java.sql.BatchUpdateException: Batch entry 0 INSERT INTO dummy (name) VALUES ('Name 0') was aborted: ERROR: null value in column "id" of relation "dummy" violates not-null constraint
  Detail: Failing row contains (null, Name 0).  Call getNextException to see other errors in the batch.
        at org.postgresql.jdbc.BatchResultHandler.handleCompletion(BatchResultHandler.java:186)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:569)
        at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:881)
        at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:904)
        at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1634)
        at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:127)
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)

The code I'm using to try to replicate the tutorial:

application.yml:

spring:
    jpa:
        hibernate:
            ddl-auto: create
        database-platform: org.hibernate.dialect.PostgreSQLDialect
        properties:
            hibernate:
                jdbc.batch_size: 30
                order_inserts: true
    datasource:
        hikari:
            connectionTimeout: 300000
            maximumPoolSize: 10

Dummy.java

@Data
@Entity
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Table(name = "dummy")
public class DummyEntity {
    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "seqGen")
    @SequenceGenerator(name = "seqGen", sequenceName = "dummy_id_seq", allocationSize = 1)
    private Long id;
    private String name;
}

DummyRepository.java

public interface DummyRepository extends JpaRepository<DummyEntity, Long> { }

Runner:

var dummies = IntStream.range(0, 50000).mapToObj(val -> DummyEntity.builder().name("Name " + val).build()).toList();
String query = "INSERT INTO dummy (name) VALUES (?)";
try (var conn = hikariSource.getConnection(); var stmt = conn.prepareStatement(query)) {
    int counter = 0;
    for (var dummy : dummies) {
        stmt.clearParameters();
        stmt.setString(1, dummy.getName());
        stmt.addBatch();
        if ((counter + 1) % batchSize == 0 || (counter + 1) == dummies.size()) {
            stmt.executeBatch();
            stmt.clearBatch();
        }
        counter++;
    }
} catch (Exception e) {
    e.printStackTrace();
}

This works fine if I change the generation type to IDENTITY, and it also works with either generation type if I don't use the connection/statement directly. Both of the following work with both generation types:

Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for (int i = 0; i < entities.size(); i++) {
    var entity = entities.get(i);
    session.merge(entity);
    if (i % 1000 == 0) {
        session.flush();
        session.clear();
    }
}
tx.commit();
session.close();
dummyRepository.saveAll(dummies);

Is Hikari bypassing Hibernate when using a connection/statement directly or something?

It might be worth noting that I don't build the DB connection itself via the properties file, I do it programmatically using the following config class (I have a need to pull the DB password from AWS Secrets Manager in production for this project):

String connectionTemplate = "jdbc:postgresql://%s:%d/%s";
String host = "postgres";
int port = 5432;
String database = "";
String username = "";
String password = "";

// overriding username/password from secrets manager if on prod profile

String connectionString = String.format(connectionTemplate, host, port, database);
var dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.driverClassName("org.postgresql.Driver");
dataSourceBuilder.url(connectionString);
dataSourceBuilder.username(username);
dataSourceBuilder.password(password);
var dataSource = dataSourceBuilder.build();

Voyen avatar Feb 02 '22 09:02 Voyen