influxdb-client-java icon indicating copy to clipboard operation
influxdb-client-java copied to clipboard
verified publisher icon influxdata

Escaping issue with tag values

Open oliverrahner opened this issue 1 year ago • 2 comments

This issue occurs in our case because a user has entered specific meta data fields as just \. According to the Influx docs, \ does not need to be escaped, but may be. In our case, however, the backslash is followed with a comma, because it is at the end of the tag value, followed by another tag value, leading to messed up interpretation of the whole record.

In my opinion, the safest way to handle this is to always escape backslashes.

Steps to reproduce: List the minimal actions needed to reproduce the behavior.

  1. Create as List<> with at least one tag containing just a backslash \
  2. Write this list to InfluxDB using client.getWriteApiBlocking().writeMeasurements(WriteParameters.DEFAULT_WRITE_PRECISION, myList);

Expected behavior: Data would be inserted correctly.

Actual behavior: Influx returns: HTTP status code: 400; Message: unable to parse 'metrics,account.companyName=\,account.contactCity=\ [...]

Specifications:

  • Client Version: 6.7.0
  • InfluxDB Version: 3.7
  • JDK Version: 17
  • Platform: Linux/Docker/openjdk:17-alpine

oliverrahner avatar May 23 '24 11:05 oliverrahner

Hi @oliverrahner,

Thank you for using our client and reaching out with your issue.

To help us better understand and address the problem related to the HTTP status code: 400; Message: unable to parse 'metrics,account.companyName=\\,account.c', could you please provide a runnable piece of code that replicates this error? This will allow us to see the issue firsthand and work on an accurate solution.

Furthermore, if you have any improvements or fixes in mind, we encourage you to contribute. We welcome Pull Requests from our community and are always ready to review and integrate useful submissions.

Thank you in advance for your cooperation and contribution. We look forward to resolving this together.

Best Regards

bednar avatar May 23 '24 11:05 bednar

This test should contain it all:

package de.dke.data.agrirouter.metricspump;

import com.influxdb.annotations.Column;
import com.influxdb.annotations.Measurement;
import com.influxdb.client.InfluxDBClientFactory;
import com.influxdb.client.InfluxDBClientOptions;
import com.influxdb.client.domain.WritePrecision;
import okhttp3.OkHttpClient;
import org.junit.jupiter.api.Test;

import java.time.Instant;
import java.util.concurrent.TimeUnit;

public class InfluxBugTest {

    private final String influxUrl = "http://localhost:8086";
    private final char[] influxToken = "6c751cea-e8dc-408a-be01-49d1fbbb02a3".toCharArray();
    private final String influxOrg = "org";
    private final String influxBucket = "test";

    @Measurement(name = "metrics")
    private class Metric {
        @Column
        private String id = "";

        @Column(tag = true, name = "account.companyName")
        private String companyName = "";

        @Column(tag = true, name = "account.contactFirstName")
        private String firstName = "";

        @Column(tag = true, name = "account.contactLastName")
        private String lastName = "";

        @Column(timestamp = true)
        private Instant createdOn = Instant.now();
    }

    @Test
    void triggerInfluxBug() {
        Metric metric = new Metric();
        metric.id = "1";
        metric.companyName = "\\";
        metric.firstName = "\\";
        metric.lastName = "\\";
        metric.createdOn = Instant.now();

        var okHttpClient = new OkHttpClient.Builder()
                .readTimeout(1, TimeUnit.MINUTES);
        var options = InfluxDBClientOptions.builder()
                .url(influxUrl)
                .authenticateToken(influxToken)
                .okHttpClient(okHttpClient)
                .org(influxOrg)
                .bucket(influxBucket)
                .build();
        var influxDBClient = InfluxDBClientFactory.create(options);
        if (!influxDBClient.ping()) {
            throw new RuntimeException("Could not connect to InfluxDB");
        }

        influxDBClient.getWriteApiBlocking().writeMeasurement(WritePrecision.MS, metric);

    }

}

The exception is:

com.influxdb.exceptions.BadRequestException: HTTP status code: 400; Message: unable to parse 'metrics,account.companyName=\,account.contactFirstName=\,account.contactLastName=\ id="1" 1716476888462': invalid tag format

	at com.influxdb.internal.AbstractRestClient.responseToError(AbstractRestClient.java:113)
	at com.influxdb.internal.AbstractRestClient.execute(AbstractRestClient.java:84)
	at com.influxdb.client.internal.AbstractWriteBlockingClient.write(AbstractWriteBlockingClient.java:90)
	at com.influxdb.client.internal.WriteApiBlockingImpl.writeMeasurements(WriteApiBlockingImpl.java:253)
	at com.influxdb.client.internal.WriteApiBlockingImpl.writeMeasurement(WriteApiBlockingImpl.java:220)
	at com.influxdb.client.internal.WriteApiBlockingImpl.writeMeasurement(WriteApiBlockingImpl.java:207)
	at com.influxdb.client.internal.WriteApiBlockingImpl.writeMeasurement(WriteApiBlockingImpl.java:198)
	at de.dke.data.agrirouter.metricspump.InfluxBugTest.triggerInfluxBug(InfluxBugTest.java:65)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

oliverrahner avatar May 23 '24 15:05 oliverrahner