dd-sdk-reactnative icon indicating copy to clipboard operation
dd-sdk-reactnative copied to clipboard

Published 20 hours ago • verified publisher icon DataDog

reverse proxy configuration is not taken into account

Open rfauglas opened this issue 1 year ago • 20 comments

Describe what happened

reverse proxy configuration is not taken into account. I create a poc project, where proxy configuration is done here: https://github.com/rfauglas/datadogrum_reactnative_poc/blob/main/App.tsx#L73 But It never goes through my nginx reverse proxy. Please note I could not find any test in the library against proxy configuration, nor code references...

Steps to reproduce the issue:

  • create an Nginx reverse proxy configuration, similar to this: (here it is with ssl, but I also tried with http...) ` server { listen 443 ssl; listen [::]:443 ssl;

    ssl_certificate /etc/nginx/ssl/server.crt; ssl_certificate_key /etc/nginx/ssl/server.key;

    location / { proxy_pass https://browser-intake-datadoghq.eu:443; } } `

  • setup react native poc as documented in README https://github.com/rfauglas/datadogrum_reactnative_poc

  • launch a reverse proxy

  • trigger some DD events throught the application...

Describe what you expected:

  • I expect to see DD Rum requests go through the proxy, it is not the case (I could validate my proxy configuration is ok with curl command). Proxy configuration seems to be ignored.

Additional context

  • Version of the SDK: used a Pixel_3a_API_34_extension_level_7_x86_64 device running on a Android 14 image using API 34
  • Version of React Native: 0.73.1
  • package.json https://github.com/rfauglas/datadogrum_reactnative_poc/blob/ea7dc89dceb02030eac8f3b4d92aaf7ec1263591/package.json
  • Podfile and Podfile.lock None (Android only)
  • android/build.gradle and android/app/build.gradle android/build.gradle https://github.com/rfauglas/datadogrum_reactnative_poc/blob/main/android/build.gradle

rfauglas avatar Dec 29 '23 10:12 rfauglas

Hi @rfauglas, thanks for reaching out!

Also thanks a lot for including a reproducible example, I've been able to try it out. I have a few hypothesis as to what could be the issue here.

First we need to figure out if the issue comes from the SDK, the device you're using or from your proxy. Do you know yet where your problem is happening? If now, can you answer the following questions:

  • Do you see logs of your requests reaching your proxy?
  • Do you see any log in logcat mentioning a failed request?

If you get an error log like this one (Unexpected response code for CONNECT: 405):

2024-01-03 15:52:58.303 22253-22682 Datadog                 com.datadogrum_reactnative_poc       E  [_dd.sdk_core.default]: Unable to execute the request; we will retry later.
2024-01-03 15:52:58.306 22253-22682 Datadog                 com.datadogrum_reactnative_poc       E  java.io.IOException: Unexpected response code for CONNECT: 405

Then the error is probably on your proxy, your proxy needs to be running on an https server to be able to reach our ingestion backend. In that case I would advise to try out with a real proxy rather than one running on your local maching.

If your error log looks like this (Failed to connect to /127.0.0.1:8000):

2024-01-03 15:52:24.983 22253-22682 Datadog                 com.datadogrum_reactnative_poc       E  [_dd.sdk_core.default]: Unable to execute the request; we will retry later.
2024-01-03 15:52:24.986 22253-22682 Datadog                 com.datadogrum_reactnative_poc       E  java.net.ConnectException: Failed to connect to /127.0.0.1:8000

Then it rather means that your device cannot access the port you're trying to reach (8000 in the example above). If you're using an emulator, running adb reverse tcp:8000 tcp:8000 could solve your issue.

If you're using a local HTTPS proxy, it could also be an issue of your device not trusting this endpoint, make sure you've added the correct certificates to your devices as well.

I hope this helps, if you're not able to fix your issue let me know which error logs you are seeing in the SDK and in your proxy so we can better help you troubleshoot!

Thanks a lot!

louiszawadzki avatar Jan 03 '24 15:01 louiszawadzki

Hey @louiszawadzki ,

Thanks for feedback!1 With a fresh eye and your great feedback , I realised I was referring to a local address in the emulator... 127.0.0.1. After correction I can get my requests:

2024/01/10 11:16:03 [info] 29#29: *30 client sent invalid request while reading client request line, client: 192.168.1.50, server: , request: "CONNECT browser-intake-datadoghq.com:443 HTTP/1.1"
192.168.1.50 - - [10/Jan/2024:11:16:03 +0000] "CONNECT browser-intake-datadoghq.com:443 HTTP/1.1" 400 157 "-" "-"

I committed my proxy code & doc.

On my side, I'm going to check the proxy documentation to see if I have done anything wrong...

rfauglas avatar Jan 10 '24 11:01 rfauglas

To my understanding connect request is used for creating a tunneling connection. I could validate proxy works with a forward proxy like squid. I have to refine my understanding this looks like a forward proxy request... In our target architecture, we will have a forward proxy and a reverse proxy. The pain point is the reverse proxy. I don't quite see how to configure a reverse proxy.

rfauglas avatar Jan 10 '24 13:01 rfauglas

Hi @rfauglas, I'm happy to see that you've made some progress here!

Can you clarify if there is anything we can do to help here?

Thanks!

louiszawadzki avatar Jan 12 '24 10:01 louiszawadzki

Hi @rfauglas, I'm happy to see that you've made some progress here!

Can you clarify if there is anything we can do to help here?

Thanks!

Hello @louiszawadzki , Yes there is ;-) , as mentionned in my last comment: we have the requirement to route all outgoing traffic through an API gaetway (aka reverse proxy). This includes the datadog rum data upload (in my case: browser-intake-datadoghq.com).
I don't know if you have that in your roadmap? If not could you point me to the code where the URL for upload is generated? So that we could submit a PR ? I know the url of the Datadog endpoint is dynamic and depends on the site configuration parameter. (i think there is a similar mechanism in the context of react web) Would a Unit test checking the generated URL enough for change validation?

rfauglas avatar Jan 15 '24 10:01 rfauglas

Hi @rfauglas, thanks for clarifying :)

The URLs for upload are generated on the native iOS and Android SDK side. Both these SDKs enable to specify a customEndpoint for each feature (RUM, Logs, Trace). I've added a task in our roadmap to bring this feature to the React Native SDK, I'll push for it so it makes it in the next release :)

If this is a blocker for you in the meantime, you can edit the native files of the SDK with XCode/Android Studio to specify the custom endpoints and use patch-package to generate a patch. The features are enables in DdSdkImplementation.swift and DdSdkImplementation.kt.

louiszawadzki avatar Jan 16 '24 09:01 louiszawadzki

Hello @louiszawadzki ,

I made further tests. I think I need more explanations and hints for a contribution/patch.

- Error 1

When running: .grradlew build under dd-sdk-reactnative/packages/core/android I get following errors: > Task :lintAnalyzeDebug e: /home/ronan/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.8.10/6d5560a229477df9406943d5feda5618e98eb64c/kotlin-stdlib-1.8.10.jar!/META-INF/kotlin-stdlib.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0. - Error 2 Then later `> Task :testDebugUnitTest OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

DdSdkTest > 𝕄 initialize native with sample rate SDK 𝕎 initialize() {}() FAILED java.lang.IllegalAccessException at DdSdkTest.kt:138 If I rebuild with: ./gradlew test --stacktrace I get error stack:java.lang.IllegalAccessException: Can not set static final java.lang.ThreadLocal field android.view.Choreographer.sThreadInstance to com.datadog.reactnative.DdSdkTestKt$mockChoreographerInstance$1 at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwFinalFieldIllegalAccessException(UnsafeFieldAccessorImpl.java:76) at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwFinalFieldIllegalAccessException(UnsafeFieldAccessorImpl.java:80) at java.base/jdk.internal.reflect.UnsafeQualifiedStaticObjectFieldAccessorImpl.set(UnsafeQualifiedStaticObjectFieldAccessorImpl.java:77) at java.base/java.lang.reflect.Field.set(Field.java:799) at com.datadog.reactnative.DdSdkTestKt.mockChoreographerInstance(DdSdkTest.kt:2560) at com.datadog.reactnative.DdSdkTest.set up(DdSdkTest.kt:138) at jdk.internal.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:126) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptBeforeEachMethod(TimeoutExtension.java:76) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:481) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$18(ClassBasedTestDescriptor.java:466) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:169) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$5(TestMethodTestDescriptor.java:197) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:197) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:166) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:133) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248) at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211) at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132) at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99) at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79) at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33) at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94) at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193) at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129) at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100) at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60) at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56) at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133) at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71) at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69) at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)` Error 3 Please note: I had to change minSdk to 24 at https://github.com/DataDog/dd-sdk-reactnative/blob/3679900ee2b225f3693645dc5fc2bab057a0678d/packages/core/android/build.gradle#L119 If not I get a Kotlin compilation error. Error 4 Please note I had to correct the bash command https://github.com/DataDog/dd-sdk-reactnative/blob/3679900ee2b225f3693645dc5fc2bab057a0678d/packages/react-navigation/fix-react-navigation-import-in-dependencies.sh#L11 It looks like a bsd compatible shell (MacOS?), that fails under Linux (Debian 12).

** proxy question ** I would have an additional question around proxy management. Datadog SDK allows to define a proxy configuration. But in our case, the react native application is deployed on customer networks that have different proxy configurations. On the device, this proxy configuration is done on the Wifi connection. This configuration is not used by the Datadog react native SDK. I suspect this is caused by the underlying http library which bypasses the system proxy settings any idea. Can you confirm this behaviour? Did you ever heard of such requirement. If yes, how was it addressed?

Environment:

java -version
openjdk version "17.0.9" 2023-10-17
OpenJDK Runtime Environment (build 17.0.9+9-Debian-1deb12u1)
OpenJDK 64-Bit Server VM (build 17.0.9+9-Debian-1deb12u1, mixed mode, sharing)

rfauglas avatar Jan 29 '24 08:01 rfauglas

Hi @rfauglas,

Thanks for coming back with all this information! I'll detail each of the errors you encountered:

Error 1: This is due to the AGP version used in the SDK. It will be bumped when we migrate the SDK to RN 0.73 very soon. However this is just a warning that should not prevent you from building nor using the SDK. Error 2: Our test suite wasn't migrated to java 17 yet, I've opened a PR to do the migration Error 3: I've also fixed this in the same PR Error4: Unfortunately we do not actively support building the SDK for contributors on Linux as all our machines and CI run on Mac. If you wish to open a PR for this you are welcome!

Regarding your proxy question I am not sure to fully understand it, could you share a bit more information on your case? Could you give an example of how the proxy is configured for a given customer, and what kind of restrictions exist on the wifi?

Thanks a lot!

louiszawadzki avatar Jan 29 '24 16:01 louiszawadzki

Hi @louiszawadzki Thanks a lot for your answer! Indeed I tried with JDK 11 and tests passed successfully: i can go forward! (I thought of it, but since jdk 11 is pretty old and there is a JDK 11 compatibilty defined in build.gradle).

To give you some context on our network requirements. We're developping a mobile application that will be deployed on customer. Some customers (at least one ;-) ), might have some network restrictions: Requirement 1: force all outgoing traffic to go through a Web Proxy. Requirement 2: All outgoing traffic, there must be some configuration on the proxy to allow traffic. Ex allow traffic to dns entry api.smarway.ai. Requirement 1 is ususally enforced, by defining a proxy on a wifi connection (lets call it System proxy configuration). image The problem is that:

  • An application can not access to the Wifi configuration for security reason (as far as I know this used to be the case).
  • Depending on the library being used in the Android SDK, the wifi proxy will be used or not.

With Datadog RUM the "System proxy" is bypassed. I suppose this is the reason why the dd-sdk-reactnative library supports Proxy configuration. It tried also with nodejs fetch, but It did not go through the System proxy I configured: fetch('https://jsonplaceholder.typicode.com/posts/1') .then(response => response.json()) .then(data => console.log(data)) .catch((error: Error) => { console.error('Error:', error);

Requirement 2: we mentionned it previvously, that would be support for an "api gateway" in the configuration. (what i'm trying to fix)...

Of course these requirements might change from customer to another. This means proxy configuration can not be made at build time. I wanted to know if you ever bumped into such requirement and if yes how you solved it.

rfauglas avatar Jan 30 '24 11:01 rfauglas

Hi @rfauglas,

Thanks for coming back with all these details!

We've recently implemented customEndpoints on the SDK: https://github.com/DataDog/dd-sdk-reactnative/pull/592. It's going to be released soon, maybe it can be used for what you're looking to achieve with API Gateway? We're not advising to use this feature as a way to achieve a proxy since it's not been designed for this purpose, but if it fits your needs I'd be interested to know if you notice any gap in data reported?

From what I understand, I would still recommend you to use the proxy configuration we provide (for instance with address 192.168.1.50 and port 3128 for the example you shared) and then it's a matter of the proxy you target to add support for HTTP CONNECT requests if it does not support them yet. Let me know if I'm missing something here in my understanding.

Finally, regarding how to apply the proxy configuration at run time, as long as you are able to identify whether to apply the proxy or not in the first moments of the app you should be able to use the delayed initialization for the RN SDK to avoid missing RUM events triggered before the initialization.

Let me know if that answers all your questions or if I missed anything!

louiszawadzki avatar Jan 30 '24 12:01 louiszawadzki

And one last thing, we've added documentation on how our Mobile SDK Proxy work, let me know if it helps clarifying things!

louiszawadzki avatar Jan 30 '24 12:01 louiszawadzki

hey @louiszawadzki This endpoint parameter looks very promising to me. It looks like what I expected. I'm happy I didn't have to code it by myself: there is quite some code! I'm going to try it asap using your node server. I suppose configuration is handed over to dd-sdk-android which holds the logic for data upload, so I have to build the dd-android-sdk aar, and then pull the dependency?

rfauglas avatar Jan 30 '24 17:01 rfauglas

Hi @rfauglas,

Unless I missed something there is no need to build the Android SDK on your side, it is already included as a gradle dependency of the React Native SDK. I you want to use an unreleased version of the SDK I would recommend the following:

  • clone the repository
  • run yarn install
  • run yarn workspace @datadog/mobile-react-native prepare
  • run yarn workspace @datadog/mobile-react-native pack
  • Move the created package.tgz to your app directory, and install it with yarn add @datadog/mobile-react-native@file:./package.tgz

Let me know if that answers your question, in any case the new version should come out this week or early next week :)

louiszawadzki avatar Jan 31 '24 09:01 louiszawadzki

Hi @louiszawadzki ,

Thanks, I guess the endpoint configuration is passed over to the dd sdk android, which performs the http requests, receiving configuration through some "generic builder pattern" ?

Anyway, I could build the library (not without issues: I had to deal with 2 JDK on my laptop: JDK 11 for SDK react native and JDK 17 for my poc,with SDKMAN that allows easy switching, I found an JDK 11 Idea that allowed me to build). Now I'm stuck, with a security issue:

2024-01-31 16:52:01.952  9025-9061  Datadog                 com.datadogrum_reactnative_poc       E  java.net.UnknownServiceException: CLEARTEXT communication not enabled for client
                                                                                                    	at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:183)
...
                                                                                                    	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
                                                                                                    	at com.datadog.android.core.internal.data.upload.GzipRequestInterceptor.intercept(GzipRequestInterceptor.kt:64)

Tried to set android:usesCleartextTraffic="true" in the AndroidManifest.xml with no effect. My Virtual device settings are (the blue hightlighted line): image Tried to retrieve the system properties of application using:

 adb shell dumpsys package  com.datadogrum_reactnative_poc
...
Packages:
  Package [com.datadogrum_reactnative_poc] (ab62dd8):
...
    flags=[ HAS_CODE ALLOW_CLEAR_USER_DATA ]
...
    pkgFlags=[ HAS_CODE ALLOW_CLEAR_USER_DATA ]

But I'm not sure these correspond to the manifest parameters. I tried using an https connection, but I had to used self signed certificates that are rejected during the request... Did you bump into similar issues when testing? (In your documentation you mention https://endpoint.datadog.com) If not can you give me your Android image and API Version? If yes can how could you fix it?

I have a few other hints to investigate, like use android port forwarding, and I few other hacks I could find on the Internet. I also had to disabled the Web proxy I did setup , due to a misconfiguration I did not investigate: it used to relay an incoming connection request, as a connection request to the endpoint, which can not handle it, note that for a proxy, Android allows http connections...

Maybe you will be interested to know that (on this android version at least, I think I did some other test with another behaviour on an Android 14, but our target is 11...), the Wifi connection system proxy is automatically used when set (maybe because it wraps java.net requests ?), Which makes the proxy configuration in the application dd sdk reactnative useless.

That's it for my brain dump ;-)

rfauglas avatar Feb 01 '24 15:02 rfauglas

Hi @rfauglas,

We released a customEndpoints configuration option in 2.1.0 version of the SDK. It enables you to specify an endpoint for data to be sent to. For instance if you specify in your configuration:

config.customEndpoints = {
  rum: 'https://my-custom-endpoint.com',
  logs: 'https://my-custom-endpoint.com',
  trace: 'https://my-custom-endpoint.com',
}

Your RUM data will be sent to https://my-custom-endpoint.com/api/v2/rum, your logs data to /api/v2/logs and your trace data to /api/v2/spans.

One important note is that this is not an official alternative to the proxy configuration yet - making it so requires more time and we have more pressing items in our backlog. So if you decide to use it for this purpose keep in mind that:

  • Although it should work well, we haven't tested all scenarios extensively. Feel free to open an issue or a PR if you notice some data isn't reported as expected.
  • As this isn't officially documented the path we send your data to might change in the future. If so, it would be mentioned in the release notes.

I've looked into your last message and I believe it would be better to go through our support team so we can more easily dedicate time from the best suited resources on your case and exchange code samples safely. Would that work for you?

louiszawadzki avatar Feb 05 '24 14:02 louiszawadzki

Hi @louiszawadzki ,

I could (finally!) set a local development environment. I could not configure HTTP requests, so I did setup a configuration with a self-signed certificate. I tried to adapt the index.js you provide to forward the request. But since the Rum Datadog Endpoint API is not documented (or I could not find it?), I still have an issue, when I try to forward the DD Rum request. indeed I get following response

Responce body
index.js:71
{"request_id":"b18a03ba-e353-48fe-a78b-e1fdacd26686","errors":[{"status":"400","title":"Bad Request","detail":"Decompression error"}]}
index.js:72
STATUS: 400

I guess it is beacause the body is decompressed by node in the listener. I'm on it.

rfauglas avatar Feb 05 '24 15:02 rfauglas

could have it work indeed by skipping decompression.

rfauglas avatar Feb 05 '24 15:02 rfauglas

Weird, i can not see my RUM session. Although I have a 202 answer from Datadog:

STATUS: 202
index.js:60
Responce body
index.js:71
{"request_id":"50beba30-1f3f-4770-a221-a877ec90663e"}

rfauglas avatar Feb 05 '24 15:02 rfauglas

Hey @louiszawadzki ,

I could make advances on my [datadog rum request forwarding].(https://github.com/rfauglas/datadogrum_reactnative_poc/commit/3ffaf657acd91cf00153ef8065cb6e4acf4c8cf2).

I have a remaining issue in the datadog intake server answer:

STATUS: 403
index.js:36
Responce body
index.js:47
{"request_id":"2c610e66-75f2-4df9-8f66-63107407fc77","errors":[{"status":"403","title":"Forbidden","detail":"Api key is either forbidden or blacklisted"}]}

Since, the rum application admin interface doesn't show any error, and I can still send data when I disable endpoint, I suppose there is still something missing when I build my request. What surprises me, and could be a hint, is that in the incoming request I can not see the application_id, whether it be on the request headers or on the request URL ( originalURL). Any clue?

rfauglas avatar Feb 07 '24 09:02 rfauglas

Hi @rfauglas, I'm happy to see that you're making progress with your setup!

Indeed you should have a DD-API-KEY header set with your clientToken (see how it is set for RUM in the Android SDK). The best way to troubleshoot this API key issue would be to go through our support team since they will be able to directly look at why your request is failing and reach out to the right people at Datadog :)

Let me know if you managed to make further progress!

louiszawadzki avatar Feb 08 '24 12:02 louiszawadzki