Josh Rosen
Josh Rosen
Short answer: probably not; you'll likely have to use an older version of `pyparsing`. According to https://github.com/JoshRosen/cmps140_creative_cooking_assistant/blob/master/requirements.txt, it looks like this might have originally used pyparsing 1.5.5, so maybe you...
Hi, I'm an engineer from Databricks who works on Databricks Runtime. In Apache Spark 3.2.0, the signature of the `Utils.fetchFile` method was changed in https://github.com/apache/spark/pull/30945 as part of code cleanup....
Does checking for filesystem closed exceptions completely fix this issue or are we vulnerable to race conditions? Skimming through the [Hadoop DFSClient code](https://github.com/apache/hadoop/blame/2dfa928a201560bc739ff5e4fe29f8bb19188927/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java), it looks like it only throws `Filesystem...
> Thanks for you guys reviews, I think @JoshRosen's way is more appropriate, we better change the implementation to "false negatives", though it may need more efforts to implement it...
@mridulm, it doesn't break local mode because there's a carve-out to preserve the existing behavior in that case: in both places where the `if(serializedOnly` check changes behavior, there's a check...
> QQ @JoshRosen, @alex-balikov - if the expectation is that the variable can be recreated (if missing) at driver - with that being a remote possibility, do we want to...
Hi @sos3k, We investigated your bug report and determined that the root-cause was a latent bug in `UnsafeHashedRelation` that was triggered more frequently following the backport of this PR. PR...
@mridulm, thanks for the ping. `CopyOnWriteArrayList` supports single-threaded writes and concurrent reads. It looks like `externalAccums` is only written from `registerAccumulator`, which is only called from `TaskContext.registerAccumulator`, which is only...
I consider this a work-in-progress until the following TODOs and questions are resolved: - [ ] Does a similar change need to be made in the older client versions, or...