Vladimir Rudnykh
Vladimir Rudnykh
На x64 deb-пакета пока нет, - нужно собирать руками из исходников.
In https://github.com/iterative/datachain/pull/1126 unpickling error was fixed, but it still being stuck in parallel mode and SaaS.
@ilongin thank you for answering all the questions 🙏❤️
> can it be expensive? are we sure it won't be doing some syscalls underneath all these normalize, as_posix - etc In this PR I am also replacing pathlib.Path with...
> anything we can do to do a very basic quick test? e.g. starts with . or .. or /? etc ... something really quick that would tell if we...
I am more worrying about modifying file path (see [tests](https://github.com/iterative/datachain/pull/1110/files#diff-2d6a9fd7e0eb8ccf0311704b0853ab94c27e1282394791e9643f317e5366c08fR372-R397) in this PR). Is this OK to modify path (e.g. `dir/../file.ext` -> `file.ext` or should we raise an exception if...
Also it looks like we should allow absolute paths, it is how it works now for `file://` source. I am going to remove the check for absolute path, but ideally...
> > Is this OK to modify path > > tbh, i think that's fine. why not? File is meant to be an actual precise object more or less /...
One more additional sanity check we can do is to check if `path` is not ends with trailing slash (`/`), which represents directories and `File` model is for file objects.
> a completely alternative option - allow any path at all, don't validate > > validate in prefetch instead (our code that deals with these files) > > let users...