[Feature] Array2D operations

[ghost]

Description

2d Array matrix operations not supported

Repro code

member inline m.ToArray2D() =
    match m with
    | Matrix m -> m
    | ZeroMatrix _ -> Array2D.zeroCreate 0 0

Expected and actual results

This code and other 2d array operators should be able to work

[alfonsogarciacaro]

It's out of the scope of this project to support the full .NET BCL. You can see what is supported by checking the docs or the tests in this repositories.

But of course if you want to add a feature we're open to contributions :+1:

[pkese]

@alfonsogarciacaro I'm looking into implementing these multidimensional arrays.

Actually, the bigger picture is that I'm checking if DiffSharp could be ported to Fable, and a requirement for that are multidimensional arrays.

Although there is an option to hack some partial array support and fiddle with the DiffSharp library a bit, in the end that doesn't buy much, because anyone who would wish to actually use DiffSharp from Fable, would need multidimensional arrays to feed their data into DiffSharp.

Same would apply for any other matrix maths or machine learning library, even if we wish to make Fable bindings for TensorflowJS rather than porting DiffSharp.

So I've been looking at what's missing and have some issues with finding my way around:

1. I don't quite understand at what compilation stage do arrays get specialized into plain old JS arrays vs JS TypedArrays

2. I need to implement array.Rank and array.GetLength(i) but I'm getting compile errors.

Here is a sample (at the bottom is a sample DiffSharp code using the array):

[<AutoOpen>]
module ArrayExtensions =
    type Array with
        member this.GetLength(i) = 42
        member this.Rank with get () = 1

let arrayShape (a:System.Array) =
    if a.Length = 0 then [||]
    else Array.init a.Rank (fun i -> a.GetLength(i))

arrayShape [|1;2;3|]

The error reported is Can not resolve System.Array.get_Rank or System.Array.GetLength.

So my first question would be where to define these System.Array extension methods?

On a larger scale, my approach would be to:

• keep the 1-dimensional array as is and implement additional N-dimensional array (some metadata like bases[] and lengths[] would be attached when creating these JS arrays)
• use the N-dimensional array when 2 or more dimensions are needed
• possibly add some polyfills to plain 1-dimensional array to implement full N-dimensional System.Array functionality (maybe just add some functions inside Array prototype).

Would you (or anyone else) be willing to mentor me at this challenge, maybe write some placeholders, or help in any other way?

[pkese]

Nah,

I spent some time looking at the code, but it turned out there are bits and pieces of array functionality scattered around all over the place and it's difficult to see where to start. I don't think I can do this on my own - or iow, it's probably beyond my capabilities.

Any help would be appreciated.

[pkese]

So @MangelMaxime suggested that @alfonsogarciacaro or @ncave might be able to provide some guidance.

And I actually even got a thumbs up from Alfonso on the previous comment, but now I'm not sure if was meant to support me to give up or keep on going... 😊

I was thinking that a possible start could be to point me to an old pull request that implemented a similar piece of functionality across the stack and possibly give some hints and pitfalls. If any such similar patch-set exists...

Anyways, I'm asking for some feedback or help.

[alfonsogarciacaro]

Well, as commented above I'm usually reluctant to increase support for the BCL because this never ends and even when someone else contributes the feature, there are always overloads, special cases... not covered that I need to fix and maintain. My preferred way is Fable-compatible libraries that someone else maintains :)

That said, we tried to make it easier to add code to the fable-library directly in F# without much JS/TS interop or compiler magic. For example, this file is supposed to replace calls to classes in the System.Text namespace (although right now there are only a few methods for StringBuilder): https://github.com/fable-compiler/Fable/blob/fc93927e882024f0bb6e98c5aad81ea2594e1825/src/fable-library/System.Text.fs

Then you only need to add a line like this in the Replacements module: https://github.com/fable-compiler/Fable/blob/fc93927e882024f0bb6e98c5aad81ea2594e1825/src/Fable.Transforms/Replacements.fs#L2878

So you can try to do something similar for Array2D (System.Array is trickier because it involves a bit more of compiler magic).

If you send a PR with some tests with the methods you need from System.Array and a couple of them for Array2D, I can try to prepare the structure to give you a head start. See for example this PR: https://github.com/fable-compiler/Fable/pull/2057/files#diff-cd91ec8228db0e79eb845cd229038c1c

[jwosty]

With regards to this problem that you speak of (which I am running into recently), does Fable have a way to "proxy" already existing .Net types, a la WebSharper's proxies? Or is the only option (other than hardcoding these things into the fable library) to create new types?

[alfonsogarciacaro]

Unfortunately it's not possible in Fable 2, although I'm currently considering ways to make it possible for users to add their own replacements (or "proxies" to their projects).

[jwosty]

That would be a massive plus for me. That way it wouldn't be a blocker for Fable to not implement some particular part of the BCL that I want to use. It would also introduce a pipeline for Fable users' implementations to eventually make it into core Fable. Definitely a win.

[alfonsogarciacaro]

Fable 3 supports plugins, though they're not currently used for extending BCL support. In principle it should be possible to adapt the plugins to enable a mechanism similar to Webpack proxies, if this is desired please reopen a new discuss about that. Although my personal preference is to have Fable-compatible libraries instead of trying to increase BCL support which usually has many caveats.

[alfonsogarciacaro]

Reopening, I thought this was from the BCL but it's from FSharp.Core so we should likely support it. We can just take the implementation from FSharpCore and tests.

[zprobinson]

I know this is old, but I'm giving it a look to see if this is something that I could work on.

I'm running into some technical challenges regarding the usage of IL in some of the FSharpCore implementation. Additionally, the fact that array2D can handle different bases for the contained arrays requires a GetLowerBound function (on the base Array type) from the dotnet runtime that I'm having trouble reconciling.

Might require some cleverness.

[MangelMaxime]

I'm running into some technical challenges regarding the usage of IL in some of the FSharpCore implementation.

Fable doesn't emit IL, so we mimic the behaviour directly in the target language.

If you need GetLowerBound you can add it to the Array.fs file of the corresponding fable-library folder (fable-library, fable-library-rust, etc.)