[Using Unity Engine's WWW] How to wait for a download to finish in Arcadia?

[markfarrell]

Hi,

I'm a bit confused about how to use in Arcadia. I want to download a texture from an online museum archive and wait for the download to finish. I'm confused about how to get around using the 'yield' keyword as seen in C# and UnityScript. I was unable to use System.Net.WebClient due to problems with authentication/decryption - am forced to use HTTPS and switching to UnityEngine.WWW 'magically' worked (but now I want to figure out how to wait for downloads to finish).

This might be a workaround, but what would you recommend doing? Will this cause issues?

(defn search
   "Search for art collections; wait until download is finished."
   [query]
   (let [client (WWW. (make-url query))]
      (do
         (some true? (repeatedly #(. client isDone)))) ; wait to finish
         (. client text))) ; text/JSON

[nasser]

Yeah, this is rough. The short answer is you'd be better off writing that logic in C# and using interop to access it.

Unity's WWW class and others depend on coroutines and the yield statement, which ClojureCLR does not support. It will probably never support it either, because the long term solution is to port core.async. That can be integrated with CLR enumerators, which would play nice with Unity's coroutines. Porting core.async is a non-trivial task, though...

Your code is close, but that's not how you want to do it. Unity is single threaded, and that will lock the whole game up until the request finishes, which I suspect is not what you want to do. You need to check for isDone once per frame, e.g. in an Update method

(defcomponent Downloader [^WWW www]
  (Start [this] (set! (.www this) (WWW. query)))
  (Update [this] (if (.isDone www)
                   (done-stuff))))

Not great, but that's the best we can do at this point in pure Clojure.

[timsgardner]

Seems a bit weird to have this lacuna in the interop situation, though, especially since ClojureCLR has such concessions to C# as gen-delegate https://github.com/richhickey/clojure-clr/wiki/CLR-Interop#gen-delegate already. I can't think offhand of a similar you-just-can't-do-that, not-even-nonperformantly thing for ClojureJVM vs Java (even inheritance is supported by proxy, for APIs that demand it); maybe something with Java Enumerations? This isn't an exotic use-case either, big important APIs expect yield. Perhaps we should lobby for a language extension.

• Tims Gardner

On Wed, Jan 7, 2015 at 9:52 PM, Ramsey Nasser [email protected] wrote:

Yeah, this is rough. The short answer is you'd be better off writing that logic in C# and using interop to access it.

Unity's WWW class and others depend on coroutines and the yield statement, which ClojureCLR does not support. It will probably never support it either, because the long term solution is to port core.async. That can be integrated with CLR enumerators, which would play nice with Unity's coroutines. Porting core.async is a non-trivial task, though...

Your code is close, but that's not how you want to do it. Unity is single threaded, and that will lock the whole game up until the request finishes, which I suspect is not what you want to do. You need to check for isDone once per frame, e.g. in an Update method

(defcomponent Downloader [^WWW www](Start [this] %28set! %28.www this%29 %28WWW. query%29%29) (Update [this](if %28.isDone www%29 %28done-stuff%29)))

Not great, but that's the best we can do at this point in pure Clojure.

— Reply to this email directly or view it on GitHub https://github.com/arcadia-unity/Arcadia/issues/98#issuecomment-69128935 .

[nasser]

That's fair. It can be done, but it's a major compiler feature, and risks duplicating effort with the core.async port if not planned correctly.

yield is a feature of the C# compiler, not the CLR. It transforms methods using yield into state machines, like core.async does to go blocks.

If we wanted Clojure fns to be consumable by C# APIs expecting yielding methods, AFunction would need to implement IEnumerator and the MoveNext method, then you'd have to break the fn's IL wherever there was a yield. The syntax would look something like this, maybe

(defn Start [this]
  (let [www (WWW. (.url this))]
    (yield www)
    (set! (.. this renderer material mainTexture) (.texture www)))

Here's a disassembly of Unity's WWW example

using UnityEngine;
using System.Collections;

public class ExampleClass : MonoBehaviour {
    public string url = "http://images.earthcam.com/ec_metros/ourcams/fridays.jpg";
    IEnumerator Start() {
        WWW www = new WWW(url);
        yield return www;
        renderer.material.mainTexture = www.texture;
    }
}

Becomes

public class ExampleClass : MonoBehaviour
{
    //
    // Fields
    //
    public string url = "http://images.earthcam.com/ec_metros/ourcams/fridays.jpg";

    //
    // Methods
    //
    [DebuggerHidden]
    private IEnumerator Start ()
    {
        ExampleClass.<Start>c__Iterator0 <Start>c__Iterator = new ExampleClass.<Start>c__Iterator0 ();
        <Start>c__Iterator.<>f__this = this;
        return <Start>c__Iterator;
    }
}

Where <Start>c__Iterator0 is a compiler generated class

[CompilerGenerated]
private sealed class <Start>c__Iterator0 : IEnumerator<object>, IEnumerator, IDisposable
{
    //
    // Fields
    //
    internal WWW <www>__0;

    internal int $PC;

    internal object $current;

    internal ExampleClass <>f__this;

    //
    // Properties
    //
    object IEnumerator.Current {
        [DebuggerHidden]
        get {
            return this.$current;
        }
    }

    object IEnumerator<object>.Current {
        [DebuggerHidden]
        get {
            return this.$current;
        }
    }

    //
    // Methods
    //
    [DebuggerHidden]
    public void Dispose ()
    {
        this.$PC = -1;
    }

    public bool MoveNext ()
    {
        uint num = (uint)this.$PC;
        this.$PC = -1;
        switch (num) {
        case 0:
            this.<www>__0 = new WWW (this.<>f__this.url);
            this.$current = this.<www>__0;
            this.$PC = 1;
            return true;
        case 1:
            this.<>f__this.get_renderer ().get_material ().set_mainTexture (this.<www>__0.get_texture ());
            this.$PC = -1;
            break;
        }
        return false;
    }

    [DebuggerHidden]
    public void Reset ()
    {
        throw new NotSupportedException ();
    }
}

[nasser]

So here's something that works in pure Clojure.

(ns corotest
  (:use arcadia.core)
  (:import IEnumerator
           Timeline
           [UnityEngine Debug WaitForSeconds]))

(defcomponent CorotestReify []
  (Start [this]
         (let [^WaitForSeconds wfs (WaitForSeconds. 5)]
           (.. this
               (StartCoroutine (reify
                                 IEnumerator
                                 (MoveNext [this]
                                           (Debug/Log "Tick")
                                           true)
                                 (get_Current [this]
                                              wfs)))))))

A coroutine in Unity is just a stateful IEnumerator. StartCoroutine puts it into an internal coroutine list, and internal machinery checks this list by calling get_Current on each coroutine and MoveNext to advance it if it is ready to move on. This Unity Answers conversation has some more details and examples.

So we can fake it with reify. I'm going to try and use a mutable local array to store state between calls to MoveNext. We could conceivably write macros that did the state-machine generation for you, but we'd still need compiler support do that they worked everywhere (i.e. where ever fns are used, e.g. in defcomponent/deftype methods).

[nasser]

Started to play around with this https://gist.github.com/nasser/3ba82302d1271c4e1890bf85b51ada77. Could make sense in core.