Type instances not mutated when using member functions


#1

Hi,

I am getting inconsistent behavior when trying to mutate instances.

Here is MyType.c3typ:

type MyType {
    idx : int
    currentIdx : int = 0

    next : member function() : int js all
}

And MyType.js:

function next() {

    if (this.currentIdx > this.idx) {
	return null;
    }
    var ret = this.currentIdx;
    this.currentIdx += 1;
    return ret;
}

From the console, running the following:

var instance = MyType.make({idx: 10});
while (true) {
   var it = instance.next();
   if (it === null) break;
   console.log(it, instance.currentIdx);
}

Works as expected: the loop exits after 10 iterations and currentIdx is updated at each iteration. Note that it only works if the method signature is js all. js server will cause an infinite loop.

Now if this code is executed from within C3, I am getting an infinite loop. instance.currentIdx stays at 0 (as logged in Splunk) and the action does not end.

Per this topic, the context should mutate since next is a member function and the instance is preserved during the loop. What am I missing?

Thanks


#2

js all indicates the code can run in the console or in the server. js server indicates it should only run in the server.

You are instantiating the object in the console, so if you run the logic in the server, it will create a copy of the object in the server execution environment, execute the logic, then return from it; as such, any changes to the object made within the logic is lost. If you made the whole code block a string and passed it to JS.exec, it would all execute server-side and you would get the expected behavior.


#3

I don’t understand. This works in the console indeed. But this does not work when I run it in the server. To make sure the code runs in the server, I have another type with a method js server called runAlgo:

var instance = MyType.make({idx: 10});
var i = 0;
while (i < 20) {
		i++;
		log.info(instance.next());
		log.info(instance.currentIdx);
}

Even though I call this from the console (MyOtherType.runAlgo()) this should exclusively run in the server so I don’t understand why isn’t the state preserved from one iteration to another in the server. This while loop runs in the server, instance is never sent to the client, so why would the context get discarded in between the iterations? (the log gets written to Splunk, not the console)


#4

That looks to me like it may be a short-coming of the current implementation of member functions in server-side execution.

// expecting currentIdx to be 1; got the expected result    
JS.exec("var instance = MyType.make({idx:10});instance.currentIdx++;instance")
"{"type":"MyType","idx":10,"currentIdx":1}"

// expecting currentIdx to be 1; got 0 instead
JS.exec("var instance = MyType.make({idx:10});instance.next();instance")
"{"type":"MyType","idx":10,"currentIdx":0}"

#5

The normal model for instances is that they are immutable once created, but we don’t enforce this strictly in JavaScript or Python because sometimes it’s useful during object setup.

However, the transitory state of an object within an execution environment (such as JavaScript) is not saved between calls. We treat calls as formal boundaries, as is necessary for a distributed system. If the JavaScript method is called from Java or Python (or from the REST API) there is nowhere to save transitory JavaScript state between calls.

To create persistent state, the type should be made an entity.

The reason it works in the Console is that you’ve created a var in the persistent global scope (window in the browser) that holds onto the otherwise transient instance.


#6

Thanks for those details. Making MyType an entity does not change this behavior, and I don’t think you’re suggesting to persist an iterator to the database at each iteration, so I’m not sure what’s the workaround.

I understand the need for formal boundaries but then, when implementing a custom Stream, how is one supposed to save the current stream state and iteration index without altering the internal state of the object?

The actual type that I am trying to implement and that does not behave as an iterator because the state is not persisted is the following:

type CsvStream mixes Stream<CsvLine> {
    // details about init & internal objects are omitted for sake of simplicity
    iterationIdx: int = 0
    hasNext: ~ js all
    next: ~ js all
}

If instance.next() cannot increase instance.iterationIdx, how is it supposed to return the next result?


S3File.writeObjs returns ArrayIndexOutOfBoundsException
#7

By declaring something as a stream, the system knows how to transport it (via materialization of the stream if necessary). Here is a simple example showing construction of a stream in JavaScript (similar to your original counter)

type Counter {
  counter: function(limit: !int): stream<int> js all
  sumTo:   function(n: !int): int js all
}
function counter(limit) {
  var iter = 0;
  return {
    hasNext: function() { return iter < limit; },
    next: function() { return iter++; }
  };
}

function sumTo(n) {
  var c = Counter.counter(n),
      s = 0;
  while (c.hasNext())
    s += c.next();
  return s;
}

Note that the stream can be serialized by materializing it (at least its result if not its internal state). The screen shot below shows using the Counter type (after provisioning) directly in the Console and remotely in the server using JS.exec.

Counter-console


S3File.writeObjs returns ArrayIndexOutOfBoundsException
#8

This solution is working as expected. I hope that transporting the stream isn’t too costly when the underlying objects (an array of instances) are large.
Thanks @JohnCoker :slight_smile: