How to use fetchObjStream?

#1

What is the purpose of fetchObjStream? How to use it? Can someone provide me a use case?

0 Likes

#2

You get an object that has methods hasNext and next so you can iterate over what would have been returned as an array by fetch. It does not allocate memory for all the elements at once, and you can iterate over the object only once (check the doc).

0 Likes

#3

What doc? How do I do c3ShowType() on a method?

0 Likes

#4

fetchObjStream is very similar to fetch but it uses the streaming mechanism to fetch data from a persistable type.
The main use case is if the you are iterating through large amounts of data it would be advisable to use fetchObjStream instead of fetch
and simple use is as below

AwsBillDetails.fetchObjStream({
    filter: Filter.ge('UsageStartDate', range.fst.toString())
                  .and.lt('UsageStartDate', range.snd.toString())
                  .and.eq('RecordType', 'LineItem'),
    order: "UsageStartDate"
  }).each(function (resource) {
    var awsCost = AwsCost.make({
      'env': awsId? awsId.env : null,
      'pod': awsId? awsId.pod : null,
      'func': awsId? awsId.func: null,
      'role': awsId? awsId.role: null,
      'subComponent': resource.subComponent(),
      'lifecycle': resource.lifecycle(),
      'start': start,
      'end': end,
      'instanceType': resource.instanceType(),
      'interval': AwsCostInterval.DAY,
      'cost': resource.BlendedCost
    });
});
2 Likes

#5
c3ShowFunc(Persistable, "fetchObjStream")
1 Like

#6

How can I configure the batch size of fetchObjStream?
While fetchObjStream can reduce the memory usage, it can run significantly slower due to the increased number of IO calls. Some batch size configuration option in FetchStreamSpec would be helpful to tune up the performance.

0 Likes