How to fetch historical versions of an entity type


#1

Is it possible for the platform to store every version of an entity type? What configuration to change to enable this?

If possible, how to fetch a historical version?


#2

@qiwei.li see c3ShowType(Ann.Db)

  /**
   * If true and version history is not disabled via {@link TenantConfig}, then any update to an obj will be recorded
   * in the {@link Persistable.versionEdits} field.  This recorded history can be used to retrieve objs at the state
   * they were in based on a previous time or version.
   *
   * @see Persistable.get
   * @see Meta.getVersion
   * @see Meta.getVersionAsOf
   */
  versionHistory:          boolean

#3

@rohit.sureka This is great! Thanks!

Just to make sure that the field to set True in TenantConfig is called ForceVersionEdits, is that correct?


#4

By default you don’t have to do anything. By default it should be enabled @trothwein can correct me if i’m wrong


#5

@qiwei.li No you don’t want to change that TenantConfig. That will force version edits for every update to every type. That is used only for debugging. If you want to enable it for a specific type then you would set the “versionHistory” for the db annotation that Rohit pointed out. Keep in mind that this has the following downsides:

  1. It is quite expensive from both a storage and performance perspective so it should not be used on types where the objects are frequently changed.
  2. Changes to stored calc fields are not tracked here, only user changes.

#6

@trothwein

if our app requires versioning on a type with no stored calcs, would you suggest we rely on this? Or is it so performance expensive that it would be worth writing our own method to track “diffs” rather than versions.


#7

@sean.summers, I would caution against writing your own diffs. The question is really how many updates there are (both total and per object). Assuming that those limits are reasonable rather than high velocity, then turning it on on a type is fine.


#8

@trothwein

Yes, wasn’t hoping to rely on writing our own diffs. The velocity of changes is slow (maybe 3 times a day at most), but the volume of data on the type is much larger than most. We’ll move forward with this and I’ll come back to the thread if we have any troubles.


#9

@sean.summers Three times a day on a large set of objs is probably not reasonable for this. It will create a child table (3 * * ) in size. Is this type in Cassandra? If it was, I’m not sure we really tested the feature on Cassandra. If not, I believe the PG table size will become unmanageable, impacting performance in numerous ways.


#10

@trothwein This will be in PG. What would you say are the upper bounds of this feature? Is there an alternative you’d suggest?


#11

@sean.summers There isn’t an exact recommendation. However, a table with ~10 million entries is “probably” ok. One with ~100 million is probably not. So if there are going to be ~ million entries in the root table. After a year there would be 300 million entries in the history table.


#12

@rohit.sureka @trothwein

Thank you for the useful notes. I can see the updates made to the type in the versionEdits field. Is there a way to directly fetch the previous version of the type? For instance, I would like to call MyType.fetch({filter:"(id=='ABC') && (version==1)}).


#13

@christian.brown Unfortunately that is not supported. Filtering only works on the latest/current value (it would be ridiculously expensive to apply it to past versions).


#14

Following the documentation for Meta.getVersion…

getVersion: int

Allows for getting a specific version of an Obj (based on its Persistable#version value} for entity types that have version history enabled. The requested version number should be specified in the input Obj instance for the Persistable#get function.

I’m not familiar with the get() function taking an object as an input. Could you clarify how to use this field to get a version of an object?


#15

Nvm, found answer here Get specific version of an object


#16

I should probably change the comment. Since Persistable.get is a member function (i.e. it has an implicit first arg of the obj it is called on), just set the value of the “meta.getVersion…” in your obj to what you are requesting and call .get() with that