Problem in mapReduce job - java.lang.Boolean cannot be cast to java.lang.String

#1

Hi all,
I have a problem in my mapReduce job but I’m not able to find where the job fails.

I have the following functions defined:

  • map
  • mapCompleted
  • reduce

Seems that the map and mapCompleted functions are executed correctly, instead of reduce function that is not called. “In the middle” the system launch the following exception :
errorLog: c3.love.exceptions.C3RuntimeException: wrapped ClassCastException: java.lang.Boolean cannot be cast to java.lang.String at c3.engine.database.async.MapReduceMethods.callCompletedCallback (MapReduceMethods.java:786) at c3.love.exceptions.C3RuntimeException.wrapIt(C3RuntimeException.java:123) at c3.love.exceptions.C3RuntimeException.wrapIt(C3RuntimeException.java:81) at c3.server.impl.Task.run(Task.java:189) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:401) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:307) at c3.love.C3.dispatch(C3.java:676) at c3.love.C3.dispatchWithArgs(C3.java:756) at c3.love.C3.dispatch(C3.java:721) at c3.love.C3.dispatch(C3.java:717) at c3.type.batch.JobBase.updateJobState(JobBase.java:192) at c3.engine.database.async.JobBaseMethods.updateStatus(JobBaseMethods.java:167) at c3.type.batch.methods.JobBaseMethodsBase$15.accept(JobBaseMethodsBase.java:158) at c3.type.batch.methods.JobBaseMethodsBase$15.accept(JobBaseMethodsBase.java:156) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:401) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:307) at c3.love.C3.dispatch(C3.java:676) at c3.love.C3.dispatchWithArgs(C3.java:756) at c3.love.C3.dispatch(C3.java:721) at c3.love.C3.dispatch(C3.java:717) at c3.type.mapr.MapReduce$Subclass.updateStatus(MapReduce.java:1067) at c3.engine.database.async.MapReduceQueueMethods.lambda$compute$0(MapReduceQueueMethods.java:60) at c3.love.exceptions.C3X.wrap(C3X.java:247) at c3.love.C3.lambda$supplier$9(C3.java:912) at c3.love.C3.asUser(C3.java:1056) at c3.love.C3.asUser(C3.java:1037) at c3.engine.database.async.MapReduceQueueMethods.compute(MapReduceQueueMethods.java:41) at c3.type.async.methods.MapReduceQueueMethodsBase$1.accept(MapReduceQueueMethodsBase.java:39) at c3.type.async.methods.MapReduceQueueMethodsBase$1.accept(MapReduceQueueMethodsBase.java:37) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:401) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:307) at c3.love.C3.dispatch(C3.java:676) at c3.engine.database.async.InvalidationQueueMethods.compute(InvalidationQueueMethods.java:1218) at c3.engine.database.async.InvalidationQueueMethods.doDispatchCompute(InvalidationQueueMethods.java:404) at c3.engine.database.async.InvalidationQueueMethods.dispatchCompute(InvalidationQueueMethods.java:367) at c3.type.async.methods.InvalidationQueueMethodsBase$5.accept(InvalidationQueueMethodsBase.java:94) at c3.type.async.methods.InvalidationQueueMethodsBase$5.accept(InvalidationQueueMethodsBase.java:92) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassCastException: java.lang.Boolean cannot be cast to java.lang.String at c3.engine.database.async.MapReduceMethods.callCompletedCallback(MapReduceMethods.java:786) at c3.engine.database.async.MapReduceMethods.updateJobState(MapReduceMethods.java:755) at c3.type.mapr.methods.MapReduceMethodsBase$9.accept(MapReduceMethodsBase.java:116) at c3.type.mapr.methods.MapReduceMethodsBase$9.accept(MapReduceMethodsBase.java:114) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) ... 62 more java.lang.Boolean cannot be cast to java.lang.String java.lang.ClassCastException: java.lang.Boolean cannot be cast to java.lang.String at c3.engine.database.async.MapReduceMethods.callCompletedCallback(MapReduceMethods.java:786) at c3.engine.database.async.MapReduceMethods.updateJobState(MapReduceMethods.java:755) at c3.type.mapr.methods.MapReduceMethodsBase$9.accept(MapReduceMethodsBase.java:116) at c3.type.mapr.methods.MapReduceMethodsBase$9.accept(MapReduceMethodsBase.java:114) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:401) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:307) at c3.love.C3.dispatch(C3.java:676) at c3.love.C3.dispatchWithArgs(C3.java:756) at c3.love.C3.dispatch(C3.java:721) at c3.love.C3.dispatch(C3.java:717) at c3.type.batch.JobBase.updateJobState(JobBase.java:192) at c3.engine.database.async.JobBaseMethods.updateStatus(JobBaseMethods.java:167) at c3.type.batch.methods.JobBaseMethodsBase$15.accept(JobBaseMethodsBase.java:158) at c3.type.batch.methods.JobBaseMethodsBase$15.accept(JobBaseMethodsBase.java:156) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:401) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:307) at c3.love.C3.dispatch(C3.java:676) at c3.love.C3.dispatchWithArgs(C3.java:756) at c3.love.C3.dispatch(C3.java:721) at c3.love.C3.dispatch(C3.java:717) at c3.type.mapr.MapReduce$Subclass.updateStatus(MapReduce.java:1067) at c3.engine.database.async.MapReduceQueueMethods.lambda$compute$0(MapReduceQueueMethods.java:60) at c3.love.exceptions.C3X.wrap(C3X.java:247) at c3.love.C3.lambda$supplier$9(C3.java:912) at c3.love.C3.asUser(C3.java:1056) at c3.love.C3.asUser(C3.java:1037) at c3.engine.database.async.MapReduceQueueMethods.compute(MapReduceQueueMethods.java:41) at c3.type.async.methods.MapReduceQueueMethodsBase$1.accept(MapReduceQueueMethodsBase.java:39) at c3.type.async.methods.MapReduceQueueMethodsBase$1.accept(MapReduceQueueMethodsBase.java:37) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:401) at c3.server.impl.InteractiveDispatcher.dispatch(InteractiveDispatcher.java:307) at c3.love.C3.dispatch(C3.java:676) at c3.engine.database.async.InvalidationQueueMethods.compute(InvalidationQueueMethods.java:1218) at c3.engine.database.async.InvalidationQueueMethods.doDispatchCompute(InvalidationQueueMethods.java:404) at c3.engine.database.async.InvalidationQueueMethods.dispatchCompute(InvalidationQueueMethods.java:367) at c3.type.async.methods.InvalidationQueueMethodsBase$5.accept(InvalidationQueueMethodsBase.java:94) at c3.type.async.methods.InvalidationQueueMethodsBase$5.accept(InvalidationQueueMethodsBase.java:92) at c3.server.engine.TypeSysEngine.execute(TypeSysEngine.java:65) at c3.server.impl.Task.doFilter(Task.java:242) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:193) at c3.server.impl.dataCache.DataCacheEngine.doFilter(DataCacheEngine.java:125) at c3.server.impl.ServerDispatcherBase$ActionFilterChainImpl.doFilter(ServerDispatcherBase.java:191) at c3.server.impl.ServerDispatcherBase.doFilter(ServerDispatcherBase.java:161) at c3.server.impl.InteractiveDispatcher.doFilter(InteractiveDispatcher.java:81) at c3.server.impl.Task.run(Task.java:178) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Do you know what kind of problem is it?

0 Likes

#2

What does your type definition look like?

0 Likes

#3

I have a type that extends MapReduce

entity type AbstractMetricResultExportJob extends MapReduce<TYPE, string, string, int> type key ‘TMREJ’

That have the following methods defined

startJob: function(
typeName : !string,
evalMetricsSpec: !EvalMetricsSpec,
startObj : !json,
endObj : !json
): AbstractMetricResultExportJob js server
createDynamicDate: function(spec: !json): datetime js all
map: ~ js server
mapCompleted: function(job: MapReduce): boolean js server
reduce: ~ js server

  • The Map phase: runs a metric evaluation, process the evaluation results and stores the result in CSV format on S3.
  • The Reduce phase: connects to Redshift and for each file exported by the Map phase it requests Redshift to execute a Copy statement.
0 Likes

#4

Hi, any suggestions?

0 Likes

#5

@lauraminicucci what does the “return” statement in the map function return?

0 Likes

#6

It returns a variable “output” defined as follow:

var output = new C3.typesys.Mapp(new C3.typesys.MappType(C3.typesys.PrimitiveType.Str, C3.typesys.PrimitiveType.Str));

It should be FixedAsset type.

0 Likes

#7

try to return an array of objects return myArray[“myobjects”] = output

0 Likes

#8

Hi,
maybe I didn’t understand well what you mean :slight_smile:
Currently I have variable “output” that is like:

{“FixedAsset”:“s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0038505-001.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048597-001.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048597-002.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048598-001.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048598-002.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048598-003.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048598-004.csv,s3://engie–stage-engiegridops3/fs/c3/c3/engie-claradomus/laura/FixedAssetMetricResultExportJob/2019_03_10/2019_03_11/FixedAsset/0048598-005.csv”}

I tryied to “convert” it before return the object, like:

var myArray = [“FixedAsset”, output.FixedAsset];
return myArray;

I have same error as before with the “simple” object.

0 Likes

#9

The error occurred inside the mapCompleted function. Changing the type returned the problem was fixed.

Thanks
Laura

0 Likes