Extract Fact Table Distinct Columns Error

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Extract Fact Table Distinct Columns Error

qiumingming
hi All:
    When I use kylin 2.0 to build cube, I got a error at Extract Fact Table Distinct Columns step, mr job failed. My Hadoop version is 2.6.0-cdh5.4.4. Below is my error info:
    Error: java.io.IOException: Initialization of all the collectors failed. Error in last collector was :Cannot find serializer for org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:423) at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:82) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:718) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:790) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:350) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.io.IOException: Cannot find serializer for org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator.init(NativeMapOutputCollectorDelegator.java:147) at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:411)
Reply | Threaded
Open this post in threaded view
|

Re: Extract Fact Table Distinct Columns Error

shaofengshi
I'm not sure but I guess it is related with Hadoop version; Start from 2.0,
Kylin upgrades Hadoop version to 2.7. That means some feature may rely on
new Hadoop APIs. While Kylin 1.6 is compiled with Hadoop 2.6, that should
support your version.

2017-05-28 22:25 GMT+08:00 qiumingming <[hidden email]>:

> hi All:
>     When I use kylin 2.0 to build cube, I got a error at Extract Fact Table
> Distinct Columns step, mr job failed. My Hadoop version is 2.6.0-cdh5.4.4.
> Below is my error info:
>     Error: java.io.IOException: Initialization of all the collectors
> failed.
> Error in last collector was :Cannot find serializer for
> org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at
> org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:423)
> at
> org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:82) at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<
> init>(MapTask.java:718)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:790) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:350) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by:
> java.io.IOException: Cannot find serializer for
> org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at
> org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelega
> tor.init(NativeMapOutputCollectorDelegator.java:147)
> at org.apache.hadoop.mapred.MapTask.createSortingCollector(
> MapTask.java:411)
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Extract-Fact-Table-Distinct-Columns-Error-tp8112.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



--
Best regards,

Shaofeng Shi 史少锋
Reply | Threaded
Open this post in threaded view
|

Re: Extract Fact Table Distinct Columns Error

qiumingming
Thank you very much. I'll have a try.
Reply | Threaded
Open this post in threaded view
|

Re: Extract Fact Table Distinct Columns Error

shaofengshi
JIRA created for this issue, we will investigate:
https://issues.apache.org/jira/browse/KYLIN-2654

2017-05-28 23:21 GMT+08:00 qiumingming <[hidden email]>:

> Thank you very much. I'll have a try.
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Extract-Fact-Table-Distinct-Columns-Error-tp8112p8119.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



--
Best regards,

Shaofeng Shi 史少锋
Reply | Threaded
Open this post in threaded view
|

Re: Extract Fact Table Distinct Columns Error

qiumingming
Thanks a lot!!!