Sample Cube Build fail at Step#3

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Sample Cube Build fail at Step#3

Barry Han
This post was updated on .
Hi Kylin developer,

my hadoop environment :
HDP 2.6
Kylin2.0

When I try to add datasource, Kylin sync hive table is okey.
But, when I try to build the cube, I got error at step 3 as follow:

Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:232)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:117)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:82)
        ... 87 more
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
        at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
        at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:213)
        ... 89 more
)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:212)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
        ... 41 more

result code:2


Please Help!

Thank you!
Barry Han
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

shaofengshi
Hi Barry, where is this error from, kylin.log or Mapper's log?

2017-06-01 11:24 GMT+08:00 Barry Han <[hidden email]>:

> Hi Kylin developer,
>
> my hadoop environment :
> HDP 2.6
> Kylin2.0
>
> When I try to add datasource, Kylin sync hive table is okey.
> But, when I try to build the cube, I got error at setp 3 as follow:
>
> *Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke
> the "BONECP" plugin to create a ConnectionPool gave an error : The
> specified
> datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH.
> Please check your CLASSPATH specification, and the name of the driver.*
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(
> ConnectionFactoryImpl.java:232)
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(
> ConnectionFactoryImpl.java:117)
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(
> ConnectionFactoryImpl.java:82)
>         ... 87 more
> *Caused by:
> org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundExcepti
> on:
> The specified datastore driver ("com.mysql.jdbc.Driver") was not found in
> the CLASSPATH. Please check your CLASSPATH specification, and the name of
> the driver.*
>         at
> org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.
> loadDriver(AbstractConnectionPoolFactory.java:58)
>         at
> org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.
> createConnectionPool(BoneCPConnectionPoolFactory.java:54)
>         at
> org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(
> ConnectionFactoryImpl.java:213)
>         ... 89 more
> )
>         at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(
> RetryingHMSHandler.java:212)
>         at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.
> invoke(RetryingHMSHandler.java:105)
>         at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.<
> init>(RetryingHMSHandler.java:78)
>         ... 41 more
>
> result code:2
>
>
> Please Help!
>
> Thank you!
> Barry Han
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Sample-Cube-Build-fail-at-Step-3-tp8134.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



--
Best regards,

Shaofeng Shi 史少锋
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

Barry Han
Hi Shaofeng,

It is from kylin.log.
I have checked that there is mysql-connector-java.jar in $HIVE_HOME/lib/ in HiveServer2

Thank you for your reply.

Barry Han
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

Barry Han

fixed now


Thank you
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

qiumingming
      Hi,I want to know what's specific version your hadoop is,My Hadoop version is 2.6.0-cdh5.4.4. I got a error at Extract Fact Table Distinct Columns step, mr job failed. The error info is :java.io.IOException: Initialization of all the collectors failed. Error in last collector was :Cannot find serializer for org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at org.apache.hadoop.mapred.MapTask.createSortingCollector.

     It seems you did not meet this problem. Did you change anything, such as maven dependency. Thank you very much!
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

shaofengshi
Mingming, Barry's environment is Hortonworks Data Platform 2.6, whose
Hadoop version should be > 2.7. So he doesn't have the problem you faced.

2017-06-01 15:12 GMT+08:00 qiumingming <[hidden email]>:

>       Hi,I want to know what's specific version your hadoop is,My Hadoop
> version is 2.6.0-cdh5.4.4. I got a error at Extract Fact Table Distinct
> Columns step, mr job failed. The error info is :java.io.IOException:
> Initialization of all the collectors failed. Error in last collector was
> :Cannot find serializer for
> org.apache.kylin.engine.mr.steps.SelfDefineSortableKey at
> org.apache.hadoop.mapred.MapTask.createSortingCollector.
>
>      It seems you did not meet this problem. Did you change anything, such
> as maven dependency. Thank you very much!
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Sample-Cube-Build-fail-at-Step-3-tp8134p8140.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



--
Best regards,

Shaofeng Shi 史少锋
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

qiumingming
Ok, I got it, I mistook HDP2.6 as hadoop 2.6 before. Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

shaofengshi
In reply to this post by Barry Han
Hi Barry, could you please share how you fixed the problem? That would
benefit other users who face the same error. Thx!

2017-06-01 14:38 GMT+08:00 Barry Han <[hidden email]>:

>
> fixed now
>
>
> Thank you
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Sample-Cube-Build-fail-at-Step-3-tp8134p8137.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



--
Best regards,

Shaofeng Shi 史少锋
Reply | Threaded
Open this post in threaded view
|

Re: Sample Cube Build fail at Step#3

Barry Han
I just add a configuration in kylin_hive_conf.xml as follow:

<property>
        <name>hive.metastore.uris</name>
        <value>thrift://localhost:9083</value>
</property>