Problem while build cube:File not found for hdfs://...

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Problem while build cube:File not found for hdfs://...

排骨瘦肉丁
when i build my cube on step 3#:
2017-02-08 16:59:04,179 ERROR [pool-9-thread-1] common.MapReduceExecutable:127 : error execute MapReduceExecutable{id=20d6e927-f2d8-490c-8beb-054fa58708a9-02, name=Extract Fact Table Distinct Columns, state=RUNNING}
java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/home/root/apache-hive-1.2.1-bin/lib/hive-exec-1.2.1.jar
        at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1072)
        at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1064)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1064)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:99)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
        at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
        at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
        at org.apache.kylin.engine.mr.common.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:149)
        at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:108)
        at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
        at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
why it look for hdfs??
/home/root/apache-hive-1.2.1-bin/ is my hive installation directory
where is my wrong configuration??
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

bertram
hi, have you fixed this problem? i got the same error
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

排骨瘦肉丁
Not yet!
People who encounter this problem seem a lot.
if you fix the problem,please update it here
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

bertram
when i start hadoop job history server and restart kylin , the error is missing ,but MR job failed
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

排骨瘦肉丁
if you build a new cube,the error will reappear..
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

bertram
i have build sample succeed when use another hadoop cluster
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

排骨瘦肉丁
It seems that the problem is still in the Hadoop environment。No wonder the official proposal to use the Enterprise Edition,Most of them are environmental problems。。
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

bertram
i agree with you, maybe you need a hadoop expert
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

Yang
Are you running a popular Hadoop release or making your own?

My experience is popular Hadoop releases like CDH or HDP usually don't have
such problems.

Yang

On Thu, Feb 16, 2017 at 9:48 AM, bertram <[hidden email]> wrote:

> i agree with you, maybe you need a hadoop expert
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Problem-while-build-cube-File-not-found-for-
> hdfs-tp7139p7217.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

排骨瘦肉丁
Neither CDH nor HDP,
i download it from http://hadoop.apache.org/releases.html
where can i download CDH or HDP version ?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

Billy Liu
google 'HDP sandbox download' will lead you to the right place.

2017-02-21 17:59 GMT+08:00 排骨瘦肉丁 <[hidden email]>:

> Neither CDH nor HDP,
> i download it from http://hadoop.apache.org/releases.html
> where can i download CDH or HDP version ?
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/Problem-while-build-cube-File-not-found-for-
> hdfs-tp7139p7266.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

shaonian
In reply to this post by 排骨瘦肉丁
hi,I got the same problem,have you fixed this?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

shaonian
In reply to this post by 排骨瘦肉丁
hi,I have solved this problem.
If you use a cluster instead single node,this problom would not occur.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem while build cube:File not found for hdfs://...

KevinLei
Hi, Shaonian


cluster here you mean is kylin cluster or hadoop cluster? hhase? hive?
Loading...