Container complete event for unknown container id

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Container complete event for unknown container id

星辰#&勿语
Hi:
when I cube today a error like that
```
0 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1494561372318_21813_01_018423```on Build N-Dimension Cuboid Data Can I help me ?


[hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Container complete event for unknown container id

星辰#&勿语
I found that at MR log

 Could not deallocate container for task attemptId attempt_1494561372318_21842_r_000010_3It seems memory is not enough for Running kylin job in mine cluster?then I google for thatyarn.nodemanager.resource.memory-mb=24576
yarn.scheduler.maximum-allocation-mb=24576
it's useful?


[hidden email]
 
From: [hidden email]
Date: 2017-05-22 21:14
To: dev
Subject: Container complete event for unknown container id
Hi:
when I cube today a error like that
```
0 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1494561372318_21813_01_018423```on Build N-Dimension Cuboid Data Can I help me ?
 
 
[hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Container complete event for unknown container id

Billy Liu
Please check if your YARN container has enough resource for MR jobs. By
default, if in-memory build is used, kylin_job_conf_inmem.xml will ask for
3072MB for MapReduce Job. You could reduce the number.

2017-05-22 21:59 GMT+08:00 [hidden email] <[hidden email]>:

> I found that at MR log
>
>  Could not deallocate container for task attemptId
> attempt_1494561372318_21842_r_000010_3It seems memory is not enough for
> Running kylin job in mine cluster?then I google for
> thatyarn.nodemanager.resource.memory-mb=24576
> yarn.scheduler.maximum-allocation-mb=24576
> it's useful?
>
>
> [hidden email]
>
> From: [hidden email]
> Date: 2017-05-22 21:14
> To: dev
> Subject: Container complete event for unknown container id
> Hi:
> when I cube today a error like that
> ```
> 0 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator:
> Container complete event for unknown container id
> container_1494561372318_21813_01_018423```on Build N-Dimension Cuboid
> Data Can I help me ?
>
>
> [hidden email]
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Re: Container complete event for unknown container id

星辰#&勿语
fast cubing (in-memory cubing) used more memory .but I found that job failed in reduce task and map tasks all be successed.
I had set
kylin.job.mr.config.override.mapreduce.map.java.opts=-Xmx10g
kylin.job.mr.config.override.mapreduce.map.memory.mb=10240
in kylin.properties and it will be replaced in kylin_job_conf_inmen.xml
All right ?


[hidden email]
 
From: Billy Liu
Date: 2017-05-23 09:38
To: dev
Subject: Re: Container complete event for unknown container id
Please check if your YARN container has enough resource for MR jobs. By
default, if in-memory build is used, kylin_job_conf_inmem.xml will ask for
3072MB for MapReduce Job. You could reduce the number.
 
2017-05-22 21:59 GMT+08:00 [hidden email] <[hidden email]>:
 

> I found that at MR log
>
>  Could not deallocate container for task attemptId
> attempt_1494561372318_21842_r_000010_3It seems memory is not enough for
> Running kylin job in mine cluster?then I google for
> thatyarn.nodemanager.resource.memory-mb=24576
> yarn.scheduler.maximum-allocation-mb=24576
> it's useful?
>
>
> [hidden email]
>
> From: [hidden email]
> Date: 2017-05-22 21:14
> To: dev
> Subject: Container complete event for unknown container id
> Hi:
> when I cube today a error like that
> ```
> 0 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator:
> Container complete event for unknown container id
> container_1494561372318_21813_01_018423```on Build N-Dimension Cuboid
> Data Can I help me ?
>
>
> [hidden email]
>
Loading...