I have the problem of cube which exceed 2G.I have edited the cogtr.xml as below
<Preference Name="MultiFileCubeThreshold" Value="100000"/></Section>.
It does work,however ,the data isn't distributed in every mdp file .For example my second file exceed 2G while the others only have less than 1G,so I have to decrease the value of MultiFileCubeThreshold when one of my mdp file exceed 2G.
Anyone have solution to make it distribute in each mdp averagely or any other good idea to the trouble of 2G?
Do need your help. Thansk :)
Perhaps someone else can chip in too, but I don't think there's an option to set the size of each mdp or have the mdp's all be exactly the same size.
Transformer decides the size of the mdp and usually, it will come close to 2 gigs on each before moving onto the next. This can change if you have partitioning etc. set manually.
But the size between each mdp differ a lot .My second mdp is almost 2GB while some the others only has less than 50mb.I have generated 31 mdp file when I change the MultiFileCubeThreshold to 100000 and the mdp file doesn't increase when I change MultiFileCubeThreshold to 50000 .
Quote from: Arsenal on 09 Dec 2010 09:11:57 AM
Perhaps someone else can chip in too, but I don't think there's an option to set the size of each mdp or have the mdp's all be exactly the same size.
Transformer decides the size of the mdp and usually, it will come close to 2 gigs on each before moving onto the next. This can change if you have partitioning etc. set manually.
But the size between each mdp differ a lot .My second mdp is almost 2GB while some the others only has less than 50mb.I have generated 31 mdp file when I change the MultiFileCubeThreshold to 100000 and the mdp file doesn't increase when I change MultiFileCubeThreshold to 50000
Setting manually, do you mean to use cubegroup .I have partition my cube using cube group in company ,then I generate 80 mdc file .now ,I could only link to each sub-mdc file ,and couldn't get the one in total .Could i link to the one in total? Thanks
The output size of the mdp is basically determined by your partition strategy and the data of the cube, I believe. It could be that the single very large mdp you're seeing contains data (categories etc.) from your fact table and the significantly smaller mdp's are data from your dimensional tables.
The link below has good info on mdp's.
http://www-01.ibm.com/support/docview.wss?uid=swg21371419
Quote from: Arsenal on 10 Dec 2010 10:52:44 AM
The output size of the mdp is basically determined by your partition strategy and the data of the cube, I believe. It could be that the single very large mdp you're seeing contains data (categories etc.) from your fact table and the significantly smaller mdp's are data from your dimensional tables.
The link below has good info on mdp's.
http://www-01.ibm.com/support/docview.wss?uid=swg21371419
Thank you indeed. :)Alas,the reference you give me also tell me the mdp was distributed evenly
As you said,the biggest cube include the data from the fact table,howerver , if I decrease the parameter to a lower number,The size of my 2nd cube surely decrease.
If you knew ,could you explain what factors would have effects on my cube ,such as desird partrition size or maximum number of passes? thanks
your desired partition size will be driven by the number of consolidated records the cube will contain or should contain. Depending upon these consolidated records, the categories which fall within this partition size are picked up by Transformer for storing in pre summarized partitions for faster query performance. This is usually a trial and error number depending upon how many categories your cube has
number of passes means Transformer will reas through the data that many number of times when dtermining the partition size for optimum query performance. Higher number of passes translates to better pratitioning in many cases but cube build time also increases so it is a trade off