Experiences on cut-down models

[sascha]

Hi folks

I know about the theoretical idea of the cut-down models. I have tried it several times and found that this really makes a big difference: the cut-down job takes its time to run and the database gets much bigger than without. What I haven't really found is any change or big performance improvement in the app. That's why I didn't use it anymore for the last  years.

I would like to hear your opinion about cut-down models. Do you use it? What is your experience? When and why do/don't you use it.

Open discussions are very welcome.

[StuartS]

Hello

An interesting post that has prompted a 30 mins discussion with another developer.

The result of our discussion is that we do not see any tangible benefit in our models for cut down.  This may be because we do not have any dimensions in our models of 000's of items.  My understanding is that the cut down models would reduce the size of the xml/structure for the user to download from the server when opening a node.  As this is the first 20% of the progress bar and this happens in about 15seconds on our models the scope for noticeable change is small.

There are also significant considerations in model design for assisting cut down models. See http://support.cognos.com/opendocs/en/html/planning/8.4/contrib_admin_id7466restrictionstocuttingdowndimensions.html

Please correct me if there are disagreement's.

Regards

Stuart

[sascha]

My understanding is that the cut down models would reduce the size of the xml/structure for the user to download from the server when opening a node.

That's what I think, too. There are two definitions that will be send to you on opening a node. One is the model definition and the other one is the data definition. Do you think this only applies to the model definition? If so that would not increase performance significally because this part is anyway only a small piece (as you said, 20% of the status bar, which is the fastest percentage ;D anyway in the whole loading process). So up to here I think this "performance increasement" is not worth the size you're increasing your database.

Looking at the data block the option 'cut-down model' does not affect it. If you are measuring your models with the appsize reader it does not make any differences if usiong cut-down or not - even if the same rules applies that are listed on you link to the support homepage.

So - why using cut-down at all?

[StuartS]

Interesting question.  I used to think it may be because you could improve performance of some hardware.  But, then we had to upgrade client machines for our users because of performance on saving and calculating within models.

[SomeClown]

Cut-downs affect both data and model block.  When done properly, a cut-down model reduces the model block for a leaf or parent node, but this reduction only exists if a dlist item is NoData (and resolves to NoData).  Thus, if a definition in the model is gone via cutdown, there would never be any data in that portion.  Presumably, the data would have been there only via Imports (since the user doesn't care about that hidden data). 

Cut-downs become more useful once you move past 1 million cells per elist item, and are nearly required once you exceed 5 million cells per elist item.  The end-user requires the smaller size to operate and it has seemed to me that administrative functions also benefit (though this is only opinion on my part).

If your model size is not reduced by cutdown, then your application (as defined) is not a good candidate for cutdown. 

The database gets larger because Cutdown creates a custom model block for each elist item (when cutdown to each elist item) or just to parent nodes (aggregrate) - leaf nodes share the use of a parent model cutdown block.  These custom models are stored in the database.

[sascha]

The end-user requires the smaller size to operate

As far as I know the workload (without cut-down) to generate the blocks that the user requests with opening his node will take place on the database server. In this case the db server will prepare the block every time a user requests that block while working with cut-down this block is pre-prepared (done by the cut-down model job before GTP).
So there should only be a enhancement for the loading(/waiting) time and not for the real performance when working in the app.

On the other handside the data block gets always cut-down regardingless if cut-down option is enabled or not.

[SomeClown]

Blocks are not generated by DB server but only saved as BLOB types - so no database work (simple read/write) - can't remember which table they are stored in.  Reconcile is the process that creates and manages data blocks (that's why you have to run a GTP after Prepare Import - get the Reconcile to read the MIBs to put it to the data blocks for each elist item).  Without Cutdowns, there is only one Model definition block (managed by the GTP process of the CAC).

Data block and cut-down: I think performance works a little different for parent nodes but it's been a while since I've had to look at it - haven't really thought about it for a while. 

In theory, once it's loaded, you shouldn't see differences in performance, but physical environment constraints may affect performance in the larger models - e.g. cutdown allows the whole model in physical RAM on the client rather than forcing it to use virtual memory

[ExpertPlanner]

I believe there are 2 points at play here:

1) the size of a leaf node may be significantly decreased with the effective use of 'No Data' Access tables.  This will have significant performance implications for the end user.  This a 'cut down model'.

2) The box labeled 'Use Cut Down Models' is a seperate issue that determines when the processing involved with implementing the 'No Data' rule is performed.  The model will still be 'cut down' whether this box is checked or not.  Whether or not to check this box is really a decision of balancing out the cost of increased GTP time vs. a small improvement on opening an elist node for the end user. 

Item 1 above may not be successfully completed without the use of epmodelsizereader.exe.  The use of 'No Data' in access tables is not always straight forward and comprehensive documentation can be tough to come by.  Using epmodelsizereader.exe is a MUST!

I will add the disclaimer that the information above is based primarily on observation.

Thanks,
Steve

[mrobby]

I have tried epmodelsizereader.exe and using extensive no data access tables and in the end I believe the model performance gains are not significant based on the same 2 points above.

Restrictions on Cut Downs and size of D-Lists.

[sascha]

@mrobby
You don't think that the performance gain is not significant? I do. Try let's say a five million cube w and w/o No Data Access Tables for cutting them down to a 500k cell cube. The preformance for the end user will be significant.
I have had models that were only GTPable when having the correct Access Tables set up.

[mrobby]

I guess I should have stated a little bit further.  Since I dont happen to have any Dlists of great length and many of my dlists are potentially restricted for cut down that I personally haven't seen large performance gains.  I do understand the concept of how cut downs are helpful for extrememly large cubes, and are a must when you use a "Dummy E List" and set up access to large cubes through access tables.  In these cases you would have to use cut downs as the model would most likely not even open at 5 million cells per slice.

In my case I have around 800k cells per slice and an e list of 230 items.  After reviewing items for cut down and testing, I happened to find that in my case, none of my large cubes that were good candidates for cut down provided me significant gains when I applied cut down models.

[blackadder]

I presume you know about the 'gotcha' where if a DList is cutdown in one cube and used as a DList formatted item in another it won't be cut-down in either cube?

My experience is that Cutdown is essential for sparse models (e.g. HR) but the lack of model definition caching often offsets model size differences in other models.

[Gunes]

Hi folks,

First post since joining the forums & an interesting topic at that...

Just to clear up a common confusion of Cutdowns being enabled or disabled/ on or off.

If you have access tables, cutdown occurs whether you have cutdowns turned on or off.

It's just a matter of where it happens and it's normally regarded as trade-off from a technical perspective.

If you have cutdown turned off, you only have 1 master model definiton that is in production irrelevant of the number of elist items you might have (you also have model definisions for Dev & Archive but I won't get into those for now). The master model definition is brought down to the client machine (along with the data block which is elist specific irrelevant of whether you have cutdowns on or off) and the cutdown will happen on the client machine (as the client has a lite version of the J calc engine used by the job servers)

If you have cutdown turned on (whether it's for aggregates or all elist items) then the whole cut down job becomes an admin task for the server hence the cut down job that is created alongside a reconcile. This creates a cut down model definition for EVERY elist item and this is stored in the Application Database/Schema (the nodesetmodel table to be specific), and when users click on their specific node, then the specific post-cutdown model definition is brought down to the client machine.

The advantages of having cut-down models turned on are most likely evident on a model/application which is very large but also very sparse (as mentioned by others above), to ideally take advantage of NO-DATA access tables to slim down each model definition. Also, as the cut down happens on the server, the run-time for the client is faster, so for end users who might have a low spec PC, the performance to resolve the definition would increase. This would also be advantageous for clients with low network bandwidth. This was a crucial & very useful bit of functionality when it was introduced sometime ago, as PC specs and technology wasn’t as advanced or as cheaply available as it is today.

On the flipside, the dis-advantages of having it switched on, causes an explosive growth in the Database size and spawns a cut down job prior to a reconcile job, which can depending on the size of the model, the number of elist items and the access tables associated with it, can take up some time to complete. As from a technology perspective, better PC specs are much easier to come by nowadays, so some testing & analysis would be my advice before turning cutdowns on or off :)

[sascha]

Hi Gunes

welcome to the board and a very big thank you for this impressive answer. I think this covers quite all sides of cutting down models and will give all of us a better understanding of its practical use.

Thanks,
Sascha

[blackadder]

One (hopefully!) final note on this topic.
I was doing an application audit for a new customer the other day, and recommended that they should try turning off cut-down models. It worked, improving model load times and of course removing the need to wait for the Cut-down job each time they GTPd.
The customer had read this thread before my visit but didnt think it fully covered why turning this feature *off* would improve the load times so I thought I'd post a brief explanation (this is as I understand it....I'm just an application guy and I'm sure the server gurus on here will correct me if I'm wrong).

As mentioned previously, the Cut-down job creates seperate model definitions with the relevant access tables applied for each eList item. This means that a ready-made data slice can be downloaded when the user clicks on the node. When cut-down is turned off, the full model definition is downloaded for all nodes, and access tables are applied client side in real time.

However this full model definition is cached after the first node has been opened, and sometimes the advantages of this cache outweigh the disadvantage of having to apply the access tables dynamically. This is why the recommendation is normally only to turn cut-down on when the model is really sparse.

Hope that helps!
p.s. Hi Gunes. Good to see you on here.

[SomeClown]

Caching would be one reason though technically, the second time a cutdown model is opened from the same client, it should be reading from local cache of the client (this is how it was supposed to work at one time, not sure if it ever did, or still does).

The other one is that complex access tables (more than 2 dimensions, lots of entries, etc) seem to make a cutdown definition worse in some cases -- not sure why or if it is true - just something noticed over the years.

[craig_karr]

Just a silly question; when using cut down models, should the xml definition of the application increase or decrease?

I have strangely enogh experienced both, i.e both that the production applications gets larger than dev when cut downs are enabled and vice versa.

My theory was that perhaps the total xml definition gets larger with cut down models than without, but the user is only donwloading a part of the total definition when cut downs are enabled.

Or is it so that the xml definition always should get smaller with cut downs enabled?

What could the reason be when it is the other way around?