Row limits for uploading files

[ssmith001]

Does anyone know if there is a data limit on the number of rows in a data file upload to then be used for a map dashboard visualization?

[MFGF]

Does anyone know if there is a data limit on the number of rows in a data file upload to then be used for a map dashboard visualization?

Hi,

There are limits on file size, but not row count. The limits are set by your Administrator, and can be changed:



Cheers!

MF.

[ssmith001]

Thank you, I didn't know that was a setting in the system. Can it be changed to any value and what are the concerns with setting it to a larger size?

[MFGF]

Thank you, I didn't know that was a setting in the system. Can it be changed to any value and what are the concerns with setting it to a larger size?

The underlying technology is using a Parquet file format - the file is loaded into memory when needed, and obviously the bigger the file, the more memory will be required and the longer the response times will be.

IBM made some changes for the 11.1.1 release and altered the format of the Parquet files used, as well as adding a new microservice utilising Spark SQL for querying data from the in-memory columnar data store. This has two benefits - faster access times all around, and also the ability to support larger data sets than 11.0.x would handle.

Cheers!

MF.