Large data sets with CloudTables

CloudTables is all about making it easy for you to work with your data. You shouldn't need to worry about performance with large data sets - even with millions of records. It should just work while still maintaining performance.

To date we've recommended that you not store more than 50'000 records in CloudTables. This wasn't a fixed limit, but rather a soft one imposed by how CloudTables operates. It would download the data to the client-side which would then apply ordering, filtering, etc, so there was a natural delay for larger data sets as the data was transferred over the network.

In order to address this, we've introduced a new server-side processing mode in CloudTables. If you have used DataTables previously, you might recognize this terminology - essentially the data processing is moved from the client to the server, so the database, which is highly optimized for actions such as ordering and filtering data, performs those actions and only the data needed for the current display is transmitted to the web-browser. Thus rather than downloading the full data set upfront, the data from it is loaded as required for display, resulting in a much more responsive end user experience.

Preview now available

By default CloudTables still operates in the original client-side processing mode, but it is possible to enable server-side processing for your application if you would like to try it out, or feel your data would benefit due to the size of the data sets you are working with.

To enable, go to Settings in your CloudTables application and enable the "Server-side processing" option under the Experimental heading. Your embedded tables will then immediately operate in server-side processing.

This feature is also available in the self-hosted package for CloudTables.

Limitations

You'll have noticed that it is labelled Experimental - this is not to cause you any concerns, but rather to highlight that not all aspects of CloudTables are fully integrated with this mode of operation yet. The four areas that are still under development are:

• Multi-row editing over multiple pages of data
• Export (Excel, PDF, CSV, Print)
• SearchBuilder component
• SearchPanes component

We will be adding these items one, by one, as development and QA on them completes.

Future

Once integration with all features of CloudTables is complete, all applications will be migrated to utilize server-side processing. The intention is that this will be a seamless transfer and you and your end users will notice no difference other than improved performance!

We welcome feedback on this new feature (as we do with all parts of CloudTables). Please get in touch if you have any questions about CloudTables with large data sets.