John Alexis Guerra Gómez

Try your data

Navio

Hi ! Very impressed by this exploring tables tool. I have rightaway tested with some data from my scope (climate models output cataloging). I have to deal with very large catalogs (2.5M records).
Is there a way to handle this with navio ?

Patrick Brockmann

I think the problem is loading it into the browser. If you can manage to load the data there, then Navio should work. This specific workbook is slower because it tries to do a lot of guessing, for example figuring out if the file is csv, json or mongo dump. Also, the default mode of navio tries to guess the types of all the attributes in the file. If you load the data yourself, and specify manually the attribute types, navio will probably load. I have successfully tested it with datasets with 2M rows.

If you go ahead and create a notebook where you have loaded your data into the browser, and you share the link, I can try giving you pointers of how to put navio on top of that. Be aware though, I haven't been able to load more than 250MB of data in the browser before it crashing

Check this tweet out, where I easily visualized 3.3M records  https://twitter.com/duto_guerra/status/1301201262323609600?s=20