10 Replies Latest reply on Oct 5, 2020 5:58 AM by Erik Little

    AWS EC2 and DEQ

    Erik Little Active Member

      We ran into a bit of performance issue with our m4.4xl (16 CPU, 64GB) DEQ profiling 550 GB snappy.parquet file.

       

      We are going to an r series EC2 instance which is memory optimized.

       

      This is the first of many business areas that will be adopting DEQ and they will be profiling large files as well.

       

      Anyone have any best practices on performance tuning the environment to handle large data sets like this?