Elasticsearch : How to do Performance Testing?

Step-1: First perform index performance testing by ingesting data with following settings applied:

"refresh_interval" : "-1"
"number_of_replicas" : 0, 
"merge.scheduler.max_thread_count" : 1,
"translog.flush_threshold_size" : "1024mb",
"translog.durability" : "async"
"thread_pool.bulk.queue_size": 1000
"bootstrap.memlockall": True


Step-2: Scale data and Elastic nodes, JVM heap memory & ingest data and measure indexing performance.

At T1: 
curl -XGET http://localhost:9200//_stats/indexing?pretty=true | grep -Ei 'index_total|index_time_in_millis'
At T2:
 curl -XGET http://localhost:9200//_stats/indexing?pretty=true | grep -Ei 'index_total|index_time_in_millis'

Indexing rate = 1000(index_total(at T2) - index_total(at T1)) /  (index_time_in_millis(at T2) - index_time_in_millis(at T1))




Step-3: Use benchmarking tools such as Rally

https://esrally.readthedocs.io/en/stable/summary_report.html
https://github.com/elastic/rally

Step-4: In all the above operations, make sure to monitor the elasticsearch with elasticsearchHQ:
https://github.com/ElasticHQ/elasticsearch-HQ

Step-5: Revert setting applied in step-1 and do queries performance testing.


Study :: https://www.youtube.com/watch?v=lWKEphKIG8U
              https://www.youtube.com/watch?v=vruklYSW4jg
              https://www.youtube.com/watch?v=UPkqFvjN-yI
python-modules: elasticsearch-py, elasticsearch-dsl, elasticsutils, pyelastisearch

Step-6: Tune further as per -> https://www.elastic.co/blog/advanced-tuning-finding-and-fixing-slow-elasticsearch-queries