This section covers various aspects of optimizing pipeline performance in Seldon Core 2, from testing methodologies to Core 2 configuration. Each subsection provides detailed guidance on different aspects of pipeline performance tuning:
Understand how Core 2 components scale with the number of deployed pipelines and models:
Dynamic scaling of dataflow engine, model gateway, and pipeline gateway
Loading and unloading of models and pipelines
Assignment of pipelines and models to replicas
Each of these aspects plays a crucial role in achieving optimal pipeline performance. We recommend starting with testing individual models in your pipeline, then using those insights to inform your Core 2 configuration and overall pipeline optimization strategies.
