Turboboot
Turboboot enables fast cold boots for your models
What is Turboboot for?
Turboboot reduces the cold boot time of your model by 40%-90%, compared to running the model on Banana without turboboot. This is especially beneficial if you're building a latency sensitive application or want to optimize costs.
Important to consider
Not all models are Turboboot compatible, you can check the requirements for compatibility here.
Turboboot doesn't improve the latency of warm calls. For example if networking is the bottleneck, switching on Turboboot won't necessarily help.
How to enable Turboboot for my model
When deploying a model on Banana, it's by default deployed with Turboboot off. You can at any time later enable Turboboot on your model. To make this happen follow these steps:
Make sure your model is Turboboot compatible
Go to model settings and switch on Turboboot
Note: switching your model to Turboboot triggers a new build during which you cannot call your model
Monitor build logs. If the model is not Turboboot compatible, it will still continue to work without Turboboot.
Once the build succeeds you'll be able to call your model just as before.
And that's it! 🍌
How to know if your model is compatible with Turboboot
The recommended & easiest way to ensure that your model works with Turboboot is by using Potassium. The Potassium framework is built to ensure compatibility with Banana infrastructure and Turboboot.
Potassium is the recommended & easiest way to ensure Turboboot compatibility
If you don't want to use Potassium you can make your model Turboboot compatible by using the legacy serverless template.
How to use templates with Turboboot?
You can identify Turboboot compatible templates by the yellow flag on the upper right corner. Note that if a template doesn't have the Turboboot flag, you won't be able to enable Turboboot on the template. Simply, if the template was made from a model which had Turboboot enabled, the template will also be enabled with Turboboot and vice versa.
If you want to use a template with Turboboot but the template doesn't have it enabled, the best way is to fork the repo, deploy the model, and enable Turboboot.
Last updated
